Skip to main content
Before launching an RFT job using the CLI or Web UI, ensure you meet these prerequisites and understand the validation process.

​
Prerequisites

Before launching an RFT job, ensure you have:
Your dataset must be in JSONL format with prompts (system and user messages). Each line represents one training example.Upload via CLI:
eval-protocol create dataset my-dataset --file dataset.jsonl
Or via the Fireworks dashboard.
Your reward function must be tested and uploaded. For local evaluators, upload via pytest:
cd evaluator_directory
pytest test_evaluator.py -vs
The test automatically registers your evaluator with Fireworks. For remote evaluators, deploy your HTTP service first.
Set your API key as an environment variable:
export FIREWORKS_API_KEY="fw_your_api_key_here"
Or store it in a .env file in your project directory.
Choose a base model that supports fine-tuning. Popular options:
  • accounts/fireworks/models/llama-v3p1-8b-instruct - Good balance of quality and speed
  • accounts/fireworks/models/qwen3-0p6b - Fast training for experimentation
  • accounts/fireworks/models/llama-v3p1-70b-instruct - Best quality, slower training
Check available models at fireworks.ai/models.

​
Job validation

Before starting training, Fireworks validates your configuration:
  • ✅ Valid JSONL format
  • ✅ Each line has messages array
  • ✅ Messages have role and content fields
  • ✅ File size within limits
  • ❌ Missing fields → error with specific line numbers
  • ❌ Invalid JSON → syntax error details
  • ✅ Evaluator code syntax is valid
  • ✅ Required dependencies are available
  • ✅ Entry point function exists
  • ✅ Test runs completed successfully
  • ❌ Import errors → missing dependencies
  • ❌ Syntax errors → code issues
  • ✅ Sufficient GPU quota
  • ✅ Base model supports fine-tuning
  • ✅ Account has RFT permissions
  • ❌ Insufficient quota → request increase
  • ❌ Invalid model → choose different base model
  • ✅ Parameters within valid ranges
  • ✅ Compatible parameter combinations
  • ❌ Invalid ranges → error with allowed values
  • ❌ Conflicting options → resolution guidance
If validation fails, you’ll receive specific error messages with instructions to fix the issues.

​
Common errors and fixes

Error: Dataset validation failed: invalid JSON on line 42Fix:
  1. Open your JSONL file
  2. Check line 42 for JSON syntax errors
  3. Common issues: missing quotes, trailing commas, unescaped characters
  4. Validate JSON at jsonlint.com
Error: Missing required field 'messages'Fix: Each dataset row must have a messages array:
{"messages": [{"role": "user", "content": "..."}]}
Error: Evaluator 'my-evaluator' not found in accountFix:
  1. Upload your evaluator first: 
    cd evaluator_directory
pytest test_evaluator.py -vs
  2. Or specify evaluator ID if using UI:
Error: Insufficient GPU quota for this jobFix:
  1. Check your current quota at Account Settings
  2. Request a quota increase through the dashboard
  3. Or choose a smaller base model to reduce GPU requirements
Error: Learning rate 1e-2 outside valid range [1e-5, 5e-4]Fix: Adjust the parameter to be within the allowed range:
--learning-rate 1e-4  # Use default value
See Parameter Reference for all valid ranges.
Error: Evaluator build timed out after 10 minutesFix:
  1. Check build logs in Evaluators dashboard
  2. Common issues:
    • Large dependencies taking too long to install
    • Network issues downloading packages
    • Syntax errors in requirements.txt
  3. Wait for build to complete, then retry launching your job
  4. Consider splitting large dependencies or using lighter alternatives

​
What happens after launching

Once your job is created, here’s what happens:
1

Job queued

Your job enters the queue and waits for available GPU resources. Queue time depends on current demand.Status: PENDING
2

Dataset validation

Fireworks validates your dataset to ensure it meets format requirements and quality standards. This typically takes 1-2 minutes.Status: VALIDATING
3

Training starts

The system begins generating rollouts, evaluating them, and updating model weights. You’ll see:
  • Rollout generation and evaluation
  • Reward curves updating in real-time
  • Training loss decreasing
Status: RUNNING
4

Monitor progress

Track training via the dashboard. See Monitor Training for details on interpreting metrics and debugging issues.Status: RUNNING → COMPLETED
5

Job completes

When training finishes, your fine-tuned model is ready for deployment.Status: COMPLETEDNext: Deploy your model for inference.

​
Next steps

Launch with CLI

Use eval-protocol CLI for fast, scriptable launches

Launch with Web UI

Use the dashboard for visual, guided job creation

Monitor training

Track job progress, inspect rollouts, and debug issues