Two ways to fine-tune
Fireworks offers two fundamentally different approaches to fine-tuning. Choose the one that fits your needs:Managed Fine-Tuning
Give Fireworks your data and configuration. The platform handles scheduling, training, checkpointing, and model output. No custom code required.Best for teams that want fast results with standard objectives (SFT, DPO, RFT).
Training SDK (Tinker compatible)
Write custom Python training loops. You control the loss function, optimizer step, checkpointing, and weight sync. Fireworks handles the distributed GPU infrastructure.Best for research teams needing custom objectives, full-parameter tuning, or inference-in-the-loop evaluation.
| Managed Fine-Tuning | Training SDK | |
|---|---|---|
| Control | Configuration-driven | Full Python loop control |
| Objectives | Built-in SFT, DPO, RFT | Any custom loss function |
| Tuning method | LoRA | Full-parameter or LoRA |
| Inference during training | Not available | Hotload + sample mid-training |
| Interface | UI, firectl, REST API | Python SDK |
| Best for | Production fine-tuning with standard methods | Research, custom RL, hybrid losses |
When to use SFT vs. RFT
In supervised fine-tuning, you provide a dataset with labeled examples of “good” outputs. In reinforcement fine-tuning, you provide a grader function that can be used to score the model’s outputs. The model is iteratively trained to produce outputs that maximize this score. Supervised fine-tuning (SFT) works well for many common scenarios, especially when:- You have a sizable dataset (~1000+ examples) with high-quality, ground-truth labels.
- The dataset covers most possible input scenarios.
- Tasks are relatively straightforward, such as:
- Classification
- Content extraction
- Your dataset is small.
- You lack ground-truth outputs (a.k.a. “golden generations”).
- The task requires multi-step reasoning.
When to use the Training SDK instead
Move from managed fine-tuning to the Training SDK when you need:- Custom loss functions — hybrid GRPO + DPO, custom reward shaping, or any non-standard objective
- Full-parameter tuning — update all model weights instead of a LoRA adapter
- Inference-in-the-loop evaluation — hotload checkpoints onto a serving deployment and sample mid-training
- Per-step control — custom gradient accumulation, dynamic learning rate schedules, or algorithm research