Reinforcement Fine-Tuning (deprecated)

This page is kept as a reference for jobs that are already running. It is no longer a starting point.

​

How RFT works

In supervised fine-tuning you provide labeled examples of good outputs. In RFT you provide prompts and an evaluator (reward function) that scores model outputs from 0 to 1. Training iteratively improves outputs that maximize that score. RFT fits when the task is verifiable (you can judge output quality), you lack golden completions, or the task needs multi-step reasoning.

​

Launch and operate

Use the docs for path choice and quickstarts. Use the Fireworks training skill for launch flags, validation, monitoring, and troubleshooting.

​

Prerequisites

Confirm account access and quota, an RFT-compatible base model, valid prompt JSONL, and a reviewed evaluator that produces non-flat scores on a small probe.

​

Evaluators

Evaluators turn a rollout into a reward. Test full-credit, partial-credit, zero, malformed-output, and edge cases locally before registration. Keep evaluator and preference-data guidance in the evaluator skill reference.

​

Parameter tuning

Resolve the current defaults from firectl rftj create --help and the live RFT parameter reference. Start with a small run, change one optimization or rollout variable at a time, and compare reward and held-out quality.

​

Weighted training

Use per-example or per-token weighting only when the selected managed contract supports it. Inspect the rendered samples and effective loss mask before launch.

​

Launch from the CLI

Use firectl rftj create for a reproducible managed job. Prepare the full base-model, dataset, evaluator, and output-model resource names, run the documented preflight, and confirm the resolved plan before creation.

​

Launch from the dashboard

Use the dashboard for a guided first run: choose the model, dataset, evaluator, rollout settings, and optional W&B project, then review the summary before launch.

​

Warm start

Warm-start from a compatible promoted LoRA when continuing SFT into RFT. Do not also supply a conflicting base-model field; verify the current CLI contract before creation.

​

Evaluator secrets

Store credentials in the Fireworks secret-management surface and reference them by environment-variable name. Never embed secret values in evaluator code, datasets, or run reports.

​

Cost planning

Estimate rollout inference, training runtime or tokens, candidate count, response length, and any deployment uptime. Use the live pricing page and Price vs Tinker; do not assume a free tier.

​

Monitor training

Track state plus actual progress: optimizer steps, rollout throughput, reward distribution, validation results, logs, and W&B metrics when enabled. Inspect representative rollouts for reward hacking and stop when quality regresses or the evaluator saturates. Parameter field reference: RFT parameters (API tab).

​

Related