Serverless Training

Serverless training is currently in private preview and access is gated per account. Request access and mention “serverless training”; we’ll enable your account and provide onboarding credits so you can start testing the same week.

Serverless training is the fastest way to run post-training on Fireworks. You connect to a shared, already-running pooled trainer through the gateway and get back a single service object that hands you both a training client and a sampling client. There is nothing to provision: no trainer job to launch, no inference deployment to stand up, and nothing to tear down.

What you need

Install the SDK (same one as the dedicated path, no separate console flow): pip install --pre "fireworks-ai[training]"
Point at the serverless endpoint: base_url="https://api.fireworks.ai/training/v1/serverless"
Pick a base model: accounts/fireworks/models/qwen3p5-9b (Qwen 3.5 9B) or accounts/fireworks/models/qwen3p6-27b (Qwen 3.6 27B)
Set your API key and run the Quickstart below.

Using a code agent? A complete, forkable serverless RL loop lives in the cookbook at training/examples/serverless_rl/. It runs out of the box against the serverless gateway.

What is serverless training?

You write the training loop, for supervised fine-tuning or reinforcement learning, and Fireworks runs the forward pass, backward pass, and optimizer on remote GPUs, then serves your latest weights for sampling in the same session.

from fireworks.training.sdk import FiretitanServiceClient

service = FiretitanServiceClient(
    api_key=api_key,
    base_url="https://api.fireworks.ai/training/v1/serverless",
)

training_client = service.create_lora_training_client(base_model, rank=8)

for step in range(steps):
    snapshot = training_client.save_weights_for_sampler(name).result().path
    sampler  = service.create_sampling_client(model_path=snapshot, tokenizer=tokenizer)
    # sample completions, score them, build training datums
    training_client.forward_backward(datums, "importance_sampling").result()
    training_client.optim_step(adam).result()

What you can run

Supervised Fine-Tuning (SFT)

Run the loop with a cross-entropy loss over your labeled data.

Reinforcement Learning (RL)

Sample completions from the current adapter, score them with your own reward function, and train with an importance-sampling loss (GRPO-style). This is the primary serverless use case.

Both run as LoRA on the shared pool. For full-parameter training, DPO, or the full menu of managed methods, use the dedicated path.

Serverless vs dedicated

The two paths share the same SDK surface and the same primitives; they differ in how compute is provisioned.

Dimension	Serverless	Dedicated
Provisioning	None. Attach to a shared pooled trainer.	SDK provisions a trainer job and an inference deployment for your run.
Tuning method	LoRA only (SFT or RL).	LoRA and full-parameter.
Capacity	Shared pool, subject to availability.	Reserved for your run.
Lifecycle	Nothing to manage or tear down.	You manage and tear down the trainer and deployment.
Idle cost	None. Pay per token.	Deployment bills per GPU-hour until scaled to zero or deleted.
Best for	Fast iteration, small-to-mid LoRA experiments, RL loops with inference in the loop, first-time training runs.	Large-model or full-parameter training, sustained production RL, long rollouts, guaranteed throughput.

Start on serverless to move fast. Move to dedicated when you need full-parameter training, reserved capacity, or the cost curve of a saturated GPU at scale.

Core concepts

Session. create_lora_training_client(base_model, rank) attaches you to a pooled trainer for that base model. That attachment is your training session (service.training_session_id) — your LoRA state lives there. Run. The training client is your run (training_client.run_id). One run is one training trajectory: the forward_backward and optim_step calls plus the checkpoints you save. LoRA adapter. Serverless is LoRA only. Pass a positive rank (e.g. rank=8); base weights stay frozen and shared across the pool, and you train an adapter on top. Checkpoint / snapshot. save_weights_for_sampler(name) writes your current adapter weights and returns a snapshot path. That path is a public sampler identity, not a raw storage URI — hand it to the sampler to serve exactly those weights. Sampling. create_sampling_client(model_path=snapshot, tokenizer=...) returns a sampler bound to that snapshot, served through Fireworks’ OpenAI-compatible completions API (/inference/v1/chat/completions). For multi-turn rollouts, session-affinity headers pin an episode to one replica so the KV cache is reused across turns (see RL rollout integration). The sampler runs in the same session, so there’s no deployment to create or hot-load.

Quickstart

Step 1: Create a key, install, and authenticate

Create an API key in the Fireworks dashboard (click Create API key and store it somewhere safe), or run firectl api-key create. Then install the SDK and export the key:

pip install --pre "fireworks-ai[training]"
export FIREWORKS_API_KEY="fw_..."   # the key you just created

Step 2: Connect to the serverless session

import os
from fireworks.training.sdk import FiretitanServiceClient

service = FiretitanServiceClient(
    api_key=os.environ["FIREWORKS_API_KEY"],
    base_url="https://api.fireworks.ai/training/v1/serverless",
)

base_model = "accounts/fireworks/models/qwen3p6-27b"
training_client = service.create_lora_training_client(base_model=base_model, rank=8)

print(f"session={service.training_session_id} run={training_client.run_id}")

This doubles as your setup check: if it prints a session id beginning with ts-, serverless routing is working. If the base URL does not end in /training/v1/serverless, sampling errors out.

Step 3: Train, checkpoint, and sample

# One optimizer step
training_client.forward_backward(datums, "importance_sampling").result()
training_client.optim_step(
    tinker.AdamParams(learning_rate=2.5e-5, beta1=0.9, beta2=0.95, eps=1e-8, weight_decay=0.0)
).result()

# Save weights and open a sampler bound to that exact snapshot
snapshot = training_client.save_weights_for_sampler("step-0001").result().path
sampler = service.create_sampling_client(model_path=snapshot, tokenizer=tokenizer)
completions = sampler.sample(prompt=prompt, num_samples=8, sampling_params=params)

Every training-client call returns a future. Always call .result(), or failures can be missed.
The tokenizer you pass must match base_model. It renders prompts and decodes sampled tokens client-side, so a mismatch silently corrupts your rewards.
Give each checkpoint a distinct name. Saving twice with the same name currently resolves to the same snapshot path.

Reinforcement learning example

The end-to-end serverless RL pattern is the standard GRPO / importance-sampling loop: each step saves the current adapter, rolls out a batch of prompts through a sampler bound to that snapshot, scores completions with your reward function, turns group-relative advantages into training datums, and takes one optimizer step.

for step in range(steps):
    snapshot = training_client.save_weights_for_sampler(f"countdown-{step:04d}").result().path
    sampler  = service.create_sampling_client(model_path=snapshot, tokenizer=tokenizer)

    # sample group_size completions per prompt, score each with your reward_fn,
    # standardize rewards within each prompt group (GRPO), drop zero-spread groups
    datums = build_importance_sampling_datums(sampler, batch, reward_fn)

    training_client.forward_backward(datums, "importance_sampling").result()
    training_client.optim_step(adam).result()

Reward should climb as the policy learns the task. The complete runnable version is the cookbook’s serverless_rl example. For a supervised loop, run the same structure with a cross-entropy loss instead of importance sampling. For the full menu of RL losses (GRPO, DAPO, GSPO, CISPO) and the dedicated provisioned path, see the cookbook RL recipes.

Pricing

Serverless training is billed per token, with three meters: prefill (reading the prompt), sample (generating tokens), and train (the tokens you learn from), each priced per 1M tokens. There is no idle cost: you pay only for tokens processed. Long-context variants are priced as separate SKUs, and MoE models are priced by active parameters. Cached prefill. Fireworks is KV-cache-aware on the sampler side and passes the benefit through to you. When an agent accumulates context turn after turn, the repeated prefill tokens hit the cache and are billed at 20% of the prefill rate (an 80% discount), so multi-turn rollouts cost materially less than the headline prefill rate suggests.

Base model	Context	Prefill (per 1M)	Cached prefill (per 1M)	Sample (per 1M)	Train (per 1M)
Qwen 3.5 9B	65K	$0.44	$0.088	$1.33	$1.33
Qwen 3.6 27B	65K	$1.24	$0.248	$3.73	$3.73

Additional models are added to the table as they come online. Checkpoint storage is billed at $0.10 / GB-month. See the pricing page for the current authoritative rates. For sustained or multi-turn RL at scale, dedicated GPU-hour billing is often cheaper overall, because session-affinity KV-cache reuse turns cache savings into more work per hour. Use the cost comparison calculator to estimate your workload on serverless per-token pricing against Fireworks Dedicated.

Supported models and limits

Models

Qwen 3.5 9B and Qwen 3.6 27B, up to 65K context. LoRA only, dense only. Additional models roll out as they come online.
For full-parameter training or MoE base models, use the dedicated path.

Capacity and rate limits (private preview)

Concurrent runs: 8 per account. Sessions auto-release ~10 minutes after the last heartbeat, so a fast iterate/retry loop can briefly hit HTTP 429: maximum concurrent serverless training runs exceeded until slots free up.
Rate limits: per-account budget shared across training and sampling — 60 requests/min, 60K prompt tokens/min, 12K generated tokens/min. RL rollouts run against raised limits. The SDK backs off on 429s, so steps may slow rather than fail. Reach out if you need higher throughput.
Shared-pool capacity: if the pool is full, create_lora_training_client returns an out-of-capacity error; retry, or switch to the dedicated path.

Behavior to know

Set max_seq_len explicitly. Serverless has no dedicated instance to infer sequence length from.
Cross-run checkpoint resume is not yet supported. A checkpoint saved during a run can only be resumed by that same run. Forking a new run from a prior run’s checkpoint (Tinker’s create_training_client_from_state) is dedicated-only today.
Serving your trained adapter. Sample in-session during the run. To serve afterward, deploy the adapter on a dedicated or preemptible deployment; serverless per-token serving of your own fine-tuned LoRA is not available yet.

Next steps

Quickstart: a custom training loop from scratch
Training and Sampling: the dedicated provisioned path
Loss Functions: built-in and custom losses
The Cookbook: ready-to-run recipes, including serverless_rl

Get Started

Serverless

Deployments

Models & Inference

Training

Fire Pass

Fireworks for Work

FireConnect

Administration

Security & Compliance

Integrations

Reference

What is serverless training?

What you can run

Supervised Fine-Tuning (SFT)

Reinforcement Learning (RL)

Serverless vs dedicated

Core concepts

Quickstart

Step 1: Create a key, install, and authenticate

Step 2: Connect to the serverless session

Step 3: Train, checkpoint, and sample

Reinforcement learning example

Pricing

Supported models and limits

Models

Capacity and rate limits (private preview)

Behavior to know

Next steps

​What is serverless training?

​What you can run

Supervised Fine-Tuning (SFT)

Reinforcement Learning (RL)

​Serverless vs dedicated

​Core concepts

​Quickstart

​Step 1: Create a key, install, and authenticate

​Step 2: Connect to the serverless session

​Step 3: Train, checkpoint, and sample

​Reinforcement learning example

​Pricing

​Supported models and limits

​Models

​Capacity and rate limits (private preview)

​Behavior to know

​Next steps

What is serverless training?

What you can run

Serverless vs dedicated

Core concepts

Quickstart

Step 1: Create a key, install, and authenticate

Step 2: Connect to the serverless session

Step 3: Train, checkpoint, and sample

Reinforcement learning example

Pricing

Supported models and limits

Models

Capacity and rate limits (private preview)

Behavior to know

Next steps