> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fireworks.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Inference for RL Rollouts
> Session affinity, weight-swap behavior, and MoE Router Replay for rollout traffic on Fireworks inference deployments.
When you use Fireworks inference to collect RL rollouts, the regular [`/v1/completions`](/api-reference/post-completions) and [`/v1/chat/completions`](/api-reference/post-chatcompletions) endpoints expose a few extra features tailored to multi-turn, stateful rollout traffic. You can use these whether or not the underlying deployment is a hot-load deployment.
These features are fully compatible with the OpenAI SDKs — they're all
attached as either request headers or optional body fields, so no SDK upgrade
is required.
## Session affinity
Multi-turn rollouts typically reuse a long prefix between turns (same system prompt, same trajectory so far). To get the KV cache to hit, all turns of a trajectory should land on the same inference replica. Two headers are relevant here:
* `x-multi-turn-session-id` — identifies the agent trajectory. Set this once per trajectory and keep it constant across turns. If both headers are present, Fireworks currently prefers this value when deriving the request's session-affinity key.
* `x-session-affinity` — fallback sticky routing key when `x-multi-turn-session-id` is absent. In most RL rollout setups, set it to the same trajectory ID.
```python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="",
base_url="https://api.fireworks.ai/inference/v1",
)
trajectory_id = "traj-42f1"
for turn in trajectory:
response = client.chat.completions.create(
model="accounts//models/",
messages=turn.messages,
extra_headers={
"x-multi-turn-session-id": trajectory_id,
"x-session-affinity": trajectory_id,
"fireworks-deployment": "accounts//deployments/",
},
)
```
```bash theme={null}
curl https://api.fireworks.ai/inference/v1/chat/completions \
-H "Authorization: Bearer " \
-H "fireworks-model: accounts//models/" \
-H "fireworks-deployment: accounts//deployments/" \
-H "x-multi-turn-session-id: traj-42f1" \
-H "x-session-affinity: traj-42f1" \
-H "Content-Type: application/json" \
-d '{
"model": "accounts//models/",
"messages": [{"role": "user", "content": "..."}]
}'
```
`x-session-affinity` on its own is already documented for general [prompt
caching](/guides/prompt-caching#optimizing-inference-request-for-caching). In
RL rollouts you typically also want `x-multi-turn-session-id` so that per-turn
metrics (TTFT, generation latency) are aggregated by trajectory, while
preserving the current serving preference when both headers are supplied.
## Behavior during weight swap
If your rollout traffic hits a hot-load deployment, a new checkpoint can arrive mid-rollout. What happens to your requests depends on the deployment's configured transition mode:
* **Async transition (recommended for RL):** in-flight requests pause then resume on the same HTTP connection using the new weights. The active turn keeps its current KV state, so it continues rather than restarting. New requests queue up. You see elevated TTFT but no errors.
* **Synchronous transition:** in-flight requests finish on the old weights; new requests get HTTP `425 Too Early` until the swap is done. Your client should retry with back-off, ideally keeping the same session-affinity key so it lands on a replica that has already finished the swap.
`reset_prompt_cache` only affects what future requests or session IDs can reuse after the swap. See [Checkpoint-swap behavior](/fine-tuning/rl-rollout-debugging#checkpoint-swap-behavior) for the full semantics.
## MoE Router Replay
For Mixture-of-Experts models, training-inference divergence often comes from the router picking different top-K experts at the same token position between trainer and inference. Aligning those choices across rollouts and training is known as [Rollout Router Replay (R3)](https://arxiv.org/abs/2510.11370).
Fireworks inference supports returning the selected MoE experts for every token and every MoE layer. Pass `include_routing_matrix: true` together with `logprobs: true` on your request:
```bash theme={null}
curl https://api.fireworks.ai/inference/v1/chat/completions \
-H "Authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"model": "accounts//models/",
"messages": [{"role": "user", "content": "..."}],
"include_routing_matrix": true,
"logprobs": true
}'
```
The selected expert indices for each token are returned alongside logprobs. For `/v1/chat/completions` you find them at `choices[i].logprobs.content[j].routing_matrix`; for `/v1/completions` the structure is analogous. Each value is a flattened, base64-encoded `uint8` array of shape `[num_layers_with_moe, num_active_experts]`.
### Example response (DeepSeek V3)
```json theme={null}
{
"object": "text_completion",
"model": "...my-deepseek-v3-model...",
"choices": [
{
"index": 0,
"logprobs": {
"content": [
{
"token": " ",
"logprob": -0.00014507,
"sampling_logprob": -0.0001450882,
"token_id": 223,
"routing_matrix": "CYvWPzaOl8g/o7q2XPVTMJ7w/Y8G..."
}
]
}
}
]
}
```
### Decoding the routing matrix
DeepSeek V3 has 58 MoE layers (the first 3 of 61 total are dense) and selects 8 active experts per token, so each decoded buffer is `58 * 8 = 464` bytes.
```python theme={null}
import base64
import numpy as np
num_layers_with_moe = 58
num_active_experts = 8
encoded = choice["logprobs"]["content"][0]["routing_matrix"]
raw_bytes = base64.b64decode(encoded)
routing_matrix = np.frombuffer(raw_bytes, dtype=np.uint8).reshape(
num_layers_with_moe, num_active_experts
)
# routing_matrix[layer_idx] -> array of 8 expert indices for that token
```
### Other API modes
* **Completions API (`/v1/completions`)**: same mechanism — `include_routing_matrix` and `logprobs` are top-level body fields.
* **Streaming (`stream: true`)**: `routing_matrix` is included on each streamed token chunk's `logprobs.content` entry.
* **Prompt tokens (`echo: true`)**: returns expert selection for the prompt tokens too. Combine with `echo_last: N` to only include expert selection for the last N prompt tokens.
## Related pages
The full bring-your-own-trainer flow that usually wraps these inference features.
Detailed semantics of request behavior across weight swaps.
Session-affinity patterns for general cache hit optimization.
Full request/response schema for `/v1/chat/completions`.