Precision
Serverless models are served using a proprietary mixed-precision configuration (FP8, FP4, and BF16) that is custom-tuned per model. Every deployment is checked against the provider’s official API and against an unquantized baseline before it serves production traffic.- Attention, MoE experts, KV cache, and projections can each run at a different precision within the same deployment, rather than one blanket format across all models.
- The specific methods are refined on an ongoing basis. The validation methodology below is applied consistently across every model we serve, and is what we encourage you to evaluate directly.
- On standard serverless, this is what lets certain models from the Kimi family run at roughly 1/10th the cost and 2-3x the speed of comparable closed frontier models.
How we validate quantization quality
We combine divergence testing with a suite of standard benchmarks.
Divergence testing is supplemented with a suite of evolving benchmarks, checked against each model provider’s own published or reproduced numbers, including side-by-side runs against the official API. These include AIME, GPQA, SWE-bench, Terminal-Bench, and DeepSWE, plus MMMU Pro and OCRBench for multimodal models.
Where our numbers diverge from a model card, we publish why. Deviations can occur when a benchmark harness assumes a decoding mode that differs from how the model is normally served.
What we test
Beyond precision itself, each model launch is validated across:- Prompt formatting: chat template output checked against the model provider’s own reference implementation.
- Tool calling: streaming, non-streaming, and grammar-constrained generation, across multi-turn workflows.
- Numerical correctness: precision matched against reference outputs via generations, logprobs, and KLD.
- System behavior: timeouts, disconnect handling, and correct error codes under load.
- SDK compatibility: common client SDKs and gateway or proxy services.
Validation stages
Benchmarks describe model capability, but they do not capture whether a serving stack implements the model correctly. Because different failure classes surface at different stages, we run multiple layers of validation:- Deterministic unit tests for formatting, parsing, and tool-call edge cases.
- Single-turn benchmark checks.
- Multi-turn agentic evaluation.
- Multimodal benchmarks for vision models.
- Production monitoring and regression reruns.
- Reasoning-trace instability in a newly released frontier model, identified pre-launch and resolved in coordination with the model provider and serving frameworks.
- A silent drift in reasoning that trapped the model in reasoning loops, which we found across all other providers.
- A tool-calling interruption under sustained load, resolved with an updated decoding safeguard.
- A load-handling edge case that could return an ambiguous response instead of a standard rate-limit signal, corrected so that normal retry behavior applies.
Verify it yourself
We encourage you to verify these results independently. The serverless Chat Completions API exposes logprobs, so you can measure divergence against a reference implementation directly, using the same method described in our quantization writeup.Custom and dedicated precision
On a dedicated deployment, a precision profile can be tailored to your workload instead of using the default serverless configuration. See Quantization for the precisions you can select yourself, and contact your account team to scope a custom configuration.Related reading
Evaluating quantization
How Fireworks evaluates quantization precisely and interpretably.
GLM 5.3 Flash
Benchmark scores are not the only indicator of performance.
DeepSeek V4 Pro
Validating frontier models for production.
Kimi K2.5
The benchmark gap: what it takes to ship Kimi K2.5.
gpt-oss
Quality first: how Fireworks is the go-to place for gpt-oss.