- Standard is the default mode. No
service_tierparameter is needed. - Priority tier is for workloads that require higher reliability during peak traffic.
- Fast is for workloads that require higher speeds.
- Ultra-fast (Preview) for workloads requiring extremely fast end-to-end generation speeds.
Priority tier
Priority tier is for workloads that require higher reliability during peak traffic periods, at a higher price point. Priority tier is prioritized above Standard traffic and is less likely to be load shed (503 server overloaded). To use priority tier, setservice_tier to "priority". Supported on OpenAI-compatible chat completions and on the Anthropic-compatible messages API:
Fast
Fast is a high-speed mode, useful for interactive applications that require fast response speeds, at a higher price point. Fast variants aim for 100+ tokens per second of generated throughput. It is not a different model and the quality of the model remains the same. Fast is available for select models. To use Fast, change themodel ID as listed below.
Ultra-fast (Preview)
Ultra-fast is a high-speed mode, useful for extreme latency sensitive applications that require fast response speeds, at a higher price point. Ultra-fast variants aim for 6-8x generation speed (up to 10x for agentic coding tasks). Like fast, it is not a different model and the quality of the model remains the same. Ultra-fast is in Preview and is available for the models listed below. Our preview modes are under active development and are subject to change quickly. To use Ultra-fast, change themodel ID as listed below.
Related
- Serverless overview
- US-only Serverless
- Serverless quickstart
- Text models
- Anthropic compatibility —
service_tieris supported on both OpenAI-compatible chat completions and the AnthropicmessagesAPI.