Enforce output formats using JSON schemas or custom grammars
Structured outputs ensure model responses conform to your specified format, making them easy to parse and integrate into your application. Fireworks supports two methods: JSON mode (using JSON schemas) and Grammar mode (using custom BNF grammars).
New to structured outputs? Check out the Serverless Quickstart for a quick introduction.
import osfrom openai import OpenAIfrom pydantic import BaseModelclient = OpenAI( api_key=os.environ.get("FIREWORKS_API_KEY"), base_url="https://api.fireworks.ai/inference/v1")# Define your schemaclass Result(BaseModel): winner: str# Make the requestresponse = client.chat.completions.create( model="accounts/fireworks/models/kimi-k2p5", response_format={ "type": "json_schema", "json_schema": { "name": "Result", "schema": Result.model_json_schema() } }, messages=[{ "role": "user", "content": "Who won the US presidential election in 2012? Reply in JSON format." }])print(response.choices[0].message.content)# Output: {"winner": "Barack Obama"}
Include the schema in both your prompt and the response_format for best results. The model doesn’t automatically “see” the schema—it’s enforced during generation.
json_object – Force any valid JSON output (no specific schema)
json_schema – Enforce a specific JSON schema (recommended)
Always instruct the model to produce JSON in your prompt. Without this, the model may generate whitespace indefinitely until hitting token limits.
Using arbitrary JSON mode
To force JSON output without a specific schema:
Copy
Ask AI
response = client.chat.completions.create( model="accounts/fireworks/models/kimi-k2p5", response_format={"type": "json_object"}, messages=[{ "role": "user", "content": "List the top 3 programming languages in JSON format." }])
Token limits: If finish_reason="length", the response may be truncated and invalid JSON. Increase max_tokens if needed.Completions API: JSON mode works with both Chat Completions and Completions APIs.Function calling: When using Tool Calling, JSON mode is enabled automatically—these guidelines don’t apply.
Fireworks automatically prevents hallucinated fields by treating schemas with properties as if "unevaluatedProperties": false is set.
Advanced: Reasoning Model JSON Mode
Some models support generating structured JSON outputs alongside their reasoning process. The Fireworks Python SDK exposes the model’s reasoning via the reasoning_content field, keeping it separate from the JSON output in the content field.
Using response_format with json_schema disables reasoning output. To get both reasoning and structured JSON, include the schema in your prompt instead and omit the response_format parameter.
import jsonfrom fireworks import Fireworksfrom pydantic import BaseModelclient = Fireworks()# Define the output schemaclass QAResult(BaseModel): question: str answer: str# Include the schema in the prompt to preserve reasoningschema = QAResult.model_json_schema()response = client.chat.completions.create( model="accounts/fireworks/models/kimi-k2p5", messages=[{ "role": "user", "content": ( "Who wrote 'Pride and Prejudice'?\n\n" f"Reply in JSON matching this schema:\n{json.dumps(schema, indent=2)}" ) }], max_tokens=1000)# The Fireworks SDK separates reasoning into its own fieldreasoning = response.choices[0].message.reasoning_contentcontent = response.choices[0].message.content# Strip markdown code fences if the model wraps the JSONjson_str = content.strip()if json_str.startswith("```"): json_str = json_str.split("\n", 1)[1].rsplit("```", 1)[0].strip()# Parse into Pydantic modelqa_result = QAResult.model_validate_json(json_str)if reasoning: print("Reasoning:", reasoning)print("Result:", qa_result.model_dump_json(indent=2))
If you don’t need reasoning and just want guaranteed schema-conformant JSON, use the response_format parameter as shown in the Quick Start above. The response_format approach enforces the schema during generation, eliminating the need to parse the JSON yourself.
