Deployments

• Better performance – Lower latency, higher throughput, and predictable performance unaffected by other users
• No hard rate limits – Only limited by your deployment’s capacity
• Cost-effective at scale – Cheaper under high utilization. Unlike serverless models (billed per token), on-demand deployments are billed by GPU-second.
• Broader model selection – Access models not available on serverless
• Custom models – Upload your own models (for supported architectures) from Hugging Face or elsewhere

​

Creating & querying deployments

# This command returns your accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID> - save it for querying
firectl deployment create accounts/fireworks/models/<MODEL_NAME> --wait

firectl deployment create accounts/fireworks/models/<MODEL_NAME> --region GLOBAL   # recommended default
firectl deployment create accounts/fireworks/models/<MODEL_NAME> --region US
firectl deployment create accounts/fireworks/models/<MODEL_NAME> --region EUROPE
firectl deployment create accounts/fireworks/models/<MODEL_NAME> --region APAC

​

Check current placement

firectl deployment get <DEPLOYMENT_ID>

​

Change placement

# 1. Create replacement with correct region
firectl deployment create accounts/fireworks/models/<MODEL_NAME> \
  --deployment-shape <shape> \
  --region GLOBAL \
  --min-replica-count 1

# 2. Verify it's healthy, then point your app at the new endpoint

# 3. Delete old deployment
firectl deployment delete <OLD_DEPLOYMENT_ID>

accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID>

accounts/alice/deployments/12345678

​

Code examples

from fireworks import Fireworks

client = Fireworks()

response = client.chat.completions.create(
  model="accounts/fireworks/models/gpt-oss-120b#<DEPLOYMENT_NAME>",
  messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("FIREWORKS_API_KEY"),
    base_url="https://api.fireworks.ai/inference/v1"
)

response = client.chat.completions.create(
    model="accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID>",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.FIREWORKS_API_KEY,
  baseURL: "https://api.fireworks.ai/inference/v1",
});

const response = await client.chat.completions.create({
  model: "accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID>",
  messages: [
    {
      role: "user",
      content: "Explain quantum computing in simple terms",
    },
  ],
});

console.log(response.choices[0].message.content);

curl https://api.fireworks.ai/inference/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FIREWORKS_API_KEY" \
  -d '{
    "model": "accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID>",
    "messages": [
      {
        "role": "user",
        "content": "Explain quantum computing in simple terms"
      }
    ]
  }'

​

Deployment status states

• CREATING - still being created
• READY - ready to be used
• UPDATING - in-progress updates happening
• DELETING - being deleted
• DELETED - soft-deleted
• FAILED - creation failed (see status for details)

• Inactive: state == READY && max_replica_count == 0 && ready_replica_count == 0
• Scaled to 0: state == READY && min_replica_count == 0 && max_replica_count > 0 && desired_replica_count == 0 && ready_replica_count == 0

​

Deployment shapes

• Fast – Low latency for interactive workloads
• Throughput – Cost-per-token at scale for high-volume workloads
• Minimal – Lowest cost for testing or light workloads

# List available shapes
firectl deployment-shape-version list --base-model <model-id>

# Create with a shape (shorthand)
firectl deployment create accounts/fireworks/models/deepseek-v3 --deployment-shape throughput

# Create with full shape ID
firectl deployment create accounts/fireworks/models/llama-v3p3-70b-instruct \
  --deployment-shape accounts/fireworks/deploymentShapes/llama-v3p3-70b-instruct-fast

# View shape details
firectl deployment-shape-version get <full-deployment-shape-version-id>

​

Managing & configuring deployments

​

Basic management

# List all deployments
firectl deployment list

# Check deployment status
firectl deployment get <DEPLOYMENT_ID>

# Delete a deployment
firectl deployment delete <DEPLOYMENT_ID>

​

GPU hardware

• NVIDIA_A100_80GB
• NVIDIA_H100_80GB
• NVIDIA_H200_141GB

​

Autoscaling

​

Multiple GPUs per replica

firectl deployment create <MODEL_NAME> --accelerator-count 2

​

Advanced

• Speculative decoding - Speed up text generation using draft models or n-gram speculation
• Quantization - Reduce model precision (e.g., FP16 to FP8) to improve speeds and reduce costs by 30-50%
• Performance benchmarking - Measure and optimize your deployment’s performance with load testing
• Managing default deployments - Control which deployment handles queries when using just the model name
• Publishing deployments - Make your deployment accessible to other Fireworks users

​

Next steps