Requirements
Fireworks currently supports the following model architectures:- DBRX
- DeepSeek V1, V2 & V3
- Falcon
- Gemma
- GPT NeoX
- Idefics3
- Llama 1,2,3,3.1,4
- LLaVA
- Mistral & Mixtral
- Phi, Phi-3, Phi-3V, Phi-4
- Pythia
- Qwen, Qwen2, Qwen2.5, Qwen2.5-VL, Qwen3
- Solar
- StableLM
- Starcoder(GPTBigCode) & Starcoder2
- Vision Llama
-
Model configuration:
config.json
.Fireworks does not support thequantization_config
option inconfig.json
. -
Model weights, in one of the following formats:
*.safetensors
*.bin
-
Weights index:
*.index.json
-
Tokenizer file(s), e.g.
tokenizer.model
tokenizer.json
tokenizer_config.json
Enabling chat completions
To enable the chat completions API for your custom base model, ensure yourtokenizer_config.json
contains a chat_template
field. See the Hugging Face guide on Templates for Chat Models for details.
Uploading the model locally (firectl)
To upload a custom base model, run the following command.Uploading models from S3 buckets (firectl)
For larger models, you can upload directly from an Amazon S3 bucket, which provides a faster transfer process than uploading from local files. To upload a model directly from an S3 bucket, run the following command.Ensure the IAM user has read access to the S3 bucket containing the model.
Uploading via REST API (python)
For more programmatic control, you can use the Fireworks REST API to upload your custom models. This involves a four-step process:- Create a model object: This creates a reference to your model in the Fireworks system.
- Get signed upload URLs: For each of your model files, you’ll get a unique URL to upload to.
- Upload files: Upload each file to its corresponding signed URL.
- Validate the upload: This tells Fireworks to verify the integrity of the uploaded files and make the model available for deployment.
Deploying
A model cannot be used for inference until it is deployed. See the Deploying models guide to deploy the model.Publishing
By default, all models you create are only visible to and deployable by users within your account. To publish a model so anyone with a Fireworks account can deploy it, you can create it with the--public
flag. This will allow it to show up in public model lists.
update