In addition to models you fine-tune on the Fireworks platform, you can also upload your own custom fine-tuned models.

Requirements

Your custom LoRA addon must contain the following files:

  • adapter_config.json - The Hugging Face adapter configuration file.
  • adapter_model.bin or adapter_model.safetensors - The saved addon file.

The adapter_config.json must contain the following fields:

  • r - The number of LoRA ranks. Must be between an integer between 4 and 64, inclusive.
  • target_modules - A list of target modules. Currently the following target modules are supported:
    • q_proj
    • k_proj
    • v_proj
    • o_proj
    • up_proj or w1
    • down_proj or w2
    • gate_proj or w3
    • block_sparse_moe.gate

Additional fields may be specified but are ignored.

Enabling chat completions

To enable the chat completions API for your LoRA addon, add a fireworks.json file directory containing:

{
  "conversation_config": {
    "style": "jinja",
    "args": {
      "template": "<YOUR_JINJA_TEMPLATE>"
    }
  }
}

Uploading the model

To upload a LoRA addon, run the following command. The MODEL_ID is an arbitrary resource ID to refer to the model within Fireworks.

Only some base models support LoRA addons.

firectl create model <MODEL_ID> /path/to/files/ --base-model "accounts/fireworks/models/<BASE_MODEL_ID>"