​
Directory structure

Your PEFT addon must contain three files:

  • adapter_config.json - The Hugging Face adapter configuration file.
  • adapter_model.binor adapter_model.safetensors- The saved addon file.
  • fireworks.json - A Fireworks configuration file.

It may also contain an optional README.md file which will be ignored during the upload process.

​
Limits

The following limits are applied by default. Contact us if you need these limits relaxed.

​
Supported base models

Currently, the following base models are supported:

  • accounts/fireworks/models/starcoder2-15b
  • accounts/fireworks/models/starcoder2-3b
  • accounts/fireworks/models/starcoder2-7b
  • accounts/fireworks/models/deepseek-coder-1b-base
  • accounts/fireworks/models/deepseek-coder-7b-base
  • accounts/fireworks/models/deepseek-coder-v2-lite-base
  • accounts/fireworks/models/llama-v2-34b-code
  • accounts/fireworks/models/codegemma-2b
  • accounts/fireworks/models/codegemma-7b
  • accounts/fireworks/models/llama-v2-13b
  • accounts/fireworks/models/llama-v2-13b-chat
  • accounts/fireworks/models/llama-v2-70b
  • accounts/fireworks/models/llama-v2-70b-chat
  • accounts/fireworks/models/llama-v2-7b
  • accounts/fireworks/models/llama-v2-7b-chat
  • accounts/fireworks/models/llama-v3-8b-instruct-hf
  • accounts/fireworks/models/llama-v3-70b-instruct-hf
  • accounts/fireworks/models/llama-v3p1-8b-instruct
  • accounts/fireworks/models/llama-v3p1-70b-instruct
  • accounts/fireworks/models/llama-guard-2-8b
  • accounts/fireworks/models/mistral-7b
  • accounts/fireworks/models/mixtral-8x22b-hf
  • accounts/fireworks/models/mixtral-8x22b-instruct-hf
  • accounts/fireworks/models/mixtral-8x7b
  • accounts/fireworks/models/mixtral-8x7b-instruct-hf
  • accounts/fireworks/models/mistral-7b-instruct-v0p2
  • accounts/fireworks/models/code-qwen-1p5-7b
  • accounts/fireworks/models/qwen2-72b-instruct
  • accounts/fireworks/models/stablecode-3b
  • accounts/yi-01-ai/models/yi-large

Additional base models (including custom models) are supported for enterprise accounts.

The base model name is specified in fireworks.json.

​
LoRA ranks

The LoRA rank must be an integer between 4 and 64, inclusive.

​
Supported target modules

Currently, the following target modules are supported:

  • Llama and Mixtral models (all linear layers)
    • q_proj
    • k_proj
    • v_proj
    • o_proj
    • up_proj/w1
    • down_proj/w2
    • gate_proj/w3
    • block_sparse_moe.gate

The target modules are specified in adapter_config.json.

Was this page helpful?