1. Deployment hanging or crashing

2. LoRA adapters vs full models

3. Performance optimization factors

Custom model issues

Autoscaling

Performance questions

Additional resources

Community

Blog

Fireworks AI Docs

API Reference

API Reference DLDE

Cookbook

Tools & SDK

Get Started

Status

Support

Troubleshooting and resolving common issues with on-demand deployments.

Deployment issues

Fireworks AI is a generative AI inference platform to run and customize models with industry-leading speed and production-readiness.

Introduction

Quickstart

This document outlines basic Fireworks AI concepts.

Concepts

Querying text models

Querying vision-language models

Querying embedding models

Using function-calling

Using grammar mode

Using JSON mode

Prompt caching

Use Predicted Outputs to boost output generation speeds for editing / rewriting use cases

Using Predicted Outputs

Overview

Uploading a custom model

Fine-tuning models

Fine-tuning models via API

A guide for downloading base models, merging them with LoRA adapters, and deploying the result using Fireworks.

Merging LoRA adapters with base models

Understand the performance impact of LoRA fine-tuning, optimization strategies, and deployment considerations.

Understanding LoRA performance

Deploying models

Quantization

On-demand deployments

Fireworks runs a global fleet of hardware on which you can deploy your models.

Regions

Reserved capacity

Direct routing enables enterprise users reduce latency to their deployments.

Direct routing

Rate limits, spend limits and quotas for serverless inference and on-demand deployments

Rate limits, spend limits and quotas

Add and delete additional users in your Fireworks account

Managing users

Set up custom Single Sign-On (SSO) authentication for Fireworks AI

Custom SSO

Data privacy & security

This page lists common error codes encountered during inference requests using the Fireworks API, their meanings, and potential resolutions.

Inference errors

Creates a model response for the given chat conversation.

Create Chat Completion

Creates a completion for the provided prompt and parameters.

Create Completion

Create embeddings

Generate an image

Transcribe audio

Translate audio

Align transcription

🛠️ [Preview] Realtime Transcribe

Start here

Step-by-step guides for hands-on exploration, ideal for interactive learning of AI techniques.

Build with Fireworks

Creative user-contributed projects that showcase innovative applications of Fireworks in diverse contexts.

Community showcase

Learn to create, deploy, and manage resources using Firectl

Getting Started

Authentication for access to your account

Authentication

List various resources in an Fireworks AI account

List Resources

Create a Deployment on Fireworks AI platform

Create a deployment

Create a fine-tuning job with a base model

Create a fine-tuning job

Create a Dataset on Fireworks AI platform

Create a Dataset

Create Model

Deletes resource(s) in a Fireworks AI account

Delete Resources

Deploy Model

Download a model from third-party locations

Download a model

Retrieves model information from Fireworks AI platform

Get Resources

Imports specified model from Fireworks AI Platform

Import Model

Undeploy a model on Fireworks AI platform

Undeploy Model

Updates Resources on Fireworks AI platform

Update Resources

OpenAI compatibility

Get Account

List Deployments

Get Deployment

Create Deployment

Update Deployment

Delete Deployment

CRUD APIs for deployed models.

List Models

Get Model

Update Model

Delete Model

Get Model Upload Endpoint

Get Model Download Endpoint

Validate Model Upload

List Users

Get User

Create User

Update User

List Datasets

Get Dataset

Create Dataset

Provides a streamlined way to upload a dataset file in a single API request. This path can handle file sizes up to 150Mb. For larger file sizes use [Get Dataset Upload Endpoint](get-dataset-upload-endpoint).


Upload Dataset Files

Get Dataset Upload Endpoint

Validate Dataset Upload

Update Dataset

Delete Dataset

List Fine-tuning Jobs

Get Fine-tuning Job

Create Fine-tuning Job

Update Fine-tuning Job

Delete Fine-tuning Job

List Batch Jobs

Get Batch Job

Create Batch Job

Update Batch Job

Cancels an existing batch job if it is queued, pending, or running.

Cancel Batch Job

Get Batch Job Logs

Delete Batch Job

Batch Delete Batch Jobs

List Clusters

Get Cluster

Create Cluster

Update Cluster

Delete Cluster

Retrieve connection settings for the cluster to be put in kubeconfig

Get Cluster Connection Info

List Environments

Create Environment

Get Environment

Update Environment

Connects the environment to a node pool.
Returns an error if there is an existing pending connection.

Connect Environment

Disconnects the environment from the node pool. Returns an error
if the environment is not connected to a node pool.

FAQ

​Custom model issues

​1. Deployment hanging or crashing

​2. LoRA adapters vs full models

​3. Performance optimization factors

​Autoscaling

​Performance questions

​Additional resources