Create Deployment

POST

accounts

{account_id}

deployments

curl --request POST \
  --url https://api.fireworks.ai/v1/accounts/{account_id}/deployments \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "displayName": "<string>",
  "description": "<string>",
  "expireTime": "2023-11-07T05:31:56Z",
  "minReplicaCount": 123,
  "maxReplicaCount": 123,
  "autoscalingPolicy": {
    "scaleUpWindow": "<string>",
    "scaleDownWindow": "<string>",
    "scaleToZeroWindow": "<string>",
    "loadTargets": {}
  },
  "baseModel": "<string>",
  "acceleratorCount": 123,
  "acceleratorType": "ACCELERATOR_TYPE_UNSPECIFIED",
  "precision": "PRECISION_UNSPECIFIED",
  "enableAddons": true,
  "draftTokenCount": 123,
  "draftModel": "<string>",
  "ngramSpeculationLength": 123,
  "deploymentTemplate": "<string>",
  "autoTune": {
    "longPrompt": true
  },
  "isNim": true
}'

{
  "name": "<string>",
  "displayName": "<string>",
  "description": "<string>",
  "createTime": "2023-11-07T05:31:56Z",
  "expireTime": "2023-11-07T05:31:56Z",
  "purgeTime": "2023-11-07T05:31:56Z",
  "deleteTime": "2023-11-07T05:31:56Z",
  "createdBy": "<string>",
  "state": "STATE_UNSPECIFIED",
  "status": {
    "code": "OK",
    "message": "<string>"
  },
  "minReplicaCount": 123,
  "maxReplicaCount": 123,
  "replicaCount": 123,
  "autoscalingPolicy": {
    "scaleUpWindow": "<string>",
    "scaleDownWindow": "<string>",
    "scaleToZeroWindow": "<string>",
    "loadTargets": {}
  },
  "baseModel": "<string>",
  "acceleratorCount": 123,
  "acceleratorType": "ACCELERATOR_TYPE_UNSPECIFIED",
  "precision": "PRECISION_UNSPECIFIED",
  "cluster": "<string>",
  "enableAddons": true,
  "draftTokenCount": 123,
  "draftModel": "<string>",
  "ngramSpeculationLength": 123,
  "numPeftDeviceCached": 123,
  "deploymentTemplate": "<string>",
  "autoTune": {
    "longPrompt": true
  },
  "region": "REGION_UNSPECIFIED",
  "isNim": true,
  "updateTime": "2023-11-07T05:31:56Z"
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

account_id

string

required

The Account Id

Query Parameters

disableAutoDeploy

boolean

By default, a deployment created with a currently undeployed base model will be deployed to this deployment. If true, this auto-deploy function is disabled.

disableSpeculativeDecoding

boolean

By default, a deployment will use the speculative decoding settings from the base model. If true, this will disable speculative decoding.

deploymentId

string

The ID of the deployment. If not specified, a random ID will be generated.

Body

application/json

The properties of the deployment being created.

baseModel

string

required

displayName

string

Human-readable display name of the deployment. e.g. "My Deployment" Must be fewer than 64 characters long.

description

string

Description of the deployment.

expireTime

string

The time at which this deployment will automatically be deleted.

state

enum<string>

default:STATE_UNSPECIFIED

The state of the deployment.

Available options:

STATE_UNSPECIFIED,

CREATING,

READY,

DELETING,

FAILED,

UPDATING,

DELETED

status

object

Detailed status information regarding the most recent operation.

minReplicaCount

integer

The minimum number of replicas. If not specified, the default is 0.

maxReplicaCount

integer

The maximum number of replicas. If not specified, the default is max(min_replica_count, 1). May be set to 0 to downscale the deployment to 0.

autoscalingPolicy

object

acceleratorCount

integer

The number of accelerators used per replica. If not specified, the default is the estimated minimum required by the base model.

acceleratorType

enum<string>

default:ACCELERATOR_TYPE_UNSPECIFIED

The type of accelerator to use. If not specified, the default is NVIDIA_A100_80GB.

Available options:

ACCELERATOR_TYPE_UNSPECIFIED,

NVIDIA_A100_80GB,

NVIDIA_H100_80GB,

AMD_MI300X_192GB,

NVIDIA_A10G_24GB,

NVIDIA_A100_40GB,

NVIDIA_L4_24GB,

NVIDIA_H200_141GB,

NVIDIA_B200_180GB

precision

enum<string>

default:PRECISION_UNSPECIFIED

The precision with which the model should be served.

Available options:

PRECISION_UNSPECIFIED,

FP16,

FP8,

FP8_MM,

FP8_AR,

FP8_MM_KV_ATTN,

FP8_KV,

FP8_MM_V2,

FP8_V2,

FP8_MM_KV_ATTN_V2,

NF4

enableAddons

boolean

If true, PEFT addons are enabled for this deployment.

draftTokenCount

integer

The number of candidate tokens to generate per step for speculative decoding. Default is the base model's draft_token_count. Set CreateDeploymentRequest.disable_speculative_decoding to false to disable this behavior.

draftModel

string

The draft model name for speculative decoding. e.g. accounts/fireworks/models/my-draft-model If empty, speculative decoding using a draft model is disabled. Default is the base model's default_draft_model. Set CreateDeploymentRequest.disable_speculative_decoding to false to disable this behavior.

ngramSpeculationLength

integer

The length of previous input sequence to be considered for N-gram speculation.

deploymentTemplate

string

The name of the deployment template to use for this deployment. Only available to enterprise accounts.

autoTune

object

The performance profile to use for this deployment.

region

enum<string>

default:REGION_UNSPECIFIED

The geographic region where the deployment is located.

Available options:

REGION_UNSPECIFIED,

US_IOWA_1,

US_VIRGINIA_1,

US_VIRGINIA_2,

US_ILLINOIS_1,

AP_TOKYO_1,

EU_LONDON_1,

US_ARIZONA_1,

US_TEXAS_1,

US_ILLINOIS_2,

EU_FRANKFURT_1,

US_TEXAS_2,

EU_PARIS_1,

EU_HELSINKI_1,

US_NEVADA_1,

EU_ICELAND_1,

EU_ICELAND_2

isNim

boolean

Whether this deployment should be created with NIM.

Response

200 - application/json

A successful response.

baseModel

string

required

name

string

displayName

string

Human-readable display name of the deployment. e.g. "My Deployment" Must be fewer than 64 characters long.

description

string

Description of the deployment.

createTime

string

The creation time of the deployment.

expireTime

string

The time at which this deployment will automatically be deleted.

purgeTime

string

The time at which the resource will be hard deleted.

deleteTime

string

The time at which the resource will be soft deleted.

createdBy

string

The email address of the user who created this deployment.

state

enum<string>

default:STATE_UNSPECIFIED

The state of the deployment.

Available options:

STATE_UNSPECIFIED,

CREATING,

READY,

DELETING,

FAILED,

UPDATING,

DELETED

status

object

Detailed status information regarding the most recent operation.

minReplicaCount

integer

The minimum number of replicas. If not specified, the default is 0.

maxReplicaCount

integer

The maximum number of replicas. If not specified, the default is max(min_replica_count, 1). May be set to 0 to downscale the deployment to 0.

replicaCount

integer

autoscalingPolicy

object

acceleratorCount

integer

The number of accelerators used per replica. If not specified, the default is the estimated minimum required by the base model.

acceleratorType

enum<string>

default:ACCELERATOR_TYPE_UNSPECIFIED

The type of accelerator to use. If not specified, the default is NVIDIA_A100_80GB.

Available options:

ACCELERATOR_TYPE_UNSPECIFIED,

NVIDIA_A100_80GB,

NVIDIA_H100_80GB,

AMD_MI300X_192GB,

NVIDIA_A10G_24GB,

NVIDIA_A100_40GB,

NVIDIA_L4_24GB,

NVIDIA_H200_141GB,

NVIDIA_B200_180GB

precision

enum<string>

default:PRECISION_UNSPECIFIED

The precision with which the model should be served.

Available options:

PRECISION_UNSPECIFIED,

FP16,

FP8,

FP8_MM,

FP8_AR,

FP8_MM_KV_ATTN,

FP8_KV,

FP8_MM_V2,

FP8_V2,

FP8_MM_KV_ATTN_V2,

NF4

cluster

string

If set, this deployment is deployed to a cloud-premise cluster.

enableAddons

boolean

If true, PEFT addons are enabled for this deployment.

draftTokenCount

integer

draftModel

string

ngramSpeculationLength

integer

The length of previous input sequence to be considered for N-gram speculation.

numPeftDeviceCached

integer

deploymentTemplate

string

The name of the deployment template to use for this deployment. Only available to enterprise accounts.

autoTune

object

The performance profile to use for this deployment.

region

enum<string>

default:REGION_UNSPECIFIED

The geographic region where the deployment is located.

Available options:

REGION_UNSPECIFIED,

US_IOWA_1,

US_VIRGINIA_1,

US_VIRGINIA_2,

US_ILLINOIS_1,

AP_TOKYO_1,

EU_LONDON_1,

US_ARIZONA_1,

US_TEXAS_1,

US_ILLINOIS_2,

EU_FRANKFURT_1,

US_TEXAS_2,

EU_PARIS_1,

EU_HELSINKI_1,

US_NEVADA_1,

EU_ICELAND_1,

EU_ICELAND_2

isNim

boolean

Whether this deployment should be created with NIM.

updateTime

string

The update time for the deployment.

Get Deployment Update Deployment

curl --request POST \
  --url https://api.fireworks.ai/v1/accounts/{account_id}/deployments \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "displayName": "<string>",
  "description": "<string>",
  "expireTime": "2023-11-07T05:31:56Z",
  "minReplicaCount": 123,
  "maxReplicaCount": 123,
  "autoscalingPolicy": {
    "scaleUpWindow": "<string>",
    "scaleDownWindow": "<string>",
    "scaleToZeroWindow": "<string>",
    "loadTargets": {}
  },
  "baseModel": "<string>",
  "acceleratorCount": 123,
  "acceleratorType": "ACCELERATOR_TYPE_UNSPECIFIED",
  "precision": "PRECISION_UNSPECIFIED",
  "enableAddons": true,
  "draftTokenCount": 123,
  "draftModel": "<string>",
  "ngramSpeculationLength": 123,
  "deploymentTemplate": "<string>",
  "autoTune": {
    "longPrompt": true
  },
  "isNim": true
}'

{
  "name": "<string>",
  "displayName": "<string>",
  "description": "<string>",
  "createTime": "2023-11-07T05:31:56Z",
  "expireTime": "2023-11-07T05:31:56Z",
  "purgeTime": "2023-11-07T05:31:56Z",
  "deleteTime": "2023-11-07T05:31:56Z",
  "createdBy": "<string>",
  "state": "STATE_UNSPECIFIED",
  "status": {
    "code": "OK",
    "message": "<string>"
  },
  "minReplicaCount": 123,
  "maxReplicaCount": 123,
  "replicaCount": 123,
  "autoscalingPolicy": {
    "scaleUpWindow": "<string>",
    "scaleDownWindow": "<string>",
    "scaleToZeroWindow": "<string>",
    "loadTargets": {}
  },
  "baseModel": "<string>",
  "acceleratorCount": 123,
  "acceleratorType": "ACCELERATOR_TYPE_UNSPECIFIED",
  "precision": "PRECISION_UNSPECIFIED",
  "cluster": "<string>",
  "enableAddons": true,
  "draftTokenCount": 123,
  "draftModel": "<string>",
  "ngramSpeculationLength": 123,
  "numPeftDeviceCached": 123,
  "deploymentTemplate": "<string>",
  "autoTune": {
    "longPrompt": true
  },
  "region": "REGION_UNSPECIFIED",
  "isNim": true,
  "updateTime": "2023-11-07T05:31:56Z"
}

API Documentation

LLM API

Embeddings API

Image API

Audio API

Accounts

Deployments

Models

LoRAs

Supervised Fine Tuning Jobs

Users

Datasets

API Keys

Authorizations

Path Parameters

Query Parameters

Body

Response