firectl
CLI tool and create an API key.
firectl
to interact with the Fireworks AI platform.Key
field to your environment variable FIREWORKS_API_KEY
.LLM()
classmain.py
and import the Fireworks AI SDK.LLM
class. The LLM class accepts a model
argument that you
can use to specify the model you want to use. For this tutorial, we will use the
Llama 4 Maverick
model."serverless"
, "on-demand"
, or "auto"
. If you pass "auto"
, the SDK will try to use serverless hosting if available, otherwise it will create an on-demand deployment. In the other cases, the SDK will try to create a deployment of the specified type and will throw an error if it’s not available for the model you selected.deployment_type
parameter, especially for "auto"
and "on-demand"
. While the SDK will try to make the most cost effective choice for you and put sensible autoscaling policies in place, it is possible to unintentionally create many deployments that lead to unwanted spend, especially when working with non-serverless models.deployment_type="on-demand"
, you must provide an id
parameter to uniquely identify your deployment. This is required to prevent accidental creation of multiple deployments.deployment_type="on-demand"
or deployment_type="on-demand-lora"
, you must call .apply()
to apply the deployment configuration to Fireworks. This is not required for serverless deployments. When using deployment_type="auto"
, the SDK will automatically handle deployment creation, but if it falls back to on-demand deployment, you may need to call .apply()
explicitly. If you do not call .apply()
, you are expected to set up the deployment through the deployment page at https://app.fireworks.ai/dashboard/deployments.LLM
class is OpenAI compatible, so you can use
the same chat completion interface to make a request to the LLM.datasets
library from Hugging Face to load the dataset.Dataset
object and upload it to Fireworks using the Dataset.from_list()
method.FIREWORKS_SDK_DEBUG
environment variable to true
to see the progress of the fine-tuning job.epochs
and learning_rate
to experiment with different fine-tuning jobs!base_model.delete_deployment(ignore_checks=True)
to delete the deployment, bypassing the check that triggers if you’ve used the deployment recently.