Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.fireworks.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
BaseCompletion Objects

class BaseCompletion()
Base class for handling completions. This class provides shared logic for creating completions,
both synchronously and asynchronously, and both streaming and non-streaming. Attributes:
  • endpoint str - API endpoint for the completion request.
  • response_class Type - Class used for parsing the non-streaming response.
  • stream_response_class Type - Class used for parsing the streaming response.

​
create

@classmethod
def create(cls,
           model,
           prompt_or_messages=None,
           request_timeout=600,
           stream=False,
           **kwargs)
Create a completion or chat completion. Arguments:
  • model str - Model name to use for the completion.
  • prompt_or_messages Union[str, List[ChatMessage]] - The prompt for Completion or a list of chat messages for ChatCompletion. If not specified, must specify either prompt or messages in kwargs.
  • request_timeout int, optional - Request timeout in seconds. Defaults to 600.
  • stream bool, optional - Whether to use streaming or not. Defaults to False.
  • **kwargs - Additional keyword arguments.
Returns: Union[CompletionResponse, Generator[CompletionStreamResponse, None, None]]:
Depending on the stream argument, either returns a CompletionResponse
or a generator yielding CompletionStreamResponse.

​
acreate

@classmethod
def acreate(cls, model, *args, request_timeout=600, stream=False, **kwargs)
Asynchronously create a completion. Arguments:
  • model str - Model name to use for the completion.
  • request_timeout int, optional - Request timeout in seconds. Defaults to 600.
  • stream bool, optional - Whether to use streaming or not. Defaults to False.
  • **kwargs - Additional keyword arguments.
Returns: Union[CompletionResponse, AsyncGenerator[CompletionStreamResponse, None]]:
Depending on the stream argument, either returns a CompletionResponse or an async generator yielding CompletionStreamResponse.

​
completion

​
Completion Objects

class Completion(BaseCompletion)
Class for handling text completions.

​
chat_completion

​
ChatCompletion Objects

class ChatCompletion(BaseCompletion)
Class for handling chat completions.

​
api

​
Choice Objects

class Choice(BaseModel)
A completion choice. Attributes:
  • index int - The index of the completion choice.
  • text str - The completion response.
  • logprobs float, optional - The log probabilities of the most likely tokens.
  • finish_reason str - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

​
CompletionResponse Objects

class CompletionResponse(BaseModel)
The response message from a /v1/completions call. Attributes:
  • id str - A unique identifier of the response.
  • object str - The object type, which is always “text_completion”.
  • created int - The Unix time in seconds when the response was generated.
  • choices List[Choice] - The list of generated completion choices.

​
CompletionResponseStreamChoice Objects

class CompletionResponseStreamChoice(BaseModel)
A streamed completion choice. Attributes:
  • index int - The index of the completion choice.
  • text str - The completion response.
  • logprobs float, optional - The log probabilities of the most likely tokens.
  • finish_reason str - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

​
CompletionStreamResponse Objects

class CompletionStreamResponse(BaseModel)
The streamed response message from a /v1/completions call. Attributes:
  • id str - A unique identifier of the response.
  • object str - The object type, which is always “text_completion”.
  • created int - The Unix time in seconds when the response was generated.
  • model str - The model used for the chat completion.
    choices (List[CompletionResponseStreamChoice]):
    The list of streamed completion choices.

​
Model Objects

class Model(BaseModel)
A model deployed to the Fireworks platform. Attributes:
  • id str - The model name.
  • object str - The object type, which is always “model”.
  • created int - The Unix time in seconds when the model was generated.

​
ListModelsResponse Objects

class ListModelsResponse(BaseModel)
The response message from a /v1/models call. Attributes:
  • object str - The object type, which is always “list”.
  • data List[Model] - The list of models.

​
ChatMessage Objects

class ChatMessage(BaseModel)
A chat completion message. Attributes:
  • role str - The role of the author of this message.
  • content str - The contents of the message.

​
ChatCompletionResponseChoice Objects

class ChatCompletionResponseChoice(BaseModel)
A chat completion choice generated by a chat model. Attributes:
  • index int - The index of the chat completion choice.
  • message ChatMessage - The chat completion message.
  • finish_reason Optional[str] - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

​
UsageInfo Objects

class UsageInfo(BaseModel)
Usage statistics. Attributes:
  • prompt_tokens int - The number of tokens in the prompt.
  • total_tokens int - The total number of tokens used in the request (prompt + completion).
  • completion_tokens Optional[int] - The number of tokens in the generated completion.

​
ChatCompletionResponse Objects

class ChatCompletionResponse(BaseModel)
The response message from a /v1/chat/completions call. Attributes:
  • id str - A unique identifier of the response.
  • object str - The object type, which is always “chat.completion”.
  • created int - The Unix time in seconds when the response was generated.
  • model str - The model used for the chat completion.
  • choices List[ChatCompletionResponseChoice] - The list of chat completion choices.
  • usage UsageInfo - Usage statistics for the chat completion.

​
DeltaMessage Objects

class DeltaMessage(BaseModel)
A message delta. Attributes:
  • role str - The role of the author of this message.
  • content str - The contents of the chunk message.

​
ChatCompletionResponseStreamChoice Objects

class ChatCompletionResponseStreamChoice(BaseModel)
A streamed chat completion choice. Attributes:
  • index int - The index of the chat completion choice.
  • delta DeltaMessage - The message delta.
  • finish_reason str - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

​
ChatCompletionStreamResponse Objects

class ChatCompletionStreamResponse(BaseModel)
The streamed response message from a /v1/chat/completions call. Attributes:
  • id str - A unique identifier of the response.
  • object str - The object type, which is always “chat.completion”.
  • created int - The Unix time in seconds when the response was generated.
  • model str - The model used for the chat completion.
    choices (List[ChatCompletionResponseStreamChoice]):
    The list of streamed chat completion choices.

​
model

​
Model Objects

class Model()

​
list

@classmethod
def list(cls, request_timeout=60)
Returns a list of available models. Arguments:
  • request_timeout int, optional - The request timeout in seconds. Default is 60.
Returns:
  • ListModelsResponse - A list of available models.

​
log

​
set_console_log_level

def set_console_log_level(level: str) -> None
Controls console logging. Arguments:
  • level - the minimum level that prints out to console.
    Supported values: [CRITICAL, FATAL, ERROR, WARN,
    WARNING, INFO, DEBUG]

​
error

​
PermissionError Objects

class PermissionError(FireworksError)
A permission denied error.

​
InvalidRequestError Objects

class InvalidRequestError(FireworksError)
A invalid request error.

​
AuthenticationError Objects

class AuthenticationError(FireworksError)
A authentication error.

​
RateLimitError Objects

class RateLimitError(FireworksError)
A rate limit error.

​
InternalServerError Objects

class InternalServerError(FireworksError)
An internal server error.

​
ServiceUnavailableError Objects

class ServiceUnavailableError(FireworksError)
A service unavailable error.