BaseCompletion Objects

class BaseCompletion()

Base class for handling completions. This class provides shared logic for creating completions,
both synchronously and asynchronously, and both streaming and non-streaming. Attributes:

endpoint str - API endpoint for the completion request.
response_class Type - Class used for parsing the non-streaming response.
stream_response_class Type - Class used for parsing the streaming response.

create

@classmethod
def create(cls,
           model,
           prompt_or_messages=None,
           request_timeout=600,
           stream=False,
           **kwargs)

Create a completion or chat completion. Arguments:

model str - Model name to use for the completion.
prompt_or_messages Union[str, List[ChatMessage]] - The prompt for Completion or a list of chat messages for ChatCompletion. If not specified, must specify either prompt or messages in kwargs.
request_timeout int, optional - Request timeout in seconds. Defaults to 600.
stream bool, optional - Whether to use streaming or not. Defaults to False.
**kwargs - Additional keyword arguments.

Returns: Union[CompletionResponse, Generator[CompletionStreamResponse, None, None]]:
Depending on the stream argument, either returns a CompletionResponse
or a generator yielding CompletionStreamResponse.

acreate

@classmethod
def acreate(cls, model, *args, request_timeout=600, stream=False, **kwargs)

Asynchronously create a completion. Arguments:

model str - Model name to use for the completion.
request_timeout int, optional - Request timeout in seconds. Defaults to 600.
stream bool, optional - Whether to use streaming or not. Defaults to False.
**kwargs - Additional keyword arguments.

Returns: Union[CompletionResponse, AsyncGenerator[CompletionStreamResponse, None]]:
Depending on the stream argument, either returns a CompletionResponse or an async generator yielding CompletionStreamResponse.

completion

Completion Objects

class Completion(BaseCompletion)

Class for handling text completions.

chat_completion

ChatCompletion Objects

class ChatCompletion(BaseCompletion)

Class for handling chat completions.

api

Choice Objects

class Choice(BaseModel)

A completion choice. Attributes:

index int - The index of the completion choice.
text str - The completion response.
logprobs float, optional - The log probabilities of the most likely tokens.
finish_reason str - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

CompletionResponse Objects

class CompletionResponse(BaseModel)

The response message from a /v1/completions call. Attributes:

id str - A unique identifier of the response.
object str - The object type, which is always “text_completion”.
created int - The Unix time in seconds when the response was generated.
choices List[Choice] - The list of generated completion choices.

CompletionResponseStreamChoice Objects

class CompletionResponseStreamChoice(BaseModel)

A streamed completion choice. Attributes:

index int - The index of the completion choice.
text str - The completion response.
logprobs float, optional - The log probabilities of the most likely tokens.
finish_reason str - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

CompletionStreamResponse Objects

class CompletionStreamResponse(BaseModel)

The streamed response message from a /v1/completions call. Attributes:

id str - A unique identifier of the response.
object str - The object type, which is always “text_completion”.
created int - The Unix time in seconds when the response was generated.
model str - The model used for the chat completion.
choices (List[CompletionResponseStreamChoice]):
The list of streamed completion choices.

Model Objects

class Model(BaseModel)

A model deployed to the Fireworks platform. Attributes:

id str - The model name.
object str - The object type, which is always “model”.
created int - The Unix time in seconds when the model was generated.

ListModelsResponse Objects

class ListModelsResponse(BaseModel)

The response message from a /v1/models call. Attributes:

object str - The object type, which is always “list”.
data List[Model] - The list of models.

ChatMessage Objects

class ChatMessage(BaseModel)

A chat completion message. Attributes:

role str - The role of the author of this message.
content str - The contents of the message.

ChatCompletionResponseChoice Objects

class ChatCompletionResponseChoice(BaseModel)

A chat completion choice generated by a chat model. Attributes:

index int - The index of the chat completion choice.
message ChatMessage - The chat completion message.
finish_reason Optional[str] - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

UsageInfo Objects

class UsageInfo(BaseModel)

Usage statistics. Attributes:

prompt_tokens int - The number of tokens in the prompt.
total_tokens int - The total number of tokens used in the request (prompt + completion).
completion_tokens Optional[int] - The number of tokens in the generated completion.

ChatCompletionResponse Objects

class ChatCompletionResponse(BaseModel)

The response message from a /v1/chat/completions call. Attributes:

id str - A unique identifier of the response.
object str - The object type, which is always “chat.completion”.
created int - The Unix time in seconds when the response was generated.
model str - The model used for the chat completion.
choices List[ChatCompletionResponseChoice] - The list of chat completion choices.
usage UsageInfo - Usage statistics for the chat completion.

DeltaMessage Objects

class DeltaMessage(BaseModel)

A message delta. Attributes:

role str - The role of the author of this message.
content str - The contents of the chunk message.

ChatCompletionResponseStreamChoice Objects

class ChatCompletionResponseStreamChoice(BaseModel)

A streamed chat completion choice. Attributes:

index int - The index of the chat completion choice.
delta DeltaMessage - The message delta.
finish_reason str - The reason the model stopped generating tokens. This will be “stop” if the model hit a natural stop point or a provided stop sequence, or “length” if the maximum number of tokens specified in the request was reached.

ChatCompletionStreamResponse Objects

class ChatCompletionStreamResponse(BaseModel)

The streamed response message from a /v1/chat/completions call. Attributes:

id str - A unique identifier of the response.
object str - The object type, which is always “chat.completion”.
created int - The Unix time in seconds when the response was generated.
model str - The model used for the chat completion.
choices (List[ChatCompletionResponseStreamChoice]):
The list of streamed chat completion choices.

model

Model Objects

class Model()

list

@classmethod
def list(cls, request_timeout=60)

Returns a list of available models. Arguments:

request_timeout int, optional - The request timeout in seconds. Default is 60.

Returns:

ListModelsResponse - A list of available models.

log

set_console_log_level

def set_console_log_level(level: str) -> None

Controls console logging. Arguments:

level - the minimum level that prints out to console.
Supported values: [CRITICAL, FATAL, ERROR, WARN,
WARNING, INFO, DEBUG]

error

PermissionError Objects

class PermissionError(FireworksError)

A permission denied error.

InvalidRequestError Objects

class InvalidRequestError(FireworksError)

A invalid request error.

AuthenticationError Objects

class AuthenticationError(FireworksError)

A authentication error.

RateLimitError Objects

class RateLimitError(FireworksError)

A rate limit error.

InternalServerError Objects

class InternalServerError(FireworksError)

An internal server error.

ServiceUnavailableError Objects

class ServiceUnavailableError(FireworksError)

A service unavailable error.

​BaseCompletion Objects

​create

​acreate

​completion

​Completion Objects

​chat_completion

​ChatCompletion Objects

​api

​Choice Objects

​CompletionResponse Objects

​CompletionResponseStreamChoice Objects

​CompletionStreamResponse Objects

​Model Objects

​ListModelsResponse Objects

​ChatMessage Objects

​ChatCompletionResponseChoice Objects

​UsageInfo Objects

​ChatCompletionResponse Objects

​DeltaMessage Objects

​ChatCompletionResponseStreamChoice Objects

​ChatCompletionStreamResponse Objects

​model

​Model Objects

​list

​log

​set_console_log_level

​error

​PermissionError Objects

​InvalidRequestError Objects

​AuthenticationError Objects

​RateLimitError Objects

​InternalServerError Objects

​ServiceUnavailableError Objects

BaseCompletion Objects

create

acreate

completion

Completion Objects

chat_completion

ChatCompletion Objects

api

Choice Objects

CompletionResponse Objects

CompletionResponseStreamChoice Objects

CompletionStreamResponse Objects

Model Objects

ListModelsResponse Objects

ChatMessage Objects

ChatCompletionResponseChoice Objects

UsageInfo Objects

ChatCompletionResponse Objects

DeltaMessage Objects

ChatCompletionResponseStreamChoice Objects

ChatCompletionStreamResponse Objects

model

Model Objects

list

log

set_console_log_level

error

PermissionError Objects

InvalidRequestError Objects

AuthenticationError Objects

RateLimitError Objects

InternalServerError Objects

ServiceUnavailableError Objects