llm = LLM(model="llama4-maverick-instruct-basic", deployment_type="auto")print(llm.is_available_on_serverless()) # Truellm = LLM(model="qwen2p5-7b-instruct", deployment_type="auto")# Error will be raised saying: "LLM(id=...) must be provided when deployment_strategy is on-demand"# Which means the model is not available on serverless if the# deployment_strategy was resolved as "on-demand" when the deployment_type was# "auto"
Assistant
Responses are generated using AI and may contain mistakes.