Errors and retries
Handle Developer API failures, timeouts, and rate limits.
The API returns an HTTP status and a JSON error body. Log the status, error code, and gateway call ID.
Common status codes
| Status | Meaning | Action |
|---|---|---|
400 | The request is invalid | Correct the request before you send it again |
401 | The API key is missing or invalid | Check the server-side secret and header |
403 | The key cannot use the requested resource | Check the approved models and access scope |
404 | The route, model, or stored resource does not exist | Check the route and model alias |
422 | The JSON body does not match the route schema | Correct the named field |
429 | A request or token limit is exhausted | Wait for the limit to reset, then retry |
500 to 504 | The gateway or provider failed | Retry a limited number of times |
Example authentication error:
{
"error": {
"message": "Authentication Error, No api key passed in.",
"type": "auth_error",
"param": "None",
"code": "401"
}
}Do not match an error by its message text. Use the HTTP status and structured error fields.
Set timeouts and retries
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["TRITONAI_API_KEY"],
base_url="https://tritonai-api.ucsd.edu/v1",
timeout=60.0,
max_retries=2,
)Retry connection failures, 429, and transient 5xx errors. Use exponential backoff with random jitter. Stop after a small retry limit.
Do not retry 400, 401, 403, 404, or 422 without a request or access change.
A retry can create a second generation
A retry can create another response and another charge. The public documentation does not define an idempotency key for generation routes.
Handle Python errors
from openai import APIConnectionError, APIStatusError, RateLimitError
try:
response = client.responses.create(
model="gpt-5.6-luna",
input="Return a short health-check response.",
store=False,
)
except RateLimitError as error:
call_id = error.response.headers.get("x-litellm-call-id")
print("Rate limit reached", call_id)
except APIStatusError as error:
call_id = error.response.headers.get("x-litellm-call-id")
print(error.status_code, call_id)
except APIConnectionError:
print("The client cannot reach the Developer API.")Do not log prompts, tool results, images, audio, or API keys with an error record.
Read rate-limit headers
The gateway can return request and token limits in x-ratelimit-* headers. Limits can differ by key and model.
When the response includes Retry-After, wait for the specified interval before another request. Otherwise, use your bounded backoff policy.
Support data
Include this data in a support request:
- UTC timestamp
- HTTP status and structured error code
x-litellm-call-id- Model alias and route
- A redacted request summary
Do not include the API key or protected request content.