Production checklist

Use this checklist before an application sends production traffic.

Access and keys

• Keep the API key in a server-side secret store.
• Use a separate key for each application when the access process permits it.
• Make sure that the authenticated model list contains every required alias.
• Remove a key from logs, traces, screenshots, and error reports.
• Stop traffic and contact support if a key is exposed.

Requests

• Set a connection timeout and a total timeout.
• Retry only the failures listed in Errors and retries.
• Limit application concurrency below the key and model limits.
• Check optional parameters against the live model catalog.
• Test streaming disconnects and partial output.
• Check every tool argument before your application runs the tool.

Models

• Measure quality, latency, and cost with representative inputs.
• Store the selected model alias in application configuration.
• Keep a tested replacement model for important workloads.
• Rebuild vector indexes after an embedding model or vector-size change.
• Read the model catalog before each planned release.

Data

• Select UC San Diego-hosted or enterprise-cloud routes for the approved data class.
• Remove data that the model does not need.
• Do not put secrets in prompts or tool results.
• Keep application logs separate from model inputs and outputs.
• Apply the required privacy and safety checks before the main model request.

Read the Triton AI trust, privacy, and hosting guidance before you send protected data.

Monitoring

• Record latency, status, model alias, token usage, and retry count.
• Record x-litellm-call-id for failed requests.
• Alert on 401, 403, 429, and repeated 5xx errors.
• Track input and output usage against the approved budget.
• Exclude request content from metrics by default.

Release and support

• Run a small production smoke test after each client or model change.
• Keep an application rollback path.
• Give operators the support address and required diagnostic fields.
• Record the model alias and client version in each release.

Email tritonai@ucsd.edu for Developer API support.