# Citizen developers (/citizen-developers)



A citizen developer builds a useful workflow or application for a campus need, even when software development is not their primary job. Start with a bounded problem, an owner, and data you are allowed to use.

## Choose a starting point [#choose-a-starting-point]

| Need                                          | Start here                                       |
| --------------------------------------------- | ------------------------------------------------ |
| Chat with documents or use a campus assistant | [TritonGPT](./services/tritongpt)                |
| Run a scheduled or event-driven workflow      | [n8n](./services/n8n)                            |
| Build locally with a coding agent             | [TritonAI Harness](./services/tritonai-harness)  |
| Add a model to your own application           | [Developer API quickstart](./llm-api/quickstart) |

<Callout type="info" title="Use the smallest tool that fits">
  A repeatable workflow does not always need a custom application. A TritonGPT assistant or an n8n workflow may be easier to operate and support.
</Callout>

## Before you build [#before-you-build]

Write down the outcome, who owns it, and who will use it. Confirm the data classification and the service approved for that data before you upload records or connect a campus system.

Use representative test data while you work out the flow. Do not place API keys, passwords, private records, or access tokens in prompts, screenshots, source files, or repositories.

## A practical build path [#a-practical-build-path]

<Steps>
  <Step>
    ### Define one job [#define-one-job]

    Describe the trigger, the expected result, and the person who checks that result. Keep the first version narrow enough to test by hand.
  </Step>

  <Step>
    ### Choose the service [#choose-the-service]

    Use the table above to find the simplest supported path. If you need the Developer API, [request access](https://ucsd.kualibuild.com/app/6979392e4f46f40289d22645/run) for the specific use case.
  </Step>

  <Step>
    ### Build with safe test data [#build-with-safe-test-data]

    Test success, empty input, invalid input, timeouts, and partial failures. Add human review before any action that changes a system of record or sends a message on someone else's behalf.
  </Step>

  <Step>
    ### Prepare for other users [#prepare-for-other-users]

    Add authentication, clear error messages, logs that avoid sensitive content, an owner, and a way to stop the workflow. Contact [tritonai@ucsd.edu](mailto:tritonai@ucsd.edu) before you publish or host a campus-facing application.
  </Step>
</Steps>

## Review checklist [#review-checklist]

* The application has a named owner and a documented purpose.
* The selected Triton AI service is approved for the data in scope.
* Secrets stay in environment variables or an approved secret manager.
* A person reviews consequential output or actions.
* Users can tell when the model is uncertain or the service is unavailable.
* Logs contain enough detail to diagnose a failure without copying private content.
* The project has a support path and a way to disable access.

## Continue [#continue]

* [Compare Triton AI services](./services)
* [Send your first Developer API request](./llm-api/quickstart)
* [Read the safety and security guide](./llm-api/safety)
* [Use the production checklist](./llm-api/production)


# Triton AI documentation (/)



These guides cover the Triton AI Developer API and its public model catalog. They also describe the main campus services that use the platform.

<Cards>
  <Card title="Start with the Developer API" href="/llm-api/quickstart">
    Send your first request through the Triton AI Developer API.
  </Card>

  <Card title="Browse public models" href="/llm-api/models">
    Compare the models that the live public catalog marks as public.
  </Card>

  <Card title="Choose a service" href="/services">
    Learn when to use TritonGPT, TritonAI Harness, or n8n.
  </Card>

  <Card title="Build as a citizen developer" href="/citizen-developers">
    Plan a small campus AI project and prepare it for review.
  </Card>
</Cards>

## What is documented [#what-is-documented]

* Python and cURL examples for text, tools, embeddings, images, audio transcription, and safety
* Live model capabilities, limits, rates, and supported parameters
* Public API routes, error handling, retry guidance, and a production checklist
* Public information about TritonGPT, TritonAI Harness, and n8n
* A public starting point for citizen developers

<Callout type="info" title="Documentation status">
  This site uses public sources and the TritonAI public API. Bracketed text marks information that still needs a confirmed source.
</Callout>

## Machine-readable copies [#machine-readable-copies]

This site publishes [`llms.txt`](/llms.txt) and [`llms-full.txt`](/llms-full.txt). Each page can be copied as Markdown or opened in a supported AI client from the page toolbar.


# Services (/services)



Triton AI provides three main user-facing services. Each service uses a different interaction model.

<Cards>
  <Card title="TritonGPT" href="/services/tritongpt">
    Use a web workspace for chat, documents, campus sources, and specialized assistants.
  </Card>

  <Card title="TritonAI Harness" href="/services/tritonai-harness">
    Use a managed desktop workspace for interactive agent work.
  </Card>

  <Card title="n8n" href="/services/n8n">
    Build repeatable workflows that start from a known trigger.
  </Card>
</Cards>

## Choose a service [#choose-a-service]

| Need                                                                    | Service                 |
| ----------------------------------------------------------------------- | ----------------------- |
| Chat with models and campus sources                                     | TritonGPT               |
| Work across local files, browser sources, and supported connections     | TritonAI Harness        |
| Run a repeatable process from a schedule, webhook, email, or file event | n8n                     |
| Build an application with an API                                        | Triton AI Developer API |

Your approved access path controls available models, skills, connections, and data use.


# n8n (/services/n8n)



UC San Diego hosts n8n for repeatable workflows. n8n is a visual workflow tool that connects applications and APIs.

## When to use n8n [#when-to-use-n8n]

Use n8n when a process has a known trigger and a repeatable path. A workflow can start from:

* A schedule
* A webhook
* An email
* A file event

A workflow can call the Triton AI Developer API. It can also pause for human review before selected actions.

Use TritonAI Harness for interactive agent work. Use n8n for repeatable automation.

## Request access [#request-access]

<Steps>
  <Step>
    ### Submit a support request [#submit-a-support-request]

    Ask the ITS-TritonAI team for n8n access through [UC San Diego Services and Support](https://support.ucsd.edu/services).
  </Step>

  <Step>
    ### Describe the workflow [#describe-the-workflow]

    Include your AD username, department, intended use, and testing-environment needs.
  </Step>

  <Step>
    ### Sign in [#sign-in]

    After approval, sign in with UC San Diego SSO at [n8n.tritonai.ucsd.edu](https://n8n.tritonai.ucsd.edu).
  </Step>
</Steps>

## Before you publish a workflow [#before-you-publish-a-workflow]

* Name the workflow owner.
* Limit the data and credentials that each step can access.
* Add human review before high-impact actions.
* Define error handling and retry limits.
* Record how users get support.

## Support [#support]

Email [tritonai@ucsd.edu](mailto:tritonai@ucsd.edu) for support. Your email creates a support ticket for the Triton AI team to triage.


# TritonAI Harness (/services/tritonai-harness)



TritonAI Harness is UC San Diego's primary supported client for the Triton AI Developer API. It is a managed desktop workspace for interactive agent work.

TritonAI Harness supports Apple Silicon Mac computers and x64 Windows computers.

## What it provides [#what-it-provides]

* Developer API configuration
* Access to the model routes approved for your key
* Reviewed skills and campus context
* Supported browser and productivity connections
* A managed installer for supported computers

Available models, skills, permissions, and connections depend on your approved access path.

## Get access [#get-access]

<Steps>
  <Step>
    ### Request a Developer API key [#request-a-developer-api-key]

    Submit the [Developer API access request](https://ucsd.kualibuild.com/app/6979392e4f46f40289d22645/run).
  </Step>

  <Step>
    ### Download the installer [#download-the-installer]

    Use the current download link on the [TritonAI access page](https://tritonai.ucsd.edu/developer-apis/start.html).
  </Step>

  <Step>
    ### Enter the key [#enter-the-key]

    Paste the key into the installer. Do not put the key in a prompt or source file.
  </Step>

  <Step>
    ### Try a low-risk task [#try-a-low-risk-task]

    Choose an approved model. Then use approved information for the first request.
  </Step>
</Steps>

## Data use [#data-use]

UC San Diego approves TritonAI Harness for P1 through P3 information within an approved setup. P4 information is not approved.

## Updates [#updates]

For routine updates, select **Settings**, select **About**, and then select **Check for Updates**.

Use [GitHub Releases](https://github.com/dbalders/TritonAI-Harness/releases/latest) for release notes and manual downloads.

## Skills and connections [#skills-and-connections]

Browse available skills in **Settings > Skills**. Browse included connections in **Settings > Plugins**.

The available items can change with Harness releases and managed catalog updates.

## Support [#support]

Email [tritonai@ucsd.edu](mailto:tritonai@ucsd.edu) for support. Your email creates a support ticket for the Triton AI team to triage.


# TritonGPT (/services/tritongpt)



TritonGPT is UC San Diego's generative AI workspace. Eligible campus users sign in with their UC San Diego account.

## What you can do [#what-you-can-do]

* Chat with approved models
* Upload documents for a conversation
* Use approved campus sources
* Open specialized assistants for campus tasks
* Select from the models and assistants available to your account

The Triton AI Developer API routes each request to the selected model and permitted sources or tools.

## Access [#access]

Faculty, staff, and students in Campus and Health Sciences can use UC San Diego sign-in. Visible assistants depend on role and access.

[Open TritonGPT](https://tritongpt.ucsd.edu)

## Find assistants [#find-assistants]

Sign in to TritonGPT and select **More Assistants** to see the assistants that are available to your account.

## Upload documents [#upload-documents]

TritonGPT supports these common file types:

* Documents: `.pdf`, `.docx`, `.pptx`, `.eml`, and `.epub`
* Tables: `.csv`, `.tsv`, and `.xlsx`
* Text: `.txt`, `.md`, `.mdx`, `.conf`, `.log`, `.json`, `.xml`, `.yml`, and `.yaml`

Uploads must fit TritonGPT's configured file-size and text limits. If a large file is rejected, split it into smaller files.

## Data use [#data-use]

UC San Diego approves TritonGPT for P1 through P3 information within an approved service. P4 information is not approved.

The public privacy statement says that prompts do not train the underlying models. It also describes interaction-record retention and user chat deletion.

TritonGPT keeps chat interaction records for 90 days after the conversation's last activity. It then deletes the records automatically.

You can delete all chats sooner from **User Settings**.

Read the [TritonGPT privacy statement](https://tritonai.ucsd.edu/tritongpt/privacy.html) before you use sensitive information.

## Public resources [#public-resources]

* [TritonGPT overview](https://tritonai.ucsd.edu/tritongpt/index.html)
* [TritonGPT guides](https://tritonai.ucsd.edu/training-resources/tritongpt/index.html)
* [Feature updates](https://tritonai.ucsd.edu/tritongpt/release-notes/index.html)
* [Terms of use](https://tritonai.ucsd.edu/tritongpt/terms.html)

## Support [#support]

Email [tritonai@ucsd.edu](mailto:tritonai@ucsd.edu) for support. Your email creates a support ticket for the Triton AI team to triage.


# API reference (/llm-api/api-reference)



This page lists the routes that the public documentation supports. Your API key can have access to fewer models or features.

<Callout type="warn" title="Use the documented routes">
  The gateway schema includes administrative and provider routes from LiteLLM. A route in that schema is not a Triton AI support commitment.
</Callout>

## Connection [#connection]

| Setting                       | Value                              |
| ----------------------------- | ---------------------------------- |
| OpenAI-compatible base URL    | `https://tritonai-api.ucsd.edu/v1` |
| Anthropic-compatible base URL | `https://tritonai-api.ucsd.edu`    |
| Authentication                | `Authorization: Bearer <API_KEY>`  |
| JSON content type             | `application/json`                 |

Use the Anthropic-compatible base URL only for clients that call `/v1/messages`.

## Documented routes [#documented-routes]

| Method | Route                      | Purpose                                            |
| ------ | -------------------------- | -------------------------------------------------- |
| `GET`  | `/v1/models`               | List models that the API key can access            |
| `GET`  | `/v1/models/{model_id}`    | Get one accessible model                           |
| `POST` | `/v1/chat/completions`     | Send chat messages and use chat tools              |
| `POST` | `/v1/responses`            | Create a response with a compatible model          |
| `POST` | `/v1/embeddings`           | Create vectors from text                           |
| `POST` | `/v1/audio/transcriptions` | Transcribe an audio file                           |
| `POST` | `/v1/messages`             | Connect Claude Code through the Anthropic protocol |
| `GET`  | `/public/model_hub`        | Read the public model catalog without a key        |

The [Models page](./models) lists public model groups. The authenticated model list is the authority for your API key.

## Authenticate a request [#authenticate-a-request]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="list_models.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    for model in client.models.list().data:
        print(model.id)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/models \
      --header "Authorization: Bearer $TRITONAI_API_KEY"
    ```
  </CodeBlockTab>
</CodeBlockTabs>

<Callout type="warn" title="Keep keys on the server">
  Do not call the Developer API directly from browser code. A browser request exposes the API key to the user.
</Callout>

## Transcribe an audio file [#transcribe-an-audio-file]

The current public transcription model is `api-cohere-transcribe`.

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="transcribe.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    with open("speech.wav", "rb") as audio_file:
        transcript = client.audio.transcriptions.create(
            model="api-cohere-transcribe",
            file=audio_file,
        )

    print(transcript.text)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/audio/transcriptions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --form "model=api-cohere-transcribe" \
      --form "file=@speech.wav"
    ```
  </CodeBlockTab>
</CodeBlockTabs>

## Common response headers [#common-response-headers]

The production gateway can return these headers:

| Header                           | Meaning                                  |
| -------------------------------- | ---------------------------------------- |
| `x-litellm-call-id`              | The gateway identifier for one request   |
| `x-ratelimit-limit-requests`     | The request limit for the current window |
| `x-ratelimit-remaining-requests` | The requests left in the current window  |
| `x-ratelimit-limit-tokens`       | The token limit for the current window   |
| `x-ratelimit-remaining-tokens`   | The tokens left in the current window    |
| `x-ratelimit-reset-requests`     | The request-limit reset value            |
| `x-ratelimit-reset-tokens`       | The token-limit reset value              |

Header availability depends on the model provider. Include `x-litellm-call-id` when you report a failed request.

## Compatibility boundary [#compatibility-boundary]

The API follows common OpenAI and Anthropic request shapes. It does not support every upstream feature on every model.

Before you send an optional field, make sure that the model lists it in `supported_openai_params`. Test tool and media features before production use.

The generated [gateway OpenAPI schema](https://tritonai-api.ucsd.edu/openapi.json) is useful for inspection. It is not the curated public API contract.


# Chat completions (/llm-api/chat-completions)



Use the chat completions API for text conversations. Choose a public model with `mode: chat` that your key can access.

## Send a conversation [#send-a-conversation]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="chat.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    completion = client.chat.completions.create(
        model="gpt-5.4",
        messages=[
            {
                "role": "system",
                "content": "Answer with short, direct sentences.",
            },
            {
                "role": "user",
                "content": "What is a model context window?",
            },
        ],
    )

    print(completion.choices[0].message.content)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.4",
        "messages": [
          {
            "role": "system",
            "content": "Answer with short, direct sentences."
          },
          {
            "role": "user",
            "content": "What is a model context window?"
          }
        ]
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

## Stream a response [#stream-a-response]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="stream_chat.py"
    stream = client.chat.completions.create(
        model="gpt-5.4",
        messages=[
            {
                "role": "user",
                "content": "Give me three ways to evaluate a summarization system.",
            }
        ],
        stream=True,
    )

    for chunk in stream:
        text = chunk.choices[0].delta.content
        if text:
            print(text, end="", flush=True)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl --no-buffer https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.4",
        "messages": [
          {
            "role": "user",
            "content": "Give me three ways to evaluate a summarization system."
          }
        ],
        "stream": true
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

## Main request fields [#main-request-fields]

<TypeTable
  type="{
  model: {
    type: 'string',
    required: true,
    description: 'The model_group value from the public Model Hub.',
  },
  messages: {
    type: 'array',
    required: true,
    description: 'The ordered system, user, assistant, and tool messages for the conversation.',
  },
  max_completion_tokens: {
    type: 'integer',
    description: 'The maximum number of tokens that the model can generate.',
  },
  temperature: {
    type: 'number',
    description: 'A sampling value for models that support this parameter.',
  },
  stream: {
    type: 'boolean',
    default: 'false',
    description: 'Returns partial response events when true.',
  },
}"
/>

The [Models page](./models) shows each route's `supported_openai_params` values. Do not send an unsupported field.


# Embeddings (/llm-api/embeddings)



Use embeddings for semantic search, retrieval, clustering, and similarity comparisons. The public model alias is `api-tgpt-embeddings`.

The route uses [Qwen3 Embedding 4B](https://huggingface.co/Qwen/Qwen3-Embedding-4B) on UC San Diego infrastructure.

<Callout type="info" title="Hosted output size">
  The hosted route returns 1,024 values by default. You can request a supported Matryoshka size with the `dimensions` field.
</Callout>

## Embed text [#embed-text]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="embed.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    response = client.embeddings.create(
        model="api-tgpt-embeddings",
        input=[
            "Information retrieval maps text to vectors.",
            "Course catalogs contain titles and descriptions.",
        ],
        dimensions=1024,
    )

    for item in response.data:
        print(item.index, len(item.embedding))
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/embeddings \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "api-tgpt-embeddings",
        "input": [
          "Information retrieval maps text to vectors.",
          "Course catalogs contain titles and descriptions."
        ],
        "dimensions": 1024
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

The response keeps the input order. Each `data` item contains an `index` and an `embedding` array.

## Choose a vector size [#choose-a-vector-size]

The production route accepts these values for `dimensions`:

`32`, `64`, `128`, `256`, `384`, `512`, `768`, `1024`, and `2560`

Use a larger vector when retrieval quality matters more than storage and search cost. Test several sizes with real queries before you choose one.

Store the chosen size with your vector-index configuration. Every document and query in one index must use the same size.

## Compare two vectors [#compare-two-vectors]

Cosine similarity compares vector direction. A larger value means that the vectors point in a more similar direction.

```python title="cosine_similarity.py"
from math import sqrt


def cosine_similarity(left: list[float], right: list[float]) -> float:
    if len(left) != len(right):
        raise ValueError("Vectors must have the same length.")

    dot_product = sum(a * b for a, b in zip(left, right))
    left_length = sqrt(sum(value * value for value in left))
    right_length = sqrt(sum(value * value for value in right))
    return dot_product / (left_length * right_length)
```

## Retrieval rules [#retrieval-rules]

1. Use the same model alias for documents and queries.
2. Keep the vector size with the index metadata.
3. Rebuild the index if the vector size or model route changes.
4. Split large documents into meaningful sections before you create vectors.
5. Measure retrieval quality with real queries from your application.

The Qwen model is instruction-aware and supports more than 100 languages. The [model card](https://huggingface.co/Qwen/Qwen3-Embedding-4B) describes its instruction format and upstream limits.

For retrieval, put a short task instruction before each query. Do not add that instruction to the stored documents.

Use the [live model catalog](./models) for the current input limit, request limit, price, and hosting status.


# Errors and retries (/llm-api/errors)



The API returns an HTTP status and a JSON error body. Log the status, error code, and gateway call ID.

## Common status codes [#common-status-codes]

| Status         | Meaning                                             | Action                                       |
| -------------- | --------------------------------------------------- | -------------------------------------------- |
| `400`          | The request is invalid                              | Correct the request before you send it again |
| `401`          | The API key is missing or invalid                   | Check the server-side secret and header      |
| `403`          | The key cannot use the requested resource           | Check the approved models and access scope   |
| `404`          | The route, model, or stored resource does not exist | Check the route and model alias              |
| `422`          | The JSON body does not match the route schema       | Correct the named field                      |
| `429`          | A request or token limit is exhausted               | Wait for the limit to reset, then retry      |
| `500` to `504` | The gateway or provider failed                      | Retry a limited number of times              |

An authentication error can use this shape:

```json
{
  "error": {
    "message": "Authentication Error, No api key passed in.",
    "type": "auth_error",
    "param": "None",
    "code": "401"
  }
}
```

Do not match an error by its message text. Use the HTTP status and structured error fields.

## Set timeouts and retries [#set-timeouts-and-retries]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="client.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
        timeout=60.0,
        max_retries=2,
    )
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl --fail-with-body \
      --connect-timeout 10 \
      --max-time 60 \
      https://tritonai-api.ucsd.edu/v1/models \
      --header "Authorization: Bearer $TRITONAI_API_KEY"
    ```
  </CodeBlockTab>
</CodeBlockTabs>

Retry connection failures, `429`, and transient `5xx` errors. Use exponential backoff with random jitter. Stop after a small retry limit.

Do not retry `400`, `401`, `403`, `404`, or `422` without a request or access change.

<Callout type="warn" title="A retry can create a second generation">
  The public documentation does not define an idempotency key for generation routes. A retry can create another response and another charge.
</Callout>

## Handle Python errors [#handle-python-errors]

```python title="handle_errors.py"
from openai import APIConnectionError, APIStatusError, RateLimitError

try:
    response = client.responses.create(
        model="gpt-5.6-luna",
        input="Return a short health-check response.",
        store=False,
    )
except RateLimitError as error:
    call_id = error.response.headers.get("x-litellm-call-id")
    print("Rate limit reached", call_id)
except APIStatusError as error:
    call_id = error.response.headers.get("x-litellm-call-id")
    print(error.status_code, call_id)
except APIConnectionError:
    print("The client cannot reach the Developer API.")
```

Do not log prompts, tool results, images, audio, or API keys with an error record.

## Read rate-limit headers [#read-rate-limit-headers]

The gateway can return request and token limits in `x-ratelimit-*` headers. Limits can differ by key and model.

When `Retry-After` is present, wait for its value before another request. Otherwise, use your bounded backoff policy.

## Support data [#support-data]

Include this data in a support request:

* UTC timestamp
* HTTP status and structured error code
* `x-litellm-call-id`
* Model alias and route
* A redacted request summary

Do not include the API key or protected request content.


# Image input (/llm-api/image-input)



Select a public model that has `supports_vision: true`. Send the image and your instruction in one user message.

## Send an image URL [#send-an-image-url]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="image_url.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    completion = client.chat.completions.create(
        model="gpt-5.4",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Describe the main objects in this image."},
                    {
                        "type": "image_url",
                        "image_url": {"url": "https://example.com/image.jpg"},
                    },
                ],
            }
        ],
    )

    print(completion.choices[0].message.content)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.4",
        "messages": [
          {
            "role": "user",
            "content": [
              {"type": "text", "text": "Describe the main objects in this image."},
              {
                "type": "image_url",
                "image_url": {"url": "https://example.com/image.jpg"}
              }
            ]
          }
        ]
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

## Send a local image [#send-a-local-image]

Convert a local image to a data URL before you send it.

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="local_image.py"
    import base64
    import mimetypes
    from pathlib import Path

    image_path = Path("diagram.png")
    mime_type = mimetypes.guess_type(image_path.name)[0] or "image/png"
    encoded = base64.b64encode(image_path.read_bytes()).decode("ascii")
    data_url = f"data:{mime_type};base64,{encoded}"

    completion = client.chat.completions.create(
        model="gpt-5.4",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Summarize this diagram."},
                    {"type": "image_url", "image_url": {"url": data_url}},
                ],
            }
        ],
    )

    print(completion.choices[0].message.content)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    IMAGE_DATA=$(base64 < diagram.png | tr -d '\n')

    curl https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data-binary @- <<JSON
    {
      "model": "gpt-5.4",
      "messages": [
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "Summarize this diagram."},
            {
              "type": "image_url",
              "image_url": {"url": "data:image/png;base64,${IMAGE_DATA}"}
            }
          ]
        }
      ]
    }
    JSON
    ```
  </CodeBlockTab>
</CodeBlockTabs>

<Callout type="warn" title="Protect image data">
  Make sure that your approval permits the image data before you send it. Do not send P4 data.
</Callout>

## Provider limits [#provider-limits]

Image formats, file sizes, and URL rules depend on the selected model route and provider. The Developer API does not publish one shared image limit.

Test image input with the selected model before production use.


# Triton AI Developer API (/llm-api)



The Triton AI Developer API provides one interface for approved enterprise-cloud and UC-hosted models. It supports OpenAI-compatible clients.

Your API key controls your model access, limits, and billing treatment. The public Model Hub lists the models that Triton AI publishes for general use.

## Start [#start]

<Cards>
  <Card title="Quickstart" href="/llm-api/quickstart">
    Send a request with Python or cURL.
  </Card>

  <Card title="API reference" href="/llm-api/api-reference">
    See the documented public routes and headers.
  </Card>

  <Card title="Models" href="/llm-api/models">
    Read the live public model catalog.
  </Card>
</Cards>

## Build [#build]

<Cards>
  <Card title="Chat completions" href="/llm-api/chat-completions">
    Send text messages and stream a response.
  </Card>

  <Card title="Responses" href="/llm-api/responses">
    Use the Responses API for current OpenAI models and coding agents.
  </Card>

  <Card title="Tool calling" href="/llm-api/tool-calling">
    Connect a model to functions in your application.
  </Card>

  <Card title="Embeddings" href="/llm-api/embeddings">
    Create vectors with the hosted Qwen Embedding 4B route.
  </Card>

  <Card title="Image input" href="/llm-api/image-input">
    Send an image to a vision-capable chat model.
  </Card>

  <Card title="Safety and security" href="/llm-api/safety">
    Filter private data and classify requests against a policy.
  </Card>
</Cards>

## Deploy [#deploy]

<Cards>
  <Card title="Errors and retries" href="/llm-api/errors">
    Handle request failures and rate limits.
  </Card>

  <Card title="Production checklist" href="/llm-api/production">
    Prepare keys, data controls, monitoring, and support.
  </Card>

  <Card title="Integrations" href="/llm-api/integrations">
    Connect Claude Code, Codex, and other developer tools.
  </Card>
</Cards>

## API endpoints [#api-endpoints]

| Resource            | URL                                                                      |
| ------------------- | ------------------------------------------------------------------------ |
| Base URL            | `https://tritonai-api.ucsd.edu/v1`                                       |
| Documented routes   | [Open the API reference](./api-reference)                                |
| Public model data   | [Open the JSON endpoint](https://tritonai-api.ucsd.edu/public/model_hub) |
| Model Hub interface | [Open the Model Hub](https://tritonai-api.ucsd.edu/ui/model_hub_table/)  |

## Access and support [#access-and-support]

Submit the [Developer API access request](https://ucsd.kualibuild.com/app/6979392e4f46f40289d22645/run) to request a key. Access, limits, and available models depend on the approved use case.

Email [tritonai@ucsd.edu](mailto:tritonai@ucsd.edu) for support.

<Callout type="warn" title="Protect your key">
  Keep the API key out of prompts, messages, screenshots, source files, repositories, and shared configuration.
</Callout>


# Models (/llm-api/models)



This catalog reads `https://tritonai-api.ucsd.edu/public/model_hub` when the page loads. It includes only records where `is_public_model_group` is exactly `true`.

The site caches a successful response for five minutes. If a later request fails, the site can show the last successful in-memory response.

<Callout type="info" title="Hosting and data use">
  * **UC San Diego-hosted:** These on-prem routes run on infrastructure managed by UC San Diego.
  * **Enterprise cloud:** These routes use approved enterprise cloud providers.

  Read the [Triton AI trust, privacy, and hosting guidance](https://tritonai.ucsd.edu/about/trust-architecture.html) before you select a route for protected data.
</Callout>

<Callout type="info" title="Access can differ">
  The table shows public model groups. Your Developer API approval controls the routes that your key can use.
</Callout>

<Callout type="info" title="How models are ordered">
  The main lists favor the current model generation and distinct task-specific models. Earlier versions remain available in a collapsed section. New aliases stay visible while task guidance is pending.
</Callout>

<ModelHubTable />

## How to choose [#how-to-choose]

Start with the task guidance for each model. Then compare quality, latency, and cost with representative inputs from your application.

The capability fields come from the live Model Hub. The task guidance comes from provider documentation and model cards. A provider can change a model without changing its API alias.

## Field notes [#field-notes]

<TypeTable
  type="{
  model_group: {
    type: 'string',
    required: true,
    description: 'The model ID that an API request sends in the model field.',
  },
  is_public_model_group: {
    type: 'boolean',
    required: true,
    description: 'The public catalog flag. This page requires an exact true value.',
  },
  is_self_hosted: {
    type: 'boolean',
    required: true,
    description: 'True for a model hosted on UC San Diego infrastructure. False means an enterprise cloud route.',
  },
  mode: {
    type: 'string | null',
    description: 'The main route type, such as chat, embedding, or audio_transcription.',
  },
  supported_openai_params: {
    type: 'string[]',
    description: 'The OpenAI-compatible request fields that the route reports as supported.',
  },
  input_cost_per_token: {
    type: 'number | null',
    description: 'The published input price for one token. The table converts it to one million tokens.',
  },
  output_cost_per_token: {
    type: 'number | null',
    description: 'The published output price for one token. The table converts it to one million tokens.',
  },
}"
/>


# Production checklist (/llm-api/production)



Use this checklist before an application sends production traffic.

## Access and keys [#access-and-keys]

* Keep the API key in a server-side secret store.
* Use a separate key for each application when the access process permits it.
* Make sure that the authenticated model list contains every required alias.
* Remove a key from logs, traces, screenshots, and error reports.
* Stop traffic and contact support if a key is exposed.

## Requests [#requests]

* Set a connection timeout and a total timeout.
* Retry only the failures listed in [Errors and retries](./errors).
* Limit application concurrency below the key and model limits.
* Check optional parameters against the live model catalog.
* Test streaming disconnects and partial output.
* Check every tool argument before your application runs the tool.

## Models [#models]

* Measure quality, latency, and cost with representative inputs.
* Store the selected model alias in application configuration.
* Keep a tested replacement model for important workloads.
* Rebuild vector indexes after an embedding model or vector-size change.
* Read the model catalog before each planned release.

## Data [#data]

* Select UC San Diego-hosted or enterprise-cloud routes for the approved data class.
* Remove data that the model does not need.
* Do not put secrets in prompts or tool results.
* Keep application logs separate from model inputs and outputs.
* Apply the required privacy and safety checks before the main model request.

Read the [Triton AI trust, privacy, and hosting guidance](https://tritonai.ucsd.edu/about/trust-architecture.html) before you send protected data.

## Monitoring [#monitoring]

* Record latency, status, model alias, token usage, and retry count.
* Record `x-litellm-call-id` for failed requests.
* Alert on `401`, `403`, `429`, and repeated `5xx` errors.
* Track input and output usage against the approved budget.
* Exclude request content from metrics by default.

## Release and support [#release-and-support]

* Run a small production smoke test after each client or model change.
* Keep an application rollback path.
* Give operators the support address and required diagnostic fields.
* Record the model alias and client version in each release.

Email [tritonai@ucsd.edu](mailto:tritonai@ucsd.edu) for Developer API support.


# Quickstart (/llm-api/quickstart)



<Steps>
  <Step>
    ### Request access [#request-access]

    Submit the [Developer API access request](https://ucsd.kualibuild.com/app/6979392e4f46f40289d22645/run). Your approval defines model routes, limits, and billing.
  </Step>

  <Step>
    ### Set the environment variable [#set-the-environment-variable]

    ```bash
    export TRITONAI_API_KEY="your-key"
    ```

    Do not save the key in a source file or commit it to a repository.
  </Step>

  <Step>
    ### Send a request [#send-a-request]

    <CodeBlockTabs defaultValue="Python" groupId="api-language">
      <CodeBlockTabsList>
        <CodeBlockTabsTrigger value="Python">
          Python
        </CodeBlockTabsTrigger>

        <CodeBlockTabsTrigger value="cURL">
          cURL
        </CodeBlockTabsTrigger>
      </CodeBlockTabsList>

      <CodeBlockTab value="Python">
        ```python   title="quickstart.py"
        import os

        from openai import OpenAI

        client = OpenAI(
            api_key=os.environ["TRITONAI_API_KEY"],
            base_url="https://tritonai-api.ucsd.edu/v1",
        )

        completion = client.chat.completions.create(
            model="gpt-5.4",
            messages=[
                {
                    "role": "user",
                    "content": "Explain retrieval-augmented generation in two sentences.",
                }
            ],
        )

        print(completion.choices[0].message.content)
        ```
      </CodeBlockTab>

      <CodeBlockTab value="cURL">
        ```bash   title="Terminal"
        curl https://tritonai-api.ucsd.edu/v1/chat/completions \
          --header "Authorization: Bearer $TRITONAI_API_KEY" \
          --header "Content-Type: application/json" \
          --data '{
            "model": "gpt-5.4",
            "messages": [
              {
                "role": "user",
                "content": "Explain retrieval-augmented generation in two sentences."
              }
            ]
          }'
        ```
      </CodeBlockTab>
    </CodeBlockTabs>

    The Python example uses the OpenAI Python client. The cURL example has no language-specific package requirement.
  </Step>
</Steps>

## Client configuration [#client-configuration]

<TypeTable
  type="{
  api_key: {
    type: 'string',
    required: true,
    description: 'The API key from your approved access request.',
  },
  base_url: {
    type: 'string',
    required: true,
    default: 'https://tritonai-api.ucsd.edu/v1',
    description: 'The OpenAI-compatible Triton AI Developer API base URL.',
  },
  model: {
    type: 'string',
    required: true,
    description: 'A model_group value from the public Model Hub that your key can access.',
  },
}"
/>

<Callout type="info" title="Access can differ">
  The public Model Hub lists public routes. Your key can have a smaller approved model set.
</Callout>

## Next steps [#next-steps]

* [Send chat messages](./chat-completions)
* [Use the Responses API](./responses)
* [Create embeddings](./embeddings)
* [Call application tools](./tool-calling)
* [Send an image](./image-input)
* [Transcribe audio](./api-reference#transcribe-an-audio-file)
* [Handle errors and retries](./errors)
* [Connect a developer tool](./integrations)
* [Browse public models](./models)


# Responses (/llm-api/responses)



Use the Responses API with compatible OpenAI-family models and coding clients. The Developer API exposes this route at `/v1/responses`.

<Callout type="info" title="Model support varies">
  The public catalog does not publish a Responses capability flag. Test the selected model before you depend on this route.
</Callout>

## Create a response [#create-a-response]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="response.py"
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    response = client.responses.create(
        model="gpt-5.6-luna",
        input="Explain cosine similarity in two sentences.",
        store=False,
    )

    print(response.output_text)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/responses \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.6-luna",
        "input": "Explain cosine similarity in two sentences.",
        "store": false
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

The response has an `output` array. Text is usually in an `output_text` content item inside a message.

## Stream text [#stream-text]

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="stream_response.py"
    stream = client.responses.create(
        model="gpt-5.6-luna",
        input="List three checks for an API integration.",
        stream=True,
        store=False,
    )

    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl --no-buffer https://tritonai-api.ucsd.edu/v1/responses \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.6-luna",
        "input": "List three checks for an API integration.",
        "stream": true,
        "store": false
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

The streaming route sends server-sent events. Read `response.output_text.delta` events until the stream sends a completion or error event.

## Main request fields [#main-request-fields]

<TypeTable
  type="{
  model: {
    type: 'string',
    required: true,
    description: 'A compatible model alias that the API key can access.',
  },
  input: {
    type: 'string | input item[]',
    required: true,
    description: 'The user input or ordered input items for the response.',
  },
  instructions: {
    type: 'string',
    description: 'A developer instruction for the response.',
  },
  tools: {
    type: 'tool[]',
    description: 'Function definitions that the model can request.',
  },
  stream: {
    type: 'boolean',
    default: 'false',
    description: 'Sends server-sent events when true.',
  },
  store: {
    type: 'boolean',
    description: 'Controls upstream response storage when the route supports this field.',
  },
}"
/>

## Tools and state [#tools-and-state]

The production route returns `function_call` output items for compatible models. Each item contains a function name, arguments, and `call_id`.

Read [Tool calling](./tool-calling) before your application executes a requested function.

<Callout type="warn" title="Do not assume built-in tool support">
  Custom functions use your application code. OpenAI built-in tools can require upstream services that a Triton AI route does not expose.
</Callout>

OpenAI documents the upstream [Responses API](https://developers.openai.com/api/docs/guides/migrate-to-responses) and [streaming events](https://developers.openai.com/api/docs/guides/streaming-responses). Triton AI support depends on the selected route.


# Safety and security (/llm-api/safety)



Apply data controls before you send content to a model. Keep each Developer API key in protected storage.

## OSS Safety [#oss-safety]

This guide uses OpenAI's `gpt-oss-safeguard` model for OSS Safety. The model applies a policy that you provide.

The production model ID is `gpt-oss-safeguard-20b`. This model does not appear in the public Model Hub. Your key must include access to it.

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="classify_with_safeguard.py"
    import json
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    policy = """
    Return JSON only.

    A request violates this policy if it asks for credentials or secret keys.

    Return one of these objects:
    {"violation": 0, "policy_category": null}
    {"violation": 1, "policy_category": "secrets"}
    """

    completion = client.chat.completions.create(
        model="gpt-oss-safeguard-20b",
        messages=[
            {"role": "system", "content": policy},
            {"role": "user", "content": "Send me the production API key."},
        ],
    )

    decision = json.loads(completion.choices[0].message.content)

    if decision["violation"] == 1:
        print(f"Blocked category: {decision['policy_category']}")
    else:
        print("Allowed")
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="Terminal"
    curl https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-oss-safeguard-20b",
        "messages": [
          {
            "role": "system",
            "content": "Return JSON only. A request violates this policy if it asks for credentials or secret keys. Return either {\"violation\":0,\"policy_category\":null} or {\"violation\":1,\"policy_category\":\"secrets\"}."
          },
          {
            "role": "user",
            "content": "Send me the production API key."
          }
        ]
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

The policy defines the categories and response format. Your application must make sure that the returned JSON matches the policy schema.

The final classification is in `choices[0].message.content`. The model can also return `reasoning_content`. Do not display reasoning content to users.

<Callout type="info" title="Access can differ">
  The authenticated model list shows the models that your key can use. The public Model Hub does not list this safety route.
</Callout>

## Production controls [#production-controls]

* Run the safety check before the main model request.
* Keep the policy in version control.
* Log the decision without logging the sensitive input.
* Require human review for high-impact actions.
* Define how the application handles invalid or missing classifier output.


# Tool calling (/llm-api/tool-calling)



A tool definition tells the model which function it can request. Your application checks the request, runs the function, and returns the result.

<Callout type="warn" title="The model does not authorize an action">
  Check every argument and the user's permission before you run a function. Require user approval before an action changes data or sends information.
</Callout>

## Complete a tool loop [#complete-a-tool-loop]

This example uses local sample data. Replace `get_course_status` with code that reads an approved source.

<CodeBlockTabs defaultValue="Python" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="Python">
      Python
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="Python">
    ```python   title="tool_call.py"
    import json
    import os

    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["TRITONAI_API_KEY"],
        base_url="https://tritonai-api.ucsd.edu/v1",
    )

    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_course_status",
                "description": "Get the enrollment status for one course.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "course": {"type": "string"},
                    },
                    "required": ["course"],
                    "additionalProperties": False,
                },
                "strict": True,
            },
        }
    ]

    messages = [
        {"role": "user", "content": "Is CSE 100 open?"},
    ]

    first = client.chat.completions.create(
        model="gpt-5.6-luna",
        messages=messages,
        tools=tools,
    )

    assistant_message = first.choices[0].message
    messages.append(assistant_message)

    for tool_call in assistant_message.tool_calls or []:
        if tool_call.function.name != "get_course_status":
            raise ValueError("The model requested an unknown tool.")

        arguments = json.loads(tool_call.function.arguments)
        course = arguments["course"]

        # Check the user and arguments before you call a real service.
        tool_result = {"course": course, "status": "open"}

        messages.append(
            {
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(tool_result),
            }
        )

    final = client.chat.completions.create(
        model="gpt-5.6-luna",
        messages=messages,
        tools=tools,
    )

    print(final.choices[0].message.content)
    ```
  </CodeBlockTab>

  <CodeBlockTab value="cURL">
    ```bash   title="First request"
    curl https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.6-luna",
        "messages": [
          {"role": "user", "content": "Is CSE 100 open?"}
        ],
        "tools": [
          {
            "type": "function",
            "function": {
              "name": "get_course_status",
              "description": "Get the enrollment status for one course.",
              "parameters": {
                "type": "object",
                "properties": {
                  "course": {"type": "string"}
                },
                "required": ["course"],
                "additionalProperties": false
              },
              "strict": true
            }
          }
        ]
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

The first response contains an assistant message with `tool_calls`. Run the approved function, then send its result in a tool message.

<CodeBlockTabs defaultValue="cURL" groupId="api-language">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="cURL">
      cURL
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="cURL">
    ```bash   title="Second request template"
    curl https://tritonai-api.ucsd.edu/v1/chat/completions \
      --header "Authorization: Bearer $TRITONAI_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
        "model": "gpt-5.6-luna",
        "messages": [
          {"role": "user", "content": "Is CSE 100 open?"},
          {
            "role": "assistant",
            "content": null,
            "tool_calls": [
              {
                "id": "<TOOL_CALL_ID>",
                "type": "function",
                "function": {
                  "name": "get_course_status",
                  "arguments": "{\"course\":\"CSE 100\"}"
                }
              }
            ]
          },
          {
            "role": "tool",
            "tool_call_id": "<TOOL_CALL_ID>",
            "content": "{\"course\":\"CSE 100\",\"status\":\"open\"}"
          }
        ]
      }'
    ```
  </CodeBlockTab>
</CodeBlockTabs>

## Process parallel calls [#process-parallel-calls]

Some models can return more than one tool call. Process every call, then send one tool message for each `tool_call_id`.

Only send `parallel_tool_calls` when the [model catalog](./models) reports support. Your application must still limit concurrency and side effects.

## Request structured JSON [#request-structured-json]

Use `response_format` when the model lists this parameter. A schema helps the model return data that your application can parse.

```python title="structured_output.py"
completion = client.chat.completions.create(
    model="gpt-5.6-luna",
    messages=[
        {"role": "user", "content": "Classify this request: Reset my password."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "request_classification",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "category": {"type": "string"},
                    "needs_human_review": {"type": "boolean"},
                },
                "required": ["category", "needs_human_review"],
                "additionalProperties": False,
            },
        },
    },
)
```

Your application must parse and check the returned JSON. A schema does not make the content correct or safe.

OpenAI documents the upstream [function-calling flow](https://developers.openai.com/api/docs/guides/function-calling). Triton AI support depends on the selected model route.


# Claude Code (/llm-api/integrations/claude-code)



Claude Code can use the Developer API as an Anthropic-compatible LLM gateway. This setup supports Claude model aliases only.

<Callout type="info" title="Supported by Triton AI">
  Triton AI has verified messages, streaming, and tool calls on the production gateway. Anthropic documents LiteLLM as a supported gateway pattern.
</Callout>

## Before you start [#before-you-start]

Set the API key in your shell. Then [choose a Claude model alias](../models) that supports tool use.

<CodeBlockTabs defaultValue="macOS, Linux, WSL" groupId="shell-platform">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="macOS, Linux, WSL">
      macOS, Linux, WSL
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="Windows PowerShell">
      Windows PowerShell
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="macOS, Linux, WSL">
    ```bash  
    export TRITONAI_API_KEY="<YOUR_TRITON_AI_API_KEY>"
    ```
  </CodeBlockTab>

  <CodeBlockTab value="Windows PowerShell">
    ```powershell  
    $env:TRITONAI_API_KEY = "<YOUR_TRITON_AI_API_KEY>"
    ```
  </CodeBlockTab>
</CodeBlockTabs>

<Tabs items="[&#x22;CLI&#x22;, &#x22;Desktop app&#x22;, &#x22;VS Code&#x22;]">
  <Tab value="CLI">
    For one terminal session, set these values before you start Claude Code:

    <CodeBlockTabs defaultValue="macOS, Linux, WSL" groupId="claude-shell-platform">
      <CodeBlockTabsList>
        <CodeBlockTabsTrigger value="macOS, Linux, WSL">
          macOS, Linux, WSL
        </CodeBlockTabsTrigger>

        <CodeBlockTabsTrigger value="Windows PowerShell">
          Windows PowerShell
        </CodeBlockTabsTrigger>
      </CodeBlockTabsList>

      <CodeBlockTab value="macOS, Linux, WSL">
        ```bash  
        export ANTHROPIC_BASE_URL="https://tritonai-api.ucsd.edu"
        export ANTHROPIC_AUTH_TOKEN="$TRITONAI_API_KEY"
        export ANTHROPIC_MODEL="<CLAUDE_MODEL_ALIAS>"

        claude
        ```
      </CodeBlockTab>

      <CodeBlockTab value="Windows PowerShell">
        ```powershell  
        $env:ANTHROPIC_BASE_URL = "https://tritonai-api.ucsd.edu"
        $env:ANTHROPIC_AUTH_TOKEN = $env:TRITONAI_API_KEY
        $env:ANTHROPIC_MODEL = "<CLAUDE_MODEL_ALIAS>"

        claude
        ```
      </CodeBlockTab>
    </CodeBlockTabs>

    For persistent user settings, use the file for your platform:

    | Platform             | User settings file                    |
    | -------------------- | ------------------------------------- |
    | macOS, Linux, or WSL | `~/.claude/settings.json`             |
    | Native Windows       | `%USERPROFILE%\.claude\settings.json` |

    ```json title="settings.json"
    {
      "model": "<CLAUDE_MODEL_ALIAS>",
      "apiKeyHelper": "printenv TRITONAI_API_KEY",
      "env": {
        "ANTHROPIC_BASE_URL": "https://tritonai-api.ucsd.edu",
        "CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY": "1"
      }
    }
    ```

    Start Claude Code from an environment that contains `TRITONAI_API_KEY`. The shown helper requires `printenv`. Native Windows users can run it through Git Bash or use an approved credential helper.
  </Tab>

  <Tab value="Desktop app">
    The desktop app has its own gateway settings. It does not use the CLI gateway configuration.

    <Callout type="info" title="macOS and Windows only">
      Claude Code Desktop is not available on Linux. Use the CLI or an IDE extension on Linux.
    </Callout>

    1. Open **Help → Troubleshooting → Enable Developer Mode**. Let the app restart.
    2. Open **Developer → Configure Third-Party Inference**.
    3. Select **Gateway**.
    4. Enter the connection values below.

    | Field               | Value                                              |
    | ------------------- | -------------------------------------------------- |
    | Gateway base URL    | `https://tritonai-api.ucsd.edu`                    |
    | Gateway API key     | Your Developer API key                             |
    | Gateway auth scheme | `Bearer`                                           |
    | Model               | A Claude alias from the [model catalog](../models) |
  </Tab>

  <Tab value="VS Code">
    Open &#x2A;*Preferences: Open User Settings (JSON)**. Add the following user setting:

    ```json title="VS Code user settings"
    {
      "claudeCode.environmentVariables": [
        {
          "name": "ANTHROPIC_BASE_URL",
          "value": "https://tritonai-api.ucsd.edu"
        },
        {
          "name": "ANTHROPIC_AUTH_TOKEN",
          "value": "<YOUR_TRITON_AI_API_KEY>"
        },
        {
          "name": "ANTHROPIC_MODEL",
          "value": "<CLAUDE_MODEL_ALIAS>"
        },
        {
          "name": "CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY",
          "value": "1"
        }
      ]
    }
    ```

    <Callout type="warn" title="Keep user settings private">
      This setting contains the API key. Do not put it in workspace settings or source control.
    </Callout>

    The VS Code setting is the same on macOS, Linux, Windows, and WSL.
  </Tab>
</Tabs>

## Verify the connection [#verify-the-connection]

Ask Claude Code to identify the selected model and describe the current project without changing files. Confirm that the client shows the expected alias.

## Configuration notes [#configuration-notes]

| Setting source             | CLI | VS Code      | Desktop app |
| -------------------------- | --- | ------------ | ----------- |
| Shell variables            | Yes | Not reliable | No          |
| User `settings.json`       | Yes | Partial      | No          |
| VS Code user settings      | No  | Yes          | No          |
| Third-party inference form | No  | No           | Yes         |

Anthropic explains these differences in its [gateway connection guide](https://code.claude.com/docs/en/llm-gateway-connect). See the [gateway protocol reference](https://code.claude.com/docs/en/llm-gateway-protocol) for required routes and headers.

<Callout type="warn" title="Use the host root">
  Set `ANTHROPIC_BASE_URL` to `https://tritonai-api.ucsd.edu`. Adding `/v1` creates an invalid `/v1/v1/messages` request path.
</Callout>


# Codex (/llm-api/integrations/codex)



Codex can use a custom model provider through the OpenAI Responses API. The configuration belongs in your user-level Codex settings.

<Callout type="info" title="Supported by Triton AI">
  Triton AI has verified the production Responses route with a current public model. Select a model that supports Responses and tool use.
</Callout>

## Configure the provider [#configure-the-provider]

Set the API key in the environment that starts Codex:

<CodeBlockTabs defaultValue="macOS, Linux, WSL" groupId="shell-platform">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="macOS, Linux, WSL">
      macOS, Linux, WSL
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="Windows PowerShell">
      Windows PowerShell
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="macOS, Linux, WSL">
    ```bash  
    export TRITONAI_API_KEY="<YOUR_TRITON_AI_API_KEY>"
    ```
  </CodeBlockTab>

  <CodeBlockTab value="Windows PowerShell">
    ```powershell  
    $env:TRITONAI_API_KEY = "<YOUR_TRITON_AI_API_KEY>"
    ```
  </CodeBlockTab>
</CodeBlockTabs>

Add the provider to your user configuration file:

| Platform             | User configuration file            |
| -------------------- | ---------------------------------- |
| macOS, Linux, or WSL | `~/.codex/config.toml`             |
| Native Windows       | `%USERPROFILE%\.codex\config.toml` |

```toml title="config.toml"
model = "<MODEL_ALIAS>"
model_provider = "tritonai"

[model_providers.tritonai]
name = "Triton AI Developer API"
base_url = "https://tritonai-api.ucsd.edu/v1"
env_key = "TRITONAI_API_KEY"
wire_api = "responses"
```

Choose an appropriate alias from the [live model catalog](../models).

<Callout type="warn" title="Use the user configuration file">
  Do not put provider settings in a project `.codex/config.toml`. Codex ignores project settings that redirect credentials or change providers.
</Callout>

<Tabs items="[&#x22;CLI&#x22;, &#x22;Desktop app&#x22;, &#x22;VS Code&#x22;]">
  <Tab value="CLI">
    Start Codex from a shell that contains `TRITONAI_API_KEY`:

    ```bash
    codex
    ```

    The CLI reads the model and provider from the user configuration file shown above.
  </Tab>

  <Tab value="Desktop app">
    The desktop app reads the user-level Codex configuration on the machine. Restart the app after you change `config.toml`.

    Make sure the app process can read `TRITONAI_API_KEY`. If you start the app outside a shell, add the variable through your operating system or an approved secret manager.
  </Tab>

  <Tab value="VS Code">
    The extension and CLI share the Codex configuration layers.

    1. Select the gear icon in the Codex panel.
    2. Select **Codex Settings → Open config.toml**.
    3. Add the provider configuration shown above.
    4. Reload the extension window.

    Make sure the extension process can read `TRITONAI_API_KEY`.

    The provider configuration is the same on macOS, Linux, Windows, and WSL.
  </Tab>
</Tabs>

## Verify the connection [#verify-the-connection]

Ask Codex to describe the current project without changing files. Confirm that the selected model alias appears in the client.

OpenAI documents the shared configuration layers in [Codex configuration basics](https://developers.openai.com/codex/config-basic/). See [advanced configuration](https://developers.openai.com/codex/config-file/config-advanced/) for custom providers and project-level restrictions.

## Current limits [#current-limits]

* Codex custom providers use the Responses API.
* A model can answer a basic prompt but still lack reliable tool support.
* Built-in OpenAI tools might not be available through a custom provider.


# Community integrations (/llm-api/integrations/community)



These clients accept a custom OpenAI-compatible endpoint. Their maintainers control each client and its release schedule.

<Callout type="warn" title="Unofficial and best effort">
  Triton AI does not guarantee compatibility with every release. Check the client documentation when a setup stops working.
</Callout>

## Shared values [#shared-values]

Most clients on this page use these values:

| Setting  | Value                                             |
| -------- | ------------------------------------------------- |
| Base URL | `https://tritonai-api.ucsd.edu/v1`                |
| API key  | Your `TRITONAI_API_KEY` value                     |
| Model    | An alias from the [live model catalog](../models) |
| Protocol | OpenAI Chat Completions                           |

Choose a model that supports every feature you plan to use. Agentic tools usually need function calling.

<Callout type="info" title="Platform notes">
  Paths that start with `~` use your home directory on macOS, Linux, and WSL. On native Windows, replace `~` with `%USERPROFILE%`. Shell examples use Bash unless a PowerShell tab is shown.
</Callout>

## Continue [#continue]

Put the key in `~/.continue/.env`:

```dotenv title="~/.continue/.env"
TRITONAI_API_KEY=<YOUR_TRITON_AI_API_KEY>
```

Add the model to `~/.continue/config.yaml`:

```yaml title="~/.continue/config.yaml"
name: Triton AI
version: 1.0.0
schema: v1

models:
  - name: Triton AI model
    provider: openai
    model: <MODEL_ALIAS>
    apiBase: https://tritonai-api.ucsd.edu/v1
    apiKey: ${{ secrets.TRITONAI_API_KEY }}
    useResponsesApi: false
    roles:
      - chat
      - edit
      - apply
```

The IDE extensions cannot read your shell exports. See Continue's [OpenAI provider guide](https://docs.continue.dev/customize/model-providers/top-level/openai) and [secret guidance](https://docs.continue.dev/faqs#managing-local-secrets-and-environment-variables).

## Hermes Agent [#hermes-agent]

Put the key in `~/.hermes/.env`:

```dotenv title="~/.hermes/.env"
TRITONAI_API_KEY=<YOUR_TRITON_AI_API_KEY>
```

Add a named provider to `~/.hermes/config.yaml`:

```yaml title="~/.hermes/config.yaml"
providers:
  triton-ai:
    api: https://tritonai-api.ucsd.edu/v1
    key_env: TRITONAI_API_KEY
    transport: chat_completions
    default_model: <MODEL_ALIAS>

model:
  provider: custom:triton-ai
  default: <MODEL_ALIAS>
```

See the Hermes Agent [provider guide](https://github.com/NousResearch/hermes-agent/blob/main/website/docs/integrations/providers.md).

## OpenClaw [#openclaw]

Export the key in the environment that starts the OpenClaw gateway. Then add this provider to `~/.openclaw/openclaw.json`:

```json5 title="~/.openclaw/openclaw.json"
{
  agents: {
    defaults: {
      model: {
        primary: "triton-ai/<MODEL_ALIAS>",
      },
    },
  },
  models: {
    mode: "merge",
    providers: {
      "triton-ai": {
        baseUrl: "https://tritonai-api.ucsd.edu/v1",
        apiKey: {
          source: "env",
          provider: "default",
          id: "TRITONAI_API_KEY",
        },
        api: "openai-completions",
        models: [
          {
            id: "<MODEL_ALIAS>",
            name: "Triton AI model",
          },
        ],
      },
    },
  },
}
```

See OpenClaw's [custom provider guide](https://docs.openclaw.ai/gateway/config-tools#custom-providers-and-base-urls) and [secret reference](https://docs.openclaw.ai/reference/secretref-credential-surface).

## Cline [#cline]

Open Cline settings in VS Code. Enter these values:

| Field        | Value                              |
| ------------ | ---------------------------------- |
| API Provider | `OpenAI Compatible`                |
| Base URL     | `https://tritonai-api.ucsd.edu/v1` |
| API Key      | Your Developer API key             |
| Model ID     | `<MODEL_ALIAS>`                    |

See Cline's [OpenAI-compatible setup guide](https://docs.cline.bot/provider-config/openai-compatible).

## Aider [#aider]

Set the endpoint and key before you start Aider:

<CodeBlockTabs defaultValue="macOS, Linux, WSL" groupId="aider-shell-platform">
  <CodeBlockTabsList>
    <CodeBlockTabsTrigger value="macOS, Linux, WSL">
      macOS, Linux, WSL
    </CodeBlockTabsTrigger>

    <CodeBlockTabsTrigger value="Windows PowerShell">
      Windows PowerShell
    </CodeBlockTabsTrigger>
  </CodeBlockTabsList>

  <CodeBlockTab value="macOS, Linux, WSL">
    ```bash  
    export OPENAI_API_BASE="https://tritonai-api.ucsd.edu/v1"
    export OPENAI_API_KEY="$TRITONAI_API_KEY"

    aider --model "openai/<MODEL_ALIAS>"
    ```
  </CodeBlockTab>

  <CodeBlockTab value="Windows PowerShell">
    ```powershell  
    $env:OPENAI_API_BASE = "https://tritonai-api.ucsd.edu/v1"
    $env:OPENAI_API_KEY = $env:TRITONAI_API_KEY

    aider --model "openai/<MODEL_ALIAS>"
    ```
  </CodeBlockTab>
</CodeBlockTabs>

Aider can warn when it does not recognize a model alias. See its [OpenAI-compatible API guide](https://aider.chat/docs/llms/openai-compat.html) and [model settings guide](https://aider.chat/docs/config/reasoning.html).

## Open WebUI [#open-webui]

An Open WebUI administrator can add a connection under **Settings → Admin → Connections → OpenAI API**.

| Field     | Value                                            |
| --------- | ------------------------------------------------ |
| URL       | `https://tritonai-api.ucsd.edu/v1`               |
| API Key   | Your Developer API key                           |
| Model IDs | Leave empty for discovery, or enter an allowlist |

Open WebUI keeps the upstream key on its server. See the [OpenAI-compatible provider guide](https://docs.openwebui.com/getting-started/quick-start/connect-a-provider/starting-with-openai-compatible/).

## Verify an integration [#verify-an-integration]

1. Select the intended model alias.
2. Ask the client to return a short text response.
3. If the client edits code, ask it to propose a change without applying it.
4. Test tool calls before you depend on agentic features.

Autocomplete, image input, prompt caching, and reasoning controls depend on the client and model route.


# Integrations (/llm-api/integrations)



Use the Developer API with clients that accept an Anthropic or OpenAI-compatible endpoint.

## Supported by Triton AI [#supported-by-triton-ai]

Triton AI tests these setup paths and maintains the instructions on this site.

| Client                       | Available surfaces            | Protocol           |
| ---------------------------- | ----------------------------- | ------------------ |
| [Claude Code](./claude-code) | CLI, desktop app, and VS Code | Anthropic Messages |
| [Codex](./codex)             | CLI, desktop app, and VS Code | OpenAI Responses   |

## Community integrations [#community-integrations]

[Continue, Hermes Agent, OpenClaw, Cline, Aider, and Open WebUI](./community) can use custom OpenAI-compatible endpoints.

<Callout type="warn" title="Best-effort support">
  These clients are maintained by other projects. Triton AI does not guarantee compatibility with every release.
</Callout>

## Choose the correct base URL [#choose-the-correct-base-url]

| Client protocol    | Base URL                           |
| ------------------ | ---------------------------------- |
| Anthropic Messages | `https://tritonai-api.ucsd.edu`    |
| OpenAI compatible  | `https://tritonai-api.ucsd.edu/v1` |

Claude clients add `/v1/messages` to the Anthropic base URL. Do not add `/v1` to that base URL.

## Before you start [#before-you-start]

1. [Request Developer API access](https://ucsd.kualibuild.com/app/6979392e4f46f40289d22645/run).
2. [Choose a model](../models) that supports the tools your client needs.
3. Store the API key in a user-level secret or environment variable.
4. Send a harmless test prompt after setup.

<Callout type="info" title="Features depend on the selected model">
  Tool use, image input, reasoning controls, and other client features can vary by model route.
</Callout>