Embeddings

Use embeddings for semantic search, retrieval, clustering, and similarity comparisons. The public model alias is api-tgpt-embeddings.

The route uses Qwen3 Embedding 4B on UC San Diego infrastructure.

Embed text

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TRITONAI_API_KEY"],
    base_url="https://tritonai-api.ucsd.edu/v1",
)

response = client.embeddings.create(
    model="api-tgpt-embeddings",
    input=[
        "Information retrieval maps text to vectors.",
        "Course catalogs contain titles and descriptions.",
    ],
    dimensions=1024,
)

for item in response.data:
    print(item.index, len(item.embedding))

The response keeps the input order. Each data item contains an index and an embedding array.

Choose a vector size

The production route accepts these values for dimensions:

32, 64, 128, 256, 384, 512, 768, 1024, and 2560

Use a larger vector when retrieval quality matters more than storage and search cost. Test several sizes with real queries before you choose one.

Store the chosen size with your vector-index configuration. Every document and query in one index must use the same size.

Compare two vectors

Cosine similarity compares vector direction. A larger value means that the vectors point in a more similar direction.

from math import sqrt


def cosine_similarity(left: list[float], right: list[float]) -> float:
    if len(left) != len(right):
        raise ValueError("Vectors must have the same length.")

    dot_product = sum(a * b for a, b in zip(left, right))
    left_length = sqrt(sum(value * value for value in left))
    right_length = sqrt(sum(value * value for value in right))
    return dot_product / (left_length * right_length)

Retrieval rules

1. Use the same model alias for documents and queries.
2. Keep the vector size with the index metadata.
3. Rebuild the index if the vector size or model route changes.
4. Split large documents into meaningful sections before you create vectors.
5. Measure retrieval quality with real queries from your application.

The Qwen model is instruction-aware and supports more than 100 languages. The model card describes its instruction format and upstream limits.

For retrieval, put a short task instruction before each query. Do not add that instruction to the stored documents.

Use the live model catalog for the current input limit, request limit, price, and hosting status.