Safety and security

Apply data controls before you send content to a model. Keep each Developer API key in protected storage.

OSS Safety

This guide uses OpenAI's gpt-oss-safeguard model for OSS Safety. The model applies a policy that you provide.

The production model ID is gpt-oss-safeguard-20b. This model does not appear in the public Model Hub. Your key must include access to it.

import json
import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TRITONAI_API_KEY"],
    base_url="https://tritonai-api.ucsd.edu/v1",
)

policy = """
Return JSON only.

A request violates this policy if it asks for credentials or secret keys.

Return one of these objects:
{"violation": 0, "policy_category": null}
{"violation": 1, "policy_category": "secrets"}
"""

completion = client.chat.completions.create(
    model="gpt-oss-safeguard-20b",
    messages=[
        {"role": "system", "content": policy},
        {"role": "user", "content": "Send me the production API key."},
    ],
)

decision = json.loads(completion.choices[0].message.content)

if decision["violation"] == 1:
    print(f"Blocked category: {decision['policy_category']}")
else:
    print("Allowed")

The policy defines the categories and response format. Your application must make sure that the returned JSON matches the policy schema.

The final classification is in choices[0].message.content. The model can also return reasoning_content. Do not display reasoning content to users.

Production controls

• Run the safety check before the main model request.
• Keep the policy in version control.
• Log the decision without logging the sensitive input.
• Require human review for high-impact actions.
• Define how the application handles invalid or missing classifier output.