OpenAI Agents

Learn about using Sentry for OpenAI Agents SDK.

This integration connects Sentry with the OpenAI Python SDK. The integration has been confirmed to work with OpenAI Agents version 0.0.19.

Once you've installed this SDK, you can use Sentry AI Agents Insights, a Sentry dashboard that helps you understand what's going on with your AI agents.

Sentry AI Agents monitoring will automatically collect information about agents, tools, prompts, tokens, and models.

Install sentry-sdk from PyPI:

Copied
pip install "sentry-sdk"

If you have the agents package in your dependencies, the OpenAI Agents integration will be enabled automatically when you initialize the Sentry SDK.

Copied
import sentry_sdk

sentry_sdk.init(
    dsn="https://examplePublicKey@o0.ingest.sentry.io/0",
    # Add data like request headers and IP for users, if applicable;
    # see https://docs.sentry.io/platforms/python/data-management/data-collected/ for more info
    send_default_pii=True,
    #  performance
    # Set traces_sample_rate to 1.0 to capture 100%
    # of transactions for tracing.
    traces_sample_rate=1.0,
    #  performance
    #  profiling
    # To collect profiles for all profile sessions,
    # set `profile_session_sample_rate` to 1.0.
    profile_session_sample_rate=1.0,
    # Profiles will be automatically collected while
    # there is an active span.
    profile_lifecycle="trace",
    #  profiling
    #  logs

    # Enable logs to be sent to Sentry
    enable_logs=True,
    #  logs
)

Verify that the integration works by running an AI agent. The resulting data should show up in your AI Agents Insights dashboard.

Copied
import asyncio
import random

import sentry_sdk
import agents
from pydantic import BaseModel  # installed by openai-agents

@agents.function_tool
def random_number(max: int) -> int:
    return random.randint(0, max)

class FinalResult(BaseModel):
    number: int

random_number_agent = agents.Agent(
    name="Random Number Agent",
    instructions="Generate a random number.",
    tools=[random_number, ],
    output_type=FinalResult,
    model="gpt-4o-mini",
)

async def main() -> None:
    sentry_sdk.init(
        dsn="https://examplePublicKey@o0.ingest.sentry.io/0",
        traces_sample_rate=1.0,
        # Add data like LLM and tool inputs/outputs;
        # see https://docs.sentry.io/platforms/python/data-management/data-collected/ for more info
        send_default_pii=True,
    )

    await agents.Runner.run(
        random_number_agent,
        input=f"Generate a random number between 0 and {10}.",
    )

if __name__ == "__main__":
    asyncio.run(main())

It may take a couple of moments for the data to appear in sentry.io.

Data on the following will be collected:

  • AI agents invocations
  • execution of tools
  • number of input and output tokens used
  • LLM models usage

Sentry considers LLM and tool inputs/outputs as PII and doesn't include PII data by default. If you want to include the data, set send_default_pii=True in the sentry_sdk.init() call. To explicitly exclude prompts and outputs despite send_default_pii=True, configure the integration with include_prompts=False as shown in the Options section below.

By adding OpenAIAgentsIntegration to your sentry_sdk.init() call explicitly, you can set options for OpenAIAgentsIntegration to change its behavior:

Copied
import sentry_sdk
from sentry_sdk.integrations.openai_agents import OpenAIAgentsIntegration

sentry_sdk.init(
    # ...
    # Add data like inputs and responses;
    # see https://docs.sentry.io/platforms/python/data-management/data-collected/ for more info
    send_default_pii=True,
    integrations=[
        OpenAIAgentsIntegration(
            include_prompts=False,  # LLM and tool inputs/outputs will be not sent to Sentry, despite send_default_pii=True
        ),
    ],
)

You can pass the following keyword arguments to OpenAIAgentsIntegration():

  • include_prompts:

    Whether LLM and tool inputs and outputs should be sent to Sentry. Sentry considers this data personal identifiable data (PII) by default. If you want to include the data, set send_default_pii=True in the sentry_sdk.init() call. To explicitly exclude prompts and outputs despite send_default_pii=True, configure the integration with include_prompts=False.

    The default is True.

  • OpenAI Agents SDK: 0.0.19+
  • Python: 3.9+
Previous
OpenAI
Next
OpenFeature
Was this helpful?
Help improve this content
Our documentation is open source and available on GitHub. Your contributions are welcome, whether fixing a typo (drat!) or suggesting an update ("yeah, this would be better").