OpenTelemetry Quick Start

Monitor custom AI agents and multi-framework agentic applications with Fiddler using OpenTelemetry’s native instrumentation.

What You’ll Learn

In this guide, you’ll learn how to:

Set up OpenTelemetry tracing for custom agent frameworks
Configure Fiddler as your OTLP endpoint with proper authentication
Map agent attributes to Fiddler’s semantic conventions
Create instrumented LLM and tool spans with required attributes
Verify traces in the Fiddler dashboard

Time to complete: ~10-15 minutes

When to Use Raw OpenTelemetryThis guide is for advanced scenarios requiring full manual OTLP control:

Multi-framework environments requiring unified observability across different agent frameworks
Existing OpenTelemetry infrastructure where you want to route Fiddler traces through your own OTel pipeline
Advanced control over trace sampling, batch processing, and attribute mapping

When to Use Fiddler SDKs Instead (recommended for most users):

Custom Python agents → Use Fiddler OTel SDK (@trace decorator, typed span wrappers, zero OTLP setup)
LangChain V1 agents → Use Fiddler LangChain SDK (one instrument() call)
LangGraph agents/workflows → Use Fiddler LangGraph SDK (auto-instrumentation)
Strands Agents → Use Strands Agents SDK (native integration)

SDKs provide automatic instrumentation and require significantly less code. Use raw OpenTelemetry only when SDKs don’t fit your use case.

Prerequisites

Before you begin, ensure you have:

Fiddler Account: An active account with a GenAI application created
Python 3.10+
OpenTelemetry Packages:
- pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
LLM Provider (for examples): OpenAI API key or similar
Fiddler Access Token: Get your token from Settings > Credentials

For a complete working example with advanced patterns, download the Advanced OpenTelemetry Notebook from GitHub or open it in Google Colab.

Create Fiddler Application

Log in to your Fiddler instance and navigate to GenAI Applications
Select “Add Application” to create a new application
Copy your Application ID - This must be a valid UUID4 format (e.g., 550e8400-e29b-41d4-a716-446655440000)
Get your Access Token from Settings > Credentials

Important: Keep your Application ID and Access Token secure. You’ll need both for the next steps.

Configure Environment Variables

Set up your environment to connect to Fiddler’s OTLP endpoint:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://your-instance.fiddler.ai"
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer <YOUR_ACCESS_TOKEN>,fiddler-application-id=<YOUR_APPLICATION_UUID>"
export OTEL_RESOURCE_ATTRIBUTES="application.id=<YOUR_APPLICATION_UUID>"

Environment Variable Breakdown:

Variable	Description	Example
`OTEL_EXPORTER_OTLP_ENDPOINT`	Your Fiddler instance URL	`https://org.fiddler.ai`
`OTEL_EXPORTER_OTLP_HEADERS`	Authentication and app ID headers	`authorization=Bearer sk-...,fiddler-application-id=550e8400...`
`OTEL_RESOURCE_ATTRIBUTES`	Resource-level application identifier	`application.id=550e8400-e29b-41d4-a716-446655440000`

Python Configuration (alternative to environment variables):

import os

os.environ['OTEL_EXPORTER_OTLP_ENDPOINT'] = 'https://your-instance.fiddler.ai'
os.environ['OTEL_EXPORTER_OTLP_HEADERS'] = 'authorization=Bearer <TOKEN>,fiddler-application-id=<UUID>'
os.environ['OTEL_RESOURCE_ATTRIBUTES'] = 'application.id=<UUID>'

Tip: Store credentials in a .env file and use python-dotenv for local development:

from dotenv import load_dotenv
load_dotenv()  # Loads variables from .env file

Initialize OpenTelemetry

Set up OpenTelemetry with Fiddler’s OTLP exporter:

import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

# Initialize tracer provider
trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

# Configure OTLP exporter for Fiddler
otlp_endpoint = os.getenv('OTEL_EXPORTER_OTLP_ENDPOINT') + '/v1/traces'
otlp_exporter = OTLPSpanExporter(endpoint=otlp_endpoint)

# Add batch span processor
otlp_processor = BatchSpanProcessor(otlp_exporter)
trace.get_tracer_provider().add_span_processor(otlp_processor)

print(f"✅ OpenTelemetry configured with endpoint: {otlp_endpoint}")

What This Does:

TracerProvider: Manages trace generation
OTLPSpanExporter: Exports spans to Fiddler via OTLP protocol
BatchSpanProcessor: Batches spans for efficient network transmission

Local Debugging: Add a console exporter to see traces locally while developing:

from opentelemetry.sdk.trace.export import ConsoleSpanExporter

console_exporter = ConsoleSpanExporter()
console_processor = BatchSpanProcessor(console_exporter)
trace.get_tracer_provider().add_span_processor(console_processor)

Instrument Your Agent

Create instrumented spans for your agent’s operations. Fiddler requires specific attributes to properly categorize and visualize your agent traces.Required Fiddler AttributesResource Level (set via environment variable):

application.id - UUID4 of your Fiddler application

Span Level (required for each span):

fiddler.span.type - Type of operation: "chain", "tool", "llm", or "agent"

Recommended: Decorate functions with start_as_current_spanOpenTelemetry’s tracer.start_as_current_span() works both as a context manager (with ... as span:) and as a function decorator. When a span should wrap an entire function — which is the common case for tools, LLM calls, and agent entry points — the decorator form is far less verbose: it starts the span on entry and ends it on return automatically. Inside the function, call trace.get_current_span() to set Fiddler attributes on the active span.Reach for the with block only when you need a span narrower than a whole function (for example, instrumenting a single section of a larger function). Both forms are pure OpenTelemetry and require no Fiddler SDK.Example: Simplified Travel Agent

import json
from openai import OpenAI
from opentelemetry import trace

client = OpenAI()

# Reuse the tracer configured in the "Initialize OpenTelemetry" step above
tracer = trace.get_tracer(__name__)

AGENT_NAME = "travel_agent"
AGENT_ID = "travel_agent_v1"

# Define tools
@tracer.start_as_current_span("book_hotel")
def book_hotel_tool(city: str, date: str):
    """Book a hotel in the specified city."""
    span = trace.get_current_span()
    span.set_attribute("fiddler.span.type", "tool")
    span.set_attribute("gen_ai.agent.name", AGENT_NAME)  # optional
    span.set_attribute("gen_ai.agent.id", AGENT_ID)      # optional

    # Tool-specific attributes
    span.set_attribute("gen_ai.tool.name", "book_hotel")
    tool_input = {"city": city, "date": date}
    span.set_attribute("gen_ai.tool.input", json.dumps(tool_input))

    # Execute tool
    result = {"status": "confirmed", "hotel": f"Grand Hotel {city}", "confirmation": "HTL123"}
    span.set_attribute("gen_ai.tool.output", json.dumps(result))

    return result

@tracer.start_as_current_span("book_flight")
def book_flight_tool(source: str, destination: str, date: str):
    """Book a flight between two cities."""
    span = trace.get_current_span()
    span.set_attribute("fiddler.span.type", "tool")
    span.set_attribute("gen_ai.agent.name", AGENT_NAME)  # optional
    span.set_attribute("gen_ai.agent.id", AGENT_ID)      # optional

    # Tool-specific attributes
    span.set_attribute("gen_ai.tool.name", "book_flight")
    tool_input = {"source": source, "destination": destination, "date": date}
    span.set_attribute("gen_ai.tool.input", json.dumps(tool_input))

    # Execute tool
    result = {"status": "confirmed", "flight": "FL456", "departure": "10:00 AM"}
    span.set_attribute("gen_ai.tool.output", json.dumps(result))

    return result

@tracer.start_as_current_span("llm_call")
def call_llm(user_request: str):
    """Call the LLM to interpret the request and choose tools."""
    span = trace.get_current_span()
    span.set_attribute("fiddler.span.type", "llm")
    span.set_attribute("gen_ai.agent.name", AGENT_NAME)  # optional
    span.set_attribute("gen_ai.agent.id", AGENT_ID)      # optional

    # LLM-specific attributes
    span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
    span.set_attribute("gen_ai.system", "openai")
    span.set_attribute("gen_ai.llm.input.user", user_request)
    span.set_attribute(
        "gen_ai.llm.input.system",
        "You are a travel agent. Parse user requests and call appropriate tools."
    )

    # Call OpenAI
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a travel agent. Parse user requests and call appropriate tools."},
            {"role": "user", "content": user_request}
        ],
        tools=[
            {
                "type": "function",
                "function": {
                    "name": "book_hotel",
                    "description": "Book a hotel in a city for a specific date",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "city": {"type": "string"},
                            "date": {"type": "string"}
                        },
                        "required": ["city", "date"]
                    }
                }
            },
            {
                "type": "function",
                "function": {
                    "name": "book_flight",
                    "description": "Book a flight between two cities",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "source": {"type": "string"},
                            "destination": {"type": "string"},
                            "date": {"type": "string"}
                        },
                        "required": ["source", "destination", "date"]
                    }
                }
            }
        ]
    )

    # Set token usage
    span.set_attribute("gen_ai.usage.input_tokens", response.usage.prompt_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", response.usage.completion_tokens)
    span.set_attribute("gen_ai.usage.total_tokens", response.usage.total_tokens)

    # Process tool calls
    tool_results = []
    if response.choices[0].message.tool_calls:
        for tool_call in response.choices[0].message.tool_calls:
            tool_name = tool_call.function.name
            tool_args = json.loads(tool_call.function.arguments)

            if tool_name == "book_hotel":
                tool_results.append(book_hotel_tool(**tool_args))
            elif tool_name == "book_flight":
                tool_results.append(book_flight_tool(**tool_args))

    span.set_attribute("gen_ai.llm.output",
                       f"Called tools and received: {tool_results}")
    return tool_results

# Agent implementation
@tracer.start_as_current_span("travel_agent_chain")
def travel_agent(user_request: str):
    """Main travel agent function."""
    span = trace.get_current_span()
    span.set_attribute("fiddler.span.type", "chain")
    span.set_attribute("gen_ai.agent.name", AGENT_NAME)  # optional
    span.set_attribute("gen_ai.agent.id", AGENT_ID)      # optional

    # call_llm runs as a child span because it is invoked within this span
    tool_results = call_llm(user_request)
    return {"status": "success", "bookings": tool_results}

# Run the agent
result = travel_agent("Book a hotel in Paris for tomorrow and a flight from London to Paris")
print(f"Agent result: {result}")

Key Implementation Details:

Decorator form: @tracer.start_as_current_span("name") wraps the whole function; retrieve the span inside with trace.get_current_span()
Chain Spans: Use fiddler.span.type = "chain" for high-level workflows
LLM Spans: Include model, system prompt, user input, output, and token usage
Tool Spans: Include tool name, input JSON, and output JSON
Nested Spans: A decorated function called from within another decorated function automatically becomes a child span, building the trace hierarchy

Simpler alternative: Install fiddler-otel to skip all the manual OTLP configuration and attribute boilerplate. The SDK’s start_as_current_span() method handles span type enforcement and attribute propagation automatically — no need to set fiddler.span.type, gen_ai.agent.name, or gen_ai.agent.id on every span manually. Typed span wrappers like FiddlerGeneration and FiddlerTool provide helper methods such as set_model() and set_tool_name() instead of raw set_attribute() calls. See Manual Instrumentation in the integration guide.

Verify Monitoring

Run your instrumented code using the example above
Wait 1-2 minutes for traces to appear in Fiddler
Navigate to GenAI Applications in your Fiddler instance
Verify application status changes to Active
View traces to see your agent spans, hierarchy, and attributes

Success Criteria:✅ Application shows as Active in GenAI Applications ✅ Traces appear in the Explorer ✅ Span hierarchy shows chain → LLM → tools relationship ✅ fiddler.span.type is set on every span ✅ LLM token usage is tracked ✅ Tool inputs and outputs are captured

Verification Tip: Check the trace timeline view to see the execution flow of your agent, including which tools were called and how long each operation took.

Attribute Reference

Required Attributes

Resource Level:

Attribute	Type	Description	Example
`application.id`	string	UUID4 of your Fiddler application	`"550e8400-e29b-41d4-a716-446655440000"`

Span Level:

Attribute	Type	Description	Valid Values
`fiddler.span.type`	string	Type of operation	`"chain"`, `"tool"`, `"llm"`, `"agent"`

Optional Attributes

Agent Identification:

Attribute	Type	Description	Example
`gen_ai.agent.name`	string	Name of the AI agent	`"travel_agent"`
`gen_ai.agent.id`	string	Unique identifier for the agent	`"travel_agent_v1"`

Set agent attributes on every span. gen_ai.agent.name and gen_ai.agent.id are optional, but if you include them, set both on every span within the trace. Fiddler uses these attributes to attribute spans to the correct agent — spans missing these fields will be unattributed even if other spans in the same trace carry them.

Conversation Tracking:

Attribute	Type	Description	Example
`gen_ai.conversation.id`	string	Session/conversation identifier	`"conv_123"`

LLM Span Attributes:

Attribute	Type	Description	Example
`gen_ai.request.model`	string	Model name	`"gpt-4o-mini"`, `"claude-3-opus"`
`gen_ai.system`	string	LLM provider	`"openai"`, `"anthropic"`
`gen_ai.llm.input.system`	string	System prompt	`"You are a helpful assistant"`
`gen_ai.llm.input.user`	string	User input	`"What's the weather?"`
`gen_ai.llm.output`	string	LLM response	`"The weather is sunny"`
`gen_ai.usage.input_tokens`	int	Input tokens used	`42`
`gen_ai.usage.output_tokens`	int	Output tokens used	`28`
`gen_ai.usage.total_tokens`	int	Total tokens used	`70`
`gen_ai.input.messages`	string (JSON array)	Chat history provided to model	`[{"role": "user", "content": "Hello"}]`
`gen_ai.output.messages`	string (JSON array)	Messages returned by model	`[{"role": "assistant", "content": "Hi"}]`

Tool Span Attributes:

Attribute	Type	Description	Example
`gen_ai.tool.name`	string	Tool/function name	`"search_database"`
`gen_ai.tool.input`	string	Tool input (JSON)	`"{"query": "hotels"}"`
`gen_ai.tool.output`	string	Tool output (JSON)	`"{"results": [...]}"`

Custom User-Defined Attributes:

Pattern	Level	Example
`fiddler.session.user.{key}`	Trace (all spans)	`fiddler.session.user.user_id = "usr_123"`
`fiddler.span.user.{key}`	Span (individual)	`fiddler.span.user.department = "sales"`

Troubleshooting

Common Issues

Problem: Application not showing as “Active” Solutions:

Verify environment variables are set correctly
Check that OTEL_EXPORTER_OTLP_ENDPOINT includes your Fiddler instance URL
Ensure OTEL_EXPORTER_OTLP_HEADERS contains valid authorization token and application ID
Add console exporter to verify spans are being generated locally
Check network connectivity: curl -I https://your-instance.fiddler.ai

Problem: ModuleNotFoundError for OpenTelemetry packages Solutions:

# Install all required packages
pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

# Verify installation
pip list | grep opentelemetry

Problem: Spans not appearing in Fiddler Solutions:

Verify required attributes are set:

# Every span MUST have this
span.set_attribute("fiddler.span.type", "llm")  # or "tool", "chain", "agent"
# Optional but recommended
span.set_attribute("gen_ai.agent.name", "your_agent")
span.set_attribute("gen_ai.agent.id", "agent_id")

Check resource attributes:

# Verify application.id is set
print(os.getenv('OTEL_RESOURCE_ATTRIBUTES'))

Enable console exporter for debugging:

from opentelemetry.sdk.trace.export import ConsoleSpanExporter
console_exporter = ConsoleSpanExporter()
console_processor = BatchSpanProcessor(console_exporter)
trace.get_tracer_provider().add_span_processor(console_processor)

Problem: Authentication errors (401 Unauthorized) Solutions:

Regenerate your access token from Fiddler Settings > Credentials
Verify header format: authorization=Bearer <token>,fiddler-application-id=<uuid>
Ensure no extra spaces in header values
Check token hasn’t expired

Problem: Invalid Application ID error Solutions:

Copy Application ID directly from Fiddler UI
Verify UUID4 format: 550e8400-e29b-41d4-a716-446655440000
Ensure no extra quotes or whitespace

Configuration Options

Basic Configuration

import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

# Set environment variables
os.environ['OTEL_EXPORTER_OTLP_ENDPOINT'] = 'https://your-instance.fiddler.ai'
os.environ['OTEL_EXPORTER_OTLP_HEADERS'] = 'authorization=Bearer <TOKEN>,fiddler-application-id=<UUID>'
os.environ['OTEL_RESOURCE_ATTRIBUTES'] = 'application.id=<UUID>'

# Initialize tracing
trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

# Configure OTLP exporter
otlp_endpoint = os.getenv('OTEL_EXPORTER_OTLP_ENDPOINT') + '/v1/traces'
otlp_exporter = OTLPSpanExporter(endpoint=otlp_endpoint)
otlp_processor = BatchSpanProcessor(otlp_exporter)
trace.get_tracer_provider().add_span_processor(otlp_processor)

Advanced Configuration

High-Volume Applications (Batch Processing Tuning):

from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Customize batch processor settings
custom_processor = BatchSpanProcessor(
    otlp_exporter,
    max_queue_size=500,           # Default: 2048
    schedule_delay_millis=500,    # Default: 5000
    max_export_batch_size=50,     # Default: 512
    export_timeout_millis=10000   # Default: 30000
)

trace.get_tracer_provider().add_span_processor(custom_processor)

Environment Variable Configuration:

# Batch processor environment variables
export OTEL_BSP_MAX_QUEUE_SIZE=500
export OTEL_BSP_SCHEDULE_DELAY_MILLIS=500
export OTEL_BSP_MAX_EXPORT_BATCH_SIZE=50
export OTEL_BSP_EXPORT_TIMEOUT=10000

Sampling for Production (Reduce Volume):

from opentelemetry.sdk.trace import sampling

# Sample 10% of traces
sampler = sampling.TraceIdRatioBased(0.1)

# Create provider with sampler
provider = TracerProvider(sampler=sampler)
trace.set_tracer_provider(provider)

Compression (Reduce Network Usage):

from opentelemetry.exporter.otlp.proto.http.trace_exporter import Compression

# Enable gzip compression
otlp_exporter = OTLPSpanExporter(
    endpoint=otlp_endpoint,
    compression=Compression.Gzip
)

Using FiddlerClient Alternative (Simplified Setup):

Deprecation notice: Importing FiddlerClient and other core symbols (trace, get_current_span, FiddlerSpan, FiddlerGeneration, FiddlerChain, FiddlerTool, get_client, set_conversation_id) directly from fiddler_langgraph is deprecated and will be removed in a future release. Import from fiddler_otel instead: from fiddler_otel import FiddlerClient.

If you have fiddler-otel installed, you can use FiddlerClient for simplified setup — it handles OTLP configuration automatically. There are two levels of abstraction:Option 1: Get a pre-configured tracer (use raw OTel spans, but skip OTLP configuration):

from fiddler_otel import FiddlerClient

client = FiddlerClient(
    application_id='<FIDDLER_APPLICATION_ID>',
    api_key='<FIDDLER_API_TOKEN>',
    url='<FIDDLER_URL>'
)

tracer = client.get_tracer()

with tracer.start_as_current_span("my_operation") as span:
    span.set_attribute("fiddler.span.type", "chain")
    # ... rest of your code

Option 2: Use SDK span wrappers (typed helper methods, automatic attribute propagation):

from fiddler_otel import FiddlerClient

client = FiddlerClient(
    application_id='<FIDDLER_APPLICATION_ID>',
    api_key='<FIDDLER_API_TOKEN>',
    url='<FIDDLER_URL>'
)

# Span type and agent attributes are set automatically
with client.start_as_current_span("llm_call", as_type="generation") as gen:
    gen.set_model("gpt-4o")
    gen.set_user_prompt(user_input)
    response = call_llm(user_input)
    gen.set_completion(response.content)
    gen.set_usage(response.usage.prompt_tokens, response.usage.completion_tokens)

Both approaches handle OTLP configuration automatically. For graceful exit (e.g. servers or short scripts), call client.shutdown() (or await client.ashutdown() in async) so buffered spans are sent before the process exits. See Manual Instrumentation for complete span wrapper documentation.

Next Steps

Now that you have OpenTelemetry integration working:

Advanced Patterns: Download the Advanced OpenTelemetry Notebook for:
- Multi-agent configurations
- Conversation tracking across sessions
- Custom user-defined attributes
- Production-ready error handling
- Comprehensive debugging techniques
Consider SDKs for Common Frameworks:
- Fiddler OTel SDK - @trace decorator for custom Python agents (no raw OTel boilerplate)
- Fiddler LangChain SDK - Auto-instrumentation for LangChain V1 agents
- Fiddler LangGraph SDK - Auto-instrumentation for LangGraph/LangChain
- Strands Agents SDK - Native Strands agent integration
Explore Fiddler Capabilities:
Production Deployment:
- Review sampling strategies for cost optimization
- Implement error handling and retry logic
- Set up monitoring alerts
- Configure custom attributes for your business context

Overview

Platform

Agentic AI Monitoring

LLM Monitoring

ML Monitoring

Experiments

Guardrails

Cookbooks

Tutorials

Client Library Reference

What You’ll Learn

Prerequisites

Attribute Reference

Required Attributes

Optional Attributes

Troubleshooting

Common Issues

Configuration Options

Basic Configuration

Advanced Configuration

Next Steps

​What You’ll Learn

​Prerequisites

​Attribute Reference

​Required Attributes

​Optional Attributes

​Troubleshooting

​Common Issues

​Configuration Options

​Basic Configuration

​Advanced Configuration

​Next Steps

What You’ll Learn

Prerequisites

Attribute Reference

Required Attributes

Optional Attributes

Troubleshooting

Common Issues

Configuration Options

Basic Configuration

Advanced Configuration

Next Steps