Agentic AI Overview

Monitor and evaluate your agentic AI applications with Fiddler’s native SDKs and framework integrations. From auto-instrumented LangGraph agents to Strands agent applications, Fiddler provides comprehensive observability for the next generation of AI systems.

​

Why Agentic Observability Matters

Agentic AI systems—autonomous agents that reason, plan, and coordinate—introduce exponential complexity compared to traditional AI applications:

• 26x more monitoring resources required than single-agent systems
• Non-deterministic behavior makes traditional debugging approaches inadequate
• Multi-step workflows require hierarchical tracing across agents, tools, and LLM calls
• Cascading failures demand root cause analysis across distributed agent architectures

Fiddler’s agentic observability provides visibility into every stage of the agent lifecycle: Thought → Action → Execution → Reflection → Alignment.

​

Native SDKs

Fiddler-built and maintained instrumentation libraries for production-grade agentic observability.

​

Fiddler OTel SDK

Core OpenTelemetry instrumentation library for framework-agnostic GenAI observability. The foundation package that all other Fiddler integrations build on. Best for: Custom Python agents with no framework dependency, or any application where you want lightweight, decorator-based instrumentation Key Features:

• @trace decorator for zero-boilerplate function instrumentation (sync and async)
• Typed span wrappers: FiddlerGeneration, FiddlerTool, FiddlerChain
• Context isolation — does not interfere with any existing OpenTelemetry setup
• set_conversation_id() for multi-turn conversation tracking
• JSONL local capture and console tracing for development

Get Started with Fiddler OTel SDK →

​

Fiddler LangChain SDK

Auto-instrumentation for LangChain V1 agents built with langchain.agents.create_agent. Best for: LangChain V1 agents that use the create_agent API Key Features:

• One call to FiddlerLangChainInstrumentor.instrument() auto-traces all agents
• Clean, flat trace hierarchy: agent → LLM calls → tool calls, no noisy Chain wrappers
• Full async support via agent.ainvoke()
• Single-trace multi-agent nesting — sub-agents nest under delegation tool spans automatically
• Retriever-as-tool support

Get Started with Fiddler LangChain SDK →

​

Fiddler LangGraph SDK

Auto-instrument LangGraph applications with OpenTelemetry-based tracing. Best for: LangChain LangGraph agent applications with complex multi-agent workflows Key Features:

• Automatic span creation for agent steps, tool calls, and LLM requests
• Hierarchical tracing across Application → Session → Agent → Span levels
• Zero-configuration setup with one environment variable
• Full context preservation for debugging non-deterministic behavior

Get Started with LangGraph SDK →

​

Strands Agents SDK

Native integration for Strands Agents applications. Best for: Teams building agents with the Strands framework Key Features:

• Purpose-built for Strands agent architecture
• Seamless integration with Strands agent runtime
• Multi-agent coordination tracking
• Platform-agnostic deployment (works on AWS, custom infrastructure, etc.)

Get Started with Strands Agents SDK →

​

Google ADK SDK

Native integration for Google ADK (Agent Development Kit) applications. Best for: Teams building agents with Google’s ADK framework and Gemini models Key Features:

• Two-line setup with FiddlerClient + GoogleADKInstrumentor
• Works with Gemini API keys and Vertex AI authentication
• Automatic capture of agent, LLM, and tool spans
• Session identity propagation across multi-turn conversations
• No monkey-patching — pure OpenTelemetry SpanProcessor approach

Get Started with Google ADK SDK →

​

LiteLLM Integration

Zero-configuration integration for teams using LiteLLM — whether calling LLM providers directly via the SDK or routing traffic through a LiteLLM proxy gateway. Best for: Teams using LiteLLM SDK or proxy who want unified cost tracking and latency monitoring across all providers — with no Fiddler-specific package required Key Features:

• LiteLLM SDK: Enable LiteLLM’s built-in OTEL integration with one line (litellm.callbacks = ["otel"]) and point it at Fiddler — no extra packages needed
• LiteLLM Proxy: Automatic detection of proxy OTel traces — no SDK or code changes needed in calling applications
• Captures prompts, responses, token usage, cost metadata, and latency
• Works with any LLM provider supported by LiteLLM (OpenAI, Anthropic, Bedrock, and more)

Get Started with LiteLLM Integration →

​

AgentGateway Integration

Zero-instrumentation integration for teams using AgentGateway as an LLM proxy — no Fiddler SDK or application code changes required. Currently supports the LLM gateway only; MCP gateway and A2A gateway are not yet supported. Best for: Teams who want observability without touching application code, or who are already routing LLM traffic through AgentGateway for auth, rate limiting, or multi-provider routing Key Features:

• Zero code changes — point your existing OpenAI client at AgentGateway; traces appear automatically
• Captures prompts, responses, token usage, model name, and latency via AgentGateway’s CEL tracing config
• Session grouping via the X-Fiddler-Conversation-Id HTTP header

Get Started with AgentGateway Integration →

​

Claude Code Integration

Zero-instrumentation integration for teams using Claude Code — Anthropic’s CLI coding agent. Captures LLM calls, tool invocations, user prompts, token usage, and permission decisions via Claude Code’s built-in OpenTelemetry tracing. Best for: Teams deploying Claude Code for software engineering and wanting visibility into coding agent sessions, tool usage patterns, and safety compliance Key Features:

• Zero code changes — Claude Code emits OTel traces natively; just set environment variables
• Captures user prompts, LLM call metadata (tokens, latency, model), and tool invocations
• Session correlation via session.id for multi-turn coding session replay
• Permission decision tracking (accept/deny) for tool invocations
• Compatible with Fiddler enrichment rules (FTL Prompt Safety scoring on agent spans)

Get Started with Claude Code Integration →

​

Fiddler Evals SDK

LLM experiments framework with pre-built evaluators and custom eval support. Best for: Offline evaluation of LLM applications and agentic workflows Key Features:

• 14+ pre-built evaluators (faithfulness, toxicity, PII, coherence, etc.)
• Custom evaluator framework for domain-specific metrics
• Batch evaluation for datasets
• Integration with the Fiddler platform for tracking and comparison

Get Started with Evals SDK →

​

Platform SDKs

Core API access for building custom integrations and monitoring workflows.

​

Python Client SDK

Comprehensive Python client for all Fiddler platform capabilities. Best for: Custom integrations, ML model monitoring, programmatic access to Fiddler features Key Features:

• Full API coverage for ML and LLM monitoring
• Dataset uploads, model publishing, event ingestion
• Alert configuration, dashboard management
• Custom metrics and enrichments

Python Client Documentation →

​

REST API

Complete HTTP API for language-agnostic platform access. Best for: Non-Python environments, webhook integrations, custom tooling REST API Reference →

​

Advanced Integrations

​

Kong AI Gateway Integration

Gateway-layer integration for teams using Kong AI Gateway (v3.13+). Fiddler integrates via Kong’s opentelemetry plugin — full LLM observability without adding any SDK to your application code. Best for: Teams already routing LLM traffic through Kong AI Gateway who want zero-instrumentation observability Key Features:

• Zero instrumentation — point your app at Kong instead of the provider; no code changes
• LLM span tracing: token counts, model name, latency, and content (with log_payloads: true)
• Multi-provider support (OpenAI, Anthropic, Cohere, Azure OpenAI, Google Gemini, and more)
• Direct OTLP export to Fiddler over HTTPS with auth headers

Get Started with Kong AI Gateway Integration →

​

S3 Trace Ingestion

Ingest pre-generated OTLP trace files from Amazon S3 into Fiddler without modifying your application. The S3 connector automatically discovers, parses, and forwards trace files to the Fiddler platform. Key Features:

• Automatic file discovery — no manual trigger required
• Supports both base64 and hex-encoded traceId/spanId fields
• IAM role-based authentication (cross-account supported)
• Per-file retry logic with status tracking via API
• Compatible with any OTLP JSON producer

Get Started with S3 Trace Ingestion →

​

OpenTelemetry Integration

Direct OTLP integration for custom agent frameworks and multi-framework environments. Best for: Multi-framework environments, custom agentic frameworks, advanced users requiring full instrumentation control Key Features:

• Vendor-neutral telemetry using OpenTelemetry standards
• Manual span creation for complete control over instrumentation
• Multi-framework support for custom and emerging agent frameworks
• Compatible with existing OpenTelemetry infrastructure
• Attribute mapping to Fiddler semantic conventions

Get Started with OpenTelemetry →

​

Exporting OTel Traces to Fiddler

Client-side export path for shipping pre-existing OpenTelemetry traces to Fiddler from your own storage or pipeline — map span attributes to Fiddler’s schema and POST to the v1/traces endpoint. Best for: Replaying traces from a data warehouse, JSONL files, or a logging pipeline; custom export pipelines; batch backfill of historical trace data Exporting OTel Traces to Fiddler →

​

Framework Support

While Fiddler provides native SDKs for LangGraph and Strands, agentic applications can be monitored regardless of framework:

​

Supported Frameworks & Tools

AI Agent Frameworks:

• LangGraph - Native SDK with auto-instrumentation ✓
• LangChain V1 (create_agent) - Native SDK with auto-instrumentation ✓
• Custom Python agents - Fiddler OTel SDK with @trace decorator ✓
• Other agentic frameworks - Fiddler OTel SDK is the recommended path for any custom or unsupported framework

LLM Provider SDKs:

• OpenAI SDK - Track via Python Client or custom instrumentation
• Anthropic SDK - Monitor Claude API calls via Python Client
• Strands Agents - Native Strands Agents SDK ✓
• LiteLLM SDK / Proxy - Zero-configuration OTel integration ✓
• AgentGateway (LLM gateway only) - Zero-instrumentation proxy integration ✓ — MCP/A2A gateways not yet supported
• Claude Code - Zero-instrumentation coding agent integration ✓ — beta OTel tracing, limited content capture

Gateways & Proxies:

• Kong AI Gateway (v3.13+) - Zero-instrumentation gateway integration ✓

Observability Standards:

• OpenTelemetry - Full OTLP support for custom instrumentation
• Custom Tracing - Python Client API for framework-agnostic monitoring

​

Integration Selector

Not sure which SDK to use? Here’s a quick decision guide:

​

Getting Started

​

Quick Start Paths

1. Custom Python Agents (Fiddler OTel SDK)
   pip install fiddler-otel
   
   from fiddler_otel import FiddlerClient, trace
   
   client = FiddlerClient(api_key="...", application_id="...", url="...")
   
   @trace(as_type="generation")
   def call_llm(prompt: str) -> str:
       ...
   
   Full Fiddler OTel SDK Guide →
2. LangChain V1 Applications
   pip install fiddler-langchain
   
   from fiddler_otel import FiddlerClient
   from fiddler_langchain import FiddlerLangChainInstrumentor
   import langchain.agents
   
   client = FiddlerClient(api_key="...", application_id="...", url="...")
   FiddlerLangChainInstrumentor(client=client).instrument()
   # All create_agent() calls are now traced automatically
   
   Full LangChain SDK Guide →
3. LangGraph Applications
   pip install fiddler-langgraph
   
   from fiddler_langgraph import FiddlerClient, LangGraphInstrumentor
   
   client = FiddlerClient(api_key="...", application_id="...", url="...")
   LangGraphInstrumentor(client).instrument()
   
   Full LangGraph Quick Start →
4. Strands Agents
   pip install fiddler-strands
   # Configure for your Strands Agent
   
   Full Strands Agents SDK Quick Start →
5. LLM Experiments
   pip install fiddler-evals
   # Run experiments on your dataset
   
   Full Evals Quick Start →
6. LiteLLM SDK
   export OTEL_EXPORTER_OTLP_ENDPOINT="https://your-fiddler-instance.com"
   export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer <token>,fiddler-application-id=<app-uuid>"
   export OTEL_RESOURCE_ATTRIBUTES="application.id=<app-uuid>"
   
   import litellm
   litellm.callbacks = ["otel"]  # Traces flow to Fiddler automatically
   
   Full LiteLLM SDK Quick Start →
7. LiteLLM Proxy
   export OTEL_EXPORTER_OTLP_ENDPOINT="https://your-fiddler-instance.com"
   export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer <token>,fiddler-application-id=<app-uuid>"
   export OTEL_RESOURCE_ATTRIBUTES="application.id=<app-uuid>"
   litellm --config config.yaml  # Traces flow to Fiddler automatically
   
   Full LiteLLM Proxy Quick Start →
8. AgentGateway (LLM proxy)
   brew install agentgateway/tap/agentgateway
   export OPENAI_API_KEY="sk-..."
   export FIDDLER_APP_ID="your-application-uuid"
   agentgateway -f agentgateway_config.yaml
   
   import os
   import uuid
   from openai import OpenAI
   
   client = OpenAI(base_url=os.getenv("AGENTGATEWAY_URL", "http://localhost:4000/v1"))
   response = client.chat.completions.create(
       model="gpt-4o-mini",
       messages=[{"role": "user", "content": "Hello"}],
       extra_headers={"X-Fiddler-Conversation-Id": str(uuid.uuid4())},
   )
   
   Full AgentGateway Integration Guide →
9. Raw OpenTelemetry (Advanced)
   pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
   # Configure OTLP endpoint and instrument your agent
   
   Full OpenTelemetry Quick Start →

​

What’s Next?

• Span and Resource Attributes - Understand required fields, value typing, custom attributes, and how they flow into metrics and alerts
• Agentic Observability Concepts - Understand the agent lifecycle and monitoring approach
• Agentic Observability Quick Start - Complete setup guide
• Centor Models Overview - Learn about the evaluation platform powering Fiddler