Overview

Complete SDK documentation and REST API reference for Fiddler AI Observability Platform.

🎯 Agentic AI SDKs

SDKs for monitoring, evaluating, and testing LLM applications and AI agents.

Fiddler Evals SDK

Evaluate and test LLM outputs with built-in and custom metrics.

Key Features:

  • Pre-built evaluators (faithfulness, toxicity, coherence, etc.)

  • Custom evaluation functions

  • Experiment tracking and comparison

  • Dataset management for test sets

Use Cases:

  • LLM output quality assessment

  • A/B testing prompts and models

  • Regression testing for LLM changes

  • Custom evaluation metrics

Quick Start:

import fiddler as fdl

# Initialize evaluator
evaluator = fdl.AnswerRelevance()

# Run evaluation
result = evaluator.evaluate(
    question="What is Fiddler?",
    answer="Fiddler is an AI observability platform."
)

View Full Documentation →

Fiddler LangGraph SDK

Monitor LangGraph agents with distributed tracing and observability.

Key Features:

  • Automatic LangGraph instrumentation

  • Distributed tracing for agent workflows

  • Span attributes for nodes and edges

  • Conversation and session tracking

Use Cases:

  • Debugging multi-step agent workflows

  • Performance analysis of agent chains

  • Monitoring production LangGraph applications

  • Understanding agent decision paths

Quick Start:

from fiddler.langchain import LangGraphInstrumentor

# Instrument your LangGraph app
instrumentor = LangGraphInstrumentor()
instrumentor.instrument()

# Your LangGraph code runs normally
# Traces are automatically sent to Fiddler

View Full Documentation →

Fiddler Strands SDK

Monitor Strands Agents with native instrumentation.

Key Features:

  • Strands Agent instrumentation

  • Session and conversation tracking

  • Span attributes for agent actions

  • Integration with Fiddler platform

Use Cases:

  • Monitoring Strands production agents

  • Debugging Strands Agent workflows

  • Tracking agent performance metrics

  • Session-based analysis

Quick Start:

from fiddler.strands import StrandsAgentInstrumentor

# Instrument Strands Agent
instrumentor = StrandsAgentInstrumentor(
    model_id="my-strands-agent"
)
instrumentor.instrument()

View Full Documentation →

🐍 Python Client SDK

Python Client SDK

Official Python SDK for comprehensive ML observability - monitor traditional ML models and LLM applications.

Key Features:

  • Model onboarding and schema definition

  • Production event publishing (batch and streaming)

  • Baseline dataset management

  • Alert configuration

  • Custom metrics and segments

  • Explainability (SHAP, feature importance)

Use Cases:

  • ML model monitoring (drift, performance, data quality)

  • Production data ingestion

  • Creating monitoring dashboards

  • Configuring alerts for model issues

  • Uploading model artifacts for explainability

Quick Start:

import fiddler as fdl

# Connect to Fiddler
client = fdl.FiddlerApi(
    url="https://app.fiddler.ai",
    api_key="fid_..."
)

# Create project and upload model
client.create_project("fraud-detection")
client.upload_model_artifact(
    project_id="fraud-detection",
    model_id="fraud_model_v1",
    model_dir="./model_artifacts"
)

# Publish production events
client.publish_events_batch(
    project="fraud-detection",
    model="fraud_model_v1",
    batch=production_data
)

View Full Documentation →

API Reference Sections:

  • Connection - Client initialization and authentication

  • Constants - Enums and configuration values

  • Entities - Data models (Model, Project, Alert, etc.)

  • Exceptions - Error handling

  • Schemas - Model schema definitions

  • Utils - Helper functions

🌐 REST API

REST API Reference

Complete HTTP API documentation for programmatic access to Fiddler platform.

Available Endpoints:

  • Models - Upload, update, and manage models

  • Events - Publish production data and predictions

  • Baselines - Create and manage baseline datasets

  • Alert Rules - Configure monitoring alerts

  • Custom Metrics - Define custom monitoring metrics

  • Explainability - Request SHAP values and explanations

  • Projects - Manage projects and environments

  • Segments - Define data segments for analysis

Use Cases:

  • Non-Python integrations (Java, Go, JavaScript, etc.)

  • Custom CI/CD pipelines

  • Integration with existing monitoring systems

  • Webhook-based automation

Quick Start (cURL):

# Publish events to Fiddler
curl -X POST https://app.fiddler.ai/api/v1/events \
  -H "Authorization: Bearer fid_..." \
  -H "Content-Type: application/json" \
  -d '{
    "project": "fraud-detection",
    "model": "fraud_model_v1",
    "events": [...]
  }'

View Full REST API Documentation →

API Guides:

Guardrails API Reference

API endpoints for Fiddler Trust Service guardrails.

🚀 Getting Started

Choose Your SDK

Your Use Case
Recommended SDK

Evaluate LLM outputs

Fiddler Evals SDK

Monitor LangGraph agents

Fiddler LangGraph SDK

Monitor Strands Agents

Fiddler Strands SDK

Monitor ML models

Python Client SDK

Non-Python integration

REST API

Custom CI/CD pipelines

REST API

Installation

Python SDKs:

# Evals SDK
pip install fiddler-evals

# LangGraph SDK
pip install fiddler-langgraph

# Strands SDK
pip install fiddler-strands

# Python Client SDK
pip install fiddler-client

REST API: No installation required - use any HTTP client.

📚 Related Documentation

💡 Common Workflows

LLM Evaluation Workflow

  1. Install Fiddler Evals SDK

  2. Create test dataset with Dataset API

  3. Define evaluators (built-in or custom)

  4. Run experiments and analyze results

Agent Monitoring Workflow

  1. Install LangGraph SDK or Strands SDK

  2. Instrument your agent application

  3. Deploy to production

  4. View traces and analytics in Fiddler platform

ML Model Monitoring Workflow

  1. Install Python Client SDK

  2. Define model schema

  3. Upload baseline dataset

  4. Publish production events

  5. Configure alerts

🔐 Authentication

All SDKs and REST API require authentication:

API Key Authentication:

# Python SDKs
client = fdl.FiddlerApi(
    url="https://app.fiddler.ai",
    api_key="fid_..."  # Get from Fiddler Settings > Credentials
)

REST API:

curl -H "Authorization: Bearer fid_..." https://app.fiddler.ai/api/v1/...

Get your API key →

📖 Additional Resources

NextIntroduction

Last updated

Was this helpful?