# Fiddler Documentation: Documentation

> Unified observability for trustworthy AI across traditional ML, LLM applications, autonomous agents, and self-hosted deployments.

## Documentation

- [Introduction to Fiddler](https://docs.fiddler.ai/index.md): The only platform delivering enterprise-grade visibility, context, and control across traditional ML models, LLM applications, and autonomous multi-agent systems.

### Getting Started

- [Onboard Your GenAI Application](https://docs.fiddler.ai/getting-started/genai-application-onboarding.md): Set up your first GenAI project and application in Fiddler. Learn how to create projects, configure applications, and start monitoring your GenAI interactions.
- [Agentic Observability](https://docs.fiddler.ai/getting-started/agentic-monitoring.md): Comprehensive monitoring, tracing, and analysis of AI agent systems that provide hierarchical visibility into agent reasoning, coordination, and decision-making across multi-agent applications
- [Experiments](https://docs.fiddler.ai/getting-started/experiments.md): Systematically evaluate and compare your LLM and agentic applications with Fiddler Experiments: built-in and custom evaluators, golden datasets, and side-by-side experiment comparison.
- [Guardrails](https://docs.fiddler.ai/getting-started/guardrails.md): Fiddler Guardrails protects GenAI and agentic applications against hallucinations, safety risks, and jailbreaks in real time — available via the Fiddler API.
- [LLM Monitoring](https://docs.fiddler.ai/getting-started/llm-monitoring.md): Monitor LLM applications in production with Fiddler. Track quality, safety, and performance enrichments, detect problematic responses, and diagnose issues before they reach users.
- [ML Observability](https://docs.fiddler.ai/getting-started/ml-observability.md): Monitor traditional ML models in production with Fiddler. Track performance, detect data drift, run root cause analysis, and ensure model fairness at scale.
- [AWS SageMaker Partner AI App](https://docs.fiddler.ai/getting-started/aws-sagemaker-partner-ai-app.md): Get started with Fiddler's Partner AI App on AWS SageMaker. Monitor, explain, and analyze your ML models and GenAI apps in your own AWS environment.
- [Evaluators & Metrics Guide](https://docs.fiddler.ai/getting-started/evaluators-metrics-guide.md): Interactive guide for selecting the right Fiddler evaluators and metrics for your use case. Filter by observability type, use case, and rating to find what to deploy.
- [Use Fiddler with AI Agents](https://docs.fiddler.ai/getting-started/use-fiddler-with-ai-agents.md): Install Fiddler's published agent skills and connect the documentation MCP server so your AI coding agent can onboard models and set up monitoring in Fiddler.

### Concepts

- [RAG Health Diagnostics](https://docs.fiddler.ai/concepts/rag-health-diagnostics.md): Understand how RAG Health Metrics diagnose Retrieval-Augmented Generation pipeline failures using Answer Relevance, Context Relevance, and RAG Faithfulness evaluators.
- [Semantic Mappings](https://docs.fiddler.ai/concepts/semantic-mappings.md): How Fiddler maps raw OTel attribute keys to canonical semantic concepts for cross-framework analytics, alerts, and dashboards

### Evaluate & Test

- [Overview](https://docs.fiddler.ai/evaluate-and-test/overview.md): Hands-on quick start guides for evaluating LLM applications, testing with custom LLM-as-a-Judge metrics, and comparing model outputs using Fiddler Experiments.
- [Evaluator Rules](https://docs.fiddler.ai/evaluate-and-test/evaluator-rules.md): Configure automated evaluations for your GenAI application spans using Evaluator Rules. Learn to map evaluators to span data, define application rules, and manage backfill configuration.
- [Evaluator Downsampling Quick Start](https://docs.fiddler.ai/evaluate-and-test/evaluator-downsampling-quick-start.md): Get started with evaluator downsampling: score only a fraction of matching traces to cut LLM-as-a-Judge cost, configured per rule from the UI or the REST API. Includes an end-to-end curl walkthrough.
- [Configure Evaluator Downsampling](https://docs.fiddler.ai/evaluate-and-test/configure-evaluator-downsampling.md): Reduce LLM-as-a-Judge evaluation cost at scale with evaluator downsampling. Learn when to use it, how the sampling rate relates to the enabled toggle, and how to configure it from the REST API and the UI.
- [Evals SDK Quick Start](https://docs.fiddler.ai/evaluate-and-test/evals-sdk-quick-start.md): Learn how to evaluate Large Language Model (LLM) applications, RAG systems, and AI agents using the Fiddler Evals SDK with built-in and custom evaluators.
- [Golden Datasets](https://docs.fiddler.ai/evaluate-and-test/golden-datasets.md): Build a golden dataset from real production traffic by promoting spans into a Fiddler Experiments dataset, then replay it against every change to catch regressions.
- [Capture Traces During Experiments](https://docs.fiddler.ai/evaluate-and-test/experiment-trace-capture.md): Link the OpenTelemetry traces your task emits to the experiment item that produced them, and score evaluators on the captured spans.
- [Prompt Specs Quick Start](https://docs.fiddler.ai/evaluate-and-test/prompt-specs-quick-start.md): Get started with Fiddler's LLM-as-a-Judge evaluation using Prompt Specs in minutes. Learn to create custom evaluations, test them, and deploy to production monitoring.
- [Compare LLM Outputs](https://docs.fiddler.ai/evaluate-and-test/llm-evaluation-example.md): Learn how to systematically compare outputs from different LLM models (GPT-3.5, Claude, etc.) using Fiddler's pre-production evaluation environment to make data-driven model selection decisions.

### Protect & Guardrails

- [Overview](https://docs.fiddler.ai/protection/index.md): Ensure AI safety and compliance with guardrails and monitoring
- [Guardrails](https://docs.fiddler.ai/protection/guardrails.md): Fiddler Guardrails is a powerful solution designed to serve as the first-line of defense to protect enterprises from costly GenAI and LLM risks in real-time environments.
- [Guardrails Quick Start](https://docs.fiddler.ai/protection/guardrails-quick-start.md): Set up access to Fiddler Guardrails in your Fiddler environment and make your first API call to protect your LLM applications.
- [Guardrails FAQ](https://docs.fiddler.ai/protection/guardrails-faq.md): Find answers to common questions about Fiddler Guardrails, including setup, implementation, and general information for protecting your LLM applications.
- [LiteLLM Guardrails](https://docs.fiddler.ai/protection/litellm-guardrails.md): Use Fiddler as a guardrail provider for the LiteLLM proxy gateway — blocking and redacting PII and secrets in real time before requests reach your LLM.
- [Kong AI Gateway Guardrails](https://docs.fiddler.ai/protection/kong-guardrails.md): Use Fiddler as a guardrail provider for the Kong AI Gateway — blocking PII and secrets in real time before requests reach your LLM.
- [AgentGateway Guardrails](https://docs.fiddler.ai/protection/agentgateway-guardrails.md): Use Fiddler as a guardrail provider for AgentGateway — redacting PII and secrets in real time before requests reach your LLM.

### Monitoring

- [Overview](https://docs.fiddler.ai/observability/monitoring.md): Monitor production models in real-time with comprehensive observability
- [Agentic Observability](https://docs.fiddler.ai/observability/agentic/index.md): Monitor AI agents and multi-step workflows with specialized dashboards, metrics, and trace visualization
- [Explorer](https://docs.fiddler.ai/observability/agentic/trace-explorer.md): Explore, filter, and search every span ingested into your GenAI application with the Explorer DataGrid.
- [Annotations](https://docs.fiddler.ai/observability/agentic/annotations.md): Add human evaluation scores to individual spans in the Explorer to review and assess LLM application outputs alongside automated evaluators.
- [Custom Metrics for Agentic Applications](https://docs.fiddler.ai/observability/agentic/custom-metrics.md): Define custom metrics for your agentic and GenAI applications using FQL and span attributes to track business KPIs, quality scores, and operational signals beyond built-in metrics.
- [Fairness](https://docs.fiddler.ai/observability/fairness.md): Explore our walkthrough of ML model fairness and bias. Review the sample calculations you can customize to your data and use with Fiddler's custom metrics.

#### LLM Monitoring

- [LLM Monitoring](https://docs.fiddler.ai/observability/llm/index.md): Explore our guide to LLM application monitoring. Learn how Fiddler generates enrichments using trust and safety metrics for alerting, analysis, and debugging.
- [LLM-Based Metrics](https://docs.fiddler.ai/observability/llm/llm-based-metrics.md): Explore our guide on LLM-specific metrics useful for evaluating AI-generated content for use cases like chatbots, writing assistants, or content creation tools.
- [Embedding Visualizations](https://docs.fiddler.ai/observability/llm/embedding-visualization-with-umap.md): Explore our guide on embedding visualization to enhance LLM monitoring. Discover UMAP techniques, analyze high-dimensional data, and uncover patterns with ease.
- [Selecting Enrichments](https://docs.fiddler.ai/observability/llm/selecting-enrichments.md): Learn about Fiddler’s enrichments and monitor key aspects of LLM applications. Discover the different factors to analyze for your specific use case.
- [Enrichments](https://docs.fiddler.ai/observability/llm/enrichments.md): Explore our guide on how Fiddler can enrich your LLM application's data to help analyze and evaluate application behavior and performance.
- [LLM Evaluation Prompt Specs](https://docs.fiddler.ai/observability/llm/llm-evaluation-prompt-specs.md): Prompt specs is a framework Fiddler provides for leveraging a general-purpose LLM to quickly create custom scoring functions without the need to manually tune an evaluation prompt.

#### Monitoring Platform

- [Monitoring Platform](https://docs.fiddler.ai/observability/platform/index.md): Dive into our guide to optimizing ML models and LLM applications with Fiddler’s monitoring tools. Learn key metrics to track data drift, performance, and more.
- [Alerts](https://docs.fiddler.ai/observability/platform/alerts-platform.md): Discover how to enhance monitoring with Alerts. Learn about alert types and how to set up and view them using the alerts tab in the navigation bar.
- [Template-Based Alerts](https://docs.fiddler.ai/observability/platform/template-based-alerts.md): Learn how to create and deploy template-based alerts in Fiddler using Google Sheets and YAML configurations for efficient model monitoring.
- [Class Imbalanced Data](https://docs.fiddler.ai/observability/platform/class-imbalanced-data.md): Explore how Fiddler uses weighting to help improve drift detection when class distribution is highly imbalanced.
- [Custom Metrics](https://docs.fiddler.ai/observability/platform/custom-metrics.md): Dive into our guide to enhancing ML and LLM insights with custom metrics. Learn to define, add, access, modify, and delete custom metrics in charts and alerts.
- [Data Drift](https://docs.fiddler.ai/observability/platform/data-drift-platform.md): Learn about data drift and how Fiddler can monitor your ML model data for drift to provide early detection of issues that could impact model performance.
- [Data Integrity](https://docs.fiddler.ai/observability/platform/data-integrity-platform.md): Dive into our guide on ensuring data integrity in ML models and LLMs. Learn to monitor violations with Fiddler’s auto-generated charts and alerts.
- [Embedding Visualization](https://docs.fiddler.ai/observability/platform/embedding-visualization-with-umap.md): Dive into our guide on embedding visualization with UMAP in Fiddler. Learn to create charts, select parameters, and interact with visualizations.
- [Fiddler Query Language](https://docs.fiddler.ai/observability/platform/fiddler-query-language.md): Explore our guide on using Fiddler Query Language to build custom metrics to drive additional business value in dashboards and extra capability in alerting.
- [Model Versions](https://docs.fiddler.ai/observability/platform/model-versions.md): Discover model versions in Fiddler. Learn structured approaches to managing related models, their use cases, capabilities, and how to create a model version.
- [Monitoring Charts](https://docs.fiddler.ai/observability/platform/monitoring-charts-platform.md): Explore our guide to the monitoring charts UI. Learn how to create charts, explore functions, customize tabs, and track LLM metrics effectively.
- [Performance Tracking](https://docs.fiddler.ai/observability/platform/performance-tracking-platform.md): Learn to track performance with Fiddler. Discover why performance metrics matter and the steps to take when your model isn’t performing as expected.
- [Segments](https://docs.fiddler.ai/observability/platform/segments.md): Learn to use model segments for monitoring diverse dimensions. Define, add, and modify segments to gain valuable insights into specific cohorts and dimensions.
- [Statistics](https://docs.fiddler.ai/observability/platform/statistics.md): Discover Fiddler’s statistical metrics guide to monitor column aggregations. Learn what’s tracked, how to monitor metrics, and how to set up alerts.
- [Traffic](https://docs.fiddler.ai/observability/platform/traffic-platform.md): Learn how Fiddler tracks your ML and GenAI models' traffic patterns and when to take action when traffic patterns deviate from normal.
- [Vector Monitoring](https://docs.fiddler.ai/observability/platform/vector-monitoring-platform.md): Dive into our vector monitoring guide to learn about model inputs represented as vectors and how to use Fiddler's custom features to monitor and detect drift.

#### Analytics

- [Analytics](https://docs.fiddler.ai/observability/analytics/index.md): Explore our UI guide to Fiddler analytics. Learn about interfaces for various analytics charts and root cause analysis to better understand your models.
- [Events Table in RCA](https://docs.fiddler.ai/observability/analytics/data-table-in-rca.md): Learn how to use Fiddler's root cause analysis features to quickly hone in on the data issues adversely impacting your ML models and LLM applications.
- [Feature Analytics](https://docs.fiddler.ai/observability/analytics/feature-analytics-chart.md): Dive into our guide on creating feature analytics charts and visualizations for important features in your ML Models and LLM applications.
- [Metric Card](https://docs.fiddler.ai/observability/analytics/metric-card.md): Dive into our guide for metric card creation. Follow step-by-step instructions to create metric cards, use custom metrics, right-side controls, and save charts.
- [Performance Charts Creation](https://docs.fiddler.ai/observability/analytics/performance-charts-creation.md): Discover our guide to creating performance charts. Learn key steps to select charts, use right-side and in-chart controls, and save your customized charts.
- [Performance Charts Visualization](https://docs.fiddler.ai/observability/analytics/performance-charts-visualization.md): Dive into our guide on performance charts and visualizations used to monitor the behavior and performance of your ML models.

#### Dashboards

- [Dashboards](https://docs.fiddler.ai/observability/dashboards/index.md): Explore our guide to Fiddler’s dashboards for centralized monitoring. Discover key features like filters, utilities, default dashboards, and performance.
- [Executive Dashboard](https://docs.fiddler.ai/observability/dashboards/executive-dashboard.md): A cross-project overview of GenAI and ML health for organization leadership, with pre-aggregated metrics that outlive raw data retention.
- [Creating Dashboards](https://docs.fiddler.ai/observability/dashboards/dashboards-creating.md): Navigate our guide for the Dashboard page. Learn how to select new or existing dashboards and access them to monitor performance, drift, integrity, and traffic.
- [Dashboard Interactions](https://docs.fiddler.ai/observability/dashboards/dashboard-interactions.md): Explore our guide to dashboard interactions. Learn to remove, edit, zoom into charts, switch between bar and line views, and undo toolbar changes.
- [Dashboard Utilities](https://docs.fiddler.ai/observability/dashboards/dashboard-utilities.md): Discover dashboard utilities on Fiddler’s platform. Learn to rename, save, share, copy links, or delete dashboards to manage your collection effortlessly.

#### Model UI

- [Model UI](https://docs.fiddler.ai/observability/model-ui/index.md): Learn about Fiddler's no-code Model Editor for streamlined ML model onboarding, featuring draft mode for iterative development and team collaboration.
- [Model Editor](https://docs.fiddler.ai/observability/model-ui/model-editor.md): Step-by-step instructions for onboarding ML models using Fiddler's UI-based editor, from dataset upload to schema validation and publication.
- [Model Schema Editing](https://docs.fiddler.ai/observability/model-ui/model-schema-editing.md): Learn how to modify numeric ranges, edit categorical features, and add metadata columns to keep your model schema aligned with evolving production data.

### Reference

- [Feature Maturity Definitions](https://docs.fiddler.ai/reference/feature-maturity-definitions.md): Review Fiddler's release and support policies for product features at different stages of maturity.
- [Python Version Support Policy](https://docs.fiddler.ai/reference/python-support-policy.md): How Fiddler's Python SDKs decide which Python versions they support, and when support for a version ends.
- [ML Metrics Reference](https://docs.fiddler.ai/reference/ml-metrics-reference.md): Complete reference of all built-in ML metrics supported by the Fiddler monitoring platform, organized by category and model task type.
- [LLM Observability Metrics Reference](https://docs.fiddler.ai/reference/llm-observability-metrics.md): Complete reference of all LLM observability metrics and enrichments supported by the Fiddler monitoring platform.

#### Administration

- [Administration](https://docs.fiddler.ai/reference/administration/settings.md): Dive into our guide to application settings in Fiddler. Learn to use the settings page to manage team setup, permissions, and credentials.
- [AWS VPC Endpoint Setup](https://docs.fiddler.ai/reference/administration/aws-vpc-endpoint-setup.md): Automated script to create AWS VPC endpoints for secure communication with Fiddler Cloud using AWS Virtual PrivateLink.
- [AWS Virtual PrivateLink Setup](https://docs.fiddler.ai/reference/administration/aws-vpl-setup.md): Step-by-step guide to configure AWS Virtual PrivateLink for secure communication between your AWS VPC and Fiddler Cloud.
- [Supported Browsers](https://docs.fiddler.ai/reference/administration/supported-browsers.md): Discover our product guide on supported web browsers for accessing Fiddler, including Google Chrome, Firefox, Safari, and Microsoft Edge.
- [LLM Gateway](https://docs.fiddler.ai/reference/administration/llm-gateway.md): Configure LLM provider credentials to enable AI-powered features in Fiddler using your own API keys from OpenAI, Anthropic, Gemini, and other providers.

#### Access Control

- [Access Control](https://docs.fiddler.ai/reference/access-control/index.md): Explore our guides on authentication options with leading IDPs like Okta and PingOne. Dive deep into authorization topics using the Fiddler UI.
- [Authentication Management](https://docs.fiddler.ai/reference/access-control/authn-authentication-management-console.md)
- [Email Login](https://docs.fiddler.ai/reference/access-control/email-login.md): This page documents the details of Fiddler's native email-based authentication including user account creation and password policy.
- [SSO Authentication Guide](https://docs.fiddler.ai/reference/access-control/sso-authentication-guide.md): Configure Single Sign-On authentication for Fiddler with Okta, Azure AD, Google, Ping, and others. Complete setup guide with troubleshooting tips.
- [Google OIDC](https://docs.fiddler.ai/reference/access-control/google-integration.md): Learn how to configure Fiddler with Google for Single Sign-On (SSO) using the OpenID Connect (OIDC) protocol.
- [Google SAML](https://docs.fiddler.ai/reference/access-control/google-saml.md): Learn how to configure Fiddler with Google Workspace for Single Sign-On (SSO) using the Security Assertion Markup Language (SAML) protocol.
- [Microsoft Entra ID OIDC](https://docs.fiddler.ai/reference/access-control/single-sign-on-with-azure-ad.md): Learn how to configure Fiddler with Microsoft Entra ID (formerly Azure AD) for Single Sign-On (SSO) using the OpenID Connect (OIDC) protocol.
- [Microsoft Entra ID SAML](https://docs.fiddler.ai/reference/access-control/entra-id-saml.md): Learn how to configure Fiddler with Microsoft Entra ID (formerly Azure AD) for Single Sign-On (SSO) using the Security Assertion Markup Language (SAML) protocol.
- [Okta OIDC](https://docs.fiddler.ai/reference/access-control/okta-integration.md): Learn how to configure Fiddler with Okta for Single Sign-On (SSO) using the OpenID Connect (OIDC) protocol.
- [Okta SAML](https://docs.fiddler.ai/reference/access-control/okta-integration-saml.md): Learn how to configure Fiddler with Okta for Single Sign-On (SSO) using the Security Assertion Markup Language (SAML) protocol.
- [PingOne SAML](https://docs.fiddler.ai/reference/access-control/ping-identity-saml.md): Learn how to configure Fiddler with PingOne for Single Sign-On (SSO) using the Security Assertion Markup Language (SAML) protocol.
- [Role-Based Access Control](https://docs.fiddler.ai/reference/access-control/role-based-access.md): Learn how Fiddler uses role-based access control with resources and roles. Discover how to manage access with resources, roles, and permissions in your company.
- [Mapping IdP Groups to Teams](https://docs.fiddler.ai/reference/access-control/mapping-ad-groups-to-fiddler-teams.md): This document describes the naming convention and rules for mapping internal AD groups to Fiddler Teams automatically.

#### Glossary

- [Glossary](https://docs.fiddler.ai/glossary/index.md): Review product concepts and terminology for the Fiddler platform to help get up to speed quickly when adopting Fiddler for your ML and GenAI monitoring.
- [Agentic Observability](https://docs.fiddler.ai/glossary/agentic-observability.md): Comprehensive monitoring, tracing, and analysis of AI agent systems that provides hierarchical visibility into agent reasoning, coordination, and decision-making across distributed multi-agent applica
- [Baseline](https://docs.fiddler.ai/glossary/baseline.md): Reference datasets in Fiddler that serve as comparison points for detecting data drift, evaluating model performance, and identifying when production data deviates from expected patterns.
- [Fiddler Centor Models](https://docs.fiddler.ai/glossary/centor-models.md): Purpose-built LLMs that evaluate AI outputs in real time, powering both monitoring metrics and real-time guardrails with significantly lower latency than general-purpose models.
- [Custom Metric](https://docs.fiddler.ai/glossary/custom-metrics.md): User-defined calculations in Fiddler that extend monitoring beyond standard metrics, allowing teams to track business-specific KPIs and specialized measurements for their AI applications.
- [Data Drift](https://docs.fiddler.ai/glossary/data-drift.md): The statistical change in data distributions over time that can impact model performance. Fiddler detects drift by comparing production data against baselines to identify degradation causes.
- [Embedding Visualization](https://docs.fiddler.ai/glossary/embedding-visualization.md): Interactive visualizations in Fiddler AI that transform complex embedding vectors into 3D displays, revealing semantic patterns, clusters, and outliers in LLM data.
- [Enrichment](https://docs.fiddler.ai/glossary/enrichment.md): Comprehensive overview of enrichments in AI monitoring and evaluation. Learn how Fiddler's enrichment framework transforms raw LLM data into actionable insights through specialized metrics and custom
- [Experiments](https://docs.fiddler.ai/glossary/experiments.md): Systematic assessment of LLM application quality through structured testing with datasets, evaluators, and experiments that enable data-driven decision-making for prompt optimization, model selection,
- [Fiddler Guardrails](https://docs.fiddler.ai/glossary/guardrails.md): Fiddler Guardrails is a real-time content-safety capability that evaluates and filters harmful LLM outputs before they reach users, powered by Fiddler Centor Models.
- [LLM Observability](https://docs.fiddler.ai/glossary/llm-observability.md): Comprehensive monitoring of LLM applications that evaluates safety, quality, and performance metrics to detect issues like hallucinations, toxicity, and drift in generative AI systems.
- [Metric](https://docs.fiddler.ai/glossary/metric.md): Metrics in Fiddler AI are quantitative measurements that evaluate model behavior, data quality, and performance over time, enabling proactive monitoring and issue detection.
- [ML Observability](https://docs.fiddler.ai/glossary/ml-observability.md): A comprehensive approach to monitoring AI systems that goes beyond performance metrics to provide insights into model behavior, data quality, and root causes of issues throughout the ML lifecycle.
- [Model Drift](https://docs.fiddler.ai/glossary/model-drift.md): Changes in model performance over time due to shifting data patterns, concept evolution, or system degradation. Fiddler detects and diagnoses model drift to maintain AI reliability.
- [Model Performance](https://docs.fiddler.ai/glossary/model-performance.md): Quantitative evaluation of AI model accuracy and effectiveness in production. Fiddler tracks performance metrics over time to detect degradation and identify opportunities for improvement.
- [Trust Score](https://docs.fiddler.ai/glossary/trust-score.md): Quantitative scores generated by Fiddler's enrichment processes that measure LLM output quality and safety. These numerical metrics enable monitoring, alerting, and real-time decision-making for AI go