LogoLogo
👨‍💻 API Reference📣 Release Notes📺 Request a Demo
  • Introduction to Fiddler
    • Monitor, Analyze, and Protect your ML Models and Gen AI Applications
  • Fiddler Doc Chatbot
  • First Steps
    • Getting Started With Fiddler Guardrails
    • Getting Started with LLM Monitoring
    • Getting Started with ML Model Observability
  • Tutorials & Quick Starts
    • LLM and GenAI
      • LLM Evaluation - Compare Outputs
      • LLM Monitoring - Simple
    • Fiddler Free Guardrails
      • Guardrails - Quick Start Guide
      • Guardrails - Faithfulness
      • Guardrails - Safety
      • Guardrails FAQ
    • ML Observability
      • ML Monitoring - Simple
      • ML Monitoring - NLP Inputs
      • ML Monitoring - Class Imbalance
      • ML Monitoring - Model Versions
      • ML Monitoring - Ranking
      • ML Monitoring - Regression
      • ML Monitoring - Feature Impact
      • ML Monitoring - CV Inputs
  • Glossary
    • Product Concepts
      • Baseline
      • Custom Metric
      • Data Drift
      • Embedding Visualization
      • Fiddler Guardrails
      • Fiddler Trust Service
      • LLM and GenAI Observability
      • Metric
      • Model Drift
      • Model Performance
      • ML Observability
      • Trust Score
  • Product Guide
    • LLM Application Monitoring & Protection
      • LLM-Based Metrics
      • Embedding Visualizations for LLM Monitoring and Analysis
      • Selecting Enrichments
      • Enrichments (Private Preview)
      • Guardrails for Proactive Application Protection
    • Optimize Your ML Models and LLMs with Fiddler's Comprehensive Monitoring
      • Alerts
      • Package-Based Alerts (Private Preview)
      • Class Imbalanced Data
      • Enhance ML and LLM Insights with Custom Metrics
      • Data Drift: Monitor Model Performance Changes with Fiddler's Insights
      • Ensuring Data Integrity in ML Models And LLMs
      • Embedding Visualization With UMAP
      • Fiddler Query Language
      • Model Versions
      • How to Effectively Use the Monitoring Chart UI
      • Performance Tracking
      • Model Segments: Analyze Cohorts for Performance Insights and Bias Detection
      • Statistics
      • Monitoring ML Model and LLM Traffic
      • Vector Monitoring
    • Enhance Model Insights with Fiddler's Slice and Explain
      • Events Table in RCA
      • Feature Analytics Creation
      • Metric Card Creation
      • Performance Charts Creation
      • Performance Charts Visualization
    • Master AI Monitoring: Create, Customize, and Compare Dashboards
      • Creating Dashboards
      • Dashboard Interactions
      • Dashboard Utilities
    • Adding and Editing Models in the UI
      • Model Editor UI
      • Model Schema Editing Guide
    • Fairness
    • Explainability
      • Model: Artifacts, Package, Surrogate
      • Global Explainability: Visualize Feature Impact and Importance in Fiddler
      • Point Explainability
      • Flexible Model Deployment
        • On Prem Manual Flexible Model Deployment XAI
  • Technical Reference
    • Python Client API Reference
    • Python Client Guides
      • Installation and Setup
      • Model Onboarding
        • Create a Project and Onboard a Model for Observation
        • Model Task Types
        • Customizing your Model Schema
        • Specifying Custom Missing Value Representations
      • Publishing Inference Data
        • Creating a Baseline Dataset
        • Publishing Batches Of Events
        • Publishing Ranking Events
        • Streaming Live Events
        • Updating Already Published Events
        • Deleting Events From Fiddler
      • Creating and Managing Alerts
      • Explainability Examples
        • Adding a Surrogate Model
        • Uploading Model Artifacts
        • Updating Model Artifacts
        • ML Framework Examples
          • Scikit Learn
          • Tensorflow HDF5
          • Tensorflow Savedmodel
          • Xgboost
        • Model Task Examples
          • Binary Classification
          • Multiclass Classification
          • Regression
          • Uploading A Ranking Model Artifact
    • Integrations
      • Data Pipeline Integrations
        • Airflow Integration
        • BigQuery Integration
        • Integration With S3
        • Kafka Integration
        • Sagemaker Integration
        • Snowflake Integration
      • ML Platform Integrations
        • Integrate Fiddler with Databricks for Model Monitoring and Explainability
        • Datadog Integration
        • ML Flow Integration
      • Alerting Integrations
        • PagerDuty Integration
    • Comprehensive REST API Reference
      • Projects REST API Guide
      • Model REST API Guide
      • File Upload REST API Guide
      • Custom Metrics REST API Guide
      • Segments REST API Guide
      • Baselines REST API Guide
      • Jobs REST API Guide
      • Alert Rules REST API Guide
      • Environments REST API Guide
      • Explainability REST API Guide
      • Server Info REST API Guide
      • Events REST API Guide
      • Fiddler Trust Service REST API Guide
    • Fiddler Free Guardrails Documentation
  • Configuration Guide
    • Authentication & Authorization
      • Adding Users
      • Overview of Role-Based Access Control
      • Email Authentication
      • Okta Integration
      • SSO with Azure AD
      • Ping Identity SAML SSO Integration
      • Mapping LDAP Groups & Users to Fiddler Teams
    • Application Settings
    • Supported Browsers
  • History
    • Release Notes
    • Python Client History
    • Compatibility Matrix
    • Product Maturity Definitions
Powered by GitBook
On this page
  • Getting Started with Fiddler’s Charts UI
  • What Are the Key Features and Functions of the Monitoring Chart UI?
  • How to Create a New Monitoring Chart
  • Exploring Chart Metric Queries & Filters
  • How to Customize Your Monitoring Chart
  • Mastering our Chart UI to Effectively Track LLM and ML Performance

Was this helpful?

  1. Product Guide
  2. Optimize Your ML Models and LLMs with Fiddler's Comprehensive Monitoring

How to Effectively Use the Monitoring Chart UI

PreviousModel VersionsNextPerformance Tracking

Last updated 1 month ago

Was this helpful?

© 2024 Fiddler Labs, Inc.

Getting Started with Fiddler’s Charts UI

The Chart UI on Fiddler AI’s platform is designed to help you effectively monitor and analyze model performance metrics. Navigating to the Charts tab in the navigation bar allows you to easily access the tools needed to visualize key data and track your LLM and ML models’ performance. Choose to open a previously saved chart for ongoing analysis or create a new one tailored to specific metrics, such as response faithfulness, jailbreak attempts, data drift, and more.

Supported Metric Types

Monitoring charts enable you to plot any combination of the following metric types for a given model:

  • Traffic

    • Model traffic volume over time provides insights into system usage patterns and overall operational health.

  • Statistics

    • Aggregated metrics that monitor trends and changes in your model's data columns

  • Data Drift

    • Data drift occurs when production data differs significantly from training data, potentially degrading model performance and accuracy.

  • Data Integrity

    • TMonitor three critical data integrity issues: missing values, incorrect data types, and out-of-range values that violate model input requirements.

  • Performance

    • Model performance measures how accurately your model accomplishes its intended task, directly impacting business outcomes and decision quality.

  • Custom Metrics

    • Build tailored monitoring metrics that align with your specific business rules and success criteria using custom formulas.

What Are the Key Features and Functions of the Monitoring Chart UI?

Multiple Charting Options

You can plot up to 20 columns and 6 metric queries for a model enabling you to easily perform model-to-model comparisons and plot multiple metrics in a single chart view.

Embedded Root Cause Analytics

Root cause analysis information covers data drift and data integrity, and performance analytics charts for binary classification, multiclass classification, and regression models.

Download Your Raw Data for Further Analysis

You can easily download the raw chart data to CSV or parquet files. This feature allows you to analyze your data further.

How to Create a New Monitoring Chart

To create a new monitoring chart, click on the Add Chart button on the Charts page. Search for and select the project to create the chart, and press Add Chart.

Save and Share Your Monitoring Chart

Save your chart by clicking the Save button in the chart studio's top right corner. You can then share your chart by copying its link and sending it to other users who have access to the project.

Undo and Redo Actions

Easily control the following actions with the undo and redo buttons:

  • Metric query selection

  • Time range selections

  • Bin size selections

To learn how to undo actions taken using the chart toolbar, see the Toolbar information in the next section.

Exploring Chart Metric Queries & Filters

Creating and Managing Metric Queries

Start building your monitoring chart by creating a metric query. First, select a model from your project to analyze. Then, specify which metrics and columns you want to visualize. Note that you can only select models that exist within your current project.

Select the type of metric you want to monitor: Performance, Data Drift, Data Integrity, Statistic, Traffic, or Customer Metric. Then choose specific metrics within that category. For instance, if you're monitoring a binary classification model, you might select Performance metrics and track accuracy over time.

Customizing Columns for Monitoring Charts

If you choose to chart data drift or data integrity, you can choose to plot up to 20 different columns from the following column categories: inputs, outputs, targets, metadata, and custom features.

Adding Multiple Metrics or Models

Add up to 6 metric queries that allow you to chart different metrics from one or more models in a single chart view.

Using Chart Filters for Better Analysis

Analyze specific slices of data using three key filtering tools: chart filters, the chart toolbar, and the zoom slider. These tools work together to help you investigate important patterns and trends in your data.

Tailored Filters

You can customize your chart view using the date range and bin size filters. The date range can be one of the pre-defined time ranges or a custom range. The bin size selected controls the frequency for which the data is displayed. So selecting Day will show daily data over the date range selected.

Customize your chart's time-based display using the date range and bin size filters. Choose from preset date ranges or define a custom period. The bin size determines how your data is grouped - for example, selecting 'Day' will show daily aggregated data across your chosen date range.

Toolbar Functions

The charts toolbar is made up of 5 functions:

  • Drag to zoom

  • Reset zoom

  • Toggle to a line chart

  • Toggle to a bar chart

  • Undo all toolbar actions

📘 Note: If the zoom reset or toolbar undo is selected, this will also undo any actions taken with the zoom slider.

Line & Bar Chart Toggles

Toggle between line and bar chart views using the toolbar icons in the top right corner. While you can freely switch between these visualization types, please note that toolbar settings are temporary and won't be saved with your chart.

Zoom Slider

Use the horizontal zoom bar at the bottom of the chart to focus on specific time periods. Select your desired time range by dragging the zoom bar handles, then slide the selected range left or right to analyze different periods. For example, to examine week-by-week data over six months, zoom to your preferred view and drag the range to explore different weeks.

Breakdown Summary

You can easily visualize your charts' raw data as a table within the fiddler chart studio, or download the content as a CSV or parquet file for further analysis. If you choose to chart multiple columns, as shown below, you can search for and sort by Model name, Metric name, Column name, or values for a specific date.

How to Customize Your Monitoring Chart

Adjusting Scale and Range

The Customize tab lets you adjust your chart's y-axis settings to improve data visualization. You can set minimum and maximum values to better utilize chart space, and apply logarithmic scaling to clearly display data with large variations in values.

Assigning Y-axis for Metric Queries

Select the y-axis for your metric queries with enhanced flexibility to customize the scale and range for each axis.

Mastering our Chart UI to Effectively Track LLM and ML Performance

Understanding how to use the chart UI is key to measuring and maintaining the performance of LLM and ML models. Fiddler’s model monitoring tools allow you to track key metrics, identify data drift, and gain valuable insights into model behavior. Customize your Fiddler charts to proactively address issues and ensure models remain accurate, reliable, and high-performing.


Questions? to a product expert or a demo.

Need help? Contact us at .

❓
💡
Talk
request
help@fiddler.ai
Plotting traffic on Fiddlers monitoring chart.
How to add a new monitoring chart to a project.
Analyzing metric queries in your monitoring chart.
Adding multiple columns to your chart.
Adding multiple metrics and/or models to a single chart.
How to add filters and use capabilities to analyze your monitoring chart.
Toolbar functions in a monitoring chart.
Visualize metrics in bar chart mode.
Focus your analysis in a specific time range.
Analyze your chart’s raw data as a table to get a breakdown summary.
Customize the scale and range of the y-axis on your monitoring chart.
Customize the scale and range of each axis on your monitoring chart.