Monitoring Charts
Last updated
Was this helpful?
Last updated
Was this helpful?
Fiddler AI’s monitoring charts allow you to easily track your models and ensure that they are performing optimally. For any of your models, monitoring charts for data drift, performance, data integrity, or traffic metrics can be displayed using Fiddler Dashboards.
Monitoring charts enable you to plot one of the following metric types for a given model:
The volume of traffic received by the model over time informs us of the overall system's health.
Metrics which can be used to monitor basic aggregations over columns.
Model performance can be poor if models trained on a specific dataset encounter different data in production.
Three types of violations can occur at model inference: missing feature values, type mismatches (e.g. sending a float input for a categorical feature type) or range mismatches (e.g. sending an unknown US State for a State categorical feature).
The model performance tells us how well a model performs on its task. A poorly performing model can have significant business implications.
Create your own custom metric formulas for ultimate flexibility in metrics monitoring.
You can plot up to 20 columns and 6 metric queries for a model enabling you to easily perform model-to-model comparisons and plot multiple metrics in a single chart view.
Root cause analysis information covers data drift and data integrity, and performance analytics charts for binary classification, multiclass classification, and regression models.
You can easily download a CSV of the raw chart data. This feature allows you to analyze your data further.
To create a new monitoring chart, click on the Add Chart button on the Charts page. Search for and select the project to create the chart, and press Add Chart.
Manually save your chart using the Save button on the top right corner of the chart studio. Copy a link to your chart and share it with other fiddler accounts who have access to the project where the chart resides.
Easily control the following actions with the undo and redo buttons:
Metric query selection
Time range selections
Time range selections
Bin size selections
To learn how to undo actions taken using the chart toolbar, see the Toolbar information in the next section.
A metric query enables you to define what model to focus on, and which metrics and columns to plot on your monitoring chart. To get started with the metric query, choose a model of choice. Note: only models within the same project as your chart are accessible.
Once a model is selected, choose a metric type from Performance, Data Drift, Data Integrity, or Traffic metrics and relevant metrics. For example, we may choose to chart accuracy for our binary classification model.
If you choose to chart data drift or data integrity, you can choose to plot up to 20 different columns from the following column categories; inputs, outputs, targets, decisions, metadata, and custom features.
Add up to 6 metric queries that allow you to chart different metrics and/or models in a single chart view.
There are three major chart filter capabilities, chart filters, chart toolbar, and zoom slider. They work together to enable you to best analyze the slices of data that may be worth investigating.
You can customize your chart view using time range, time zone, and bin size chart filters. The data range can be one of the pre-defined time ranges or a custom range. The bin size selected controls the frequency for which the data is displayed. So selecting Day will show daily data over the date range selected.
The charts toolbar is made up of 5 functions:
Drag to zoom
Reset zoom
Toggle to a line chart
Toggle to a bar chart
Undo all toolbar actions
📘 Note: If the zoom reset or toolbar undo is selected, this will also undo any actions taken with the zoom slider.
Line & Bar Chart Toggle
You can switch between visualizing your chart as a line or bar chart using the toolbar icons. Click on the line chart icon on the top right of the chart to switch to the line chart view. Likewise, select the bar chart icon in the toolbar to switch to the bar chart view. However, note that these views are only temporary and any settings you specify using the toolbar will not be saved to the chart.
Zoom Slider
You can also use the horizontal zoom bar to zoom, located at the base of the chart. Once you've identified the time range you want to focus on, you can use the zoom bar to drag the range across time. For instance, if you want to analyze your data week-by-week over the past six months, you can use the toolbar or horizontal zoom bar to zoom in on the desired time range and then click and drag the selected range using the base horizontal zoom bar.
You can easily visualize your charts' raw data as a table within the fiddler chart studio, or download the content as a CSV for further analysis. If you choose to chart multiple columns, as shown below, you can search for and sort by Model name, Metric name, Column name, or values for a specific date.
The Customize tab enables users to adjust the scale and range of the y-axis on their monitoring charts. In the example below, we have adjusted the minimum value of the y-axis for the plotted traffic to make more use of the chart space. For values with large variance, logarithmic scale can be applied to more clearly analyze the chart.
Select the y-axis for your metric queries with enhanced flexibility to customize the scale and range for each axis.
