Skip to content

Alerts

Fiddler allows you to set up alerts for a model or group of models in a project. View your alerts by clicking Alerts in the Monitor page.

Alert_Dashboard

What kinds of alerts are supported??

You can get alerts for the following metrics: * Drift score - predictions and all features * Data integrity - all features * Service metrics

Support for alerts about outliers is coming soon

You have two options for deciding when to be alerted:

  • Compare the metric to an absolute value. For example, if traffic for a given hour is less than 1000, then alert.
  • Compare the metric to a previous time period. For example, if traffic is over 10% lower than the corresponding time period a week ago, then alert.

You can set a threshold for either comparison, between 10%-90%.

Why do we need alerts?

  • It’s not possible to manually track all metrics 24/7
  • Sensible alerts are your first line of defense, and they are meant to warn about issues in production.

What should I do when I receive an alert?

  • Click on the link in the email to go to the tab where the alert originated (e.g. Data Drift).
  • In Fiddler’s Monitor UI, more information can usually be obtained from the drill down below the main chart.
  • You can also examine the data in the Analyze tab. You can use SQL to slice and dice the data, and use custom visualization tools and operators to make sense of the model’s behavior on these segments.

Setting up Alerts

Create a new alert by clicking the *Add Alert* button in the top right inside the Monitor pane. Configure the alert, and click *Save* when you are done.

Create_New_Alert

Alert Service Metrics Data Drift
Type Service metrics Data drift
Metric Traffic, Latency, Error Prediction or Feature
Name N/A Prediction or Feature names
Window Size 1 hour 1 hour
Compare To Previous window or Abs value Previous window or Abs Value
Condition Increased or Decreased Drift greater or lesser
Threshold 10-90 in Percentage 10-90 in Percentage
Email User email User email

Once an alert is set, it can be viewed under ‘Alert Configuration’ page.

Delete an existing alert by clicking on the three dots under the Actions column. You can only edit the email for an alert; to make any other changes, you need to delete the alert and create a new one.

Edit_Delete_Alert

Email Alert

Here is a sample of the email you would receive when an alert is triggered: Email_Alert

Reference


  1. Join our community Slack to ask any questions 

Back to top