Fiddler allows you to set up alerts for your model. View your alerts by clicking Alerts on the Monitor page.
What kinds of alerts are supported?¶
You can get alerts for the following metrics:
- Data drift — Predictions and all features
- Data integrity — All features
- Service metrics
You have two options for deciding when to be alerted:
- Compare the metric to an absolute value (e.g. if traffic for a given hour is less than 1000, then alert).
- Compare the metric to a previous time period (e.g. if traffic is down 10% or more than it was at the same time one week ago, then alert).
You can set the alert threshold in either case.
Why do we need alerts?¶
- It’s not possible to manually track all metrics 24/7.
- Sensible alerts are your first line of defense, and they are meant to warn about issues in production.
What should I do when I receive an alert?¶
- Click on the link in the email to go to the tab where the alert originated (e.g. Data Drift).
- In Fiddler’s Monitor UI, more information can be obtained from the drill down below the main chart.
- You can also examine the data in the Analyze tab. You can use SQL to slice and dice the data, and use custom visualization tools and operators to make sense of the model’s behavior within the time range under consideration.
Setting up Alerts¶
Create a new alert by clicking the Add Alert button on the top-right corner of the Monitor page. Configure the alert, and click Save when you're done.
Once an alert is set, it can be viewed under the Alerts tab of the Monitor page.
Delete an existing alert by clicking on the three dots under the Actions column. You can only edit the email for an alert; to make any other changes, you need to delete the alert and create a new one.
Sample Alert Email¶
Here's a sample of an email that's sent if an alert is triggered:
- See our article on The Rise of MLOps Monitoring