Alerts with Fiddler Client

In addition to using the Fiddler UI, users have the flexibility to set up alert rules using the Fiddler API client to enable the following workflows:

• Add alert rule(s)
• Delete alert rule(s)
• Get the list of all alert rule(s)
• Get the list of triggered alert(s)

📘

The user guide for setting up alert rules in the Fiddler UI here.

Add an Alert Rule

The Fiddler client can be used to create a variety of alert rules viz. Data Drift, Performance, Data Integrity, and Service Metrics

Example 1: Data Integrity Alert Rule to compare against a raw value

Let's sets up a Data Integrity alert that triggers an email notification when published events have 5% null values in a 1 day bin for the age column. Notice compare_to = 'raw_value'.

The fdl.AlertRule.create() API is used to create alert rules.

import fiddler as fdl

MODEL_ID='4531bfd9-2ca2-4a7b-bb5a-136c8da09ca2'
alert_rule = fdl.AlertRule(
        name='alert_name',
        model_id=MODEL_ID,
        metric_id='null_violation',
        priority=fdl.Priority.HIGH,
        compare_to=fdl.CompareTo.RAW_VALUE,
  			compare_bin_delta=0,
        condition=fdl.AlertCondition.GREATER,
        bin_size=fdl.BinSize.Day,
        critical_threshold=0.1,
        warning_threshold=0.05,
        columns=['age'],
).create()

notifications = alert_rule.set_notification_config(
        emails=['[email protected]', '[email protected]'],
    )

Please note, the possible values for bin_size are 'one_hour', 'one_day', and 'one_week'. When alert_type is 'data_integrity', use one of 'missing_value', 'range_violation', or 'type_violation' for metric type.

Example 2: Performance Alert Rule to compare against a previous time window

And the following API call sets up a Performance alert rule which triggers an email notification when precision metric is 5% higher than that from 1 day bin one day ago. Notice compare_to = 'time_period' and compare_bin_delta = '1 day'.

import fiddler as fdl

MODEL_ID='4531bfd9-2ca2-4a7b-bb5a-136c8da09ca2'
alert_rule = fdl.AlertRule(
        name='alert_name',
        model_id=MODEL_ID,
        metric_id='precision',
        priority=fdl.Priority.HIGH,
        compare_to=fdl.CompareTo.TIME_PERIOD,
  			compare_bin_delta=1,
        condition=fdl.AlertCondition.GREATER,
        bin_size=fdl.BinSize.Day,
        critical_threshold=0.1,
        warning_threshold=0.05,
).create()

notifications = alert_rule.set_notification_config(
        emails=['[email protected]', '[email protected]'],
    )

Please note, the possible values for compare_bin_delta vs bin_size are:

Get Alert Rules

The fdl.AlertRule.get() API can be used to get a list of all alert rules with respect to the filtering parameters and it returns a paginated list of alert rules.

import fiddler as fdl

MODEL_ID='4531bfd9-2ca2-4a7b-bb5a-136c8da09ca2'
alert_rules = fdl.AlertRule.list(
    project_id = PROJECT_ID,# Optional parameter
    model_id = MODEL_ID,# Optional parameter
    alert_type = fdl.AlertType.DATA_INTEGRITY, # Optional parameter
    metric = fdl.Metric.MISSING_VALUE,# Optional parameter
    column = "age", 
    ordering = ['critical_threshold'], #['-critical_threshold'] for descending
    limit= 4, ## to set the number of rules to show in one go
    offset = 0, # page offset (multiple of limit)
)

Here is an example output of get_alert_rules() API:

[AlertRule(alert_rule_uuid='9b8711fa-735e-4a72-977c-c4c8b16543ae',
           organization_name='some_org_name',
           project_id='some_project_id',
           model_id='some_model_id',
           name='age-null-1hr',
           alert_type=AlertType.DATA_INTEGRITY,
           metric=Metric.MISSING_VALUE,
           column='age',
           priority=Priority.HIGH,
           compare_to=CompareTo.RAW_VALUE,
           compare_period=None,
           warning_threshold=0.05,
           critical_threshold=0.1,
           condition=AlertCondition.GREATER,
           bin_size=BinSize.ONE_HOUR)]

Delete an Alert Rule

To delete an alert rule we need the corresponding unique alert_rule_uuid which is part of the output we get from fdl.AlertRule.get(). Then we can delete a rule by calling the fdl.AlertRuledelete() API as shown below:

ALERT_RULE_ID='ed8f18e6-c319-4374-8884-71126a6bab85'
rule = fdl.AlertRule.get(id_=ALERT_RULE_ID)
rule.delete()

Get Triggered Alerts

Finally, to get a paginated list of triggered alerts for a given alert rule in a given time range we can call the fdl.AlertRecord.list() API as the following:

triggered_alerts = fdl.AlertRecord.list(
        alert_rule_id=ALERT_RULE_ID,
        start_time=datetime(2023, 12, 18),
        end_time=datetime(2023, 12, 25),
        ordering = ['alert_time_bucket'], #['-alert_time_bucket'] for descending, optional.
)

Notifications

After creating a new alert rule, users choose the type of the notification that will be leveraged by Fiddler when an alert is raised. Currently Fiddler client supports email, PagerDuty and Webhook services as notifications. To create a notification configuration we add the notification config to Alert Rule. For example, the following code snippet creates a notification configuration using a comma separated list of email addresses.

To create a notification configuration using both email addresses and pager duty.

ALERT_RULE_ID = 'ed8f18e6-c319-4374-8884-71126a6bab85'
alert_rule = fdl.AlertRule.get(id_=ALERT_RULE_ID)
notifications = alert_rule.set_notification_config(
        emails=['[email protected]', '[email protected]'],
        pagerduty_services = ['pagerduty_service_1','pagerduty_service_2']
			  pagerduty_severity = 'critical'
    )

↪ Questions? Join our community Slack to talk to a product expert