Package-Based Alerts (Private Preview)

Private Preview Notice This feature is in early access, and we are actively enhancing it. We recommend testing on a single model first before enabling it more broadly. Please reach out to the Fiddler team for guidance.

Private Preview Limitations

• This feature is not enabled by default and requires manual configuration.

• Changes to existing alerts may require manual intervention.

• Auto-thresholding logic is still being refined based on real-world feedback.

The Package-Based Alerts feature streamlines the process of configuring and generating alert rules, significantly reducing manual effort. By leveraging a Google Sheet to input key data and generate a recipe YAML file, this feature enables efficient and accurate alert rule creation. This ensures more relevant alerts, enhancing monitoring capabilities and allowing for proactive issue resolution.

Overview

The Packaged Based Alerts feature is designed to facilitate the configuration and generation of alert rules with minimal manual intervention. The process involves:

1. Populating a multi-sheet Google Sheet with necessary data.

2. Generating a recipe YAML file by clicking a dedicated button.

3. Downloading and referencing the YAML file in an alert rules notebook.

4. Updating configuration settings before running the notebook to generate the alerts.

Google Sheet Components

The feature uses several interconnected sheets within a single Google Sheet document:

1. Main Sheet

• Purpose: Central dashboard for creating the recipe YAML file.

• Functionality: Hosts the “Generate Alert Recipe” button that triggers the script.

2. Recipe Parameters

• Purpose: Input key details such as:

  • Project Name

  • Model Name

  • Feature Priority Lists: Define:

    • High-priority features (for high-priority alerts)

    • Medium-priority features (for medium-priority alerts)

3. Mapping Sheets

• Examples: Metric ID Map, Metric Type Map, Threshold Type Map, etc.

• Purpose: Provide dropdown values and mappings (e.g., monitor types, threshold algorithms like standard deviation mapped to a specific threshold type).

• Note: These maps should not be modified unless you need to add new monitor types or update corresponding metrics.

Generating the Recipe YAML File

Follow these steps to generate the YAML file that will be used to create alert rules:

1. Enter Data in Recipe Parameters:

   • Provide the project name, model name, and list of high-priority features.

   • The remaining features will automatically be treated as medium priority.

2. Click the “Generate Alert Recipe” Button:

   • Located on the main sheet.

   • Clicking this button triggers an integrated script.

   • The script collects data from all sheets (including mapping data) and generates alert_recipes.yaml.

3. Script Execution & Permissions:

   • When you click the button, you may be prompted to log in and grant permission for the script to access your Google Drive.

   • Once permissions are granted, the script runs and creates the YAML file in your Google Drive.

Downloading and Integrating the YAML File

1. Download the YAML File:

   • After generation, use the provided link or navigate to your Google Drive to download the YAML file.

   • Save the file in an easy-to-find location, as you'll need to refer to its path in the notebook.

2. Reference the File in the Notebook:

   • Open the alert rules notebook.

   • Update the file path in the notebook configuration to point to the downloaded alert_recipes.yaml.

Configuring and Running the Notebook

Before executing the notebook to generate alert rules, perform the following configuration:

1. Update the Configuration File:

   • Use the provided config.py example as a starting point.

   • Create your configuration file (e.g., copy config.py.example to config.py).

   • Fill in the necessary details:

     • Token

     • Path to the alert_recipes.yaml file

   • Note: Notification channel settings are optional for this process.

2. Execute the Notebook:

   • With the configuration in place, run the notebook.

   • The notebook will parse the alert_recipe.yaml file (which includes your project name, model name, and alert configurations) and generate the corresponding alert rules.

Final Notes

• Permission Flow: Ensure you complete any Google Drive authentication prompts during script execution.

• Mapping Integrity: Do not modify the mapping sheets unless you are intentionally adding or updating monitor types.

• Troubleshooting:

  • If alert rules are not generated as expected, verify:

    • Data accuracy in the Google Sheet.

    • The correct path and token entries in your configuration file.

    • You have provided the necessary permissions for the script.

---

❓ Questions? Talk to a product expert or request a demo.

💡 Need help? Contact us at [email protected].