Skip to main content
Custom metric for monitoring business-specific and domain-specific KPIs. CustomMetric enables creation of user-defined metrics that calculate specific values from model data using SQL-like expressions. Custom metrics extend Fiddler’s built-in monitoring capabilities to support business requirements, domain-specific quality measures, and complex performance indicators.

​
Example

# Business conversion rate metric
conversion_rate = CustomMetric(
    name="weekly_conversion_rate",
    model_id=model.id,
    definition="sum(if(prediction_score > 0.7 and converted == 1, 1, 0)) / sum(if(prediction_score > 0.7, 1, 0))",
    description="Conversion rate for high-confidence predictions"
).create()

# Data quality metric
missing_rate = CustomMetric(
    name="feature_missing_rate",
    model_id=model.id,
    definition="sum(if(is_null(income), 1, 0)) / count(income)",
    description="Percentage of records with missing income values"
).create()

# Fairness metric
fairness_metric = CustomMetric(
    name="demographic_parity",
    model_id=model.id,
    definition="abs((sum(if(gender == 'Male', predicted_churn, 0)) / sum(if(gender == 'Male', 1, 0))) - (sum(if(gender == 'Female', predicted_churn, 0)) / sum(if(gender == 'Female', 1, 0))))",
    description="Demographic parity difference between gender groups"
).create()

# Use in alert rule
alert_rule = AlertRule(
    name="conversion_rate_alert",
    model_id=model.id,
    metric_id=conversion_rate.id,
    priority=Priority.HIGH,
    compare_to=CompareTo.TIME_PERIOD,
    condition=AlertCondition.LESSER,
    bin_size=BinSize.DAY,
    critical_threshold=0.15,  # Alert if conversion drops below 15%
    compare_bin_delta=7
).create()
Custom metrics are calculated during data ingestion and monitoring cycles. Complex expressions may impact performance, so optimize for efficiency. Test expressions thoroughly before using in production alert rules.

​
create()

Create a new CustomMetric on the Fiddler platform. Registers this CustomMetric with the Fiddler platform. The expression must have a name, model_id, and definition specified before calling create().

​
Returns

The same CustomMetric instance with updated server-side attributes (id, created_at, etc.).

​
Raises

  • ApiError – If there’s an error communicating with the Fiddler API.
  • Conflict – If a CustomMetric with the same name already exists for this model.

​
delete()

Delete this CustomMetric from the Fiddler platform. Permanently removes the CustomMetric. This action cannot be undone. Any alert rules or monitors using this CustomMetric must be deleted first.

​
Raises

  • NotFound – If the CustomMetric no longer exists.
  • ApiError – If there’s an error communicating with the Fiddler API.
  • Conflict – If the CustomMetric is still being used by alert rules or monitors.

​
classmethod from_name()

Retrieve a CustomMetric by name and model. Fetches a CustomMetric from the Fiddler platform using its name and associated model ID.

​
Parameters

​
name
str
required
The name of the CustomMetric to retrieve.
​
model_id
UUID | str
required
UUID or string identifier of the associated Model.

​
Returns

The CustomMetric instance for the provided parameters.

​
Raises

  • NotFound – If no CustomMetric exists with the specified name and model.
  • ApiError – If there’s an error communicating with the Fiddler API.

​
classmethod get()

Retrieve a CustomMetric by its unique identifier. Fetches a CustomMetric from the Fiddler platform using its UUID.

​
Parameters

​
id_
UUID | str
required
The unique identifier (UUID) of the CustomMetric to retrieve. Can be provided as a UUID object or string representation.

​
Returns

The CustomMetric instance with all its configuration and metadata.

​
Raises

  • NotFound – If no CustomMetric exists with the specified ID.
  • ApiError – If there’s an error communicating with the Fiddler API.

​
classmethod get_organization_id()

Get the organization UUID from the global connection.

​
Returns

Unique identifier of the organization associated with the current connection.

​
classmethod get_organization_name()

Get the organization name from the global connection.

​
Returns

Name of the organization associated with the current connection.

​
classmethod list()

List all CustomMetric instances for a model. Retrieves all CustomMetric instances associated with a specific model.

​
Parameters

​
model_id
UUID | str
required
UUID or string identifier of the Model.

​
Yields

CustomMetric instances for each CustomMetric in the model.

​
Raises

ApiError – If there’s an error communicating with the Fiddler API.

​
Returns

Iterator[CustomMetric]

​
init()

Construct a custom metric instance.