API Methods 3.x

Alerts

Alert Rule

Alert rule object contains the below fields.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the alert rule.

name

str

-

Unique name of the alert rule.

model

-

Details of the model.

project

-

Details of the project to which the dataset belongs.

baseline

Optional[Baseline]

None

Details of the baseline for the alert.

segment

Optional[Segment]

None

Details of segment for the alert.

priority

Union[str, Priority]

-

To set the priority for the alert rule. Select from: 1. Priority.LOW 2. Priority.MEDIUM 3. Priority.HIGH.

compare_to

Union[str, CompareTo]

-

Select from the two: 1. CompareTo.RAW_VALUE 2. CompareTo.TIME_PERIOD

metric_id

Union[str, UUID]

-

Type of alert metric UUID or string denoting metric name.

critical_threshold

float

-

Threshold value to crossing which a critical level severity alert will be triggered.

condition

Union[str, AlertCondition]

-

Select from: 1. AlertCondition.LESSER 2. AlertCondition.GREATER

bin_size

Union[str, BinSize]

-

Size of the bin for alert rule.

columns

Optional[List[str]]

None

List of column names on which alert rule is to be created. It can take ['ANY'] to check for all columns.

baseline_id

Optional[UUID]

None

UUID of the baseline for the alert.

segment_id

Optional[UUID]

None

UUID of segment for the alert

compare_bin_delta

Optional[int]

None

Number of bin_size to compare the metric to a previous time period.

warning_threshold

Optional[float]

None

Threshold value to crossing which a warning level severity alert will be triggered.

created_at

datetime

-

Time at which alert rule was created.

updated_at

datetime

-

Latest time at which alert rule was updated.

evaluation_delay

int

0

To introduce a delay in the evaluation of the alert, specifying the duration in hours. The delay period must not exceed one year(8760 hours).

constructor()

Initialise a new alert rule on Fiddler Platform.

Parameters

ParameterTypeDefaultDescription

name

str

-

Unique name of the model

model_id

UUID

-

Details of the model.

metric_id

Union[str, UUID]

-

Type of alert metric UUID or enum.

columns

Optional[List[str]]

None

List of column names on which alert rule is to be created. It can take ['ANY'] to check for all columns.

baseline_id

Optional[UUID]

None

UUID of the baseline for the alert.

segment_id

Optional[UUID]

None

UUID of the segment for the alert.

priority

Union[str, Priority]

-

To set the priority for the alert rule. Select from: 1. Priority.LOW 2. Priority.MEDIUM 3. Priority.HIGH.

compare_to

Union[str, CompareTo]

-

Select from the two: 1. CompareTo.RAW_VALUE (absolute alert) 2. CompareTo.TIME_PERIOD (relative alert)

compare_bin_delta

Optional[int]

None

Compare the metric to a previous time period in units of bin_size.

warning_threshold

Optional[float]

None

Threshold value to crossing which a warning level severity alert will be triggered.

critical_threshold

float

-

Threshold value to crossing which a critical level severity alert will be triggered.

condition

Union[str, AlertCondition]

-

Select from: 1. AlertCondition.LESSER 2. AlertCondition.GREATER

bin_size

Union[str, BinSize]

-

Size of the bin for alert rule.

evaluation_delay

int

0

To introduce a delay in the evaluation of the alert, specifying the duration in hours. The delay period must not exceed one year(8760 hours).

Usage

MODEL_NAME = 'test_model'
PROJECT_NAME = 'test_project'
SEGMENT_NAME = 'test_segment'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)
segment = fdl.Segment.from_name(name=SEGMENT_NAME, model_id=model.id)

alert_rule = fdl.AlertRule(
        name='alert_name',
        model_id=model.id,
        metric_id='drift',
        priority=fdl.Priority.HIGH,
        compare_to=fdl.CompareTo.TIME_PERIOD,
        compare_bin_delta=1,
        condition=fdl.AlertCondition.GREATER,
        bin_size=fdl.BinSize.HOUR,
        critical_threshold=0.5,
        warning_threshold=0.1,
        columns=['gender', 'creditscore'],
        segment_id=segment.id,
        evaluation_delay=10
    )

create()

Set a new alert rule.

Parameters

No

Usage

PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
SEGMENT_NAME = 'test_segment'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)
segment = fdl.Segment.from_name(name=SEGMENT_NAME, model_id=model.id)

alert_rule = fdl.AlertRule(
        name='alert_name',
        model_id=model.id,
        metric_id='drift',
        priority=fdl.Priority.HIGH,
        compare_to=fdl.CompareTo.TIME_PERIOD,
        compare_bin_delta=1,
        condition=fdl.AlertCondition.GREATER,
        bin_size=fdl.BinSize.HOUR,
        critical_threshold=0.5,
        warning_threshold=0.1,
        columns=['gender', 'creditscore'],
        segment_id=segment.id,
    ).create()

Returns

Return TypeDescription

Alert rule instance.

get()

Get a single alert rule.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique identifier for the alert rule.

Usage

ALERT_RULE_ID='ed8f18e6-c319-4374-8884-71126a6bab85'

alert = fdl.AlertRule.get(id_=ALERT_RULE_ID)

Returns

Return TypeDescription

Alert rule instance.

Raises

Error codeIssue

NotFound

Alert rule with given identifier not found.

Forbidden

Current user may not have permission to view details of alert rule.


list()

Get a list of all alert rules in the organization.

Parameters

ParameterTypeDefaultDescription

model_id

Optional[UUID]

None

Unique identifier for the model to which alert rule belongs.

project_id

Optional[UUID]

None

Unique identifier for the project to which alert rule belongs

metric_id

Union[str, UUID]

-

Type of alert metric UUID or enum.

columns

Optional[List[str]]

None

List of column names on which alert rule is to be created. It can take ['ANY'] to check for all columns.

baseline_id

Optional[UUID]

None

UUID of the baseline for the alert.

ordering

Optional[List[str]]

None

List of Alert Rule fields to order by. Eg. [‘alert_time_bucket’] or [‘- alert_time_bucket’] for descending order.

Usage

PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts = fdl.AlertRule.list(model_id=model.id)

Returns

Return TypeDescription

Iterator[AlertRule]

Iterable of alert rule instances.


delete()

Delete an existing alert rule.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique UUID of the alert rule .

Usage

ALERT_NAME = "YOUR_ALERT_NAME"
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts_list = fdl.AlertRule.list(model_id=model.id)
alert_rule = None

for alert in alerts_list:
    if ALERT_NAME == alert.name:
    alert_rule = alert

alert_rule.delete()

Returns

No

Raises

Error codeIssue

NotFound

Alert rule with given identifier not found.

Forbidden

Current user may not have permission to view details of alert rule.


enable_notifications()

Enable notification for an alert rule.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique UUID of the alert rule .

Usage

ALERT_NAME = "YOUR_ALERT_NAME"
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts_list = fdl.AlertRule.list(model_id=model.id)
alert_rule = None

for alert in alerts_list:
    if ALERT_NAME == alert.name:
    alert_rule = alert

alert_rule.enable_notifications()

Returns

None

Raises

Error codeIssue

NotFound

Alert rule with given identifier not found.

Forbidden

Current user may not have permission to view details of alert rule.


diable_notification()

Disable notification for an alert rule.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique UUID of the alert rule .

Usage

ALERT_NAME = "YOUR_ALERT_NAME"
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts_list = fdl.AlertRule.list(model_id=model.id)
alert_rule = None

for alert in alerts_list:
    if ALERT_NAME == alert.name:
    alert_rule = alert

alert_rule.diable_notification()

Returns

None

Raises

Error codeIssue

NotFound

Alert rule with given identifier not found.

Forbidden

Current user may not have permission to view details of alert rule.


AlertNotifications

Alert notifications for an alert rule.

ParameterTypeDefaultDescription

emails

Optional[List[str]]

None

List of emails to send notification to.

pagerduty_services

Optional[List[str]]

None

List of pagerduty services to trigger the alert to.

pagerduty_severity

Optional[str]

None

Severity of pagerduty.

webhooks

Optional[List[UUID]]

None

List of webhook UUIDs.

set_notification_config()

Set notifications for an alert rule.

Parameters

ParameterTypeDefaultDescription

emails

Optional[List[str]]

None

List of emails to send notification to.

pagerduty_services

Optional[List[str]]

None

List of pagerduty services to trigger the alert to.

pagerduty_severity

Optional[str]

None

Severity of pagerduty.

webhooks

Optional[List[UUID]]

None

List of webhook UUIDs.

Usage

ALERT_NAME = "YOUR_ALERT_NAME"
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts_list = fdl.AlertRule.list(model_id=model.id)
alert_rule = None

for alert in alerts_list:
    if ALERT_NAME == alert.name:
    alert_rule = alert

notifications = alert_rule.set_notification_config(
        emails=['abc@xyz.com', 'admin@xyz.com'],
        webhooks=[
            'e20bf4cc-d2cf-4540-baef-d96913b14f1b',
            '6e796fda-0111-4a72-82cd-f0f219e903e1',
        ],
    )

Returns

Return TypeDescription

Alert notifications for an alert rule.

If we pagerduty_severity is passed without pagerduty_services then the pagerduty_severity is ignored.

Raises

Error codeIssue

BadRequest

All 4 input parameters are empty.

ValueError

Webhook ID is incorrect.

get_notification_config()

Get notification configuration for an alert rule.

Parameters

None

Usage

ALERT_NAME = "YOUR_ALERT_NAME"
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts_list = fdl.AlertRule.list(model_id=model.id)
alert_rule = None

for alert in alerts_list:
    if ALERT_NAME == alert.name:
    alert_rule = alert

notification_config = alert_rule.get_notification_config()

Returns

Return TypeDescription

Notification Config

Alert notifications for an alert rule.

Raises

Error codeIssue

BadRequest

All 4 input parameters are empty.

ValueError

Webhook ID is incorrect.


TriggeredAlert

Alert records triggered for an alert rule.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the triggered alert rule.

alert_rule_id

UUID

-

Unique identifier for the alert rule which needs to be triggered.

alert_run_start_time

int

-

Timestamp of alert rule evaluation in epoch.

alert_time_bucket

int

-

Timestamp pointing to the start of the time bucket in epoch.

alert_value

float

-

Value of the metric for alert_time_bucket.

baseline_time_bucket

Optional[int]

None

Timestamp pointing to the start of the baseline time bucket in epoch, only if alert rule is of 'time period' based comparison.

baseline_value

Optional[float]

None

Value of the metric for baseline_time_bucket.

is_alert

bool

-

Boolean to indicate if alert was supposed to be triggered.

severity

str

-

Severity of alert represented by SeverityEnum, calculated based on value of metric and alert rule thresholds.

failure_reason

str

-

String message if there was a failure sending notification.

message

str

-

String message sent as a part of email notification.

feature_name

Optional[str]

None

Name of feature for which alert was triggered.

alert_record_main_version

int

-

Main version of triggered alert record in int, incremented when the value of severity changes.

alert_record_sub_version

int

-

Sub version of triggered alert record in int, incremented when another alert with same severity as before is triggered.

created_at

datetime

-

Time at which trigger alert rule was created.

updated_at

datetime

-

Latest time at which trigger alert rule was updated.

list()

List alert records triggered for an alert rule.

Parameters

ParameterTypeDefaultDescription

alert_rule_id

UUID

-

Unique identifier for the alert rule which needs to be triggered.

start_time

Optional[datetime]

None

Start time to filter trigger alerts in yyyy-MM-dd format, inclusive.

end_time

Optional[datetime]

None

End time to filter trigger alerts in yyyy-MM-dd format, inclusive.

ordering

Optional[List[str]]

None

List of Alert Rule fields to order by. Eg. [‘alert_time_bucket’] or [‘- alert_time_bucket’] for descending order.

Usage

ALERT_NAME = "YOUR_ALERT_NAME"
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

alerts_list = fdl.AlertRule.list(model_id=model.id)
alert_rule = None

for alert in alerts_list:
    if ALERT_NAME == alert.name:
    alert_rule = alert

alert_records = fdl.AlertRecord.list(
        alert_rule_id=alert_rule.id,
        start_time=datetime(2023, 12, 18),
        end_time=datetime(2023, 12, 25),
    )

Returns

Return TypeDescription

Iterator[AlertRecord]

Iterable of triggered alert rule instances for an alert rule.


Baselines

Baseline datasets are used for making comparisons with production data.

A baseline dataset should be sampled from your model's training set, so it can serve as a representation of what the model expects to see in production.

Baseline

Baseline object contains the below fields.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the baseline.

name

str

-

Baseline name.

type_

str

-

Type of baseline. Type can be static(Pre-production or production) or rolling(production).

start_time

Optional[int]

None

Epoch to be used as start time for STATIC baseline.

end_time

Optional[int]

None

Epoch to be used as end time for STATIC baseline.

offset

Optional[int]

None

Offset in seconds relative to current time to be used for ROLLING baseline.

window_size

Optional[int]

None

Span of window in seconds to be used for ROLLING baseline.

row_count

Optional[int]

None

Number of rows in baseline.

model

-

Details of the model.

project

-

Details of the project to which the baseline belongs.

dataset

-

Details of the dataset from which baseline is derived.

created_at

datetime

-

Time at which baseline was created.

updated_at

datetime

-

Latest time at which baseline was updated.

constructor()

Initialize a new baseline instance.

Parameters

ParameterTypeDefaultDescription

name

str

-

Unique name of the baseline.

model_id

UUID

-

Unique identifier for the model to add baseline to.

environment

-

Type of environment. Can either be PRE_PRODUCTION or PRODUCTION.

type_

str

-

Type of Baseline. Type can be static(Pre-production or production) or rolling(production).

dataset_id

Optional[UUID]

None

Unique identifier for the dataset on which the baseline is created.

start_time

Optional[int]

None

Epoch to be used as start time for STATIC baseline.

end_time

Optional[int]

None

Epoch to be used as end time for STATIC baseline.

offset_delta

Optional[int]

None

Number of times of WindowBinSize to be used for ROLLING baseline. offset = offset_delta * window_bin_size

window_bin_size

Optional[str]

None

Span of window in seconds to be used for ROLLING baseline using WindowBinSize

Usage

BASELINE_NAME = 'YOUR_BASELINE_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
DATASET_NAME = 'YOUR_DATASET_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)
dataset = fdl.Dataset.from_name(name=DATASET_NAME, model_id=model.id)

baseline = fdl.Baseline(
        name=BASELINE_NAME,
        model_id=model.id,
        environment=fdl.EnvType.PRE_PRODUCTION,
        dataset_id=dataset.id,
        type_=fdl.BaselineType.STATIC,
    )

create()

Adds a baseline to Fiddler.

Parameters

No

Usage

BASELINE_NAME = 'YOUR_BASELINE_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
DATASET_NAME = 'YOUR_DATASET_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)
dataset = fdl.Dataset.from_name(name=DATASET_NAME, model_id=model.id)

baseline = fdl.Baseline(
        name=BASELINE_NAME,
        model_id=model.id,
        environment=fdl.EnvType.PRE_PRODUCTION,
        dataset_id=dataset.id,
        type_=fdl.BaselineType.STATIC,
    ).create()

Returns

Return TypeDescription

Baseline instance.

Raises

Error codeIssue

Conflict

Baseline with same name may exist in project .

NotFound

Given dataset may not exist in for the input model.

ValueError

Validation failures like wrong window size, start_time, end_time etc

get()

Get baseline from Fiddler Platform based on UUID.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique identifier for the baseline.

Usage

BASELINE_ID = 'af05646f-0cef-4638-84c9-0d195df2575d'
baseline = fdl.Baseline.get(id_=BASELINE_ID)

Returns

Return TypeDescription

Baseline instance.

Raises

Error codeIssue

NotFound

Baseline with given identifier not found.

Forbidden

Current user may not have permission to view details of baseline.


from_name()

Get baseline from Fiddler Platform based on name.

Parameters

ParameterTypeDefaultDescription

name

str

-

Name of the baseline.

model_id

UUID | str

-

Unique identifier for the model.

Usage

BASELINE_NAME = 'YOUR_BASELINE_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'


project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)


baseline = fdl.Baseline.from_name(
    name=BASELINE_NAME,
    model_id=model.id
)

Returns

Return TypeDescription

Baseline instance.

Raises

Error codeIssue

NotFound

Baseline with given identifier not found.

Forbidden

Current user may not have permission to view details of baseline.


list()

List all baselines accessible to user.

Parameters

ParameterTypeDefaultDescription

model_id

UUID

-

UUID of the model associated with baseline.

Usage

MODEL_ID = '4531bfd9-2ca2-4a7b-bb5a-136c8da09ca2'
baselines = fdl.Baseline.list(model_id=MODEL_ID)

Returns

Return TypeDescription

Iterable[Baseline]

Iterable of all baseline objects.

Raises

Error codeIssue

Forbidden

Current user may not have permission to view details of baseline.


delete()

Deletes a baseline.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique UUID of the baseline .

Usage

BASELINE_NAME = 'YOUR_BASELINE_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'


project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)
baseline = fdl.Baseline.from_name(name=BASELINE_NAME, model_id=model.id)

baseline.delete()

Returns

None

Raises

Error codeIssue

NotFound

Baseline with given identifier not found.

Forbidden

Current user may not have permission to delete baseline.


CustomMetrics

Customized metrics for your specific use case.

CustomMetric

CustomMetric object contains the below parameters.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the custom metric.

name

str

-

Custom metric name.

model_id

UUID

-

UUID of the model in which the custom metric is being added.

definition

str

-

Definition of the custom metric.

description

Optional[str]

None

Description of the custom metric.

created_at

datetime

-

Time of creation of custom metric.

constructor()

Initialise a new custom metric.

Parameters

ParameterTypeDefaultDescription

name

str

-

Custom metric name.

model_id

UUID

-

UUID of the model in which the custom metric is being added.

definition

str

-

Definition of the custom metric.

description

Optional[str]

None

Description of the custom metric.

Usage

METRIC_NAME = 'YOUR_CUSTOM_METRIC_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

metric = fdl.CustomMetric(
        name=METRIC_NAME,
        model_id=model.id,
        definition="average(if(\"spend_amount\">1000, \"spend_amount\", 0))", #Use Fiddler Query Languge (FQL) to define your custom metrics
        description='Get average spend for users spending over $1000',
    )

get()

Get CustomMetric from Fiddler Platform based on model UUID.

Parameters

ParameterTypeDefaultDescription

model_id

UUID

-

UUID of the model associated with the custom metrics.

Usage

PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

metrics = fdl.CustomMetric.list(model_id=model.id)

Returns

Return TypeDescription

Iterable[CustomMetric]

Iterable of all custom metric objects.

Raises

Error codeIssue

Forbidden

Current user may not have permission to view details of custom metric.

from_name()

Get CustomMetric from Fiddler Platform based on name and model UUID.

Parameters

ParameterTypeDefaultDescription

name

str

-

Name of the custom metric.

model_id

UUID | str

-

Unique identifier for the model.

Usage

METRIC_NAME = 'YOUR_CUSTOM_METRIC_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

metric = fdl.CustomMetric.from_name(
        name=METRIC_NAME,
        model_id=model.id
    )

Returns

Return TypeDescription

Custom Metric instance.

Raises

Error codeIssue

NotFound

Custom metric with given identifier not found.

Forbidden

Current user may not have permission to view details of custom metric.

create()

Creates a custom metric for a model on Fiddler Platform.

Parameters

None

Usage

METRIC_NAME = 'YOUR_CUSTOM_METRIC_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

metric = fdl.CustomMetric(
        name=METRIC_NAME,
        model_id=model.id,
        definition="average(if(\"spend_amount\">1000, \"spend_amount\", 0))", #Use Fiddler Query Languge (FQL) to define your custom metrics
        description='Get average spend for users spending over $1000',
    ).create()

Returns

Return TypeDescription

Custom Metric instance.

Raises

Error codeIssue

Conflict

Custom metric with same name may exist in project .

BadRequest

Invalid definition.

NotFound

Given model may not exist.

delete()

Delete a custom metric.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique UUID of the custom metric.

Usage

METRIC_NAME = 'YOUR_CUSTOM_METRIC_NAME'
PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

metric = fdl.CustomMetric.from_name(name=METRIC_NAME, model_id=model.id)
metric.delete()

Returns

No

Raises

Error codeIssue

NotFound

Custom metric with given identifier not found.

Forbidden

Current user may not have permission to delete custom metric.


Datasets

Datasets (or baseline datasets) are used for making comparisons with production data.

A baseline dataset should be sampled from your model's training set, so it can serve as a representation of what the model expects to see in production.

For more information, see Uploading a Baseline Dataset.

For guidance on how to design a baseline dataset, see Designing a Baseline Dataset.


Dataset

Dataset object contains the below parameters.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the dataset.

name

str

-

Dataset name.

row_count

Optional[int]

None

Number of rows in dataset.

model

-

Details of the model.

project

-

Details of the project to which the dataset belongs.

organization

-

Details of the organization to which the dataset belongs.


get()

Get dataset from Fiddler Platform based on UUID.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique identifier for the dataset.

Usage

DATASET_ID = 'ba6ec4e4-7188-44c5-ba84-c2cb22b4bb00'
dataset = fdl.Dataset.get(id_=DATASET_ID)

Returns

Return TypeDescription

Dataset instance.

Raises

Error codeIssue

NotFound

Dataset with given identifier not found.

Forbidden

Current user may not have permission to view details of dataset.


from_name()

Get dataset from Fiddler Platform based on name and model UUID.

Usage params

ParameterTypeDefaultDescription

name

str

-

Name of the dataset.

model_id

UUID | str

-

Unique identifier for the model.

Usage

PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
DATASET_NAME = 'YOUR_DATASET_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

DATASET = fdl.Dataset.from_name(name=DATASET_NAME,
                                model_id=model.id
                               )

Returns

Return TypeDescription

Dataset instance.

Raises

Error codeIssue

NotFound

Dataset not found in the given project name.

Forbidden

Current user may not have permission to view details of dataset.


list()

Get a list of all datasets associated to a model.

Parameters

ParameterTypeDefaultDescription

model_id

UUID

-

UUID of the model associated with baseline.

Usage

PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'
DATASET_NAME = 'YOUR_DATASET_NAME'

project = fdl.Project.from_name(name=PROJECT_NAME)
model = fdl.Model.from_name(name=MODEL_NAME, project_id=project.id)

datasets = fdl.Dataset.list(model_id=model.id)

Returns

Return TypeDescription

Iterable[Dataset]

Iterable of all dataset objects.

Raises

Error codeIssue

Forbidden

Current user may not have permission to view details of dataset.


Jobs

Get job details.

Job

Job object contains the below fields.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the job.

name

str

-

Name of the job.

status

str

-

Current status of job.

progress

float

-

Progress of job completion.

info

dict

-

Dictionary containing resource_type, resource_name, project_name.

error_message

Optional[str]

None

Message for job failure, if any.

error_reason

Optional[str]

None

Reason for job failure, if any.

extras

Optional[dict]

None

Metadata regarding the job.

get()

Get the job instance using job UUID.

Parameters

ParameterTypeDefaultDescription

id_

UUID

-

Unique UUID of the project to which model is associated.

verbose

bool

False

Flag to get extras metadata about the tasks executed.

Usage

JOB_ID = '1531bfd9-2ca2-4a7b-bb5a-136c8da09ca1'
job = fdl.Job.get(id_=JOB_ID)

Returns

Return TypeDescription

Single job object for the input params.

Raises

Error codeIssue

Forbidden

Current user may not have permission to view details of job.


wait()

Wait for job to complete either with success or failure status.

Parameters

ParameterTypeDefaultDescription

interval

Optional[int]

3

Interval in seconds between polling for job status.

timeout

Optional[int]

1800

Timeout in seconds for iterator to stop.

Usage

JOB_ID = '1531bfd9-2ca2-4a7b-bb5a-136c8da09ca1'
job = fdl.Job.get(id_=JOB_ID)
job.wait()

Returns

Return TypeDescription

Single job object for the input params.

Raises

Error codeIssue

Forbidden

Current user may not have permission to view details of job.

TimeoutError

When the default time out of 1800 secs.

watch()

Watch job status at given interval and yield job object.

Parameters

ParameterTypeDefaultDescription

interval

Optional[int]

3

Interval in seconds between polling for job status.

timeout

Optional[int]

1800

Timeout in seconds for iterator to stop.

Usage

JOB_ID = '1531bfd9-2ca2-4a7b-bb5a-136c8da09ca1'
job = fdl.Job.get(id_=JOB_ID)
job.watch()

Returns

Return TypeDescription

Iterator[Job]

Iterator of job objects.

Raises

Error codeIssue

Forbidden

Current user may not have permission to view details of job.

TimeoutError

When the default time out of 1800 secs.


Models

A model is a representation of your machine learning model. Each model can be used for monitoring, explainability, and fairness capabilities.

You do not need to upload your model artifact in order to onboard your model, but doing so will significantly improve the quality of explanations generated by Fiddler.

Model

Model object contains the below parameters.

ParameterTypeDefaultDescription

id

UUID

-

Unique identifier for the model.

name

str

-

Unique name of the model (only alphanumeric and underscores are allowed).

input_type

ModelInputType.TABULAR

Input data type used by the model.

task

ModelTask.NOT_SET

Task the model is designed to address.

task_params

-

Task parameters given to a particular model.

schema

-

Model schema defines the details of each column.

version

Optional[str]

-

Unique version name within a model

spec

-

Model spec defines how model columns are used along with model task.

description

str

-

Description of the model.

event_id_col

str

-

Column containing event id.

event_ts_col

str

-

Column containing event timestamp.

event_ts_format

str

-

Event time stamp format.

xai_params

-

Explainability parameters of the model.

artifact_status

str

-

Artifact Status of the model.

artifact_files

list[dict]

-

Dictionary containing file details of model artifact.

is_binary_ranking_model

bool

-

True if model is ModelTask.RANKING and has only 2 target classes.

created_at

datetime

-

Time at which model was created.

updated_at

datetime

-

Latest time at which model was updated.

created_by

-

Details of the who created the model.

updated_by

-

Details of the who last updated the model.

project

-

Details of the project to which the model belongs.

organization

-

Details of the organization to which the model belongs.

constructor()

Initialize a new model instance.

Parameters

ParameterTypeDefaultDescription

name

str

-

Unique name of the model

project_id

UUID

-

Unique identifier for the project to which model belongs.

input_type

ModelInputType.TABULAR

Input data type used by the model.

task

ModelTask.NOT_SET

Task the model is designed to address.

schema

-

Model schema defines the details of each column.

spec

-

Model spec defines how model columns are used along with model task.

version

Optional[str]

-

Unique version name within a model

task_params

-

Task parameters given to a particular model.

description

str

-

Description of the model.

event_id_col

str

-

Column containing event id.

event_ts_col

str

-

Column containing event timestamp.

event_ts_format

str

-

Event time stamp format.

xai_params

-

Explainability parameters of the model.

from_data()

Build model instance from the given dataframe or file(csv/parquet).

Parameters

ParameterTypeDefaultDescription

source

pd.DataFrame | Path | str

-

Pandas dataframe or path to csv/parquet file

name

str

-

Unique name of the model

project_id

UUID | str

-

Unique identifier for the project to which model belongs.

input_type

ModelInputType.TABULAR

Input data type used by the model.

task

ModelTask.NOT_SET

Task the model is designed to address.

spec

-

Model spec defines how model columns are used along with model task.

version

Optional[str]

-

Unique version name within a model

task_params

-

Task parameters given to a particular model.

description

Optional[str]

-

Description of the model.

event_id_col

Optional[str]

-

Column containing event id.

event_ts_col

Optional[str]

-

Column containing event timestamp.

event_ts_format

Optional[str]

-

Event time stamp format.

xai_params

-

Explainability parameters of the model.

max_cardinality

Optional[int]

None

Max cardinality to detect categorical columns.

sample_size

Optional[int]

-

No. of samples to use for generating schema.

Usage

MODEL_NAME = 'example_model'
PROJECT_ID = '1531bfd9-2ca2-4a7b-bb5a-136c8da09ca1'
MODEL_SPEC = {
  'custom_features': [],
  'decisions': ['Decisions'],
  'inputs': [
    'CreditScore',
    'Geography',
  ],
  'metadata': [],
  'outputs': ['probability_churned'],
  'schema_version': 1,
  'targets': ['Churned'],
}

# Without version
model = fdl.Model.from_data(
  source=<file_path>,
  name=MODEL_NAME,
  project_id=PROJECT_ID,
  spec=fdl.ModelSpec(**MODEL_SPEC),
)

# With version
model = fdl.Model.from_data(
  source=<file_path>,
  name=MODEL_NAME,
  version='v2',
  project_id=PROJECT_ID,
  spec=fdl.ModelSpec(**MODEL_SPEC),
)

Returns

Return TypeDescription

Model instance.

Notes

  • from_data will not create a model entry on Fiddler Platform. Instead this method only returns a model instance which can be edited, call .create() to onboard the model to Fiddler Platform.

  • spec is optional to from_data method. However, a spec with at least inputs is required for model onboarding.

  • Make sure spec is passed to from_data method if model requires custom features. This method generates centroids which are needed for custom feature drift computation

  • If version is not explicitly passed, Fiddler Platform will treat it as v1 version of the model.

create()

Onboard a new model to Fiddler Platform

Parameters

No

Usage

model = fdl.Model.from_data(...)
model.create()

Returns

Return TypeDescription

Model instance.

Raises

Error codeIssue

Conflict

Model with same name may exist in project .

get()

Get model from Fiddler Platform based on UUID.

Parameters

ParameterTypeDefaultDescription

id_

UUID | str

-

Unique identifier for the model.

Returns

Return TypeDescription

Model instance.

Raises

Error codeIssue

NotFound

Model with given identifier not found.

Forbidden

Current user may not have permission to view details of model.

Usage

MODEL_ID = '4531bfd9-2ca2-4a7b-bb5a-136c8da09ca2'
model = fdl.Model.get(id_=MODEL_ID)

from_name()

Get model from Fiddler Platform based on name and project UUID.

Parameters

ParameterTypeDefaultDescription

name

str

-

Name of the model.

project_id

UUID | str

-

Unique identifier for the project.

version

Optiona[str]

-

Unique version name within a model

version parameter is available from fiddler-client==3.1 onwards

\

Usage

PROJECT_NAME = 'YOUR_PROJECT_NAME'
MODEL_NAME = 'YOUR_MODEL_NAME'

PROJECT = fdl.Project.from_name(name=PROJECT_NAME)

# Without version
MODEL = fdl.Model.from_name(name=MODEL_NAME, project_id=PROJECT.id)

# With version
MODEL = fdl.Model.from_name(name=MODEL_NAME, project_id=PROJECT.id, version='v2')

Returns

Return TypeDescription

Model instance.

Notes

  • When the version is not passed, then the model created without any version will be fetched. Fiddler internally assigns version=v1 when not passed.

  • When the version is passed, method will fetch the model corresponding to that specific version.

\

Raises

Error codeIssue

NotFound

Model not found in the given project name.

Forbidden

Current user may not have permission to view details of model.

list()

Gets all models of a project.

Parameters

ParameterTypeDefaultDescription

project_id

Optional[UUID]

-

Unique UUID of the project to which model is associated.

name

Optiona[str]

-

Model name. Pass this to fetch all versions of a model.

Returns

Return TypeDescription

Iterable[Model Compact]

Iterable of model compact objects.

Errors