API Methods 3.x
Alerts
AlertRule
AlertRule object contains the below fields.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier of the AlertRule. |
name | str | - | Unique name of the AlertRule. |
model | - | The associated model details. | |
project | - | The associated project details | |
baseline | Optional[Baseline] | None | The associated baseline. |
segment | Optional[Segment] | None | Details of segment for the alert. |
priority | Union[str, Priority] | - | To set the priority for the AlertRule. Select from: 1. Priority.LOW 2. Priority.MEDIUM 3. Priority.HIGH. |
compare_to | Union[str, CompareTo] | - | Select from the two: 1. CompareTo.RAW_VALUE 2. CompareTo.TIME_PERIOD |
metric_id | Union[str, UUID] | - | Type of alert metric UUID or string denoting metric ID. |
critical_threshold | float | - | Critical alert is triggered when this value satisfies the condition to the selected metric_id. |
condition | Union[str, AlertCondition] | - | Select from: 1. AlertCondition.LESSER 2. AlertCondition.GREATER |
bin_size | Union[str, BinSize] | - | Bin size for example fdl.BinSize.HOUR. |
columns | Optional[List[str]] | None | List of 1 or more column names for the rule to evaluate. Use ['__ANY__'] to evaluate all columns. |
baseline_id | Optional[UUID] | None | UUID of the baseline for the alert. |
segment_id | Optional[UUID] | None | UUID of segment for the alert |
compare_bin_delta | Optional[int] | None | Indicates previous period for comparison e.g. for fdl.BinSize.DAY, compare_bin_delta=1 will compare 1 day back, compare_bin_delta=7 will compare 7 days back. |
warning_threshold | Optional[float] | None | Warning alert is triggered when this value satisfies the condition to the selected metric_id. |
created_at | datetime | - | The creation timestamp. |
updated_at | datetime | - | The timestampe of most recent update. |
evaluation_delay | int | 0 | Specifies a delay in hours before AlertRule is evaluated. The delay period must not exceed one year(8760 hours). |
constructor()
Initialize a new AlertRule on Fiddler Platform.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Unique name of the model |
model_id | UUID | - | Details of the model. |
metric_id | Union[str, UUID] | - | Type of alert metric UUID or enum. |
columns | Optional[List[str]] | None | List of column names on which AlertRule is to be created. It can take ['__ANY__'] to check for all columns. |
baseline_id | Optional[UUID] | None | UUID of the baseline for the alert. |
segment_id | Optional[UUID] | None | UUID of the segment for the alert. |
priority | Union[str, Priority] | - | To set the priority for the AlertRule. Select from: 1. Priority.LOW 2. Priority.MEDIUM 3. Priority.HIGH. |
compare_to | Union[str, CompareTo] | - | Select from the two: 1. CompareTo.RAW_VALUE (absolute alert) 2. CompareTo.TIME_PERIOD (relative alert) |
compare_bin_delta | Optional[int] | None | Compare the metric to a previous time period in units of bin_size. |
warning_threshold | Optional[float] | None | Threshold value to crossing which a warning level severity alert will be triggered. |
critical_threshold | float | - | Threshold value to crossing which a critical level severity alert will be triggered. |
condition | Union[str, AlertCondition] | - | Select from: 1. AlertCondition.LESSER 2. AlertCondition.GREATER |
bin_size | Union[str, BinSize] | - | Size of the bin for AlertRule. |
evaluation_delay | int | 0 | To introduce a delay in the evaluation of the alert, specifying the duration in hours. The delay period must not exceed one year(8760 hours). |
Usage
create()
Create a new AlertRule.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
AlertRule instance. |
get()
Get a single AlertRule.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique identifier for the AlertRule. |
Usage
Returns
Return Type | Description |
---|---|
AlertRule instance. |
Raises
Error code | Issue |
---|---|
NotFound | AlertRule with given identifier not found. |
Forbidden | Current user may not have permission to view details of AlertRule. |
list()
Get a list of AlertRules .
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_id | Union[str, UUID] | None | Unique identifier for the model to which AlertRule belongs. |
project_id | Optional[UUID] | None | Unique identifier for the project to which AlertRule belongs |
metric_id | Optional[UUID] | None | Type of alert metric UUID or enum. |
columns | Optional[List[str]] | None | List of column names on which AlertRule is to be created. It can take ['ANY'] to check for all columns. |
baseline_id | Optional[UUID] | None | UUID of the baseline for the AlertRule. |
ordering | Optional[List[str]] | None | List of AlertRule fields to order by. Eg. [‘alert_time_bucket’] or [‘- alert_time_bucket’] for descending order. |
Usage
Returns
Return Type | Description |
---|---|
Iterator[AlertRule] | Iterator of AlertRule instances. |
delete()
Delete an existing AlertRule.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the AlertRule . |
Usage
Returns
No
Raises
Error code | Issue |
---|---|
NotFound | AlertRule with given identifier not found. |
Forbidden | Current user may not have permission to view details of AlertRule. |
enable_notifications()
Enable an AlertRule's notification.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the AlertRule . |
Usage
Returns
None
Raises
Error code | Issue |
---|---|
NotFound | AlertRule with given identifier not found. |
Forbidden | Current user may not have permission to view details of AlertRule. |
disable_notifications()
Disable notifications for an AlertRule.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the AlertRule . |
Usage
Returns
None
Raises
Error code | Issue |
---|---|
NotFound | AlertRule with given identifier not found. |
Forbidden | Current user may not have permission to view details of AlertRule. |
Alert Notifications
Alert notifications for an AlertRule.
Parameter | Type | Default | Description |
---|---|---|---|
emails | Optional[List[str]] | None | List of emails to send notification to. |
pagerduty_services | Optional[List[str]] | None | List of pagerduty services to trigger the alert to. |
pagerduty_severity | Optional[str] | None | Severity of pagerduty. |
webhooks | Optional[List[UUID]] | None | List of webhook UUIDs. |
set_notification_config()
Set NotificationConfig for an AlertRule.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
emails | Optional[List[str]] | None | List of emails to send notification to. |
pagerduty_services | Optional[List[str]] | None | List of pagerduty services to trigger the alert to. |
pagerduty_severity | Optional[str] | None | Severity of pagerduty. |
webhooks | Optional[List[UUID]] | None | List of webhook UUIDs. |
Usage
Returns
Return Type | Description |
---|---|
NotificationConfig | Alert notification settings for an AlertRule. |
If
pagerduty_severity
is passed without specifyingpagerduty_services
then thepagerduty_severity
is ignored.
Raises
Error code | Issue |
---|---|
BadRequest | All 4 input parameters are empty. |
ValueError | Webhook ID is incorrect. |
get_notification_config()
Get notification configuration for an AlertRule.
Parameters
None
Usage
Returns
Return Type | Description |
---|---|
NotificationConfig | Alert notification settings for an AlertRule. |
Raises
Error code | Issue |
---|---|
BadRequest | All 4 input parameters are empty. |
ValueError | Webhook ID is incorrect. |
Triggered Alerts
AlertRecord
An AlertRecord details an AlertRule's triggered alert.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the triggered AlertRule. |
alert_rule_id | UUID | - | Unique identifier for the AlertRule which needs to be triggered. |
alert_run_start_time | int | - | Timestamp of AlertRule evaluation in epoch. |
alert_time_bucket | int | - | Timestamp pointing to the start of the time bucket in epoch. |
alert_value | float | - | Value of the metric for alert_time_bucket. |
baseline_time_bucket | Optional[int] | None | Timestamp pointing to the start of the baseline time bucket in epoch, only if AlertRule is of 'time period' based comparison. |
baseline_value | Optional[float] | None | Value of the metric for baseline_time_bucket. |
is_alert | bool | - | Boolean to indicate if alert was supposed to be triggered. |
severity | str | - | Severity of alert represented by Severity, calculated based on value of metric and AlertRule thresholds. |
failure_reason | str | - | String message if there was a failure sending notification. |
message | str | - | String message sent as a part of email notification. |
feature_name | Optional[str] | None | Name of feature for which alert was triggered. |
alert_record_main_version | int | - | Main version of triggered alert record in int, incremented when the value of severity changes. |
alert_record_sub_version | int | - | Sub version of triggered alert record in int, incremented when another alert with same severity as before is triggered. |
created_at | datetime | - | Time at which trigger AlertRule was created. |
updated_at | datetime | - | Latest time at which trigger AlertRule was updated. |
list()
List AlertRecords triggered for an AlertRule.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
alert_rule_id | UUID | - | Unique identifier for the AlertRule which needs to be triggered. |
start_time | Optional[datetime] | None | Start time to filter trigger alerts in yyyy-MM-dd format, inclusive. |
end_time | Optional[datetime] | None | End time to filter trigger alerts in yyyy-MM-dd format, inclusive. |
ordering | Optional[List[str]] | None | List of AlertRule fields to order by. Eg. [‘alert_time_bucket’] or [‘- alert_time_bucket’] for descending order. |
Usage
Returns
Return Type | Description |
---|---|
Iterator[AlertRecord] | Iterable of triggered AlertRule instances for an AlertRule. |
Baselines
Baseline datasets are used for making comparisons with production data.
A baseline dataset should be sampled from your model's training set, so it can serve as a representation of what the model expects to see in production.
Baseline
Baseline object contains the below fields.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the baseline. |
name | str | - | Baseline name. |
type_ | - | Baseline type can be static (Pre-production or production) or rolling(production). | |
start_time | Optional[int] | None | Epoch to be used as start time for STATIC baseline. |
end_time | Optional[int] | None | Epoch to be used as end time for STATIC baseline. |
offset | Optional[int] | None | Offset in seconds relative to current time to be used for ROLLING baseline. |
window_size | Optional[int] | None | Span of window in seconds to be used for ROLLING baseline. |
row_count | Optional[int] | None | Number of rows in baseline. |
model | - | Details of the model. | |
project | - | Details of the project to which the baseline belongs. | |
dataset | - | Details of the dataset from which baseline is derived. | |
created_at | datetime | - | Time at which baseline was created. |
updated_at | datetime | - | Latest time at which baseline was updated. |
constructor()
Initialize a new baseline instance.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Unique name of the baseline. |
model_id | UUID | - | Unique identifier for the model to add baseline to. |
environment | - | Type of environment. Can either be PRE_PRODUCTION or PRODUCTION. | |
type_ | - | Baseline type can be static (pre-production or production) or rolling(production). | |
dataset_id | Optional[UUID] | None | Unique identifier for the dataset on which the baseline is created. |
start_time | Optional[int] | None | Epoch to be used as start time for STATIC baseline. |
end_time | Optional[int] | None | Epoch to be used as end time for STATIC baseline. |
offset_delta | Optional[int] | None | Number of times of WindowBinSize to be used for ROLLING baseline. offset = offset_delta * window_bin_size |
window_bin_size | Optional[str] | None | Span of window in seconds to be used for ROLLING baseline using WindowBinSize |
Usage
create()
Adds a baseline to Fiddler.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Baseline instance. |
Raises
Error code | Issue |
---|---|
Conflict | Baseline with same name may exist in project . |
NotFound | Given dataset may not exist in for the input model. |
ValueError | Validation failures like wrong window size, start_time, end_time etc |
get()
Get baseline from Fiddler Platform based on UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique identifier for the baseline. |
Usage
Returns
Return Type | Description |
---|---|
Baseline instance. |
Raises
Error code | Issue |
---|---|
NotFound | Baseline with given identifier not found. |
Forbidden | Current user may not have permission to view details of baseline. |
from_name()
Get baseline from Fiddler Platform based on name.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Name of the baseline. |
model_id | UUID | str | - | Unique identifier for the model. |
Usage
Returns
Return Type | Description |
---|---|
Baseline instance. |
Raises
Error code | Issue |
---|---|
NotFound | Baseline with given identifier not found. |
Forbidden | Current user may not have permission to view details of baseline. |
list()
List all baselines accessible to user.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_id | UUID | - | UUID of the model associated with baseline. |
Usage
Returns
Return Type | Description |
---|---|
Iterable[Baseline] | Iterable of all baseline objects. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of baseline. |
delete()
Deletes a baseline.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the baseline . |
Usage
Returns
None
Raises
Error code | Issue |
---|---|
NotFound | Baseline with given identifier not found. |
Forbidden | Current user may not have permission to delete baseline. |
Custom Metrics
User-defined metrics to extend Fiddler's built-in metrics.
CustomMetric
CustomMetric object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the custom metric. |
name | str | - | Custom metric name. |
model_id | UUID | - | UUID of the model in which the custom metric is being added. |
definition | str | - | Definition of the custom metric. |
description | Optional[str] | None | Description of the custom metric. |
created_at | datetime | - | Time of creation of custom metric. |
constructor()
Initialise a new custom metric.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Custom metric name. |
model_id | UUID | - | UUID of the model in which the custom metric is being added. |
definition | str | - | Definition of the custom metric. |
description | Optional[str] | None | Description of the custom metric. |
Usage
get()
Get CustomMetric from Fiddler Platform based on model UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_id | UUID | - | UUID of the model associated with the custom metrics. |
Usage
Returns
Return Type | Description |
---|---|
Iterable[CustomMetric] | Iterable of all custom metric objects. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of custom metric. |
from_name()
Get CustomMetric from Fiddler Platform based on name and model UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Name of the custom metric. |
model_id | UUID | str | - | Unique identifier for the model. |
Usage
Returns
Return Type | Description |
---|---|
Custom Metric instance. |
Raises
Error code | Issue |
---|---|
NotFound | Custom metric with given identifier not found. |
Forbidden | Current user may not have permission to view details of custom metric. |
create()
Creates a custom metric for a model on Fiddler Platform.
Parameters
None
Usage
Returns
Return Type | Description |
---|---|
Custom Metric instance. |
Raises
Error code | Issue |
---|---|
Conflict | Custom metric with same name may exist in project . |
BadRequest | Invalid definition. |
NotFound | Given model may not exist. |
delete()
Delete a custom metric.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the custom metric. |
Usage
Returns
No
Raises
Error code | Issue |
---|---|
NotFound | Custom metric with given identifier not found. |
Forbidden | Current user may not have permission to delete custom metric. |
Datasets
Datasets (or baseline datasets) are used for making comparisons with production data.
Dataset
Dataset object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the dataset. |
name | str | - | Dataset name. |
row_count | int | None | Number of rows in dataset. |
model_id | - | Unique identifier of the associated model | |
project_id | - | Unique identifier of the associated project |
get()
Get dataset from Fiddler Platform based on UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique identifier for the dataset. |
Usage
Returns
Return Type | Description |
---|---|
Dataset instance. |
Raises
Error code | Issue |
---|---|
NotFound | Dataset with given identifier not found. |
Forbidden | Current user may not have permission to view details of dataset. |
from_name()
Get dataset from Fiddler Platform based on name and model UUID.
Usage params
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Name of the dataset. |
model_id | UUID | str | - | Unique identifier for the model. |
Usage
Returns
Return Type | Description |
---|---|
Dataset instance. |
Raises
Error code | Issue |
---|---|
NotFound | Dataset not found in the given project name. |
Forbidden | Current user may not have permission to view details of dataset. |
list()
Get a list of all datasets associated to a model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_id | UUID | - | UUID of the model associated with baseline. |
Usage
Returns
Return Type | Description |
---|---|
Iterable[Dataset] | Iterable of all dataset objects. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of dataset. |
Jobs
A Job is used to track asynchronous processes such as batch publising of data.
Job
Job object contains the below fields.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the job. |
name | str | - | Name of the job. |
status | str | - | Current status of job. |
progress | float | - | Progress of job completion. |
info | dict | - | Dictionary containing resource_type, resource_name, project_name. |
error_message | Optional[str] | None | Message for job failure, if any. |
error_reason | Optional[str] | None | Reason for job failure, if any. |
extras | Optional[dict] | None | Metadata regarding the job. |
get()
Get the job instance using job UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the project to which model is associated. |
verbose | bool | False | Flag to get |
Usage
Returns
Return Type | Description |
---|---|
Single job object for the input params. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of job. |
wait()
Wait for job to complete either with success or failure status.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
interval | Optional[int] | 3 | Interval in seconds between polling for job status. |
timeout | Optional[int] | 1800 | Timeout in seconds for iterator to stop. |
Usage
Returns
Return Type | Description |
---|---|
Single job object for the input params. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of job. |
TimeoutError | When the default time out of 1800 secs. |
watch()
Watch job status at given interval and yield job object.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
interval | Optional[int] | 3 | Interval in seconds between polling for job status. |
timeout | Optional[int] | 1800 | Timeout in seconds for iterator to stop. |
Usage
Returns
Return Type | Description |
---|---|
Iterator[Job] | Iterator of job objects. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of job. |
TimeoutError | When the default time out of 1800 secs. |
Models
A Model is a representation of your machine learning model which can be used for monitoring, explainability, and more. You do not need to upload your model artifact in order to onboard your model, but doing so will significantly improve the quality of explanations generated by Fiddler.
Model
Model object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the model. |
name | str | - | Unique name of the model (only alphanumeric and underscores are allowed). |
input_type | ModelInputType.TABULAR | Input data type used by the model. | |
task | ModelTask.NOT_SET | Task the model is designed to address. | |
task_params | - | Task parameters given to a particular model. | |
schema | - | Model schema defines the details of each column. | |
version | Optional[str] | - | Unique version name within a model |
spec | - | Model spec defines how model columns are used along with model task. | |
description | str | - | Description of the model. |
event_id_col | str | - | Column containing event id. |
event_ts_col | str | - | Column containing event timestamp. |
xai_params | - | Explainability parameters of the model. | |
artifact_status | str | - | Artifact Status of the model. |
artifact_files | list[dict] | - | Dictionary containing file details of model artifact. |
is_binary_ranking_model | bool | - | True if model is ModelTask.RANKING and has only 2 target classes. |
created_at | datetime | - | Time at which model was created. |
updated_at | datetime | - | Latest time at which model was updated. |
created_by | - | Details of the who created the model. | |
updated_by | - | Details of the who last updated the model. | |
project | - | Details of the project to which the model belongs. | |
organization | - | Details of the organization to which the model belongs. |
constructor()
Initialize a new model instance.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Unique name of the model |
project_id | UUID | - | Unique identifier for the project to which model belongs. |
input_type | ModelInputType.TABULAR | Input data type used by the model. | |
task | ModelTask.NOT_SET | Task the model is designed to address. | |
schema | - | Model schema defines the details of each column. | |
spec | - | Model spec defines how model columns are used along with model task. | |
version | Optional[str] | - | Unique version name within a model |
task_params | - | Task parameters given to a particular model. | |
description | str | - | Description of the model. |
event_id_col | str | - | Column containing event id. |
event_ts_col | str | - | Column containing event timestamp. |
xai_params | - | Explainability parameters of the model. |
from_data()
Build model instance from the given dataframe or file(csv/parquet).
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
source | pd.DataFrame | Path | str | - | Pandas dataframe or path to csv/parquet file |
name | str | - | Unique name of the model |
project_id | UUID | str | - | Unique identifier for the project to which model belongs. |
input_type | ModelInputType.TABULAR | Input data type used by the model. | |
task | ModelTask.NOT_SET | Task the model is designed to address. | |
spec | - | Model spec defines how model columns are used along with model task. | |
version | Optional[str] | - | Unique version name within a model |
task_params | - | Task parameters given to a particular model. | |
description | Optional[str] | - | Description of the model. |
event_id_col | Optional[str] | - | Column containing event id. |
event_ts_col | Optional[str] | - | Column containing event timestamp. |
xai_params | - | Explainability parameters of the model. | |
max_cardinality | Optional[int] | None | Max cardinality to detect categorical columns. |
sample_size | Optional[int] | - | No. of samples to use for generating schema. |
Usage
Returns
Return Type | Description |
---|---|
Model instance. |
Notes
from_data
will not create a model entry on Fiddler Platform. Instead this method only returns a model instance which can be edited, call.create()
to onboard the model to Fiddler Platform.spec
is optional tofrom_data
method. However, aspec
with at leastinputs
is required for model onboarding.Make sure
spec
is passed tofrom_data
method if model requires custom features. This method generates centroids which are needed for custom feature drift computationIf
version
is not explicitly passed, Fiddler Platform will treat it asv1
version of the model.
create()
Onboard a new model to Fiddler Platform
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Model instance. |
Raises
Error code | Issue |
---|---|
Conflict | Model with same name may exist in project . |
get()
Get model from Fiddler Platform based on UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | str | - | Unique identifier for the model. |
Returns
Return Type | Description |
---|---|
Model instance. |
Raises
Error code | Issue |
---|---|
NotFound | Model with given identifier not found. |
Forbidden | Current user may not have permission to view details of model. |
Usage
from_name()
Get model from Fiddler Platform based on name and project UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Name of the model. |
project_id | UUID | str | - | Unique identifier for the project. |
version | Optiona[str] | - | Unique version name within a model |
version
parameter is available fromfiddler-client==3.1
onwards
Usage
Returns
Return Type | Description |
---|---|
Model instance. |
Notes
When the version is not passed, then the model created without any version will be fetched. Fiddler internally assigns version=v1 when not passed.
When the version is passed, method will fetch the model corresponding to that specific version.
Raises
Error code | Issue |
---|---|
NotFound | Model not found in the given project name. |
Forbidden | Current user may not have permission to view details of model. |
list()
Gets all models of a project.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
project_id | Optional[UUID] | - | Unique UUID of the project to which model is associated. |
name | Optiona[str] | - | Model name. Pass this to fetch all versions of a model. |
Returns
Return Type | Description |
---|---|
Iterable[Model Compact] | Iterable of model compact objects. |
Errors
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to the given project. |
Usage example
Notes
Since
Model
contains a lot of information, list operations does not return all the fields of a model. Instead this method returnsModelCompact
objects on which.fetch()
can be called to get the completeModel
instance. For most of the use-cases,ModelCompact
objects are sufficient.
update()
Update an existing model. Only following fields are allowed to be updated, backend will ignore if any other field is updated on the instance.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
version | Optional[str] | None | Model version name |
xai_params | Optional[XaiParams] | None | Explainability parameters of the model. |
description | Optional[str] | None | Description of the model. |
event_id_col | Optional[str] | None | Column containing event id. |
event_ts_col | Optional[str] | None | Column containing event timestamp. |
version
parameter is available fromfiddler-client==3.1
onwards
Usage
Returns
No
Raises
Error code | Issue |
---|---|
BadRequest | If field is not updatable. |
duplicate()
Duplicate the model instance with the given version name.
This call will not save the model on Fiddler Platform. After making changes to the model instance, call .create()
to add the model version to Fiddler Platform.
This method is available from
fiddler-client==3.1
onwards.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
version | Optional[str] | None | Model version name |
Usage
Returns
Return Type | Description |
---|---|
Model instance. |
No
Raises
Error code | Issue |
---|---|
BadRequest | If field is not updatable. |
delete()
Delete a model.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Async job details for the delete job. |
Notes
Model deletion is an async process, hence a job object is returned on
delete()
call. Calljob.wait()
to wait for the job to complete. If you are planning to create a model with the same name, please wait for the job to complete, otherwise backend will not allow new model with same name.
add_surrogate()
Add surrogate existing model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
dataset_id | UUID | str | - | Dataset identifier |
deployment_params | Optional[DeploymentParams] | - | Model deployment parameters. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the add surrogate job. |
Raises
Error code | Issue |
---|---|
BadRequest | Invalid deployment parameter is passed |
update_surrogate()
Update surrogate existing model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
dataset_id | UUID | str | - | Dataset identifier |
deployment_params | Optional[DeploymentParams] | None | Model deployment parameters. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the update surrogate job. |
add_artifact()
Add artifact files to existing model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_dir | str | - | Path to directory containing artifacts for upload. |
deployment_params | Optional[DeploymentParams] | None | Model deployment parameters. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the add artifact job. |
update_artifact()
Update existing artifact files in a model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_dir | str | - | Path to directory containing artifacts for upload. |
deployment_params | Optional[DeploymentParams] | None | Model deployment parameters. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the add artifact job. |
download_artifact()
Download existing artifact files in a model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
output_dir | str | - | Path to directory to download the artifacts. |
Usage
Returns
No
Properties
datasets
List all datasets associated with a model.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Iterable[Dataset] | Iterable of dataset instances. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of model. |
model_deployment
Get the model deployment object associated with the model.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Model deployment instance. |
Raises
Error code | Issue |
---|---|
NotFound | Model with given identifier not found. |
Forbidden | Current user may not have permission to view details of model. |
publish()
Publish Pre-production or production events.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
source | Union[list[dict[str, Any]], str, Path, pd.DataFrame] | - | Source can be: 1. Path or str path: path for data file. 2. list[dict]: list of event dicts. EnvType.PRE_PRODUCTION not supported. 3. dataframe: events dataframe. |
environment | EnvType | EnvType.PRODUCTION | Either EnvType.PRE_PRODUCTION or EnvType.PRODUCTION |
dataset_name | Optional[str] | None | Name of the dataset. Not supported for EnvType.PRODUCTION |
update | Optional[bool] | False | If True, the events data passed in the publish call will be used to update previously published event records matched by their event_ids (note that only updating target and metadata columns is supported). For more details refer to Updating Events |
Usage
Pre-requisite
Publish dataset (pre-production data) from file
Publish dataset (pre-production data) from dataframe
Publish production events from list
List is only supported for production data but not for pre-production.
Events are published as a stream. This mode is recommended If you have a high volume of continuous real-time traffic of events, as it allows for more efficient processing on our backend.
It returns a list of event_id
for each of the published events.
Notes
In this example where
model.event_id_col
=event_id
, we expectevent_id
as the required key of the dictionary. Otherwise if you keepmodel.event_id_col=None
, our backend will generate unique event ids and return these back to you. Same formodel.event_ts_col
, we assign current time as event timestamp in case ofNone
.
Publish production events from file
Batch events is faster if you want to publish a large-scale set of historical data.
Publish production events from dataframe
Update events
if you need to update the target or metadata columns for a previously published production event, set update
=True. For more details please refer to Updating Events. Note only production events can be updated.
Update production events from list
Update production events from dataframe
Returns
In case of streaming publish
Return Type | Source | Description |
---|---|---|
list[UUID|str] | list[dict] | List of event identifier |
In case of batch publish
Return Type | Source | Description |
---|---|---|
Union[str, Path, pd.DataFrame] | Job object for file/dataframe published |
Model Compact
Model object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the model. |
name | str | - | Unique name of the model |
version | Optional[str] | - | Unique version name within a model |
fetch()
Fetch the model instance from Fiddler Platform.
Parameters
No
Returns
Return Type | Description |
---|---|
Model instance. |
Raises
Error code | Issue |
---|---|
NotFound | Model not found for the given identifier |
Forbidden | Current user may not have permission to view details of model. |
Model deployment
Get model deployment object of a particular model.
Model deployment:
Model deployment object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the model. |
model | - | Details of the model. | |
project | - | Details of the project to which the model belongs. | |
organization | - | Details of the organization to which the model belongs. | |
artifact_type | - | Task the model is designed to address. | |
deployment_type | - | Type of deployment of the model. | |
image_uri | Optional[str] | md-base/python/python-311:1.0.0 | Reference to the docker image to create a new runtime to serve the model. Check the available images on the Model Deployment page. |
active | bool | True | Status of the deployment. |
replicas | Optional[str] | 1 | The number of replicas running the model. Minimum value: 1 Maximum value: 10 Default value: 1 |
cpu | Optional[str] | 100 | The amount of CPU (milli cpus) reserved per replica. Minimum value: 10 Maximum value: 4000 (4vCPUs) Default value: 100 |
memory | Optional[str] | 256 | The amount of memory (mebibytes) reserved per replica. Minimum value: 150 Maximum value: 16384 (16GiB) Default value: 256 |
created_at | datetime | - | Time at which model deployment was created. |
updated_at | datetime | - | Latest time at which model deployment was updated. |
created_by | - | Details of the user who created the model deployment. | |
updated_by | - | Details of the user who last updated the model deployment. |
Update model deployment
Update an existing model deployment.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
active | Optional[bool] | True | Status of the deployment. |
replicas | Optional[str] | 1 | The number of replicas running the model. Minimum value: 1 Maximum value: 10 Default value: 1 |
cpu | Optional[str] | 100 | The amount of CPU (milli cpus) reserved per replica. Minimum value: 10 Maximum value: 4000 (4vCPUs) Default value: 100 |
memory | Optional[str] | 256 | The amount of memory (mebibytes) reserved per replica. Minimum value: 150 Maximum value: 16384 (16GiB) Default value: 256 |
Usage
Returns
No
Raises
Error code | Issue |
---|---|
BadRequest | If field is not updatable. |
Organizations
Organization in which all the projects, models are present.
Organization:
Organization object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the organization. |
name | str | - | Unique name of the organization. |
created_at | datetime | - | Time at which organization was created. |
updated_at | datetime | - | Latest time at which organization was updated. |
Projects
Projects are used to organize your models and datasets. Each project can represent a machine learning task (e.g. predicting house prices, assessing creditworthiness, or detecting fraud).
A project can contain one or more models (e.g. lin_reg_house_predict, random_forest_house_predict).
Project
Project object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | None | Unique identifier for the project. |
name | str | None | Unique name of the project. |
created_at | datetime | None | Time at which project was created. |
updated_at | datetime | None | Latest time at which project was updated. |
created_by | None | Details of the who created the project. | |
updated_by | None | Details of the who last updated the project. | |
organization | None | Details of the organization to which the project belongs. |
create()
Creates a project using the specified name.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | None | Unique name of the project. |
Usage
Returns
Return Type | Description |
---|---|
Project instance. |
Raises
Error code | Issue |
---|---|
Conflict | Project with same name may exist. |
get()
Get project from Fiddler Platform based on UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | None | Unique identifier for the project. |
Usage
Returns
Return Type | Description |
---|---|
Project instance. |
Raises
Error code | Issue |
---|---|
NotFound | Project with given identifier not found. |
Forbidden | Current user may not have permission to view details of project. |
from_name()
Get project from Fiddler Platform based on name.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
project_name | str | None | Name of the project. |
Usage
Returns
Return Type | Description |
---|---|
Project instance. |
Raises
Error code | Issue |
---|---|
NotFound | Project not found in the given project name. |
Forbidden | Current user may not have permission to view details of project. |
list()
Gets all projects in an organization.
Parameters
No
Returns
Return Type | Description |
---|---|
Iterable[Project ] | Iterable of project objects. |
Errors
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to the given project. |
Usage example
delete()
Delete a project.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | None | Unique UUID of the project . |
Usage
Returns
None
Properties
List models()
List all models associated with a project.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | None | Unique UUID of the project . |
Usage
Returns
Return Type | Description |
---|---|
Iterable[Model] | Iterable of model objects. |
Raises
Error code | Issue |
---|---|
NotFound | Project with given identifier not found. |
Forbidden | Current user may not have permission to view details of project. |
Segments
Fiddler offers the ability to segment your data based on a custom condition.
Segment
Segment object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the segment. |
name | str | - | Segment name. |
model_id | UUID | - | UUID of the model to which segment belongs. |
definition | str | - | Definition of the segment. |
description | Optional[str] | None | Description of the segment. |
created_at | datetime | - | Time of creation of segment. |
constructor()
Initialise a new segment.
Usage params
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Segment name. |
model_id | UUID | - | UUID of the model to which segment belongs. |
definition | str | - | Definition of the segment. |
description | Optional[str] | None | Description of the segment. |
Usage
get()
Get segment from Fiddler Platform based on UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique identifier for the segment. |
Usage
Returns
Return Type | Description |
---|---|
Segment instance. |
Raises
Error code | Issue |
---|---|
NotFound | Segment with given identifier not found. |
Forbidden | Current user may not have permission to view details of segment. |
from_name()
Get segment from Fiddler Platform based on name and model UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Name of the segment. |
model_id | UUID | str | - | Unique identifier for the model. |
Usage
Returns
Return Type | Description |
---|---|
Segment instance. |
Raises
Error code | Issue |
---|---|
NotFound | Segment with given identifier not found. |
Forbidden | Current user may not have permission to view details of segment. |
list()
List all segments in the given model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
model_id | UUID | - | UUID of the model associated with the segment. |
Usage
Returns
Return Type | Description |
---|---|
Iterable[Segment] | Iterable of all segment objects. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of segment. |
create()
Adds a segment to a model.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Segment instance. |
Raises
Error code | Issue |
---|---|
Conflict | Segment with same name may exist for the model. |
BadRequest | Invalid definition. |
NotFound | Given model may not exist . |
delete()
Delete a segment.
Parameters
No
Usage
Returns
No
Raises
Error code | Issue |
---|---|
NotFound | Segment with given identifier not found. |
Forbidden | Current user may not have permission to delete segment. |
Webhooks
Webhooks integration for alerts to be posted on Slack or other apps.
Webhook()
Webhook object contains the below parameters.
Parameter | Type | Default | Description |
---|---|---|---|
id | UUID | - | Unique identifier for the webhook. |
name | str | - | Unique name of the webhook. |
url | str | - | Webhook integration URL. |
provider | - | App in which the webhook needs to be integrated. Either 'SLACK' or 'OTHER' | |
created_at | datetime | - | Time at which webhook was created. |
updated_at | datetime | - | Latest time at which webhook was updated. |
constructor()
Initialise a new webhook.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Unique name of the webhook. |
url | str | - | Webhook integration URL. |
provider | - | App in which the webhook needs to be integrated. |
Usage
get()
Gets all details of a particular webhook from UUID.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique identifier for the webhook. |
Usage
Returns
Return Type | Description |
---|---|
Webhook instance. |
Raises
Error code | Issue |
---|---|
NotFound | Webhook with given identifier not found. |
Forbidden | Current user may not have permission to view details of webhook. |
from_name()
Get Webhook from Fiddler Platform based on name.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Name of the webhook. |
Usage
Returns
Return Type | Description |
---|---|
Webhook instance. |
Raises
Error code | Issue |
---|---|
NotFound | Webhook with given name not found. |
Forbidden | Current user may not have permission to view details of webhook. |
list()
Gets all webhooks accessible to a user.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Iterable[Webhook] | Iterable of webhook objects. |
Raises
Error code | Issue |
---|---|
Forbidden | Current user may not have permission to view details of webhook. |
create()
Create a new webhook.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Webhook object. |
update()
Update an existing webhook.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
name | str | - | Unique name of the webhook. |
url | str | - | Webhook integration URL. |
provider | - | App in which the webhook needs to be integrated. |
Usage
Returns
None
Raises
Error code | Issue |
---|---|
BadRequest | If field is not updatable. |
delete()
Delete a webhook.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
id_ | UUID | - | Unique UUID of the webhook. |
Usage
Returns
None
Explainability
Explainability methods for models.
precompute_feature_importance
Pre-compute feature importance for a model on a dataset. This is used in various places in the UI. A single feature importance can be precomputed (computed and cached) for a model.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
dataset_id | UUID | - | The unique identifier of the dataset. |
num_samples | Optional[int] | None | The number of samples used. |
num_iterations | Optional[int] | None | The maximum number of ablated model inferences per feature. |
num_refs | Optional[int] | None | The number of reference points used in the explanation. |
ci_level | Optional[float] | None | The confidence level (between 0 and 1). |
update | Optional[bool] | False | Flag to indicate whether the precomputed feature importance should be recomputed and updated. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the pre-compute job . |
get_precomputed_feature_importance
Get pre-computed global feature importance for a model over a dataset or a slice.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Tuple | A named tuple with the feature importance results . |
get_feature_importance()
Get global feature importance for a model over a dataset or a slice.
Usage params
Parameter | Type | Default | Description |
---|---|---|---|
data_source | - | DataSource for the input dataset to compute feature importance on (DatasetDataSource or SqlSliceQueryDataSource). | |
num_iterations | Optional[int] | None | The maximum number of ablated model inferences per feature. |
num_refs | Optional[int] | None | The number of reference points used in the explanation. |
ci_level | Optional[float] | None | The confidence level (between 0 and 1). |
Usage
Returns
Return Type | Description |
---|---|
Tuple | A named tuple with the feature importance results . |
Raises
Error code | Issue |
---|---|
BadRequest | If dataset id is not specified. |
precompute_feature_impact()
Pre-compute feature impact for a model on a dataset. This is used in various places in the UI. A single feature impact can be precomputed (computed and cached) for a model.
Usage params
Parameter | Type | Default | Description |
---|---|---|---|
dataset_id | UUID | - | The unique identifier of the dataset. |
num_samples | Optional[int] | None | The number of samples used. |
num_iterations | Optional[int] | None | The maximum number of ablated model inferences per feature. |
num_refs | Optional[int] | None | The number of reference points used in the explanation. |
ci_level | Optional[float] | None | The confidence level (between 0 and 1). |
min_support | Optional[int] | 15 | Only used for NLP (TEXT inputs) models. Specify a minimum support (number of times a specific word was present in the sample data) to retrieve top words. Default to 15. |
update | Optional[bool] | False | Flag to indicate whether the precomputed feature impact should be recomputed and updated. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the pre-compute job . |
upload_feature_impact()
Upload a custom feature impact for a model of input type TABULAR
. All input features need to be passed for the method to run successfully. Partial upload of feature impacts are not supported.
Usage params
Parameter | Type | Default | Description |
---|---|---|---|
feature_impact_map | dict | - | Feature impacts dictionary with feature name as key and impact as value. Impact value is of type float and can be positive, negative or zero. |
update | Optional[bool] | False | Flag to indicate whether the feature impact is being uploaded or updated. |
Usage
Returns
Return Type | Description |
---|---|
Dict | Dictionary with feature_names, feature_impact_scores, system_generated, model_task, model_input_type, created_at. |
get_precomputed_feature_impact()
Get pre-computed global feature impact for a model over a dataset or a slice.
Parameters
No
Usage
Returns
Return Type | Description |
---|---|
Tuple | A named tuple with the feature impact results . |
get_feature_impact()
Get global feature impact for a model over a dataset or a slice.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
data_source | - | DataSource for the input dataset to compute feature importance on (DatasetDataSource or SqlSliceQueryDataSource). | |
num_iterations | Optional[int] | None | The maximum number of ablated model inferences per feature. |
num_refs | Optional[int] | None | The number of reference points used in the explanation. |
ci_level | Optional[float] | None | The confidence level (between 0 and 1). |
min_support | Optional[int] | 15 | Only used for NLP (TEXT inputs) models. Specify a minimum support (number of times a specific word was present in the sample data)to retrieve top words. Default to 15. |
output_columns | Optional[list[str]] | None | Only used for NLP (TEXT inputs) models. Output column names to compute feature impact on. |
Usage
Returns
Return Type | Description |
---|---|
Tuple | A named tuple with the feature impact results . |
Raises
Error code | Issue |
---|---|
BadRequest | If dataset id is not specified or query is not valid. |
precompute_predictions()
Pre-compute predictions for a model on a dataset.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
dataset_id | UUID | - | Unique identifier of the dataset used for prediction. |
chunk_size | Optional[int] | None | Chunk size for fetching predictions. |
update | Optional[bool] | False | Flag to indicate whether the pre-computed predictions should be re-computed and updated for this dataset. |
Usage
Returns
Return Type | Description |
---|---|
Async job details for the prediction job . |
explain()
Get explanation for a single observation.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
input_data_source | Union[RowDataSource, EventIdDataSource] | - | DataSource for the input data to compute explanation on (RowDataSource, EventIdDataSource). |
ref_data_source | Optional[Union[DatasetDataSource, SqlSliceQueryDataSource]] | None | DataSource for the reference data to compute explanationon (DatasetDataSource, SqlSliceQueryDataSource). Only used for non-text models and the following methods: 'SHAP', 'FIDDLER_SHAP', 'PERMUTE', 'MEAN_RESET'. |
method | Optional[Union[ExplainMethod, str]] | ExplainMethod.FIDDLER_SHAP | Explanation method name. Could be your custom explanation method or one of the following method: 'SHAP', 'FIDDLER_SHAP', 'IG', 'PERMUTE', 'MEAN_RESET', 'ZERO_RESET'. |
num_permutations | Optional[int] | None | For Fiddler SHAP, that corresponds to the number of coalitions to sample to estimate the Shapley values of each single-reference game. For the permutation algorithms, this corresponds to the number of permutations from the dataset to use for the computation. |
ci_level | Optional[float] | None | The confidence level (between 0 and 1) to use for the confidence intervals in Fiddler SHAP. Not used for other methods. |
top_n_class | Optional[int] | None | For multiclass classification models only, specifying if only the n top classes are computed or all classes (when parameter is None). |
Usage
Return params
Return Type | Description |
---|---|
Tuple | A named tuple with the explanation results. |
Raises
Error code | Issue |
---|---|
NotSupported | If specified source type is not supported. |
get_slice()
Fetch data with slice query. 1M rows is the max size that can be fetched.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
query | str | - | An SQL query that begins with the keyword 'SELECT'. |
sample | Optional[bool] | False | Whether rows should be sample or not from the database. |
max_rows | Optional[int] | None | Number of maximum rows to fetch. |
columns | Optional[list[str]] | None | Allows caller to explicitly specify list of columns to select overriding columns selected in the query. |
Usage
Returns
Return Type | Description |
---|---|
Dataframe | A pandas DataFrame containing the slice returned by the query. |
Raises
Error code | Issue |
---|---|
BadRequest | If given query is wrong. |
📘 Info
Only read-only SQL operations are supported. Certain SQL operations like aggregations and joins might not result in a valid slice.
download_slice()
Download data with slice query to parquet file. 10M rows is the max size that can be downloaded.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
output_dir | Union[Path, str] | - | Path to download the file. |
query | str | - | An SQL query that begins with the keyword 'SELECT'. |
sample | Optional[bool] | False | Allows caller to explicitly specify list of columns to select overriding columns selected in the query. |
max_rows | Optional[int] | None | Number of maximum rows to fetch. |
columns | Optional[list[str]] | None | Whether rows should be sample or not from the database. |
Usage
Returns
Parquet file with slice query contents downloaded to the Path mentioned in output_dir.
Raises
Error code | Issue |
---|---|
BadRequest | If given query is wrong. |
predict()
Run model on an input dataframe.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
df | pd.DataFrame | None | Feature dataframe. |
chunk_size | Optional[int] | None | Chunk size for fetching predictions. |
Usage
Returns
Return Type | Description |
---|---|
Dataframe | A pandas DataFrame of the predictions. |
get_mutual_info()
Get mutual information.
The Mutual information measures the dependency between two random variables. It's a non-negative value. If two random variables are independent MI is equal to zero. Higher MI values means higher dependency.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
query | str | - | Slice query to compute Mutual information on. |
column_name | str | - | Column name to compute mutual information with respect to all the variables in the dataset. |
num_samples | Optional[int] | None | Number of samples to select for computation. |
normalized | Optional[bool] | False | If set to True, it will compute Normalized Mutual Information. |
Usage
Returns
Return Type | Description |
---|---|
Dictionary | Contains mutual information w.r.t the given feature for each column given. |
Raises
Error code | Issue |
---|---|
BadRequest | If given query is wrong. |
Constants
ModelInputType
Input data type used by the model.
Enum Value | Description |
---|---|
ModelInputType.TABULAR | For tabular models. |
ModelInputType.TEXT | For text models. |
ModelInputType.MIXED | For models which can be a mixture of text and tabular. |
ModelTask
The model’s algorithm type.
Enum Value | Description |
---|---|
ModelTask.REGRESSION | For regression models. |
ModelTask.BINARY_CLASSIFICATION | For binary classification models. |
ModelTask.MULTICLASS_CLASSIFICATION | For multiclass classification models. |
ModelTask.RANKING | For ranking classification models. |
ModelTask.LLM | For LLM models. |
ModelTask.NOT_SET | For other model tasks or no model task specified. |
DataType
The available data types when defining a model Column.
Enum Value | Description |
---|---|
DataType.FLOAT | For floats. |
DataType.INTEGER | For integers. |
DataType.BOOLEAN | For booleans. |
DataType.STRING | For strings. |
DataType.CATEGORY | For categorical types. |
DataType.TIMESTAMP | For timestamps. |
DataType.VECTOR | For vector types |
CustomFeatureType
This is an enumeration defining the types of custom features that can be created.
Enum | Value |
---|---|
CustomFeatureType.FROM_COLUMNS | Represents custom features derived directly from columns. |
CustomFeatureType.FROM_VECTOR | Represents custom features derived from a vector column. |
CustomFeatureType.FROM_TEXT_EMBEDDING | Represents custom features derived from text embeddings. |
CustomFeatureType.FROM_IMAGE_EMBEDDING | Represents custom features derived from image embeddings. |
CustomFeatureType.ENRICHMENT | Represents custom features derived from an enrichment. |
ArtifactType
Indicator of type of a model artifact.
Enum Value | Description |
---|---|
ArtifactType.SURROGATE | For surrogates. |
ArtifactType.PYTHON_PACKAGE | For python package. |
DeploymentType
Indicator of how the model was deployed.
Enum Value | Description |
---|---|
DeploymentType.BASE_CONTAINER | For base containers. |
DeploymentType.MANUAL | For manual deployment. |
EnvType
Environment type of a dataset.
Enum Value | Description |
---|---|
EnvType.PRODUCTION | For production events. |
EnvType.PRE_PRODUCTION | For pre production events. |
BaselineType
Type of a baseline.
Enum Value | Description |
---|---|
BaselineType.STATIC | For static production baseline. |
BaselineType.ROLLING | For rolling production baseline. |
WindowBinSize
Window for rolling baselines.
Enum Value | Description |
---|---|
WindowBinSize.HOUR | For rolling window to be 1 hour. |
WindowBinSize.DAY | For rolling window to be 1 day. |
WindowBinSize.WEEK | For rolling window to be 1 week. |
WindowBinSize.MONTH | For rolling window to be 1 month. |
WebhookProvider
Specifies the integration provider or OTHER for generic callback response.
Enum Value | Description |
---|---|
WebhookProvider.SLACK | For slack. |
WebhookProvider.OTHER | For any other app. |
AlertCondition
Specifies the comparison operator to use for an alert threshold value.
Enum Value | Description |
---|---|
AlertCondition.GREATER | The greater than operator. |
AlertCondition.LESSER | the less than operator. |
BinSize
Specifies the comparison operator to use for an alert threshold value.
Enum Value | Description |
---|---|
BinSize.HOUR | The 1 hour bin. |
BinSize.DAY | the 1 day bin. |
BinSize.WEEK | The 7 day bin. |
BinSize.MONTH | The 30 day bin. |
CompareTo
Specifies the type of evaluation to use for an alert.
Enum Value | Description |
---|---|
CompareTo.RAW_VALUE | For an absolute comparison of a specified value to the alert metric |
CompareTo.TIME_PERIOD | For a relative comparison of the alert metric to the same metric from a previous time period. |
Priority
Priority level label for alerts.
Enum Value | Description |
---|---|
Priority.LOW | The low priority label. |
Priority.MEDIUM | The medium priority label. |
Priority.HIGH | The high priority label. |
Severity
Severity level for alerts.
Enum Value | Description |
---|---|
Severity.DEFAULT | For AlertRule when none of the thresholds have passed. |
Severity.WARNING | For AlertRule when alert crossed the warning_threshold but not the critical_threshold. |
Severity.CRITICAL | For AlertRule when alert crossed the critical_raw_threshold. |
Alert Metric ID
AlertRule metric_id parameter constants.
Metric Type | Metric Id Constant | Metric Name |
---|---|---|
Drift | jsd | Jensen-Shannon Distance |
psi | Population Stability Index | |
Service Metrics | traffic | Traffic |
Data Integrity | null_violation_count | Missing Value Violation |
type_violation_count | Type Violation | |
range_violation_count | Range Violation | |
any_violation_count | Any Violation | |
null_violation_percentage | % Missing Value Violation | |
type_violation_percentage | % Type Violation | |
range_violation_percentage | % Range Violation | |
any_violation_percentage | % Any Violation | |
Statistics | sum | Sum |
average | Average | |
frequency | Frequency | |
Performance | accuracy | Accuracy |
log_loss | Log Loss | |
map | MAP | |
ndcg_mean | NDhorCG | |
query_count | Query Count | |
precision | Precision | |
recall | Recall / TPR | |
f1_score | F1 | |
geometric_mean | Geometric Mean | |
data_count | Total Count | |
expected_calibration_error | Expected Calibration Error | |
auc | AUC | |
auroc | AUROC | |
calibrated_threshold | Calibrated Threshold | |
fpr | False Positive Rate | |
Custom Metrics | UUID of custom metric | Custom Metric Name |
Schemas
Column
A model column representation.
Parameter | Type | Default | Description |
---|---|---|---|
name | str | None | Column name provided by the customer. |
data_type | list[Datatype] | None | List of columns. |
min | Union[int, float] | None | Min value of integer/float column. |
max | Union[int, float] | None | Max value of integer/float column. |
categories | list | None | List of unique values of a categorical column. |
bins | list[Union[int, float]] | None | Bins of integer/float column. |
replace_with_nulls | list | None | Replace the list of given values to NULL if found in the events data. |
n_dimensions | int | None | Number of dimensions of a vector column. |
fdl.Enrichment (Private Preview)
Input Parameter | Type | Default | Description |
---|---|---|---|
name | str | The name of the custom feature to generate | |
enrichment | str | The enrichment operation to be applied | |
columns | List[str] | The column names on which the enrichment depends | |
config | Optional[List] | {} | (optional): Configuration specific to an enrichment operation which controls the behavior of the enrichment |
Note
Enrichments are disabled by default. To enable them, contact your administrator. Failing to do so will result in an error during the add_model
call.
Embedding (Private Preview)
Supported Models:
model_name | size | Type | pooling_method | Notes |
---|---|---|---|---|
BAAI/bge-small-en-v1.5 | small | Sentence Transformer | ||
sentence-transformers/all-MiniLM-L6-v2 | med | Sentence Transformer | ||
thenlper/gte-base | med | Sentence Transformer | (default) | |
gpt2 | med | Encoder NLP Transformer | last_token | |
distilgpt2 | small | Encoder NLP Transformer | last_token | |
EleuteherAI/gpt-neo-125m | med | Encoder NLP Transformer | last_token | |
google/bert_uncased_L-4_H-256_A-4 | small | Decoder NLP Transformer | first_token | Smallest Bert |
bert-base-cased | med | Decoder NLP Transformer | first_token | |
distilroberta-base | med | Decoder NLP Transformer | first_token | |
xlm-roberta-large | large | Decoder NLP Transformer | first_token | Multilingual |
roberta-large | large | Decoder NLP Transformer | first_token |
\
The above example will lead to generation of new column:
Column | Type | Description |
---|---|---|
FDL Question Embedding | vector | Embeddings corresponding to string column |
\
Note
In the context of Hugging Face models, particularly transformer-based models used for generating embeddings, the pooling_method determines how the model processes the output of its layers to produce a single vector representation for input sequences (like sentences or documents). This is crucial when using these models for tasks like sentence or document embedding, where you need a fixed-size vector representation regardless of the input length.
Centroid Distance (Private Preview)
\
The above example will lead to generation of new column:
Column | Type | Description |
---|---|---|
FDL Centroid Distance (question_embedding) | float | Distance from the nearest K-Means centroid present in
|
Note
Does not calculate membership for preproduction data, so you cannot calculate drift. Centroid Distance is automatically added if the TextEmbedding
enrichment is created for any given model.
Personally Identifiable Information (Private Preview)
List of PII entities
Entity Type | Description | Detection Method | Example |
---|---|---|---|
CREDIT_CARD | A credit card number is between 12 to 19 digits. https://en.wikipedia.org/wiki/Payment_card_number | Pattern match and checksum |
|
CRYPTO | A Crypto wallet number. Currently only Bitcoin address is supported | Pattern match, context and checksum |
|
DATE_TIME | Absolute or relative dates or periods or times smaller than a day. | Pattern match and context | ../2024 |
EMAIL_ADDRESS | An email address identifies an email box to which email messages are delivered | Pattern match, context and RFC-822 validation |
|
IBAN_CODE | The International Bank Account Number (IBAN) is an internationally agreed system of identifying bank accounts across national borders to facilitate the communication and processing of cross border transactions with a reduced risk of transcription errors. | Pattern match, context and checksum |
|
IP_ADDRESS | An Internet Protocol (IP) address (either IPv4 or IPv6). | Pattern match, context and checksum |
|
LOCATION | Name of politically or geographically defined location (cities, provinces, countries, international regions, bodies of water, mountains | Custom logic and context | PALO ALTO Japan |
PERSON | A full person name, which can include first names, middle names or initials, and last names. | Custom logic and context | Joanna Doe |
PHONE_NUMBER | A telephone number | Custom logic, pattern match and context |
|
URL | A URL (Uniform Resource Locator), unique identifier used to locate a resource on the Internet | Pattern match, context and top level url validation | |
US SSN | A US Social Security Number (SSN) with 9 digits. | Pattern match and context |
|
US_DRIVER_LICENSE | A US driver license according to https://ntsi.com/drivers-license-format/ | Pattern match and context | |
US_ITIN | US Individual Taxpayer Identification Number (ITIN). Nine digits that start with a "9" and contain a "7" or "8" as the 4 digit. | Pattern match and context | 912-34-1234 |
US_PASSPORT | A US passport number begins with a letter, followed by eight numbers | Pattern match and context | L12345678 |
The above example will lead to generation of new columns:
Column | Type | Description |
---|---|---|
FDL Rag PII (question) | bool | Whether any PII was detected. |
FDL Rag PII (question) Matches | str | What matches in raw text were flagged as potential PII (ex. ‘Douglas MacArthur,Korean’)? |
FDL Rag PII (question) Entities | str | What entites these matches were tagged as (ex. 'PERSON')? |
Note
PII enrichment is integrated with Presidio
Evaluate (Private Preview)
Here is a summary of the three evaluation metrics for natural language generation:
Metric | Description | Strengths | Limitations |
---|---|---|---|
bleu | Measures precision of word n-grams between generated and reference texts | Simple, fast, widely used | Ignores recall, meaning, and word order |
rouge | Measures recall of word n-grams and longest common sequences | Captures more information than BLEU | Still relies on word matching, not semantic similarity |
meteor | Incorporates recall, precision, and additional semantic matching based on stems and paraphrasing | More robust and flexible than BLEU and ROUGE | Requires linguistic resources and alignment algorithms |
\
The above example generates 6 new columns:
Column | Type |
---|---|
FDL QA Evaluate (bleu) | float |
FDL QA Evaluate (rouge1) | float |
FDL QA Evaluate (rouge2) | float |
FDL QA Evaluate (rougel) | float |
FDL QA Evaluate (rougelsum) | float |
FDL QA Evaluate (meteor) | floa |
Textstat (Private Preview)
**Supported Statistics **
Statistic | Description | Usage |
---|---|---|
char_count | Total number of characters in text, including everything. | Assessing text length, useful for platforms with character limits. |
letter_count | Total number of letters only, excluding numbers, punctuation, spaces. | Gauging text complexity, used in readability formulas. |
miniword_count | Count of small words (usually 1-3 letters). | Specific readability analyses, especially for simplistic texts. |
words_per_sentence | Average number of words in each sentence. | Understanding sentence complexity and structure. |
polysyllabcount | Number of words with more than three syllables. | Analyzing text complexity, used in some readability scores. |
lexicon_count | Total number of words in the text. | General text analysis, assessing overall word count. |
syllable_count | Total number of syllables in the text. | Used in readability formulas, measures text complexity. |
sentence_count | Total number of sentences in the text. | Analyzing text structure, used in readability scores. |
flesch_reading_ease | Readability score indicating how easy a text is to read (higher scores = easier). | Assessing readability for a general audience. |
smog_index | Measures years of education needed to understand a text. | Evaluating text complexity, especially for higher education texts. |
flesch_kincaid_grade | Grade level associated with the complexity of the text. | Educational settings, determining appropriate grade level for texts. |
coleman_liau_index | Grade level needed to understand the text based on sentence length and letter count. | Assessing readability for educational purposes. |
automated_readability_index | Estimates the grade level needed to comprehend the text. | Evaluating text difficulty for educational materials. |
dale_chall_readability_score | Assesses text difficulty based on a list of familiar words for average American readers. | Determining text suitability for average readers. |
difficult_words | Number of words not on a list of commonly understood words. | Analyzing text difficulty, especially for non-native speakers. |
linsear_write_formula | Readability formula estimating grade level of text based on sentence length and easy word count. | Simplifying texts, especially for lower reading levels. |
gunning_fog | Estimates the years of formal education needed to understand the text. | Assessing text complexity, often for business or professional texts. |
long_word_count | Number of words longer than a certain length (often 6 or 7 letters). | Evaluating complexity and sophistication of language used. |
monosyllabcount | Count of words with only one syllable. | Readability assessments, particularly for simpler texts. |
The above example leads to the creation of two additional columns:
Column | Type | Description |
---|---|---|
FDL Text Statistics (question) char_count | int | Character count of string in |
FDL Text Statistics (question) dale_chall_readability_score | float | Readability score of string in |
Sentiment (Private Preview)
The above example leads to creation of two columns:
Column | Type | Description |
---|---|---|
FDL Question Sentiment (question) compound | float | Raw score of sentiment. |
FDL Question Sentiment (question) sentiment | string | One of |
Profanity (Private Preview)
\
The above example leads to creation of two columns:
Column | Type | Description |
---|---|---|
FDL Profanity (prompt) contains_profanity | bool | To indicate if input contains profanity in the value of the prompt column. |
FDL Profanity (response) contains_profanity | bool | To indicate if input contains profanity in the value of the response column. |
Answer Relevance (Private Preview)
The above example will lead to the generation of a new column
Column | Type | Description |
---|---|---|
FDL Answer Relevance | bool | Binary metric, which is True if |
Faithfulness (Private Preview)
The above example will lead to generation of new column:
Column | Type | Description |
---|---|---|
FDL Faithfulness | bool | Binary metric, which is True if the facts used in |
Coherence (Private Preview)
\
The above example will lead to generation of new column:
Column | Type | Description |
---|---|---|
FDL Coherence | bool | Binary metric, which is True if |
Conciseness (Private Preview)
The above example will lead to generation of new column:
Column | Type | Description |
---|---|---|
FDL Conciseness | Binary metric, which is True if |
Toxicity (Private Preview)
Dataset | PR-AUC | Precision | Recall |
---|---|---|---|
Toxic-Chat | 0.4 | 0.64 | 0.24 |
Usage
The code snippet shows how to enable toxicity scoring on the prompt
and response
columns for each event published to Fiddler.
The above example leads to creation of two columns each for prompt and response that contain the prediction probability and the model decision.
For example for the prompt column following two columns will be generated
Column | Type | Description |
---|---|---|
FDL Toxicity (prompt) toxicity_prob | float | Model prediction probability between 0-1. |
FDL Toxicity (prompt) contains_toxicity | bool | Model prediction either 0 or 1. |
Regex Match (Private Preview)
The above example will lead to generation of new column
Column | Type | Description |
---|---|---|
FDL Regex - only digits | category | Match or No Match, depending on the regex specified in config matching in the string. |
Topic (Private Preview)
\
The above example leads to creation of two columns -
Column | Type | Description |
---|---|---|
FDL Topics (response) topic_model_scores | list[float] | Indicating probability of the given column in each of the topics specified in the Enrichment config. Each float value indicate probability of the given input classified in the corresponding topic, in the same order as topics. Each value will be between 0 and 1. The sum of values does not equal to 1, as each classification is performed independently of other topics. |
FDL Topics (response) max_score_topic | string | Topic with the maximum score from the list of topic names specified in the Enrichment config. |
Banned Keyword Detector (Private Preview)
\
The above example leads to creation of two columns -
Column | Type | Description |
---|---|---|
FDL Banned KW (prompt) contains_banned_kw | bool | To indicate if input contains one of the specified banned keywords in the value of the prompt column. |
FDL Banned KW (response) contains_banned_kw | bool | To indicate if input contains one of the specified banned keywords in the value of the response column. |
Language Detector (Private Preview)
Language detector leverages fasttext models for language detection.
\
The above example leads to creation of two columns -
Column | Type | Description |
---|---|---|
FDL Language (prompt) language | string | Language prediction for input text |
FDL Language (prompt) language_probability | float | To indicate the confidence probabillity of language prediction |
Fast Safety (Private Preview)
The Fast safety enrichment evaluates the safety of the text along ten different dimensions: illegal, hateful, harassing, racist, sexist, violent, sexual, harmful, unethical, jailbreaking
. These dimensions are all returned by default, but can be selectively chosen as needed. Fast safety is generated through the Fast Trust Models.
\
The above example leads to creation of a column for each dimension. -
Column | Type | Description |
---|---|---|
FDL Prompt Safety (prompt) | bool | Binary metric, which is True if the input is deemed unsafe, False otherwise |
FDL Prompt Safety (prompt) | float | To indicate the confidence probabillity of safety prediction |
Fast Faithfulness (Private Preview)
The Fast faithfulness enrichment is designed to evaluate the accuracy and reliability of facts presented in AI-generated text responses. Fast safety is generated through the Fast Trust Models.
\
The above example leads to creation of two columns -
Column | Type | Description |
---|---|---|
FDL Faithfulness faithful | bool | Binary metric, which is True if the facts used in |
FDL Faithfulness faithful score | float | To indicate the confidence probabillity of faithfulness prediction |
SQL Validation (Private Preview)
Query validation is syntax based and does not check against any existing schema or databases for validity.
The SQL Validation enrichment is designed to evaluate different query dialects for syntax correctness.
The above example leads to creation of two columns -
Column | Type | Description |
---|---|---|
SQL Validator valid | bool | True if the query string is syntactically valid for the specified dialect, False if not. |
SQL Validator errors | str | If syntax errors are found they will be present as a JSON serialized string containing a list of dictionaries describing the errors. |
JSON Validation (Private Preview)
The JSON Validation enrichment is designed to evaluate strings for correct JSON syntax and optionally against a user-defined schema for validation.
This enrichment uses the python-jsonschema library for json schema validation. The defined validation_schema
must be a valid python-jsonschema schema.
The above example leads to creation of two columns -
Column | Type | Description |
---|---|---|
JSON Validator valid | bool | String is valid JSON. |
JSON Validator errors | str | If the string failed to parse to JSON any parsing errors will be returned as a serialized json list of dictionaries, |
ModelTaskParams
Task parameters given to a particular model.
Parameter | Type | Default | Description |
---|---|---|---|
binary_classification_threshold | float | None | Threshold for labels. |
target_class_order | list | None | Order of target classes. |
group_by | str | None | Query/session id column for ranking models. |
top_k | int | None | Top k results to consider when computing ranking metrics. |
class_weights | list[float] | None | Weight of each class. |
weighted_ref_histograms | bool | None | Whether baseline histograms must be weighted or not when calculating drift metrics. |
ModelSchema
Model schema defines the list of columns associated with a model version.
Parameter | Type | Default | Description |
---|---|---|---|
schema_version | int | 1 | Schema version. |
columns | list[Column] | None | List of columns. |
ModelSpec
Model spec defines how model columns are used along with model task.
Parameter | Type | Default | Description |
---|---|---|---|
schema_version | int | 1 | Schema version. |
inputs | list[str] | None | Feature columns. |
outputs | list[str] | None | Prediction columns. |
targets | list[str] | None | Label columns. |
decisions | list[str] | None | Decisions columns. |
metadata | list[str] | None | Metadata columns |
custom_features | list[CustomFeature] | None | Custom feature definitions. |
CustomFeature
The base class for derived features such as Multivariate, VectorFeature, etc.
Parameter | Type | Default | Description |
---|---|---|---|
name | str | None | The name of the custom feature. |
type | None | The type of custom feature. Must be one of the | |
n_clusters | Optional[int] | 5 | The number of clusters. |
centroids | Optional[List] | None | Centroids of the clusters in the embedded space. Number of centroids equal to |
Multivariate
Represents custom features derived from multiple columns.
Parameter | Type | Default | Description |
---|---|---|---|
type | CustomFeatureType.FROM_COLUMNS | Indicates this feature is derived from multiple columns. | |
columns | List[str] | None | List of original columns from which this feature is derived. |
monitor_components | bool | False | Whether to monitor each column in |
VectorFeature
Represents custom features derived from a single vector column.
Parameter | Type | Default | Description |
---|---|---|---|
type | CustomFeatureType.FROM_VECTOR | Indicates this feature is derived from a single vector column. | |
source_column | Optional[str] | None | Specifies the original column if this feature is derived from an embedding. |
column | str | None | The vector column name. |
TextEmbedding
Represents custom features derived from text embeddings.
Parameter | Type | Default | Description |
---|---|---|---|
type | CustomFeatureType.FROM_TEXT_EMBEDDING | Indicates this feature is derived from a text embedding. | |
n_tags | Optional[int] | 5 | How many tags(tokens) the text embedding uses in each cluster as the |
ImageEmbedding
Represents custom features derived from image embeddings.
Parameter | Type | Default | Description |
---|---|---|---|
type | CustomFeatureType.FROM_IMAGE_EMBEDDING | Indicates this feature is derived from an image embedding. |
Enrichment
Represents custom features derived from enrichment.
Parameter | Type | Default | Description |
---|---|---|---|
type | CustomFeatureType.ENRICHMENT | Indicates this feature is derived from enrichment. | |
columns | List[str] | None | List of original columns from which this feature is derived. |
enrichment | str | None | A string identifier for the type of enrichment to be applied. |
config | Dict[str, Any] | None | A dictionary containing configuration options for the enrichment. |
XaiParams
Represents the explainability parameters.
Parameter | Type | Default | Description |
---|---|---|---|
custom_explain_methods | List[str] | None | User-defined explain_custom methods of the model object defined in package.py. |
default_explain_method | Optional[str] | NOne | Default explanation method. |
DeploymentParams
Deployment parameters of a particular model.
Parameter | Type | Default | Description |
---|---|---|---|
artifact_type | str | Type of artifact upload. | |
deployment_type | None | Type of deployment. | |
image_uri | Optional[str] | md-base/python/python-311:1.0.0 | Reference to the docker image to create a new runtime to serve the model. Check the available images on the Model Deployment page. |
replicas | Optional[str] | 1 | The number of replicas running the model. Minimum value: 1 Maximum value: 10 Default value: 1 |
cpu | Optional[str] | 100 | The amount of CPU (milli cpus) reserved per replica. Minimum value: 10 Maximum value: 4000 (4vCPUs) Default value: 100 |
memory | Optional[str] | 256 | The amount of memory (mebibytes) reserved per replica. Minimum value: 150 Maximum value: 16384 (16GiB) Default value: 256 |
RowDataSource
Explainability input source for row data.
Parameter | Type | Default | Description |
---|---|---|---|
row | Dict | None | Dictionary containing row details. |
EventIdDataSource
Explainability input source for event data.
Parameter | Type | Default | Description |
---|---|---|---|
event_id | str | None | Unique ID for event. |
env_id | Optional[Union[str, UUID]] | None | Unique ID for environment. |
env_type | None | Environment type. |
DatasetDataSource
Reference data source for explainability.
Parameter | Type | Default | Description |
---|---|---|---|
env_type | None | Environment type. | |
num_samples | Optional[int] | None | Number of samples to select for computation. |
env_id | Optional[Union[str, UUID] | None | Unique ID for environment. |
SqlSliceQueryDataSource
Sql data source for explainability.
Parameter | Type | Default | Description |
---|---|---|---|
query | str | None | Query for slice. |
num_samples | Optional[int] | None | Number of samples to select for computation. |
Helper functions
set_logging
Set app logger at given log level.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
level | int | logging.INFO | Logging level from python logging module |
Usage
Returns
None
group_by
Group the events by a column. Use this method to form the grouped data for ranking models.
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
df | pd.DataFrame | - | Unique identifier for the AlertRule. |
group_by_col | str | - | The column to group the data by. |
output_path | Optional[Path | str] | - |
Usage
Returns
Return Type | Description |
---|---|
pd.Dataframe | Dataframe in grouped format. |
Last updated