Skip to content

How To Debug Model Upload

View In Github

Initialize Fiddler Client

We begin this section as usual by establishing a connection to our Fiddler instance. We can establish this connection either by specifying our credentials directly, or by utilizing our fiddler.ini file. More information can be found in the setup section.

import fiddler as fdl
import logging

# True for logging debug message
verbose = True

if verbose:
    logging.basicConfig(level=logging.DEBUG)

# client = fdl.FiddlerApi(url=url, org_id=org_id, auth_token=auth_token, verbose=verbose)
client = fdl.FiddlerApi(verbose=verbose)
INFO:root:url =https://<name>.fiddler.ai
INFO:root:running api call as POST request
to https://<name>.fiddler.ai/v1/project/get/<org_id>
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload None
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/v1/project/get/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2', 'Content-Type': 'application/json'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <name>.fiddler.ai:443
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /v1/project/get/<org_id> HTTP/1.1" 200 None
INFO:root:response: {
  "status": "SUCCESS",
  "result": [
    "tutorial-re",
    "tutorial",
    "titanic",
    "iris_classification",
    "sample",
    "newsgroup",
    "heart_disease",
    "lending",
    "imdb_rnn",
    "tutorial_2",
    "wine_quality",
    "bank_churn",
    "newsgroup_text_topics"
  ]
}
INFO:root:API call to https://<name>.fiddler.ai/v1/project/get/<org_id> succeeded.
Request response:   {
    "status": "SUCCESS",
    "result": [
      "tutorial-re",
      "tutorial",
      "titanic",
      "iris_classification",
      "sample",
      "newsgroup",
      "heart_disease",
      "lending",
      "imdb_rnn",
      "tutorial_2",
      "wine_quality",
      "bank_churn",
      "newsgroup_text_topics"
    ]
  }
Request:
  url: https://<name>.fiddler.ai/v1/project/get/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2', 'Content-Type': 'application/json'}

Create Project

Here we will create a project, a convenient container for housing the models and datasets associated with a given ML use case. Uploading our dataset in the next step will depend on our created project's project_id.

project_id = 'debug_model_upload'
# Creating our project using project_id
if project_id not in client.list_projects():
    client.create_project(project_id)

Load Dataset

Here we will load in our baseline dataset from a csv called train.csv. We will also create a schema using this information.

import pandas as pd
df = pd.read_csv('/app/fiddler_samples/samples/datasets/winequality/train.csv')
df_schema = fdl.DatasetInfo.from_dataframe(df, max_inferred_cardinality=1000)

Upload Dataset

To upload a model, you first need to upload a sample of the data of the model’s inputs, targets, and additional metadata that might be useful for model analysis. This data sample helps us (among other things) to infer the model schema and the data types and values range of each feature.

if 'wine_quality' not in client.list_datasets(project_id):
    upload_result = client.upload_dataset(
        project_id=project_id,
        dataset={'train': df},
        dataset_id='wine_quality')
INFO:root:running api call as POST request
to https://<name>.fiddler.ai/v1/dataset/get/<org_id>
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload None
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/v1/dataset/get/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2', 'Content-Type': 'application/json'}
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /v1/dataset/get/<org_id> HTTP/1.1" 200 233
INFO:root:response: {
  "status": "SUCCESS",
  "result": [
    "sample-data",
    "20news",
    "bank_churn",
    "heart_disease",
    "imdb_rnn",
    "iris",
    "p2p_loans",
    "titanic",
    "winequality",
    "wine_quality",
    "loan_status"
  ]
}
INFO:root:API call to https://<name>.fiddler.ai/v1/dataset/get/<org_id> succeeded.
Request response:   {
    "status": "SUCCESS",
    "result": [
      "sample-data",
      "20news",
      "bank_churn",
      "heart_disease",
      "imdb_rnn",
      "iris",
      "p2p_loans",
      "titanic",
      "winequality",
      "wine_quality",
      "loan_status"
    ]
  }
Request:
  url: https://<name>.fiddler.ai/v1/dataset/get/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2', 'Content-Type': 'application/json'}

Create Model Schema

As you must have noted, in the dataset upload step we did not ask for the model’s features and targets, or any model specific information. That’s because we allow for linking multiple models to a given dataset schema. Hence we require an Infer model schema step which helps us know the features relevant to the model and the model task. Here you can specify the input features, the target column, decision columns and metadata columns, and also the type of model.

target = 'quality'
train_input = df.drop(columns=['row_id', 'quality'])
train_target = df[target]

feature_columns = list(train_input.columns)

model_info = fdl.ModelInfo.from_dataset_info(
    dataset_info=client.get_dataset_info(project_id, 'wine_quality'),
    target=target,
    features=feature_columns,
    display_name='debug model',
    description='this is a sklearn model from tutorial that shows how to debug model upload'
)
INFO:root:running api call as POST request
to https://<name>.fiddler.ai/v1/dataset/get/<org_id>
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload   {
    "dataset_id": "wine_quality"
  }
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/v1/dataset/get/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '30', 'Content-Type': 'application/json'}
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /v1/dataset/get/<org_id> HTTP/1.1" 200 None
INFO:root:response: {
  "status": "SUCCESS",
  "result": {
    "dataset": {
      "name": "",
      "columns": [
        {
          "column-name": "row_id",
          "data-type": "int",
          "is-nullable": false,
          "value-range-min": 0,
          "value-range-max": 1597
        },
        {
          "column-name": "fixed acidity",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 4.7,
          "value-range-max": 15.9
        },
        {
          "column-name": "volatile acidity",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.12,
          "value-range-max": 1.58
        },
        {
          "column-name": "citric acid",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.0,
          "value-range-max": 1.0
        },
        {
          "column-name": "residual sugar",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.9,
          "value-range-max": 15.5
        },
        {
          "column-name": "chlorides",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.012,
          "value-range-max": 0.611
        },
        {
          "column-name": "free sulfur dioxide",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 1.0,
          "value-range-max": 72.0
        },
        {
          "column-name": "total sulfur dioxide",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 6.0,
          "value-range-max": 289.0
        },
        {
          "column-name": "density",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.9900700000000001,
          "value-range-max": 1.00369
        },
        {
          "column-name": "pH",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 2.74,
          "value-range-max": 4.01
        },
        {
          "column-name": "sulphates",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.37,
          "value-range-max": 2.0
        },
        {
          "column-name": "alcohol",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 8.4,
          "value-range-max": 14.9
        },
        {
          "column-name": "quality",
          "data-type": "int",
          "is-nullable": false,
          "value-range-min": 3,
          "value-range-max": 8
        }
      ],
      "files": [
        "train.csv"
      ]
    }
  }
}
INFO:root:API call to https://<name>.fiddler.ai/v1/dataset/get/<org_id> succeeded.
Request response:   {
    "status": "SUCCESS",
    "result": {
      "dataset": {
        "name": "",
        "columns": [
          {
            "column-name": "row_id",
            "data-type": "int",
            "is-nullable": false,
            "value-range-min": 0,
            "value-range-max": 1597
          },
          {
            "column-name": "fixed acidity",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 4.7,
            "value-range-max": 15.9
          },
          {
            "column-name": "volatile acidity",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.12,
            "value-range-max": 1.58
          },
          {
            "column-name": "citric acid",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.0,
            "value-range-max": 1.0
          },
          {
            "column-name": "residual sugar",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.9,
            "value-range-max": 15.5
          },
          {
            "column-name": "chlorides",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.012,
            "value-range-max": 0.611
          },
          {
            "column-name": "free sulfur dioxide",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 1.0,
            "value-range-max": 72.0
          },
          {
            "column-name": "total sulfur dioxide",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 6.0,
            "value-range-max": 289.0
          },
          {
            "column-name": "density",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.9900700000000001,
            "value-range-max": 1.00369
          },
          {
            "column-name": "pH",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 2.74,
            "value-range-max": 4.01
          },
Request:
  url: https://<name>.fiddler.ai/v1/dataset/get/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '30', 'Content-Type': 'application/json'}

Train Model

Build and train your model.

import sklearn.linear_model
import sklearn.pipeline
import sklearn.preprocessing


regressor = sklearn.linear_model.LinearRegression()

full_model = sklearn.pipeline.Pipeline(steps=[
        ('standard_scaling', sklearn.preprocessing.StandardScaler()),
        ('model_name', regressor),
    ])

full_model.fit(train_input, train_target)
full_model.predict(train_input)
array([6.50506782, 6.65427237, 5.35208865, ..., 5.27678525, 6.36560789,
       4.44488096])

Save Model and Schema

Next step, we need to save the model and any pre-processing step you had on the input features (for example Categorical encoder, Tokenization, ...).

import pathlib
import shutil
import pickle
import yaml

project_id = 'tutorial'
model_id = 'debug_model'

# create temp dir
model_dir = pathlib.Path(model_id)
shutil.rmtree(model_dir, ignore_errors=True)
model_dir.mkdir()

# save model
with open(model_dir / 'model.pkl', 'wb') as pkl_file:
    pickle.dump(full_model, pkl_file)

# save model schema
with open(model_dir / 'model.yaml', 'w') as yaml_file:
    yaml.dump({'model': model_info.to_dict()}, yaml_file)

Write package.py

A wrapper is needed between Fiddler and the model. This wrapper can be used to translate the inputs and outputs to fit what the model expects and what Fiddler is able to consume. More information can be found here

%%writefile debug_model/package.py

import pickle
from pathlib import Path
import pandas as pd
import logging

PACKAGE_PATH = Path(__file__).parent

class SklearnModelPackage:
    is_classifier = False
    output_columns = ['predicted_quality']

    def __init__(self):
        with open(PACKAGE_PATH / 'model.pkl', 'rb') as infile:
            self.model = pickle.load(infile)

    def predict(self, input_df):

        # this will log the dataframe after transforming
        logging.info(f'log dataframe {input_df}')
        f = self.model.predict if not self.is_classifier else self.model.predict_proba
        return pd.DataFrame(f(input_df), columns=self.output_columns)

def get_model():
    return SklearnModelPackage()
Writing debug_model/package.py

Show Generated Files

!ls -l debug_model
total 12
-rw-r--r-- 1 jovyan users 1408 Oct 20 22:48 model.pkl
-rw-r--r-- 1 jovyan users 1817 Oct 20 22:48 model.yaml
-rw-r--r-- 1 jovyan users  706 Oct 20 22:49 package.py

Test package.py Locally Before Uploading

You may have to restart the kernel for the class to be loaded locally

from debug_model import package as model_class
model_class.get_model().predict(train_input[0:5])
INFO:root:log dataframe    fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \
0            8.6              0.49         0.29             2.0      0.110
1            8.0              0.18         0.37             0.9      0.049
2            6.8              0.67         0.00             1.9      0.080
3           10.1              0.31         0.35             1.6      0.075
4            7.3              0.45         0.36             5.9      0.074

   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \
0                 19.0                 133.0  0.99720  2.93       1.98
1                 36.0                 109.0  0.99007  2.89       0.44
2                 22.0                  39.0  0.99701  3.40       0.74
3                  9.0                  28.0  0.99672  3.24       0.83
4                 12.0                  87.0  0.99780  3.33       0.83

   alcohol
0      9.8
1     12.7
2      9.7
3     11.2
4     10.5
predicted_quality
0 6.505068
1 6.654272
2 5.352089
3 6.377989
4 5.834986

Upload Model

Now that we have all the parts that we need, we can go ahead and upload the model to the Fiddler platform. You can use the upload_model_package to upload this entire directory in one shot. We need the following for uploading a model: - The path to the directory - The project_id to which the model belongs - The model_id, which is the name you want to give the model. You can access it in Fiddler henceforth via this ID - The dataset which the model is linked to (optional)

In total, we will have a model.yaml, a *.pkl, and a package.py file within our model directory.

client.delete_model(project_id, model_id)
client.upload_model_package(model_dir, project_id, model_id)
INFO:root:running api call as POST request
to https://<name>.fiddler.ai/v1/model/delete/<org_id>
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload   {
    "project_id": "tutorial",
    "model_id": "debug_model",
    "delete_prod": false,
    "delete_pred": true
  }
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/v1/model/delete/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '96', 'Content-Type': 'application/json'}
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /v1/model/delete/<org_id> HTTP/1.1" 200 43
INFO:root:response: {
  "status": "SUCCESS",
  "result": true
}
INFO:root:API call to https://<name>.fiddler.ai/v1/model/delete/<org_id> succeeded.
Request response:   {
    "status": "SUCCESS",
    "result": true
  }
Request:
  url: https://<name>.fiddler.ai/v1/model/delete/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '96', 'Content-Type': 'application/json'}

INFO:root:[debug_model] model upload: uploading custom model from artifacts in debug_model tarred to /tmp/tmpk5e1quv3/files
INFO:root:running api call as POST request
to https://<name>.fiddler.ai/v1/model/upload/<org_id>
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload   {
    "project": "tutorial",
    "model": "debug_model",
    "model_schema": {
      "model": {
        "name": "debug model",
        "input-type": "structured",
        "model-task": "regression",
        "inputs": [
          {
            "column-name": "fixed acidity",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 4.7,
            "value-range-max": 15.9
          },
          {
            "column-name": "volatile acidity",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.12,
            "value-range-max": 1.58
          },
          {
            "column-name": "citric acid",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.0,
            "value-range-max": 1.0
          },
          {
            "column-name": "residual sugar",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.9,
            "value-range-max": 15.5
          },
          {
            "column-name": "chlorides",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.012,
            "value-range-max": 0.611
          },
          {
            "column-name": "free sulfur dioxide",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 1.0,
            "value-range-max": 72.0
          },
          {
            "column-name": "total sulfur dioxide",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 6.0,
            "value-range-max": 289.0
          },
          {
            "column-name": "density",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.9900700000000001,
            "value-range-max": 1.00369
          },
          {
            "column-name": "pH",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 2.74,
            "value-range-max": 4.01
          },
          {
            "column-name": "sulphates",
            "data-type": "f
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/v1/model/upload/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '12545', 'Content-Type': 'multipart/form-data; boundary=6be49b1374655a60393dab891d7ee06d'}
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /v1/model/upload/<org_id> HTTP/1.1" 200 None
INFO:root:response: {
  "status": "SUCCESS",
  "result": {
    "model": {
      "name": "debug model",
      "input-type": "structured",
      "model-task": "regression",
      "inputs": [
        {
          "column-name": "fixed acidity",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 4.7,
          "value-range-max": 15.9
        },
        {
          "column-name": "volatile acidity",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.12,
          "value-range-max": 1.58
        },
        {
          "column-name": "citric acid",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.0,
          "value-range-max": 1.0
        },
        {
          "column-name": "residual sugar",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.9,
          "value-range-max": 15.5
        },
        {
          "column-name": "chlorides",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.012,
          "value-range-max": 0.611
        },
        {
          "column-name": "free sulfur dioxide",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 1.0,
          "value-range-max": 72.0
        },
        {
          "column-name": "total sulfur dioxide",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 6.0,
          "value-range-max": 289.0
        },
        {
          "column-name": "density",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.9900700000000001,
          "value-range-max": 1.00369
        },
        {
          "column-name": "pH",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 2.74,
          "value-range-max": 4.01
        },
        {
          "column-name": "sulphates",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 0.37,
          "value-range-max": 2.0
        },
        {
          "column-name": "alcohol",
          "data-type": "float",
          "is-nullable": false,
          "value-range-min": 8.4,
          "value-range-max": 14.9
        }
      ],
      "outputs": [
        {
          "column-name": "predicted_quality",
          "data-type": "float",
          "is-nullable": false
        }
      ],
      "datasets": [
        "wine_quality"
      ],
      "targets": [
        {
          "column-name": "quality",
          "data-type": "int",
          "is-nullable": false,
          "value-range-min": 3,
          "value-range-max": 8
        }
      ],
      "description": "this is a sklearn model from tutorial that shows how to debug model upload"
    }
  }
}
INFO:root:API call to https://<name>.fiddler.ai/v1/model/upload/<org_id> succeeded.
Request response:   {
    "status": "SUCCESS",
    "result": {
      "model": {
        "name": "debug model",
        "input-type": "structured",
        "model-task": "regression",
        "inputs": [
          {
            "column-name": "fixed acidity",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 4.7,
            "value-range-max": 15.9
          },
          {
            "column-name": "volatile acidity",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.12,
            "value-range-max": 1.58
          },
          {
            "column-name": "citric acid",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.0,
            "value-range-max": 1.0
          },
          {
            "column-name": "residual sugar",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.9,
            "value-range-max": 15.5
          },
          {
            "column-name": "chlorides",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.012,
            "value-range-max": 0.611
          },
          {
            "column-name": "free sulfur dioxide",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 1.0,
            "value-range-max": 72.0
          },
          {
            "column-name": "total sulfur dioxide",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 6.0,
            "value-range-max": 289.0
          },
          {
            "column-name": "density",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 0.9900700000000001,
            "value-range-max": 1.00369
          },
          {
            "column-name": "pH",
            "data-type": "float",
            "is-nullable": false,
            "value-range-min": 2.74,
            "value-range-max": 4.01
          },
          {
            "column-name": "sulphates",
            "data-type": "float",
            "is-nullable": fa
Request:
  url: https://<name>.fiddler.ai/v1/model/upload/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'data_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '12545', 'Content-Type': 'multipart/form-data; boundary=6be49b1374655a60393dab891d7ee06d'}

Run Model

Now, let's test out our model by interfacing with the client and calling run model.

prediction_input = train_input[0: 10]
result = client.run_model(project_id, model_id, prediction_input, log_events=True)
INFO:root:running api call as POST request
to https://<name>.fiddler.ai/v1/execution/run/<org_id>
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload   {
    "project_id": "tutorial",
    "model_id": "debug_model",
    "data": [
      {
        "fixed acidity": 8.6,
        "volatile acidity": 0.49,
        "citric acid": 0.29,
        "residual sugar": 2.0,
        "chlorides": 0.11,
        "free sulfur dioxide": 19.0,
        "total sulfur dioxide": 133.0,
        "density": 0.9972,
        "pH": 2.93,
        "sulphates": 1.98,
        "alcohol": 9.8
      },
      {
        "fixed acidity": 8.0,
        "volatile acidity": 0.18,
        "citric acid": 0.37,
        "residual sugar": 0.9,
        "chlorides": 0.049,
        "free sulfur dioxide": 36.0,
        "total sulfur dioxide": 109.0,
        "density": 0.9900700000000001,
        "pH": 2.89,
        "sulphates": 0.44,
        "alcohol": 12.7
      },
      {
        "fixed acidity": 6.8,
        "volatile acidity": 0.67,
        "citric acid": 0.0,
        "residual sugar": 1.9,
        "chlorides": 0.08,
        "free sulfur dioxide": 22.0,
        "total sulfur dioxide": 39.0,
        "density": 0.9970100000000001,
        "pH": 3.4,
        "sulphates": 0.74,
        "alcohol": 9.7
      },
      {
        "fixed acidity": 10.1,
        "volatile acidity": 0.31,
        "citric acid": 0.35,
        "residual sugar": 1.6,
        "chlorides": 0.075,
        "free sulfur dioxide": 9.0,
        "total sulfur dioxide": 28.0,
        "density": 0.99672,
        "pH": 3.24,
        "sulphates": 0.83,
        "alcohol": 11.2
      },
      {
        "fixed acidity": 7.3,
        "volatile acidity": 0.45,
        "citric acid": 0.36,
        "residual sugar": 5.9,
        "chlorides": 0.07400000000000001,
        "free sulfur dioxide": 12.0,
        "total sulfur dioxide": 87.0,
        "density": 0.9978,
        "pH": 3.33,
        "sulphates": 0.83,
        "alcohol": 10.5
      },
      {
        "fixed acidity": 8.4,
        "volatile acidity": 0.34,
        "citric acid": 0.42,
        "residual sugar": 2.1,
        "chlorides": 0.07200000000000001,
        "free sulfur dioxide": 23.0,
        "total sulfur dioxide": 36.0,
        "density": 0.99392,
        "pH": 3.11,
        "sulphates": 0.78,
        "alcohol": 12.4
      },
      {
        "fixed acidity": 8.
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/v1/execution/run/<org_id>
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'executor_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2551', 'Content-Type': 'application/json'}
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /v1/execution/run/<org_id> HTTP/1.1" 400 2590
INFO:root:response: {
    "errors": {
        "data": "[{'fixed acidity': 8.6, 'volatile acidity': 0.49, 'citric acid': 0.29, 'residual sugar': 2.0, 'chlorides': 0.11, 'free sulfur dioxide': 19.0, 'total sulfur dioxide': 133.0, 'density': 0.9972, 'pH': 2.93, 'sulphates': 1.98, 'alcohol': 9.8}, {'fixed acidity': 8.0, 'volatile acidity': 0.18, 'citric acid': 0.37, 'residual sugar': 0.9, 'chlorides': 0.049, 'free sulfur dioxide': 36.0, 'total sulfur dioxide': 109.0, 'density': 0.9900700000000001, 'pH': 2.89, 'sulphates': 0.44, 'alcohol': 12.7}, {'fixed acidity': 6.8, 'volatile acidity': 0.67, 'citric acid': 0.0, 'residual sugar': 1.9, 'chlorides': 0.08, 'free sulfur dioxide': 22.0, 'total sulfur dioxide': 39.0, 'density': 0.9970100000000001, 'pH': 3.4, 'sulphates': 0.74, 'alcohol': 9.7}, {'fixed acidity': 10.1, 'volatile acidity': 0.31, 'citric acid': 0.35, 'residual sugar': 1.6, 'chlorides': 0.075, 'free sulfur dioxide': 9.0, 'total sulfur dioxide': 28.0, 'density': 0.99672, 'pH': 3.24, 'sulphates': 0.83, 'alcohol': 11.2}, {'fixed acidity': 7.3, 'volatile acidity': 0.45, 'citric acid': 0.36, 'residual sugar': 5.9, 'chlorides': 0.07400000000000001, 'free sulfur dioxide': 12.0, 'total sulfur dioxide': 87.0, 'density': 0.9978, 'pH': 3.33, 'sulphates': 0.83, 'alcohol': 10.5}, {'fixed acidity': 8.4, 'volatile acidity': 0.34, 'citric acid': 0.42, 'residual sugar': 2.1, 'chlorides': 0.07200000000000001, 'free sulfur dioxide': 23.0, 'total sulfur dioxide': 36.0, 'density': 0.99392, 'pH': 3.11, 'sulphates': 0.78, 'alcohol': 12.4}, {'fixed acidity': 8.8, 'volatile acidity': 0.41, 'citric acid': 0.64, 'residual sugar': 2.2, 'chlorides': 0.09300000000000001, 'free sulfur dioxide': 9.0, 'total sulfur dioxide': 42.0, 'density': 0.9986, 'pH': 3.54, 'sulphates': 0.66, 'alcohol': 10.5}, {'fixed acidity': 8.7, 'volatile acidity': 0.7, 'citric acid': 0.24, 'residual sugar': 2.5, 'chlorides': 0.226, 'free sulfur dioxide': 5.0, 'total sulfur dioxide': 15.0, 'density': 0.9991, 'pH': 3.32, 'sulphates': 0.6, 'alcohol': 9.0}, {'fixed acidity': 7.5, 'volatile acidity': 0.57, 'citric acid': 0.02, 'residual sugar': 2.6, 'chlorides': 0.077, 'free sulfur dioxide': 11.0, 'total sulfur dioxide': 35.0, 'density': 0.9955700000000001, 'pH': 3.36, 'sulphates': 0.62, 'alcohol': 10.8}, {'fixed acidity': 7.2, 'volatile acidity': 0.62, 'citric acid': 0.06, 'residual sugar': 2.5, 'chlorides': 0.078, 'free sulfur dioxide': 17.0, 'total sulfur dioxide': 84.0, 'density': 0.99746, 'pH': 3.51, 'sulphates': 0.53, 'alcohol': 9.7}] is not of type 'object'"
    },
    "message": "Input payload validation failed"
}

DEBUG:root:API call to https://<name>.fiddler.ai/v1/execution/run/<org_id> failed with status 400: The full response message was {
    "errors": {
        "data": "[{'fixed acidity': 8.6, 'volatile acidity': 0.49, 'citric acid': 0.29, 'residual sugar': 2.0, 'chlorides': 0.11, 'free sulfur dioxide': 19.0, 'total sulfur dioxide': 133.0, 'density': 0.9972, 'pH': 2.93, 'sulphates': 1.98, 'alcohol': 9.8}, {'fixed acidity': 8.0, 'volatile acidity': 0.18, 'citric acid': 0.37, 'residual sugar': 0.9, 'chlorides': 0.049, 'free sulfur dioxide': 36.0, 'total sulfur dioxide': 109.0, 'density': 0.9900700000000001, 'pH': 2.89, 'sulphates': 0.44, 'alcohol': 12.7}, {'fixed acidity': 6.8, 'volatile acidity': 0.67, 'citric acid': 0.0, 'residual sugar': 1.9, 'chlorides': 0.08, 'free sulfur dioxide': 22.0, 'total sulfur dioxide': 39.0, 'density': 0.9970100000000001, 'pH': 3.4, 'sulphates': 0.74, 'alcohol': 9.7}, {'fixed acidity': 10.1, 'volatile acidity': 0.31, 'citric acid': 0.35, 'residual sugar': 1.6, 'chlorides': 0.075, 'free sulfur dioxide': 9.0, 'total sulfur dioxide': 28.0, 'density': 0.99672, 'pH': 3.24, 'sulphates': 0.83, 'alcohol': 11.2}, {'fixed acidity': 7.3, 'volatile acidity': 0.45, 'citric acid': 0.36, 'residual sugar': 5.9, 'chlorides': 0.07400000000000001, 'free sulfur dioxide': 12.0, 'total sulfur dioxide': 87.0, 'density': 0.9978, 'pH': 3.33, 'sulphates': 0.83, 'alcohol': 10.5}, {'fixed acidity': 8.4, 'volatile acidity': 0.34, 'citric acid': 0.42, 'residual sugar': 2.1, 'chlorides': 0.07200000000000001, 'free sulfur dioxide': 23.0, 'total sulfur dioxide': 36.0, 'density': 0.99392, 'pH': 3.11, 'sulphates': 0.78, 'alcohol': 12.4}, {'fixed acidity': 8.8, 'volatile acidity': 0.41, 'citric acid': 0.64, 'residual sugar': 2.2, 'chlorides': 0.09300000000000001, 'free sulfur dioxide': 9.0, 'total sulfur dioxide': 42.0, 'density': 0.9986, 'pH': 3.54, 'sulphates': 0.66, 'alcohol': 10.5}, {'fixed acidity': 8.7, 'volatile acidity': 0.7, 'citric acid': 0.24, 'residual sugar': 2.5, 'chlorides': 0.226, 'free sulfur dioxide': 5.0, 'total sulfur dioxide': 15.0, 'density': 0.9991, 'pH': 3.32, 'sulphates': 0.6, 'alcohol': 9.0}, {'fixed acidity': 7.5, 'volatile acidity': 0.57, 'citric acid': 0.02, 'residual sugar': 2.6, 'chlorides': 0.077, 'free sulfur dioxide': 11.0, 'total sulfur dioxide': 35.0, 'density': 0.9955700000000001, 'pH': 3.36, 'sulphates': 0.62, 'alcohol': 10.8}, {'fixed acidity': 7.2, 'volatile acidity': 0.62, 'citric acid': 0.06, 'residual sugar': 2.5, 'chlorides': 0.078, 'free sulfur dioxide': 17.0, 'total sulfur dioxide': 84.0, 'density': 0.99746, 'pH': 3.51, 'sulphates': 0.53, 'alcohol': 9.7}] is not of type 'object'"
    },
    "message": "Input payload validation failed"
}

INFO:root:running api call as POST request
to https://<name>.fiddler.ai/execute/<org_id>/tutorial/debug_model
with headers {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s'}
with payload   {
    "data": [
      {
        "fixed acidity": 8.6,
        "volatile acidity": 0.49,
        "citric acid": 0.29,
        "residual sugar": 2.0,
        "chlorides": 0.11,
        "free sulfur dioxide": 19.0,
        "total sulfur dioxide": 133.0,
        "density": 0.9972,
        "pH": 2.93,
        "sulphates": 1.98,
        "alcohol": 9.8
      },
      {
        "fixed acidity": 8.0,
        "volatile acidity": 0.18,
        "citric acid": 0.37,
        "residual sugar": 0.9,
        "chlorides": 0.049,
        "free sulfur dioxide": 36.0,
        "total sulfur dioxide": 109.0,
        "density": 0.9900700000000001,
        "pH": 2.89,
        "sulphates": 0.44,
        "alcohol": 12.7
      },
      {
        "fixed acidity": 6.8,
        "volatile acidity": 0.67,
        "citric acid": 0.0,
        "residual sugar": 1.9,
        "chlorides": 0.08,
        "free sulfur dioxide": 22.0,
        "total sulfur dioxide": 39.0,
        "density": 0.9970100000000001,
        "pH": 3.4,
        "sulphates": 0.74,
        "alcohol": 9.7
      },
      {
        "fixed acidity": 10.1,
        "volatile acidity": 0.31,
        "citric acid": 0.35,
        "residual sugar": 1.6,
        "chlorides": 0.075,
        "free sulfur dioxide": 9.0,
        "total sulfur dioxide": 28.0,
        "density": 0.99672,
        "pH": 3.24,
        "sulphates": 0.83,
        "alcohol": 11.2
      },
      {
        "fixed acidity": 7.3,
        "volatile acidity": 0.45,
        "citric acid": 0.36,
        "residual sugar": 5.9,
        "chlorides": 0.07400000000000001,
        "free sulfur dioxide": 12.0,
        "total sulfur dioxide": 87.0,
        "density": 0.9978,
        "pH": 3.33,
        "sulphates": 0.83,
        "alcohol": 10.5
      },
      {
        "fixed acidity": 8.4,
        "volatile acidity": 0.34,
        "citric acid": 0.42,
        "residual sugar": 2.1,
        "chlorides": 0.07200000000000001,
        "free sulfur dioxide": 23.0,
        "total sulfur dioxide": 36.0,
        "density": 0.99392,
        "pH": 3.11,
        "sulphates": 0.78,
        "alcohol": 12.4
      },
      {
        "fixed acidity": 8.8,
        "volatile acidity": 0.41,
        "citric acid": 0
DEBUG:root:Request:
  url: https://<name>.fiddler.ai/execute/<org_id>/tutorial/debug_model
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'executor_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2498', 'Content-Type': 'application/json'}
DEBUG:urllib3.connectionpool:https://<name>.fiddler.ai:443 "POST /execute/<org_id>/tutorial/debug_model HTTP/1.1" 200 None
INFO:root:response: {
  "status": "SUCCESS",
  "result": [
    {
      "predicted_quality": 6.505067816095054
    },
    {
      "predicted_quality": 6.654272369837178
    },
    {
      "predicted_quality": 5.3520886518006625
    },
    {
      "predicted_quality": 6.377989322254154
    },
    {
      "predicted_quality": 5.834985553017842
    },
    {
      "predicted_quality": 6.697743954798386
    },
    {
      "predicted_quality": 5.6555750895722285
    },
    {
      "predicted_quality": 4.7949125962300005
    },
    {
      "predicted_quality": 5.7286604971200035
    },
    {
      "predicted_quality": 5.056228085483125
    }
  ]
}
INFO:root:API call to https://<name>.fiddler.ai/execute/<org_id>/tutorial/debug_model succeeded.
Request response:   {
    "status": "SUCCESS",
    "result": [
      {
        "predicted_quality": 6.505067816095054
      },
      {
        "predicted_quality": 6.654272369837178
      },
      {
        "predicted_quality": 5.3520886518006625
      },
      {
        "predicted_quality": 6.377989322254154
      },
      {
        "predicted_quality": 5.834985553017842
      },
      {
        "predicted_quality": 6.697743954798386
      },
      {
        "predicted_quality": 5.6555750895722285
      },
      {
        "predicted_quality": 4.7949125962300005
      },
      {
        "predicted_quality": 5.7286604971200035
      },
      {
        "predicted_quality": 5.056228085483125
      }
    ]
  }
Request:
  url: https://<name>.fiddler.ai/execute/<org_id>/tutorial/debug_model
  method: POST
  headers: {'Authorization': 'Bearer pD8COC39v0_O-dKBCijuOnpvGwsDr_xSvGTfSCy2B5s', 'x-fdlr-fwd': 'executor_service', 'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2498', 'Content-Type': 'application/json'}
result
predicted_quality
0 6.505068
1 6.654272
2 5.352089
3 6.377989
4 5.834986
5 6.697744
6 5.655575
7 4.794913
8 5.728660
9 5.056228
Back to top