ML Flow Integration
Fiddler allows your team to onboard, monitor, explain, and analyze your models developed with MLFlow.
This guide shows you how to ingest the model metadata and artifacts stored in your MLFlow model registry and use them to set up model observability in the Fiddler Platform:
- Exporting Model Metadata from MLFlow to Fiddler
- Uploading Model Artifacts to Fiddler for XAI
Adding Model Information
Using the MLFlow API you can query the model registry and get the model signature which describes the inputs and outputs as a dictionary. You can use this dictionary to build out the ModelInfo object required to the model to Fiddler:
import mlflow
from mlflow.tracking import MlflowClient
client = MlflowClient() #initiate MLFlow Client
#Get the model URI
model_version_info = client.get_model_version(model_name, model_version)
model_uri = client.get_model_version_download_uri(model_name, model_version_info)
#Get the Model Signature
mlflow_model_info = mlflow.models.get_model_info(model_uri)
model_inputs_schema = model_info.signature.inputs.to_dict()
model_inputs = [ sub['name'] for sub in model_inputs_schema ]
Now you can use the model signature to build the Fiddler ModelInfo object:
features = model_inputs
model_task = fdl.ModelTask.BINARY_CLASSIFICATION
model_info = fdl.ModelInfo.from_dataset_info(
dataset_info = client.get_dataset_info(YOUR_PROJECT,YOUR_DATASET),
target = "TARGET COLUMN",
dataset_id=DATASET_ID,
model_task=model_task,
features=features,
outputs=['output_column'])
Uploading Model Files
Sharing your model artifacts helps Fiddler explain your models. By leveraging the MLFlow API you can download these model files:
import os
import mlflow
from mlflow.store.artifact.models_artifact_repo import ModelsArtifactRepository
model_name = "example-model-name"
model_stage = "Staging" # Should be either 'Staging' or 'Production'
mlflow.set_tracking_uri("databricks")
os.makedirs("model", exist_ok=True)
local_path = ModelsArtifactRepository(
f'models:/{model_name}/{model_stage}').download_artifacts("", dst_path="model")
print(f'{model_stage} Model {model_name} is downloaded at {local_path}')
Once you have the model file, you can create a package.py file in this model directory that describes how to access this model.
Finally, you can upload all the model artifacts to Fiddler:
client.add_model_artifact(
project_id=PROJECT_ID,
model_id=MODEL_ID,
model_dir='model/',
)
Alternatively, you can skip uploading your model and use Fiddler to generate a surrogate model to get low-fidelity explanations for your model.
Updated 17 days ago
Once your MLFlow models are on-boarded successfully you can start publishing production events