Flexible Model Deployment
Fiddler supports explainability for models with varying dependencies. This is achieved by running each model in its own dedicated container to provide the resources and dependencies that are unique to that model. For example, if your team has two models developed with the same libraries but using different versions you can run both those models by specifying the exact version they were built with.
📘 Note
For models that require monitoring features only, there is no need to upload your model artifact or create a surrogate model as these are only used to support explainability features.
When adding a model artifact to your Fiddler model (see add_artifact), you specify the deployment configuration needed to run it using the DeploymentParams argument. Fiddler has a set of starter images from which to select the configuration most appropriate for running your model. These images vary by included libraries and Python versions. Note you can also customize an image by including your own requirements.txt file along with the model artifact package.
DeploymentParams Arguments
📘 Image upgrades
These Docker images are upgraded routinely to resolve security vulnerabilities and the image tag is updated accordingly. Unsupported Python versions are not provided.
🚧 Be aware
Model version features are supported with the image versions listed above. Images below 2.x for
python-39
,java
andrpy2
will continue to work for existing models using a single version. From 24.5 onwards, model version first class support is added and these require the new model deployment base image tag versions.
Each base image comes with a few pre-installed libraries and these can be overridden and added to by specifying a requirements.txt file inside your model artifact directory where package.py is defined.
Note that the old images deep-learning
and machine-learning
are deprecated (All current versions are still working, but we stopped maintaining and upgrading those). We encourage users to select any plain Python image, and add the necessary libraries in requirements.txt
.
🚧 Be aware
Installing new dependencies at runtime will take time and is prone to network errors.
🚧 Be aware
Your model might require more memory than the default setting. Please ensure you set a sufficient amount of resources. If you see a
ModelServeError
error when adding a model, it means the current settings were not enough to run your model.
cpu
: The amount of CPU (milli cpus) reserved per replica. Both number of features and model complexity can require more CPU allocation.
Both add_artifact and update_artifact methods support passing deployment_params
. For example:
Once the model is added in Fiddler, you can fine-tune the model deployment based on the scaling requirements, using update_model_deployment. This function allows you to:
Last updated