Release Notes
Release 24.12 Notes
By Rohan Sharma • Aug 29, 2024
Surfacing Error Messages for Failed Jobs
Error messages for failed jobs are now visible directly on the UI job status page, simplifying the process of diagnosing and resolving issues.
User Selected Default Dashboards
Any dashboard within a project can now be assigned as the default dashboard for a model, with all insights leading directly to the assigned default dashboard.
Custom Feature Impact Feature Release Notes
Introducing Custom Feature Impact: Upload custom feature impact scores for your models, leveraging domain-specific knowledge or external data without requiring the corresponding model artifact.
Easy data upload via API endpoint with required parameters: Model UUID, Feature Names, and Impact Scores.
View updated feature impact scores in:
Model details page
Charts page
Explain page
Flexible update options: Update existing feature impact data by uploading new data for the same model and Seamless integration with existing model artifacts.
Flexible Model Deployment
The python-38
base image is no longer supported.
Release 24.11 Notes
By Sabina Cartacio • Aug 6, 2024
Client Version
Client version 3.3+ is required for the updates and features mentioned in this release.
What's New and Improved:
Performance Analytics (Preview) Embedded in Monitoring Charts
Visualize performance analytics charts as part of the root cause analysis flow for Binary Classification, Multiclass Classification, and Regression models, spanning from confusion matrices, precision recall charts, prediction scatterplots and more.
Release 24.10 Notes
By Sabina Cartacio • July 23, 2024
Client Version
Client version 3.3+ is required for the updates and features mentioned in this release.
What's New and Improved:
Support for applied segments in monitoring charts
Create and apply segments dynamically in monitoring charts for exploratory analysis without requiring them to be saved to the model.
User-Defined Feature Impact
The User-Defined Feature Impact enables you to upload custom feature impact for models. This feature addresses several issues reported by our customers, including model artifact size, onboarding complexity, and the need for custom feature impact.
Key highlights
New method: UploadFeatureImpact
Improved Fiddler UI to display uploaded feature impact
Release 24.9 Note
By Rohan Sharma • July 12, 2024
What's New and Improved
Enhanced access controls
Control access with precision: Manage user access to resources with Role-Based Access Control (RBAC), ensuring the right users have the right permissions.
Simplify user management: Assign roles to users and teams to streamline access control and enhance collaboration. *≠ Protect sensitive resources: Restrict access to sensitive resources, such as models and project settings, with granular permissions.
Work efficiently: Focus on your work without worrying about unauthorized access or data breaches.
Release 24.8 Notes
By Sabina Cartacio • June 18, 2024
Release of Fiddler Platform Version 24.8:
Performance Analytics Charts (Public Preview)
Visualize charts to aid in analyzing model performance for Binary Classification, Multiclass Classification, and Regression models.
Leverage applied segments in Performance Analytics charts to explore problematic cohorts of data.
Release 24.7 Notes
By Parth Domadia • June 4, 2024
What's New and Improved
TBD
Release 24.6 Notes
By Parth Domadia • May 22, 2024
Release of Fiddler Platform Version 24.6:
Performance improvements
Improved the performance of various modules / APIs.
Improved observability which can help monitor health and performance of the operations.
Client Version
Client version 3.1.2+ is required for the updates and features mentioned in this release.
Release 24.5 Notes
By Sabina Cartacio • May 2, 2024
Release of Fiddler Platform Version 24.5:
Support for model versions for streamlined model management
What's New and Improved:
Model Versions
Efficiently manage related models by creating structured versions, facilitating tasks like retraining and comparison analyses.
Users can maintain model lineage, efficiently manage updates, flexibly modify schemas, and adjust parameters.
See: Model Versions for more information.
Airgapped Enrichments (alpha)
For privacy sensitive use cases, all data getting enriched stays within customer premises.
New Deployment Base Images
We have added new deployment base images to support model versioning.
See here.
Client Version
Client version 3.1.0+ is required for the updates and features mentioned in this release.
Release 24.4 Notes
By Parth Domadia • April 26, 2024
Release of Fiddler Platform Version 24.4:
UMAP UI changes
SSO integration changes
New concept: Environments
Fundamental changes to product concepts
What's New and Improved:
UMAP UI
Vertical scrolling instead of horizontal scrolling for data cards
"View More" option to open data cards in maximized modal
Ability to toggle between data cards in the maximized modal
SSO integration changes
Fiddler now integrates with Azure AD SSO, allowing you to leverage existing user roles for access control within Fiddler. This eliminates the need for manual user creation and simplifies user management within your organization. See Azure AD SSO Support for details
Environments
Each Model now has two Envrionments (Pre-Production and Production) used to house data in different ways.
A Model's Pre-Production Environment is used to house non-time series data (Datasets).
A Model's Production Environment is used to house time series data.
See Data and Environments for more
Product concept changes
Datasets are no longer stored at the Project level. Instead, they're stored at the Model level under the Pre-Production Environment.
The Model Details page has been updated with a new design. See more here.
Client Version
Client version 3.0+ is required for the updates and features mentioned in this release.
Client 3.x Release:
We are launching Client 3.x, this is revamped client 2.x as we move to more object oriented based methods. This means, any pipeline setup in client 2.x would eventually be required to upgrade to the new methods. Client 2.x will sunset approximately 6 months post this release. Please take a look at the below resources to help you understand client 3.x and also how you can upgrade your pipelines:
Deprecations and Removals:
All IDs will be UUIDs instead of strings.
Dataset deletion is not allowed anymore.
For API level changes and updates please check client history
Release 24.3 Notes
By Sabina Cartacio • March 19, 2024
Release of Fiddler Platform Version 24.3:
Ability to update metadata column values
Percentage thresholds on incoming traffic for data integrity alerts
Additions to the enrichment framework
What's New and Improved:
Ability to update metadata columns
Support updating metadata columns with new values.
You can observe the changes in Analytics but the Performance metrics of the columns won't be reflected.
See Updating Events for more information.
Percentage thresholds on incoming traffic for data integrity alerts
Added an alert category that quantifies data integrity violations as a percentage-based threshold relative to the total traffic within the time bin. details here Data Integrity Alert Example API usage
New enrichments for LLM monitoring (beta)
New Enrichments:
Profanity
Answer Relevance : LLM based metric, requires OPENAI_API_KEY
Faithfulness : LLM based metric, requires OPENAI_API_KEY
Coherence : LLM based metric, requires OPENAI_API_KEY
Conciseness : LLM based metric, requires OPENAI_API_KEY
Toxicity
Topic
RegexMatch
KeywordDetector
BannedKeywords
See Fiddler Enrichments for more information.
Client Version
Client version 2.5+ is required for the updates and features mentioned in this release.
Release 24.2 Notes
By Sabina Cartacio • January 30, 2024
Release of Fiddler Platform Version 24.2:
Automatically generate a rich set of LLM enrichments
Investigate an alerts' revision history
What's New and Improved:
Alert Revision History
Easily view the revision history of alerts within the monitoring charts UI, including informations on the alerts severity, thresholds, and more.
See: Triggered Alert Revisions for more information.
LLM Enrichments & Embeddings (beta)
Leverage the Enrichments framework to enhance your dataset by adding columns to reflect:
Automatic generation of embeddings
Generation of distance of an embedding from the cluster centroid
PII (Personal Identifiable Information)
Sentiment
Various text statistics
See LLM Monitoring for more information.
Client Version
Client version 2.5+ is required for the updates and features mentioned in this release.
Release 24.1 Notes
By Sabina Cartacio • December 29, 2023
Release of Fiddler Platform Version 24.1:
Support for segments on monitoring charts and alerting
Support for Azure Active Directory
Support for default dashboards for reporting
What's New and Improved:
New Support for Segments
Provides the ability to filter data in monitoring charts using Fiddler Query Language and set alerts on the filtered data.
See Segments for more information.
Azure AD Support
Fiddler now supports integration with Microsoft Azure Active Directory for user authorization
See Single sign on with Azure for more information
Tailored Insights via Default Dashboards
Users can now explore tailored model monitoring dashboards under
Insights
on the homepage, featuring charts on Performance, Traffic, Data Drift, and Data Integrity.With the introduction of Default Dashboards, past monitoring charts have been deprecated.
See Default Dashboards for more information.
Client Version
Client version 2.4+ is required for the updates and features mentioned in this release.
Release 23.7 Notes
By Sabina Cartacio • December 11, 2023
This page enumerates the new features and updates in Release 23.7 of Fiddler.
Release of Fiddler Platform Version 23.7:
Support for multi-target models
Support for alert rule creation on the statistics frequency metric
What's New and Improved:
Custom Target Support in Custom Metrics
Custom Metrics now supports the ability to specify which column to use as the target for all built-in performance metrics
Custom Metrics now support functions for identifying True Positives, True Negatives, False Positives, and False Negatives for both binary classification and multi-class classification model tasks.
See Custom Metrics for more information
Alert Rules on Frequency
Support for alert rules on
Frequency
statistics metricsSee Statistics Guide for more information
Client Version
Client version 2.3+ is required for the updates and features mentioned in this release. This page enumerates the new features and updates in Release 23.6 of the Fiddler platform.
Release 23.6 Notes
By Sabina Cartacio • November 21, 2023
Release of Fiddler Platform Version 23.6:
Embedding visualization with UMAP is now available for all customers
Support for user-defined metrics via custom metrics
Improved alert messaging for Webhook notifications
Improved standalone job status page
What's New and Improved:
Custom Metrics
Ability to define flexible metrics via the UI and client
Custom metrics can be used to create monitoring charts and alerts
Metrics are defined using the Fiddler Query Language (FQL)
Improved Webhook Messaging
Refining "Why Fiddler Generated" alerts for Webhook with enhanced details
Learn how to enable webhook notification.
**Improved Job Status **
Access all your models, datasets, and events jobs through the Jobs page in the navigation bar.
Learn more on the Product Tour
Embedding Visualization with UMAP
Visualize your custom features with high dimensional data using UMAP
Client Version
Client version 2.2+ is required for the updates and features mentioned in this release.
Release 23.5 Notes
By Sabina Cartacio • October 19, 2023
This page enumerates the new features and updates in Release 23.5 of the Fiddler platform.
Release of Fiddler Platform Version 23.5:
Support for new
Vector
data typeSupport for new
Frequency
statistics metricSupport for new
NOT_SET
task typeNew standalone bookmarks page
Contact your customer success team to get access and documentation to the below:
Addition of LLM task
Auto embeddings on Text (Prompt, Response, etc) and Image data
Auto enrichments of Toxicity, PII, and Hallucination on LLM task
3D UMAP visualization of high-dimensional embeddings
Support for user-defined metrics -> Custom metrics. Alerts can be set on these custom metrics and plotted in charts and dashboards.
What's New and Improved:
New Vector input type
You can now create custom features from Vector data type
The DI violation functionality on vector data types is now available to quickly detect any issues in your data pipelines. As vector and embedding pipelines are complex and prone to errors, we hope this functionality will be of value.
Violation of value - Dimension does not match the expected dimension
Violation of nullable - Vector length equals zero or received NULL value
Violation of type - Value is not of the type of VECTOR or VECTOR contains a non-numerical type
Clusters of Text Embeddings have a short summary to understand which of your clusters have the most problematic drift.
Events of a particular cluster can be queried in the 'Analyze' tab of the model. This can be used to pinpoint the most problematic prompts/responses in an LLM scenario or images in a CV scenario. This information can be used to improve your models.
New Task Type for Task-less Models
You can now add models without a task by choosing the
NOT_SET
task type when onboarding your model.Note that task-less models have no restrictions on output and target columns, but performance metrics are disabled when a task is not specified.
Standalone Bookmarks Page
Access all your bookmarked projects, models, datasets, charts, and dashboards through the Bookmarks page in the navigation bar.
Learn more on the Product Tour
Client Version
Client version 2.1+ is required for the updates and features mentioned in this release.
Last updated