Selecting Enrichments

Fiddler offers enrichments out of the box to monitor different aspects of LLM applications. Use the below table to select the right enrichment for your specific use case.

This table provides high level information on the metric, the enrichment to use to measure the metric, if the metric uses LLMs, and if so, what LLM it uses.

If you have a use case not covered by the below enrichments out of the box, please contact your administrator.

Metric
Metric Category
Description
Enrichment
LLM Used?
LLM Type

Hallucination

This enrichment identifies the accuracy and reliability of facts presented in AI-generated texts

faithfulness

Yes

OpenAI

Hallucination

This enrichment identifies the accuracy and reliability of facts presented in AI-generated texts. It is generated by Fiddler's Fast Trust Models

ftl_response_faithfulness

Yes

Fiddler Fast Trust Model

Hallucination

This enrichment measures the pertinence of AI-generated responses to their inputs

answer_relevance

Yes

OpenAI

Hallucination

This enrichment evaluates the brevity and clarity of AI-generated responses

conciseness

Yes

OpenAI

Hallucination

This enrichment assesses the logical flow and clarity of AI-generated responses

coherence

Yes

OpenAI

Safety

This enrichment classifies whether a piece of text is toxic or not

toxicity

Yes

OpenAI

Safety

This enrichment detects the presence of jailbreak attempts by the user. It is generated by Fiddler's Fast Trust Models.

ftl_prompt_safety

Yes

Fiddler Fast Trust Model

Safety

This enrichment generates 10 different safety metrics to measure texts upon. These metrics are: illegal, hateful, harassing, racist, sexist, violent, sexual, harmful, unethical, jailbreaking

ftl_prompt_safety

Yes

Fiddler Fast Trust Model

Safety

This enrichment flags the presence of sensitive information within textual data

pii

No

Safety

This enrichment compares the text with a regular expression string

regex_match

No

Safety

This enrichment classifies the text into several preset dimensions using a zero-shot classifier

topic_model

No

Safety

This enrichment detects the presence of banned keywords configured by the user

banned_keywords

No

Safety

This enrichment flags the use of offensive or inappropriate language

profanity

No

Safety

This enrichment identifies the language of the source text

language_detection

No

Text Statistics

This enrichment provides classic text evaluation methods such as BLEU, ROUGE, and Meteor

evaluate

No

Text Statistics

This enrichment provides sentiment analysis of the target text

sentiment

No

Text Statistics

This enrichment provides various text statistics such as character/letter count, flesh kinkaid, and others

textstat

No

Text Validation

Evaluates different query dialects for syntax correctness.

sql_validation

No

Text Validation

Validates JSON for correctness and optionally against a user-defined schema.

json_validation

No

Last updated

© 2024 Fiddler Labs, Inc.