# FTLResponseFaithfulness

Evaluator to assess response faithfulness using Fiddler Centor Models.

The FTLResponseFaithfulness evaluator uses Fiddler's proprietary Centor Model to evaluate how faithful an LLM response is to the provided context. This evaluator helps ensure that responses accurately reflect the information in the source context and don't contain hallucinated or fabricated information.

Key Features:

* **Faithfulness Assessment**: Evaluates how well the response reflects the context
* **Probability-Based Scoring**: Returns probability scores (0.0-1.0) for faithfulness
* **Context-Response Alignment**: Compares response against provided context
* **Fiddler Centor Model**: Uses Fiddler's proprietary faithfulness evaluation model
* **Hallucination Detection**: Identifies responses that go beyond the context

Faithfulness Categories Evaluated:

* **faithful\_prob**: Probability that the response is faithful to the context

Use Cases:

* **RAG Systems**: Ensuring responses stay grounded in retrieved context
* **Document Q\&A**: Verifying answers are based on provided documents
* **Fact-Checking**: Validating that responses don't contain fabricated information
* **Content Validation**: Ensuring responses accurately reflect source material
* **Hallucination Detection**: Identifying responses that go beyond the context

Scoring Logic: : The faithfulness score represents the probability that the response is faithful to the context:

* **0.0-0.3**: Low faithfulness (likely contains hallucinated information)
* **0.3-0.7**: Medium faithfulness (some information may not be grounded)
* **0.7-1.0**: High faithfulness (response accurately reflects context)

## Parameters

* **response** (*str*) – The LLM response to evaluate for faithfulness.
* **context** (*str*) – The source context that the response should be faithful to.
* **score\_name\_prefix** (*str* *|* *None*)
* **score\_fn\_kwargs\_mapping** (*ScoreFnKwargsMappingType* *|* *None*)

## Returns

A list of Score objects containing:

* name: The faithfulness category name ("faithful\_prob")
* evaluator\_name: "FTLResponseFaithfulness"
* value: Probability score (0.0-1.0) for faithfulness

**Return type:** list\[Score]

## Raises

**ValueError** – If the response or context is empty or None.

## Example

```python
from fiddler_evals.evaluators import FTLResponseFaithfulness
evaluator = FTLResponseFaithfulness()
```

```python
# Faithful response
context = "The capital of France is Paris. It is located in northern Europe."
response = "Paris is the capital of France."
scores = evaluator.score(response=response, context=context)
for score in scores:

    print(f"{score.name}: {score.value}")

# faithful_prob: 0.95

# Unfaithful response with hallucination
context = "The capital of France is Paris."
response = "The capital of France is Paris, and it has a population of 2.1 million people."
scores = evaluator.score(response=response, context=context)
for score in scores:

    print(f"{score.name}: {score.value}")

# faithful_prob: 0.65 (population info not in context)

# Highly unfaithful response
context = "The capital of France is Paris."
response = "The capital of France is London."
scores = evaluator.score(response=response, context=context)
for score in scores:

    print(f"{score.name}: {score.value}")

# faithful_prob: 0.05

# Filter based on faithfulness threshold
faithful_score = next(s for s in scores if s.name == "faithful_prob")
if faithful_score.value < 0.7:

    print("Response flagged as potentially unfaithful")
```

{% hint style="info" %}
This evaluator is designed for response faithfulness assessment and should be used in conjunction with other evaluation metrics for comprehensive response quality assessment. The probability scores should be interpreted in context and combined with other quality measures for robust response validation.
{% endhint %}

## name *= 'ftl\_response\_faithfulness'*

## score()

Score the faithfulness of a response to its context.

### Parameters

| Parameter  | Type  | Required | Default | Description                                                 |
| ---------- | ----- | -------- | ------- | ----------------------------------------------------------- |
| `response` | `str` | ✓        | `-`     | The `LLM` response to evaluate for faithfulness.            |
| `context`  | `str` | ✓        | `-`     | The source context that the response should be faithful to. |

### Returns

A Score object for faithfulness probability.

**Return type:** Score


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fiddler.ai/api/fiddler-evals-sdk/evaluators/ftl-response-faithfulness.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.