AnswerRelevance

API reference for AnswerRelevance

AnswerRelevance

AnswerRelevance

Evaluator to assess how well an answer addresses a given question.

The AnswerRelevance evaluator measures whether an LLM’s answer is relevant and directly addresses the question being asked. This is a critical metric for ensuring that LLM responses stay on topic and provide meaningful value to users.

Key Features:

• Relevance Assessment: Determines if the answer directly addresses the question

• Binary Scoring: Returns 1.0 for relevant answers, 0.0 for irrelevant ones

• Detailed Reasoning: Provides explanation for the relevance assessment

• Fiddler API Integration: Uses Fiddler’s built-in relevance evaluation model

Use Cases:

• Q&A Systems: Ensuring answers stay on topic

• Customer Support: Verifying responses address user queries

• Educational Content: Checking if explanations answer the question

• Research Assistance: Validating that responses are relevant to queries

Scoring Logic:

• 1.0 (Relevant): Answer directly addresses the question with relevant information

• 0.0 (Irrelevant): Answer doesn’t address the question or goes off-topic

Parameters

Parameter	Type	Required	Default	Description
prompt	str	✗	None	The question being asked.
response	str	✗	None	The LLM’s response to evaluate.

Returns

A Score object containing: : - value: 1.0 if relevant, 0.0 if irrelevant

• label: String representation of the boolean result

• reasoning: Detailed explanation of the assessment Return type: Score

Example

AnswerRelevance

AnswerRelevance

Evaluator to assess how well an answer addresses a given question.

The AnswerRelevance evaluator measures whether an LLM’s answer is relevant and directly addresses the question being asked. This is a critical metric for ensuring that LLM responses stay on topic and provide meaningful value to users.

Key Features:

• Relevance Assessment: Determines if the answer directly addresses the question

• Binary Scoring: Returns 1.0 for relevant answers, 0.0 for irrelevant ones

• Detailed Reasoning: Provides explanation for the relevance assessment

• Fiddler API Integration: Uses Fiddler’s built-in relevance evaluation model

Use Cases:

• Q&A Systems: Ensuring answers stay on topic

• Customer Support: Verifying responses address user queries

• Educational Content: Checking if explanations answer the question

• Research Assistance: Validating that responses are relevant to queries

Scoring Logic:

• 1.0 (Relevant): Answer directly addresses the question with relevant information

• 0.0 (Irrelevant): Answer doesn’t address the question or goes off-topic

Parameters

Parameter	Type	Required	Default	Description
prompt	str	✗	None	The question being asked.
response	str	✗	None	The LLM’s response to evaluate.

Returns

A Score object containing: : - value: 1.0 if relevant, 0.0 if irrelevant

• label: String representation of the boolean result

• reasoning: Detailed explanation of the assessment Return type: Score

Example

from fiddler_evals.evaluators import AnswerRelevance
evaluator = AnswerRelevance()

# Relevant answer
score = evaluator.score(
prompt="What is the capital of France?",
response="The capital of France is Paris."
)
print(f"Relevance: {score.value}")  # 1.0
print(f"Reasoning: {score.reasoning}")

# Irrelevant answer
score = evaluator.score(
prompt="What is the capital of France?",
response="I like pizza and Italian food."
)
print(f"Relevance: {score.value}")  # 0.0

{% hint style="info" %}
This evaluator uses Fiddler’s built-in relevance assessment model
and requires an active connection to the Fiddler API.
{% endhint %}

#### name *= 'answer_relevance'*

#### score()

Score the relevance of an answer to a question.

#### Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `prompt` | `str` | ✗ | `None` | The question being asked. |
| `response` | `str` | ✗ | `None` | The LLM’s response to evaluate. |

#### Returns
A Score object containing:
  : - value: 1.0 if relevant, 0.0 if irrelevant
- label: String representation of the boolean result
- reasoning: Detailed explanation of the assessment
**Return type:** Score