AnswerRelevance

• Relevance Assessment: Determines if the answer directly addresses the question
• Three-Level Scoring: Returns high (1.0), medium (0.5), or low (0.0) relevance scores
• Context-Aware: Can use retrieved documents to assess relevance more accurately
• Detailed Reasoning: Provides explanation for the relevance assessment
• Fiddler API Integration: Uses Fiddler’s built-in relevance evaluation model

• RAG Systems: Evaluating if generated answers are relevant to user queries
• Q&A Systems: Ensuring answers stay on topic
• Customer Support: Verifying responses address user queries
• Educational Content: Checking if explanations answer the question
• Research Assistance: Validating that responses are relevant to queries

• 1.0 (High): Answer is fully relevant and directly addresses the question
• 0.5 (Medium): Answer partially addresses the question but may miss some aspects
• 0.0 (Low): Answer does not address the question or is off-topic

​

Parameters

• user_query (str) – The question or query being asked.
• rag_response (str) – The LLM’s response to evaluate.
• retrieved_documents (list[str], optional) – Reference documents for context.
• model (str)
• credential (str | None)
• kwargs (Any)

​

Returns

​

Example

from fiddler_evals.evaluators import AnswerRelevance
evaluator = AnswerRelevance(model="openai/gpt-4o")

# High relevance answer
score = evaluator.score(
    user_query="What is the capital of France?",
    rag_response="The capital of France is Paris."
)
print(f"Relevance: {score.label}")  # "high"
print(f"Score: {score.value}")      # 1.0

# With context documents
score = evaluator.score(
    user_query="What is our refund policy?",
    rag_response="Our refund policy allows returns within 30 days.",
    retrieved_documents=[
        "Refund Policy: Customers may return items within 30 days of purchase.",
        "All returns must include original packaging."
    ]
)

​

name

​

score()

​

Parameters

​

Returns