> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fiddler.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# RAG Evaluation Fundamentals
> Evaluate RAG application quality using Fiddler's built-in evaluators with direct scoring for rapid iteration on retrieval and generation quality.
Evaluate your RAG application's retrieval and generation quality using Fiddler's built-in evaluators. This cookbook demonstrates the direct `.score()` API for rapid iteration on test cases before scaling to full experiments.
**Use this cookbook when:** You have a RAG application and want to quickly assess whether responses are faithful to retrieved documents and relevant to user queries.
**Time to complete**: \~15 minutes
```mermaid theme={null}
graph LR
A["Define Test Cases"] --> B["Score with Evaluators"]
B --> C{"Faithfulness?"}
B --> D{"Relevance?"}
C -->|Yes| E["Grounded"]
C -->|No| F["Hallucination"]
D -->|High| G["On-topic"]
D -->|Low| H["Off-topic"]
style F fill:#f96,stroke:#333
style H fill:#f96,stroke:#333
style E fill:#6f9,stroke:#333
style G fill:#6f9,stroke:#333
```
**Prerequisites**
* Fiddler account with API access
* LLM credential configured in **Settings > LLM Gateway**
* `pip install fiddler-evals pandas`
***
Replace `URL`, `TOKEN`, and credential names with your Fiddler account details. Find your credentials in **Settings > Access Tokens** and **Settings > LLM Gateway**.
```python theme={null}
import pandas as pd
from fiddler_evals import init
from fiddler_evals.evaluators import RAGFaithfulness, AnswerRelevance
URL = 'https://your-org.fiddler.ai'
TOKEN = 'your-access-token'
LLM_CREDENTIAL_NAME = 'your-llm-credential' # From Settings > LLM Gateway
LLM_MODEL_NAME = 'openai/gpt-4o' # Or your preferred model
init(url=URL, token=TOKEN)
# Initialize evaluators
faithfulness = RAGFaithfulness(model=LLM_MODEL_NAME, credential=LLM_CREDENTIAL_NAME)
relevance = AnswerRelevance(model=LLM_MODEL_NAME, credential=LLM_CREDENTIAL_NAME)
```
Define representative test cases that cover both successful and failing RAG scenarios:
```python theme={null}
test_cases = pd.DataFrame(
[
{
'scenario': 'Perfect Match',
'user_query': 'What is the capital of France?',
'retrieved_documents': ['Paris is the capital of France.'],
'rag_response': 'The capital of France is Paris.',
},
{
'scenario': 'Hallucination',
'user_query': 'What are the office hours?',
'retrieved_documents': ['We are closed on weekends.'],
'rag_response': 'We are open 9 AM to 5 PM every day.',
},
{
'scenario': 'Irrelevant Answer',
'user_query': 'How do I reset my password?',
'retrieved_documents': ['To reset, click "Forgot Password".'],
'rag_response': 'Our system is very secure and uses 256-bit encryption.',
},
]
)
```
Use the `.score()` method to evaluate each test case directly. Each evaluator returns a `Score` object with `value`, `label`, and `reasoning`:
```python theme={null}
def evaluate_row(row):
f_score = faithfulness.score(
user_query=row['user_query'],
rag_response=row['rag_response'],
retrieved_documents=row['retrieved_documents'],
)
r_score = relevance.score(
user_query=row['user_query'],
rag_response=row['rag_response'],
)
return pd.Series(
{
'Faithfulness': f_score.label,
'Relevance': r_score.label,
'Status': 'HEALTHY'
if f_score.label == 'yes' and r_score.value >= 0.5
else 'ISSUE DETECTED',
}
)
results = test_cases.join(test_cases.apply(evaluate_row, axis=1))
```
```python theme={null}
results[['scenario', 'Faithfulness', 'Relevance', 'Status']]
```
**Expected output:**
| scenario | Faithfulness | Relevance | Status |
| ----------------- | ------------ | --------- | -------------- |
| Perfect Match | yes | high | HEALTHY |
| Hallucination | no | high | ISSUE DETECTED |
| Irrelevant Answer | yes | low | ISSUE DETECTED |
The hallucination case scores high on relevance (it addresses the question) but fails faithfulness (the response fabricates hours not in the context). The irrelevant answer is faithful to the context but doesn't actually answer the user's question.
***
## Understanding the Evaluators
### [RAG Faithfulness](/sdk-api/evals/rag-faithfulness)
RAG Faithfulness checks whether the response is grounded in the retrieved documents.
* **Inputs**: `user_query`, `rag_response`, `retrieved_documents`
* **Scoring**: Binary — Yes (1.0) / No (0.0)
* **Use for**: Detecting hallucinations where the LLM generates plausible but unsupported claims
### [Answer Relevance](/sdk-api/evals/answer-relevance)
Answer Relevance measures how well the response addresses the user's query.
* **Inputs**: `user_query`, `rag_response` (+ optional `retrieved_documents`)
* **Scoring**: Ordinal — High (1.0), Medium (0.5), Low (0.0)
* **Use for**: Detecting off-topic responses where the LLM answers a different question
**RAG Faithfulness vs. Faithfulness (Centor Model):** This cookbook uses `RAGFaithfulness`, an LLM-as-a-Judge evaluator. Fiddler also offers `FTLResponseFaithfulness`, powered by a proprietary Fiddler Centor Model with different inputs (`context`, `response`) and probability-based scoring (`faithful_prob` 0.0–1.0). These are separate evaluators — see the [Evaluators Glossary](/glossary/enrichment) for details.
***
## Next Steps
* [Running RAG Experiments at Scale](/developers/cookbooks/rag-experiments-at-scale) — Use Datasets and Experiments to evaluate systematically across larger test sets
* [Detecting Hallucinations in RAG](/developers/cookbooks/hallucination-detection-pipeline) — Set up continuous hallucination monitoring in production
* [RAG Health Diagnostics](/concepts/rag-health-diagnostics) — Conceptual guide to the diagnostic triad
***
**Source notebook**: [Fiddler Cookbook: RAG Evaluation Fundamentals](https://github.com/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_RAG_Evaluation_Fundamentals.ipynb)