Introduction

​

Overview

Complete API reference documentation for the fiddler-evals package.

​

Components

​

Connection

Connection management and initialization

• Connection
• init

​

Entities

Core entity classes for interacting with the platform

• Application
• Dataset
• Experiment
• ExperimentItemStatus
• ExperimentStatus
• Project

​

Evaluators

Built-in and custom evaluators for LLM assessment

• AnswerRelevance
• Coherence
• Conciseness
• ContextRelevance
• CustomJudge
• CustomJudgeSpec
• EvalFn
• Evaluator
• FTLPromptSafety
• FTLResponseFaithfulness
• InputFieldSpec
• Message
• OutputField
• OutputFieldTransform
• RAGFaithfulness
• RegexMatch
• RegexSearch
• Sentiment
• TopicClassification

​

Pydantic Models

Pydantic data models and validation schemas

• DatasetItem
• NewDatasetItem
• Score
• ScoreStatus

​

Runner

Evaluation execution and orchestration

• evaluate