> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fiddler.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# CustomJudgeSpec

> Reusable prompt specification for CustomJudge evaluators.

Reusable prompt specification for CustomJudge evaluators.

Provides a structured, validated way to define evaluation prompts with
input/output schemas, transforms, and multi-message templates. A
`CustomJudgeSpec` can be defined once and reused across multiple evaluator
instances or shared across a codebase.

## Parameters

<ParamField path="prompt_template" type="str | list[Message]" required={true}>
  The evaluation prompt. Can be a plain string (wrapped
  in a single user message) or a list of [`Message`](/sdk-api/evals/message) dicts.
</ParamField>

<ParamField path="output_fields" type="Dict[str, OutputField]" required={true}>
  Schema defining the expected output fields.
</ParamField>

<ParamField path="inputs" type="Dict[str, InputFieldSpec] | None" required={false} default="None">
  Optional metadata for template variables.
</ParamField>

<ParamField path="llm_response_fields" type="Dict[str, OutputField] | None" required={false} default="None">
  Optional schema for the LLM response before
  transformation. Required when output fields use `transform`.
</ParamField>

## Example

Defining a reusable faithfulness evaluator spec:

```python theme={null}
from fiddler_evals.evaluators.custom_judge import (
    CustomJudge, CustomJudgeSpec, Message, InputFieldSpec,
    OutputFieldTransform,
)

FAITHFULNESS_SPEC = CustomJudgeSpec(
    prompt_template=[
        Message(role='system', content='Judge faithfulness.'),
        Message(role='user', content=(
            'Question: {{ query }}\n'
            'Documents: {{ docs }}\n'
            'Answer: {{ answer }}'
        )),
    ],
    inputs={
        'query': InputFieldSpec(required=True),
        'docs': InputFieldSpec(required=True),
        'answer': InputFieldSpec(required=True),
    },
    llm_response_fields={
        'is_faithful': {
            'type': 'string',
            'choices': ['faithful', 'not_faithful'],
        },
    },
    output_fields={
        'label': {
            'type': 'string',
            'choices': ['yes', 'no'],
            'transform': OutputFieldTransform(
                source_field='is_faithful',
                value_map={
                    'faithful': 'yes',
                    'not_faithful': 'no',
                },
            ),
        },
    },
)

evaluator = CustomJudge(
    prompt_spec=FAITHFULNESS_SPEC,
    model='openai/gpt-4o',
)
```

## prompt\_template

## output\_fields

## inputs

## llm\_response\_fields

## model\_config

Configuration for the model, should be a dictionary conforming to \[ConfigDict]\[pydantic.config.ConfigDict].