Configure Evaluator Downsampling

​

How it works

• The decision is deterministic and trace-atomic — every span within the same trace receives the same evaluate-or-skip outcome, so multi-step and agentic traces are always scored as a unit.
• A rule left at the default sampling_rate = 1.0 behaves exactly as it does today, with no change in coverage.

​

When to use evaluator downsampling

• Cost reduction at scale. LLM-as-a-Judge evaluators are expensive at production volumes. A representative sample typically preserves drift and quality signal at a fraction of the cost — a 5–20% sample is often equivalent to full coverage for trend monitoring. See Recommended minimum trace volume for choosing a rate.
• High-throughput applications. Reducing evaluation volume relieves backpressure on the enrichment pipeline during traffic spikes.
• Multiple evaluators on the same traffic. When several rules score the same high-volume application, downsampling each one keeps total evaluation cost manageable. (Each application supports up to 100 evaluator rules.)

​

When not to use evaluator downsampling

• Your application is low-volume. On low-traffic applications, sampling removes too many data points and produces noisy, unreliable drift and quality signals.
• You need a very low rate (below ~1%). Rates under roughly 1% introduce statistical noise that can swamp the signal you are trying to measure. If cost is still a concern at 1%, your application volume is probably too low to downsample at all.
• You need every trace scored — for example, for compliance, exhaustive evaluation, or guardrail-style use cases. In those cases, leave the rule at 1.0.
• You want to pause evaluation. Do not use a tiny sampling_rate to “almost turn off” a rule. Deactivate the rule instead (see below).

​

Recommended minimum trace volume

• Aim for at least ~1,000 evaluated traces per monitoring window (the period over which you read drift or quality metrics).
• Avoid downsampling applications that match fewer than a few thousand traces per day for a given rule.
• Keep sampling_rate at or above 0.01 (1%).

​

sampling_rate vs. deactivating a rule

• sampling_rate = 0.0 is rejected — to stop evaluation entirely, deactivate the rule instead of using a near-zero rate.
• A rule is created enabled; the create request does not accept an enabled field. To pause a rule, deactivate it afterward — toggle it off in the UI or send a PATCH with {"enabled": false}.
• Deactivating a rule preserves its configured sampling_rate, so reactivating it restores your previous setting.
• Changing sampling_rate never reactivates a deactivated rule.

​

Configure downsampling

​

Using the REST API

curl -X POST https://<your-fiddler-url>/v3/evaluator-rules \
  -H "Authorization: Bearer $FIDDLER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "custom-judge-sampled",
    "application_id": "<application_id>",
    "evaluator_id": "<evaluator_id>",
    "mapped_input_keys": {"response": "fiddler.contents.gen_ai.llm.output"},
    "sampling_rate": 0.1
  }'

curl -X PATCH https://<your-fiddler-url>/v3/evaluator-rules/<evaluator_rule_id> \
  -H "Authorization: Bearer $FIDDLER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sampling_rate": 0.25}'

​

Using the UI

1. Open your application’s Evaluator Rules tab and click Add Rule.
2. Configure the evaluator, input mappings, and application rules as usual (see Evaluator Rules).
3. Set the Sampling rate to the percentage of matching traces you want evaluated — for example, enter 10 for 10%. Leave it at the default (100) to evaluate everything.
4. Save the rule.

​

Python SDK

​

Related Documentation

• Evaluator Rules - Create and manage Evaluator Rules
• Evaluator Rules REST API - Full create/update API contract, including the sampling_rate field
• Explorer - Inspect ingested traces, including sampled-out traces