LiteLLM Guardrails

Overview

Fiddler implements the LiteLLM Generic Guardrail API spec, allowing you to plug Fiddler’s guardrails directly into a LiteLLM proxy gateway. Once configured, every LLM request routed through the proxy is checked by Fiddler — in the recommended pre_call mode, before it ever reaches the model. This page takes you from a clean LiteLLM installation to a verified block, redact, and allow decision in about ten minutes. Pick your path before you start:

Goal	Recommended path
Trace prompts, responses, tokens, costs, and latency	LiteLLM Integration
Block or redact PII and secrets inline	This page
Observe and protect the same traffic	Configure both — they are independent and share no settings

The configuration on this page checks for:

Secrets — API keys, tokens, credentials, and connection strings
PII — personally identifiable information (emails, phone numbers, SSNs, credit cards, etc.)

Each check independently blocks or redacts — see Check Reference for details.

How It Works

Request flow: a client sends POST /v1/chat/completions to the LiteLLM Proxy, which calls the Fiddler Guardrails API at /v3/guardrails/litellm/beta/litellm_basic_guardrail_api as a pre_call or post_call check. Fiddler returns one of three actions — NONE (clean, forwarded unchanged), GUARDRAIL_INTERVENED (sensitive content redacted, then forwarded), or BLOCKED (request rejected, error returned) — and the proxy forwards the sanitized request to the LLM provider.

LiteLLM calls the Fiddler endpoint with the extracted text from the request (or response). Fiddler returns one of three actions:

Action	Meaning	LiteLLM behavior
`NONE`	No issues detected	Forwards the request unchanged
`GUARDRAIL_INTERVENED`	Sensitive content redacted	Forwards the request with redacted `texts`
`BLOCKED`	Request must not proceed	Returns an HTTP 400 error to the client

Prerequisites

Before you start, you need:

A Fiddler environment with Guardrails enabled. Your instance base URL (for example https://your-company.fiddler.ai) is the api_base prefix. If the endpoint returns HTTP 403, Guardrails is not enabled for your deployment — contact Fiddler.
A Fiddler API key, created under Settings → Credentials.
LiteLLM ≥ 1.91.0 (this page was verified against LiteLLM 1.96.2).
An LLM provider API key — the quick start uses OpenAI.
uv installed, or another way to run the LiteLLM proxy.
Network access from the proxy host to your Fiddler instance and your LLM provider, over HTTPS.

Expect the quick start to take about ten minutes.

Minimum supported LiteLLM version: 1.91.0. Older versions accept the settings on this page without an error and then silently ignore some of them: fail_on_error is only honored from 1.91.0, and versions below 1.87.0 return HTTP 500 instead of HTTP 400 for blocked requests (fixed upstream in BerriAI/litellm#27617). A misconfigured version floor fails silently, not loudly — pin your LiteLLM version.

Quick Start

Set Environment Variables

# Fiddler API key — create one under Settings → Credentials
export FIDDLER_API_KEY="<your-fiddler-api-key>"

# LiteLLM resolves os.environ/ references as whole values, so the
# guardrail's Authorization header is exported as one complete string.
export FIDDLER_AUTH_HEADER="Bearer $FIDDLER_API_KEY"

# LLM provider key (name varies by provider)
export OPENAI_API_KEY="<your-openai-api-key>"

FIDDLER_AUTH_HEADER exists because the config below references it with os.environ/FIDDLER_AUTH_HEADER — the key stays in your environment and never appears in the config file.

Create a Minimal config.yaml

model_list:
  - model_name: gpt-5-mini
    litellm_params:
      model: openai/gpt-5-mini
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "fiddler"
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call]
      api_base: https://<your-fiddler-instance>/v3/guardrails/litellm
      default_on: true
      # All three failure controls are set explicitly, even where the
      # value matches the default — see Failure Behavior for what each
      # one covers.
      unreachable_fallback: fail_closed   # LiteLLM default: fail_closed
      fail_on_error: true                 # LiteLLM default: true
      headers:
        Authorization: os.environ/FIDDLER_AUTH_HEADER
      additional_provider_specific_params:
        # Fiddler's default is open — closed is the secure override.
        failure_mode: closed
        pii:
          enabled: true
          config:
            threshold: 0.5
            mode: redact          # "redact" (default) or "block"
        secrets:
          enabled: true
          config:
            mode: block

LiteLLM appends /beta/litellm_basic_guardrail_api to api_base automatically. The full endpoint called is https://<your-fiddler-instance>/v3/guardrails/litellm/beta/litellm_basic_guardrail_api.

Start LiteLLM

uvx --from "litellm[proxy]" --with "fastapi<0.140.7" litellm --config config.yaml --port 4000

The proxy is ready when the startup log ends with:

INFO:     Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)

The "fastapi<0.140.7" pin works around a LiteLLM 1.96.x dependency gap: its declared FastAPI range admits versions that removed an internal import the proxy still uses, so an unpinned install fails at startup with ImportError: cannot import name 'get_flat_dependant'. Remove the pin once upstream ships a fix. To run the proxy in Docker instead, see the LiteLLM Docker guide — the guardrails configuration above is identical.

Verify Block, Redact, and Allow

1. A synthetic secret is blocked. Secrets use mode: block, so the request never reaches the model:

curl -sS -i http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"Here is my key: sk-ant-api03-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}]}'

Expected response — HTTP 400 with the guardrail’s reason:

{
  "error": {
    "message": "Secret detected: Anthropic API Key (1 total)",
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

2. Synthetic PII is redacted in place. PII uses mode: redact, so the request succeeds and the model receives sanitized text:

curl -sS http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"My email is jane.doe@example.com and my phone is 555-867-5309."}]}'

Expected: HTTP 200 and a normal model response. The redaction happens between the proxy and the model, so to see it directly, call the Fiddler endpoint with the same text:

curl -sS https://<your-fiddler-instance>/v3/guardrails/litellm/beta/litellm_basic_guardrail_api \
  -H "Authorization: Bearer $FIDDLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"texts":["My email is jane.doe@example.com and my phone is 555-867-5309."],"request_data":{},"input_type":"request","additional_provider_specific_params":{"pii":{"enabled":true,"config":{"threshold":0.8,"mode":"redact"}},"secrets":{"enabled":true,"config":{"mode":"block"}}}}'

Expected response — the exact sanitized text the model would receive:

{"action": "GUARDRAIL_INTERVENED", "texts": ["My email is [REDACTED EMAIL] and my phone is [REDACTED PHONE_NUMBER]."]}

3. Benign traffic passes through. Ordinary requests reach the model unchanged:

curl -sS http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"What is the capital of France?"}]}'

Expected: HTTP 200 with a normal completion (for example, “The capital of France is Paris.”).You now have a working guardrail: secrets are blocked with HTTP 400, PII is redacted before the model sees it, and clean traffic is unaffected.

What This Configuration Enables

The quick-start configuration runs exactly two checks on every request, before the LLM call:

Check	Mode	Effect on detection
Secrets	`block`	Request rejected with HTTP 400 and a reason like `Secret detected: Anthropic API Key (1 total)`
PII	`redact` (threshold 0.8)	Detected values replaced in place, for example `[REDACTED EMAIL]`; request proceeds

Check selection is explicit: only the checks you list with enabled: true run.

If any check key is present in additional_provider_specific_params, only the checks explicitly listed with enabled: true will run. To tune one check without silently disabling another, list every check you want with an explicit enabled value. Two related traps: a check must be a mapping with enabled: true — pii: {} or pii: true enables nothing — and listing a single check with enabled: false and nothing else disables all checks.

If you omit additional_provider_specific_params entirely, server-side defaults apply instead. Always configure checks explicitly so the behavior matches this page. The three failure controls in the config are also explicit, even where they match the default, so that a secure posture survives upstream default changes:

Setting	Owner	Default	The quick-start value
`unreachable_fallback`	LiteLLM	`fail_closed`	`fail_closed` (explicit)
`fail_on_error`	LiteLLM	`true`	`true` (explicit)
`failure_mode`	Fiddler	`open`	`closed` — an explicit secure override of Fiddler’s default

See Failure Behavior for what each control covers and how they compose.

What Gets Scanned

Check	Free-text messages	Tool-call arguments
PII	Redact or block, per `mode`	Always blocked on detection
Secrets	Redact or block, per `mode`	Always blocked on detection

Free-text messages — user, assistant, and system message text forwarded by LiteLLM in texts[]. With mode: redact, detected values are replaced in place (for example [REDACTED EMAIL]).
Tool-call arguments — structured JSON in tool_calls[].function.arguments. Detections always block, because redacting inside structured JSON is unsafe — see Tool Call Handling.

Only texts and tool_calls[].function.arguments are scanned. Other request fields (tool definitions, images, headers, metadata) are accepted for protocol compatibility but never scanned — see the API Reference.

Supported Modes and Endpoints

Fiddler supports all three LiteLLM guardrail modes. Set mode in your proxy config to one or more of:

Mode	When it runs	Can modify text?
`pre_call`	Before the LLM call, on the request	Yes — redact in place
`post_call`	After the LLM call, on the response	Yes — redact in place
`during_call`	In parallel with the LLM call, on the request	No — block or allow only

during_call hides guardrail latency behind the LLM round-trip: the response is held until the check completes, but because the LLM call has already started, the input cannot be modified. Use it only when block-or-allow is sufficient. You can combine modes. For example, mode: [pre_call, post_call] scans both input and output:

guardrails:
  - guardrail_name: "fiddler"
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]

Supported Endpoints

The guardrail runs on LiteLLM proxy endpoints that carry text content, including /v1/chat/completions (OpenAI format) and /v1/messages (Anthropic format). LiteLLM extracts text from the request and forwards it to the Fiddler endpoint regardless of the upstream provider format, and applies redacted text before the provider call on both endpoint families.

Per-Request Control

With default_on: true (the quick-start configuration), the guardrail runs on every request automatically. When default_on is omitted, LiteLLM treats it as false, so set it explicitly.

Selective Activation

Set default_on: false in the proxy config, then activate the guardrail on individual requests by passing guardrails in the request body:

{
  "model": "gpt-5-mini",
  "messages": [{"role": "user", "content": "..."}],
  "guardrails": ["fiddler"]
}

Requests without "guardrails": ["fiddler"] bypass the guardrail entirely — the Fiddler endpoint is never called.

Check Reference

Each check is configured entirely in the LiteLLM proxy’s additional_provider_specific_params:

enabled (true / false) — whether the check runs at all.
mode (redact / block) — what happens on a detection. The default is redact for both checks.
threshold and entities (PII only) — detection sensitivity and entity selection.

Secrets

Detects credentials, API keys, and tokens.

Mode	Free-text messages	Tool-call arguments
`redact` (default)	Value replaced, for example `[REDACTED ANTHROPIC_API_KEY]`; request forwarded	Always blocked
`block`	Request blocked with `Secret detected: <type> (N total)`	Request blocked

secrets:
  enabled: true
  config:
    mode: block             # "redact" (default) or "block"

PII

Detects personally identifiable information.

Mode	Free-text messages	Tool-call arguments
`redact` (default)	Value replaced, for example `[REDACTED EMAIL]`; request forwarded	Always blocked
`block`	Request blocked entirely	Request blocked

pii:
  enabled: true
  config:
    mode: redact            # "redact" (default) or "block"
    threshold: 0.5          # detection confidence threshold (default: 0.5)
    entities:               # optional: REPLACES the default entity set
      - email
      - phone number

By default the check uses Fiddler’s curated PII entity set. Supplying entities switches the check to exactly the entities you list — it replaces the default set rather than extending it. For the available entity types, see the PII Detection tutorial. Detection is confidence-based: values below threshold are left untouched, so entities like personal names may pass through at the default threshold while high-precision entities like emails and phone numbers redact reliably.

Failure Behavior

Three settings control what happens when a guardrail check cannot complete. Two belong to LiteLLM and one to Fiddler, and each observes a different class of failure.

Setting	Owner	Default	Governs
`unreachable_fallback`	LiteLLM	`fail_closed`	The Fiddler endpoint being unreachable: network and DNS errors, timeouts, and HTTP 502/503/504
`fail_on_error`	LiteLLM	`true`	Any guardrail call failure: everything above, plus other non-2xx statuses (400, 401, 500), malformed or unparseable response bodies, and request serialization errors
`failure_mode`	Fiddler	`open`	Internal Fiddler check failures — detector timeouts and errors that LiteLLM cannot see, because the endpoint still returns HTTP 200

How the LiteLLM Controls Compose

LiteLLM lets a failed guardrail call through when either condition holds: the failure is in the unreachable class and unreachable_fallback: fail_open, or fail_on_error: false. There is no conflict case — fail-open wins if either setting grants it. In practice:

With the quick-start values (fail_closed + true), every guardrail call failure rejects the request.
unreachable_fallback: fail_open with fail_on_error: true tolerates outages (unreachable endpoint, gateway 502/503/504) while still rejecting on malformed responses and other non-2xx errors.
fail_on_error: false lets every guardrail failure through, regardless of unreachable_fallback — it is the broadest bypass and deserves deliberate consideration.

fail_on_error: false cannot override a valid decision: a successfully parsed action: BLOCKED response always blocks.

What the Client Sees

Outcome	HTTP status to the client	Body
Parsed `BLOCKED` decision	400	`{"error": {"message": "<blocked_reason>", "type": "None", "param": "None", "code": "400"}}`
Guardrail call failed, fail-closed	500	`{"error": {"message": "Generic Guardrail API failed: <cause>", ...}}`
Guardrail call failed, fail-open	200	Normal LLM response — the request proceeded unscanned; LiteLLM logs a CRITICAL `Generic Guardrail API error (fail-open)` line

A blocked request is a 400 for non-streaming calls and for a pre_call block on a streaming call — the check runs before generation starts, so “before the first chunk” is trivially true there. It does not hold for a post_call block at default settings: post_call needs the complete response before it can check anything, and on a streaming call the client has already received every chunk by the time that check even runs. A post_call block on a streamed response does not produce a 400 at all. The client gets a 200, the model’s full output streams through as normal delta.content chunks, and only after that does a synthetic data: {"error": {...}} event get appended, which most SSE clients (including the OpenAI SDK) will surface as an exception, but only after the real content already arrived intact. GUARDRAIL_INTERVENED (redact) behaves the same way at default settings. LiteLLM forwards a streamed response to the client chunk-by-chunk as the model generates it, and by default it does not hold those chunks back behind a post_call guardrail’s verdict. The first chunk reaches the client in about a second, well before the check’s roughly twelve-second window can complete, so a secret or PII value the model streams back reaches the client before any redaction decision exists to apply. Streaming protection is available, but it is off by default and covers one route. Fiddler’s integration does not define LiteLLM’s streaming-aware async_post_call_streaming_iterator_hook itself. It does not need to: because it implements apply_guardrail, the proxy routes its streamed responses through the shared UnifiedLLMGuardrails implementation of that hook. The shared hook’s default mode, block_only, is the unprotected one — raw chunks stream out as the model produces them, text rewrites are discarded, and only an end-of-stream BLOCK terminates the stream, by which point the content has already been delivered. Setting streaming_transform_mode: incremental_diff changes that:

guardrails:
  - guardrail_name: "fiddler"
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      streaming_transform_mode: incremental_diff   # LiteLLM default: block_only

In this mode the proxy withholds the raw chunks, scans the accumulated text, and emits the guardrailed text as new deltas instead — so a redaction reaches the client in place of the raw value, and a BLOCK ends the stream before the offending text is emitted. Two limits apply. It covers the OpenAI chat completions streaming path only (string delta.content); Anthropic /v1/messages and the other streaming surfaces fall back to the unprotected block_only behavior. And it is ignored when streaming_end_of_stream_only is set. LiteLLM carries a second, buffer-everything mechanism — streaming_buffer_until_moderated, which withholds every chunk until the assembled response has been moderated, the same shape as the opt-in reject mitigation AgentGateway’s guardrail page describes. It is not reachable from generic_guardrail_api configuration today: the flag is read from an attribute on the guardrail object, and the integration neither accepts it as a parameter nor sets it. Its buffered-replay design also disables itself for content-rewriting guardrails, so it would cover block and not redaction even if it were exposed. If you stream on a route incremental_diff does not cover, either disable streaming for that route or treat post_call as best-effort there.

Fiddler’s `failure_mode`

failure_mode controls what happens when an internal check fails to complete — an inference timeout, an inference server error, or content that exceeds the scan limit. LiteLLM cannot see these failures because the Fiddler endpoint still returns HTTP 200 with a normal action, so the LiteLLM-side controls never fire.

additional_provider_specific_params:
  failure_mode: closed  # default when omitted: open

Value	Behavior
`open` (default)	The failed check is skipped — the request proceeds, potentially unscanned
`closed`	The request is blocked with a reason like `Guardrail check failed to complete (...)` — unscanned content never reaches the LLM

The default is open for backward compatibility. For security-sensitive deployments, set failure_mode: closed, as the quick start does. With all three controls at their secure values (fail_closed, fail_on_error: true, failure_mode: closed), no request bypasses scanning, whether the failure is at the transport, protocol, or detector level. One caveat: an unexpected exception in the Fiddler pipeline under failure_mode: open surfaces as an HTTP 5xx rather than a 200, and is then handled by LiteLLM’s fail_on_error. A deliberate fail-open posture (failure_mode: open plus fail_on_error: false) trades protection for availability: traffic keeps flowing during a guardrail outage, unscanned. Choose it only for traffic where an unscanned request is acceptable, and monitor the LiteLLM logs for the fail-open CRITICAL entries.

Timeouts

The guardrail check has one customer-facing timeout: the wall-clock deadline for the whole check pipeline, set per request via timeout (seconds) in additional_provider_specific_params:

additional_provider_specific_params:
  timeout: 12               # default: 12, max: 60

What you set is what you get — there is no hidden padding. Omitted or invalid values fall back to the default (12 seconds); values above the maximum are clamped to 60. When the deadline expires before the checks complete, failure_mode governs the outcome: under open the request proceeds unscanned, under closed it is blocked.

Text Length Limits

The maximum total text length scanned per request is controlled by the GUARDRAILS_MAX_TEXT_LENGTH environment variable on the Fiddler server (default: 50,000 characters). Messages are concatenated until the cap is reached; segments beyond the cap are skipped, never truncated mid-segment:

Content type	`failure_mode: open`	`failure_mode: closed`
Free-text messages	Skipped — request proceeds with the oversized entry unscanned	Blocked
Tool-call arguments	Always blocked	Blocked

Tool Call Handling

When tool_calls contain PII or secrets, Fiddler blocks rather than redacts. Why: tool-call arguments are structured JSON that the downstream application will parse and execute. Replacing a value like an email address with [REDACTED EMAIL] would cause send_email to attempt delivery to a nonsensical address — producing unpredictable behavior that is worse than blocking outright. Example — blocked tool call:

{
  "role": "assistant",
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "send_email",
      "arguments": "{\"to\": \"jane.doe@example.com\", \"body\": \"Phone: 555-867-5309\"}"
    }
  }]
}

Fiddler returns:

{"action": "BLOCKED", "blocked_reason": "PII or secrets detected in tool call arguments. Cannot redact tool call arguments — blocking request."}

LiteLLM translates this into an HTTP 400 error for the client.

Coverage Differs by Client Wire Format

This blocking guarantee holds for requests sent to LiteLLM in OpenAI Chat Completions format (/v1/chat/completions), where message.tool_calls is a native field that LiteLLM forwards to Fiddler on the pre-call hook. It does not currently hold for the Anthropic Messages format (/v1/messages), used by Claude Code and other Anthropic-native clients, and the gap is narrower than “enable post-call”: LiteLLM’s guardrail-translation layer only extracts an assistant tool_use block’s arguments into tool_calls on the post-call (response) hook, and only for the tool call the model is generating in that response. Enabling post-call protects that one turn, at the moment the model produces the tool call — but a tool_use block already sitting in conversation history (replayed in every later request, which is how agentic clients like Claude Code send it) is never re-extracted on the pre-call hook, whether or not post-call is also enabled. Fiddler’s LLM Gateway runs pre-call only, so neither hook scans it. The root cause is in LiteLLM itself (AnthropicMessagesHandler._extract_content_block has no case for tool_use, only for tool_result), and Fiddler has no visibility into content LiteLLM never forwards, so there is no Fiddler-side workaround. Upstream status (as of 2026-09-16). The gap applies to every LiteLLM release up to and including 1.98.0, the version Fiddler’s LLM Gateway pins. BerriAI/litellm#40380 was closed by BerriAI/litellm#40984, which makes the Anthropic handler extract historical tool_use arguments into tool_calls. That fix is merged but has not yet appeared in a stable LiteLLM release, and BerriAI/litellm#41265 reports that it does not reach guardrails that read only the texts[] input — so do not assume the gap is closed on the strength of #40984 alone. Re-verify against your own gateway version. If you depend on this guarantee and your traffic goes through /v1/messages, treat tool call arguments in conversation history as unscanned.

Tool Results

Fiddler scans tool result content — the contents of a file an agent read, or the output of an MCP tool call — on LiteLLM >= 1.97.0, which is when LiteLLM’s guardrail-translation layer began extracting Anthropic tool_result content blocks and OpenAI role: "tool" messages into the texts[] array it sends to Fiddler. Fiddler’s own scanning is role-agnostic: every entry in texts[] is scanned identically regardless of which message role it came from, so no separate configuration is needed once LiteLLM extracts it. Fiddler has verified this behavior on 1.98.0; 1.97.0 is the version LiteLLM’s own changelog gives for the extraction, not a boundary Fiddler has tested. Confirm against the version you run. On an older LiteLLM version, tool result content is not extracted into texts[] at all — it is visible only inside the structured_messages field, which Fiddler accepts but never scans (see Request/Response Schema below). If your gateway predates 1.97.0, tool result content — including file reads and MCP output — is not scanned, and the fix is to upgrade LiteLLM rather than to change any Fiddler configuration. Two caveats hold even on a current version:

Tool results are typed as any in the GenAI semantic conventions — they can be plain strings, JSON objects, or arrays. In-place character-span redaction on a serialized JSON payload is not safe if the downstream application re-parses the result as structured data.
This depends on LiteLLM’s own extraction behavior, not a contract Fiddler controls. Test against your specific gateway version before relying on it.

For reliable protection against secrets in tool output, use mode: block on secrets and treat tool result content as untrusted.

Troubleshooting

Symptom	Likely cause	Fix
HTTP 401 from the Fiddler endpoint	Invalid or expired API key	Create a new key under Settings → Credentials and update `FIDDLER_API_KEY`
HTTP 403 from the Fiddler endpoint	Guardrails is not enabled for your deployment	Contact Fiddler
HTTP 404 from the guardrail call	Wrong `api_base`	Use `https://<instance>/v3/guardrails/litellm` — LiteLLM appends `/beta/litellm_basic_guardrail_api` itself
Every blocked request returns HTTP 500	LiteLLM < 1.87.0	Upgrade to ≥ 1.91.0
`fail_on_error` appears to have no effect	LiteLLM < 1.91.0 silently ignores it — config loads without error	Upgrade to ≥ 1.91.0
Proxy fails to start: `ImportError: cannot import name 'get_flat_dependant'`	LiteLLM 1.96.x installed with an incompatible FastAPI	Install with the `"fastapi<0.140.7"` pin shown in the quick start
Requests bypass the guardrail entirely	`default_on: false` and the request body has no `"guardrails": ["fiddler"]`; or a guardrail failure passed through under `fail_on_error: false`	Set `default_on: true`; check the proxy log for CRITICAL `Generic Guardrail API error (fail-open)` entries
HTTP 500 with `Generic Guardrail API failed: ...`	The guardrail call failed (unreachable endpoint, non-2xx, malformed response) and the LiteLLM controls are fail-closed	Fix connectivity or the failing endpoint; see Failure Behavior for which control observes which failure
Blocked with `Guardrail check failed to complete (...)`	An internal Fiddler check timed out or failed under `failure_mode: closed`	Raise `timeout` (max 60 seconds) or investigate detector health; this block is generated by Fiddler, not LiteLLM
A check you expected did not run, or a block cites a check you did not configure	Check selection is explicit: listing any check runs only the listed ones, and omitting `additional_provider_specific_params` applies server-side defaults instead	Configure every check you want explicitly, as the quick start does
Requests with large prompts are blocked or pass unscanned	Content exceeded the 50,000-character scan cap	See Text Length Limits

For issues with a specific request, the safe identifiers to include in a support ticket are litellm_call_id and litellm_trace_id — never include prompts, completions, or credentials.

API Reference

Endpoint

POST /v3/guardrails/litellm/beta/litellm_basic_guardrail_api

Authentication: Authorization: Bearer <your-fiddler-api-key>

Request

{
  "texts": ["string"],
  "tool_calls": [{"id": "...", "type": "function", "function": {"name": "...", "arguments": "..."}}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}],
  "structured_messages": [{"role": "user", "content": "..."}],
  "images": ["base64-encoded-string"],
  "model": "gpt-5-mini",
  "request_data": {"user_api_key_alias": "my-key", "user_api_key_team_id": "team-123"},
  "request_headers": {"content-type": "application/json"},
  "input_type": "request",
  "additional_provider_specific_params": {
    "failure_mode": "closed",
    "timeout": 12,
    "pii": {"enabled": true, "config": {"threshold": 0.5, "mode": "redact"}},
    "secrets": {"enabled": true, "config": {"mode": "redact"}}
  },
  "litellm_call_id": "uuid",
  "litellm_trace_id": "uuid",
  "litellm_version": "1.96.2"
}

Only texts and tool_calls[].function.arguments are scanned by guardrail checks. The remaining fields are accepted for LiteLLM protocol compatibility, and unknown fields are ignored.

Field	Type	Required	Description
`texts`	`string[]`	Yes	Extracted text strings from the request messages (scanned)
`request_data`	`object`	Yes	LiteLLM virtual key metadata (user, team, org, key hash)
`input_type`	`"request"` \| `"response"`	Yes	Whether this is a pre-call or post-call check
`tool_calls`	`object[]` \| `null`	No	Tool invocations in OpenAI format (arguments are scanned)
`tools`	`object[]` \| `null`	No	Tool definitions in OpenAI format (not scanned)
`structured_messages`	`object[]` \| `null`	No	Full messages array in OpenAI format (not scanned)
`images`	`string[]` \| `null`	No	Base64-encoded images (not scanned)
`model`	`string` \| `null`	No	Model name the proxy is routing to (not scanned)
`request_headers`	`object` \| `null`	No	Inbound request headers, sanitized by LiteLLM (not scanned)
`additional_provider_specific_params`	`object` \| `null`	No	Per-check configuration plus `failure_mode` and `timeout` (see Quick Start and Failure Behavior)
`litellm_call_id`	`string` \| `null`	No	LiteLLM call ID for tracing
`litellm_trace_id`	`string` \| `null`	No	LiteLLM trace ID for tracing
`litellm_version`	`string` \| `null`	No	LiteLLM library version

Response

Fields with null values are omitted from the wire. The response shape varies by action:

// action: NONE
{"action": "NONE"}

// action: BLOCKED
{"action": "BLOCKED", "blocked_reason": "Secret detected: Anthropic API Key (1 total)"}

// action: GUARDRAIL_INTERVENED
{"action": "GUARDRAIL_INTERVENED", "texts": ["My email is [REDACTED EMAIL] and my phone is [REDACTED PHONE_NUMBER]."]}

Field	Type	Description
`action`	`string`	One of `NONE`, `BLOCKED`, or `GUARDRAIL_INTERVENED`
`blocked_reason`	`string`	Human-readable reason; present only when `action` is `BLOCKED`
`texts`	`string[]`	The full texts array with redacted strings substituted per index; present only when `action` is `GUARDRAIL_INTERVENED`

Guardrail decisions are observable. With the Fiddler OTel callback enabled on the same proxy, each guardrail execution is traced as its own span next to the model call, so every block or redact decision — and its reason — shows up in Explorer as an audit trail.

Next Steps

Guardrails overview — understand the detection models behind these checks
LiteLLM Integration — add tracing, token, and cost observability to the same proxy
LiteLLM Generic Guardrail API spec — the upstream contract this integration implements
AgentGateway Guardrails — the same Fiddler checks via AgentGateway’s webhook protocol
Kong AI Gateway Guardrails — Fiddler guardrails via Kong (block-only, no masking)

Getting Started

Concepts

Evaluate & Test

Reference

Overview

How It Works

Prerequisites

Quick Start

What This Configuration Enables

What Gets Scanned

Supported Modes and Endpoints

Supported Endpoints

Per-Request Control

Selective Activation

Check Reference

Secrets

PII

Failure Behavior

How the LiteLLM Controls Compose

What the Client Sees

Fiddler’s `failure_mode`

Timeouts

Text Length Limits

Tool Call Handling

Coverage Differs by Client Wire Format

Tool Results

Troubleshooting

API Reference

Endpoint

Request

Response

Next Steps

​Overview

​How It Works

​Prerequisites

​Quick Start

​What This Configuration Enables

​What Gets Scanned

​Supported Modes and Endpoints

​Supported Endpoints

​Per-Request Control

​Selective Activation

​Check Reference

​Secrets

​PII

​Failure Behavior

​How the LiteLLM Controls Compose

​What the Client Sees

​Fiddler’s failure_mode

​Timeouts

​Text Length Limits

​Tool Call Handling

​Coverage Differs by Client Wire Format

​Tool Results

​Troubleshooting

​API Reference

​Endpoint

​Request

​Response

​Next Steps

Overview

How It Works

Prerequisites

Quick Start

What This Configuration Enables

What Gets Scanned

Supported Modes and Endpoints

Supported Endpoints

Per-Request Control

Selective Activation

Check Reference

Secrets

PII

Failure Behavior

How the LiteLLM Controls Compose

What the Client Sees

Fiddler’s `failure_mode`

Timeouts

Text Length Limits

Tool Call Handling

Coverage Differs by Client Wire Format

Tool Results

Troubleshooting

API Reference

Endpoint

Request

Response

Next Steps