AgentGateway Guardrails

Overview

AgentGateway can call Fiddler’s guardrail webhook adapter before and after every LLM call, redacting PII and secrets at the proxy layer. This is a separate integration from AgentGateway tracing — guardrails run inline on the request path (they can block or rewrite the call), while tracing exports spans out-of-band for observability. You can use either independently or both together.

Guardrails on Fiddler’s canonical, versioned endpoints require AgentGateway ≥ v1.4.0 — the first release containing Fiddler’s upstream OSS contribution that lets the webhook’s headers CEL config route calls to a path other than the hardcoded /request / /response. See Prerequisites for the earlier-version workaround.

Fiddler checks for:

PII — personal identifiable information (names, emails, phone numbers, SSNs, credit cards, etc.)
Secrets — API keys, tokens, credentials, and connection strings

AgentGateway’s webhook protocol supports masking: detected PII and secrets are redacted in place and the (sanitized) request still reaches the model, rather than being blocked outright.

How It Works

Each gateway speaks its own wire format — AgentGateway, Kong, and LiteLLM each wrap the prompt/response in a different envelope and expect a differently shaped answer back. Fiddler exposes a dedicated adapter per gateway (under /v3/guardrails/agentgateway/*, /v3/guardrails/kong, and /v3/guardrails/litellm/* respectively — see each page’s own Endpoints/API Reference section for the exact paths) so each protocol is translated to and from Fiddler’s guardrail checks without the gateways needing to agree on a shared format. AgentGateway wraps each request/response in its own envelope format and expects one of three actions back:

Action	Meaning	AgentGateway behavior
`pass`	No issues detected	Forwards the request/response unchanged
`mask`	Sensitive content redacted	Forwards the request/response with redacted content
`reject`	Request must not proceed	Returns an HTTP error to the client

Prerequisites

AgentGateway ≥ v1.4.0 — earlier versions cannot route webhook calls to a path other than the hardcoded /request / /response on the target host. Fiddler worked with the AgentGateway maintainers on this (agentgateway/agentgateway#2368, fixed in agentgateway/agentgateway#2595); on 1.4.0+ the webhook’s headers CEL config can set :path to call Fiddler’s canonical, versioned endpoints directly.
A named backend entry pointing at your Fiddler instance, referenced from the LLM model’s guardrails block.

Step 1: Configure AgentGateway

Add a named backend for the Fiddler guardrail webhook, then reference it from each model’s guardrails block:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
backends:
  - name: fiddler-guardrail
    host: <your-fiddler-instance>:443
    policies:
      # Required for any HTTPS backend host — without it AgentGateway speaks
      # plain HTTP to the host:port above and the connection hangs.
      backendTLS: {}
      backendAuth:
        key: "<your-fiddler-api-key>"

llm:
  models:
    - name: gpt-5-mini
      provider: openai
      params:
        model: gpt-5-mini
      guardrails:
        request:
          - webhook:
              target:
                backend: /fiddler-guardrail
              headers:
                # AgentGateway defaults webhook calls to the backend's
                # root-level /request and /response paths. Setting :path
                # via a CEL expression routes to Fiddler's canonical,
                # versioned endpoints instead.
                ":path": '"/v3/guardrails/agentgateway/request"'
        response:
          - webhook:
              target:
                backend: /fiddler-guardrail
              headers:
                ":path": '"/v3/guardrails/agentgateway/response"'

A named backend: reference (rather than an inline host: on the webhook target) is required to attach backendAuth/backendTLS policies to the guardrail call. On standalone (non-Kubernetes) AgentGateway deployments, backend names with no namespace are stored with a leading slash — reference them as /fiddler-guardrail, not fiddler-guardrail (see agentgateway/agentgateway#2220).

The headers CEL config for setting :path requires AgentGateway ≥ v1.4.0. On earlier versions, omit the headers block — the webhook falls back to calling /request and /response at the backend’s root, which requires either a dedicated host/ingress rule for those exact paths or a rewrite rule in front of Fiddler.

Step 2: Start AgentGateway

agentgateway -f agentgateway_config.yaml

Step 3: Verify

With no gateways block, AgentGateway’s llm.models config implies a default gateway serving LLM traffic on port 4000. Send a request with a known, dummy secret (correctly formatted, but not a real credential):

curl -i http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"My API key is sk-ant-api03-abcdefghijklmnopqrstu"}]}'

The secret is redacted before the request reaches the model — the completion is returned normally, with the model having only seen the sanitized prompt.

What Gets Scanned

Check	Request guard	Response guard	Redaction Support
PII	✓	✓	Redact
Secrets	✓	✓	Redact

Free-text messages — every message’s content field. PII/secrets are redacted in place (e.g. [REDACTED EMAIL_ADDRESS]) and the sanitized message array is returned to AgentGateway.

AgentGateway’s webhook protocol sends a simplified message shape — {role, content} only, no tool_calls, name, or other OpenAI chat-completions fields. AgentGateway strips these before calling the webhook, so there is nothing beyond message text for Fiddler to scan or redact on this integration.

Check Behavior

Checks are configured server-side via the same environment variables and thresholds used across all Fiddler guardrail integrations — see Guardrails for the underlying PII model, and the secrets detection tutorial for secrets. Since AgentGateway’s wire body carries no per-request config field, those defaults can be overridden per route via static HTTP headers in the webhook’s headers CEL config — the same mechanism used for the :path override in Step 1. This gives AgentGateway access to the same set of checks and overrides as the LiteLLM integration’s additional_provider_specific_params, but applied per route rather than per individual request — see Header-Based Configuration.

PII and Secrets

Detection	Behavior
PII detected	Message content redacted to `[REDACTED <type>]`; request/response proceeds with the masked content
Secret detected	Message content redacted to `[REDACTED <type>]`; request/response proceeds with the masked content

Header-Based Configuration

Header	Applies to	Value	Effect
`x-fiddler-guardrails`	Both	Comma-separated check names — `pii`, `secrets`	Restricts which checks run. Case-insensitive and whitespace-tolerant (`"PII, Secrets"` works). Absent, blank, or naming no recognized check at all runs every check with defaults.
`x-fiddler-pii-mode`	`pii`	`redact` (default) or `block`	Overrides the PII action.
`x-fiddler-pii-threshold`	`pii`	Float, e.g. `0.8`	Overrides the PII detection threshold. An unparseable value is ignored (falls back to the default threshold), not rejected.
`x-fiddler-pii-entities`	`pii`	Comma-separated entity names	Restricts which PII entity types are checked — see the PII Detection tutorial for the full list.
`x-fiddler-secrets-mode`	`secrets`	`redact` (default) or `block`	Overrides the secrets action.
`x-fiddler-failure-mode`	Both	`open` or `closed`	Overrides what happens when a check itself fails to complete (e.g. a detector timeout or error) — default `open`. This is separate from AgentGateway’s own `failureMode` webhook-policy setting, which governs what happens when the webhook call itself is unreachable or errors — see Failure Mode.
`x-fiddler-timeout`	Both	Seconds, e.g. `12`	Overrides Fiddler’s internal wall-clock budget for running checks — default 12s, capped at 60s.

guardrails:
  request:
    - webhook:
        target:
          backend: /fiddler-guardrail
        headers:
          ":path": '"/v3/guardrails/agentgateway/request"'
          "x-fiddler-guardrails": '"pii,secrets"'
          "x-fiddler-pii-mode": '"block"'

Each header value must be a CEL string literal — note the nested quotes ('"pii,secrets"'), matching the :path override above. A bare, unquoted value (e.g. "x-fiddler-pii-mode": "block") is parsed as an unresolvable CEL field reference; AgentGateway drops that header instead of raising an error, so the corresponding override is silently ignored.

A header naming a check that isn’t pii or secrets (for example, a typo) is dropped with a server-side warning, not treated as valid — but this only falls back to running every check when none of the requested names are recognized. A partial typo, like pii,screts, still recognizes pii and runs only that check — secrets is silently skipped, with no signal visible outside the server logs. Per-check override headers (x-fiddler-pii-mode, etc.) only take effect when the corresponding check is present in x-fiddler-guardrails; if x-fiddler-guardrails itself is absent, override headers are never read at all, so every check runs with system defaults regardless of what other x-fiddler-* headers are set.

x-fiddler-timeout sets Fiddler’s own check budget, not AgentGateway’s webhook call timeout — see Failure Mode. Since AgentGateway’s hardcoded webhook timeout (10s) is shorter than Fiddler’s default check budget (12s), raising x-fiddler-timeout above 10s has no effect unless the check also completes within AgentGateway’s own window.

Action Mapping

AgentGateway’s webhook protocol uses serde untagged deserialization on the response body — there is no explicit "type" discriminator field. The three action shapes are distinguished by their JSON structure:

// pass (allow)
{"action": {}}

// pass, with a reason
{"action": {"reason": "..."}}

// reject (block) — body is a string
{"action": {"body": "Request blocked by guardrail.", "status_code": 403, "reason": "..."}}

// mask (redact), request guard — body is an object with messages
{"action": {"body": {"messages": [{"role": "user", "content": "my email is [REDACTED EMAIL_ADDRESS]"}]}, "reason": "PII/secrets redacted"}}

// mask (redact), response guard — body is an object with choices
{"action": {"body": {"choices": [{"message": {"role": "assistant", "content": "[REDACTED]"}}]}, "reason": "PII/secrets redacted"}}

mask and reject are disambiguated by the type of body: an object ({"messages": [...]} or {"choices": [...]}) means mask, a string means reject.

Failure Mode

The webhook has a fixed 10-second wall-clock timeout, hardcoded by AgentGateway itself (with_default_timeout in crates/agentgateway/src/llm/policy/mod.rs) — there is currently no config field to override it. If your guardrail backend is slower than this (for example, a cold-starting GPU inference worker), requests will time out before the check completes. AgentGateway’s webhook policy defaults to failClosed: if the webhook is unreachable or returns an error (including a timeout), the request is rejected rather than allowed through unscanned. Set failureMode: failOpen on the webhook config to allow requests through instead when the guardrail is unavailable — weigh this against your security posture, since fail-open means unscanned content can reach the model during an outage.

guardrails:
  request:
    - webhook:
        target:
          backend: /fiddler-guardrail
        headers:
          ":path": '"/v3/guardrails/agentgateway/request"'
        failureMode: failOpen   # default: failClosed

Known Limitations

Limitation	Details
Fixed 10-second timeout	Hardcoded by AgentGateway itself, with no config field to override it — see Failure Mode.
Per-route, not per-request, configuration	Check selection and overrides are static headers baked into the webhook policy YAML, so they apply to every request AgentGateway sends to that route — there is no way to vary them per individual request the way the LiteLLM integration’s `additional_provider_specific_params` can, since AgentGateway’s wire body carries no per-request config field — see Header-Based Configuration.
Text content only	AgentGateway strips `tool_calls`, `name`, and other OpenAI chat-completions fields before calling the webhook, so Fiddler only sees and scans message `content` — see What Gets Scanned.
AgentGateway ≥ v1.4.0 required for canonical endpoints	Earlier versions cannot route webhook calls to a path other than the hardcoded `/request` / `/response` — see Prerequisites.

Endpoints

POST /v3/guardrails/agentgateway/request
POST /v3/guardrails/agentgateway/response

Authentication: Authorization: Bearer <your-fiddler-api-key> (set via backendAuth.key on the AgentGateway backend — AgentGateway injects the Bearer scheme itself, so configure the raw token value, not Bearer <token>).

Request Body

/request — the pre-LLM prompt guard:

{
  "body": {
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "hello world"}
    ]
  }
}

/response — the post-LLM response guard:

{
  "body": {
    "choices": [
      {"message": {"role": "assistant", "content": "Hello! How can I help?"}}
    ]
  }
}

Field	Type	Description
`body.messages`	`object[]`	(Request only) Chat messages, each `{role, content}`
`body.choices`	`object[]`	(Response only) Model choices, each `{message: {role, content}}`

Only content fields are scanned. AgentGateway’s simplified webhook message shape has no other fields to accept.

Response Body

See Action Mapping above for the full shape of each variant. Summarized:

Action	HTTP status	Shape
`pass`	200	`{"action": {}}` (optionally with `"reason"`)
`mask`	200	`{"action": {"body": {"messages": [...]} \| {"choices": [...]}, "reason": "..."}}`
`reject`	200	`{"action": {"body": "<reason>", "status_code": 403, "reason": "..."}}`

Fiddler’s adapter always returns HTTP 200 to AgentGateway — the status_code field inside a reject action tells AgentGateway what to return to the client (default 403).

Guardrails overview
AgentGateway Integration (observability)
LiteLLM Guardrails — Fiddler guardrails via the LiteLLM proxy gateway
Kong AI Gateway Guardrails — Fiddler guardrails via Kong (block-only, no masking)
AgentGateway webhook guardrails documentation

Getting Started

Concepts

Evaluate & Test

Reference

Overview

How It Works

Prerequisites

Step 1: Configure AgentGateway

Step 2: Start AgentGateway

Step 3: Verify

What Gets Scanned

Check Behavior

PII and Secrets

Header-Based Configuration

Action Mapping

Failure Mode

Known Limitations

Endpoints

Request Body

Response Body

​Overview

​How It Works

​Prerequisites

​Step 1: Configure AgentGateway

​Step 2: Start AgentGateway

​Step 3: Verify

​What Gets Scanned

​Check Behavior

​PII and Secrets

​Header-Based Configuration

​Action Mapping

​Failure Mode

​Known Limitations

​Endpoints

​Request Body

​Response Body

​Related Documentation

Overview

How It Works

Prerequisites

Step 1: Configure AgentGateway

Step 2: Start AgentGateway

Step 3: Verify

What Gets Scanned

Check Behavior

PII and Secrets

Header-Based Configuration

Action Mapping

Failure Mode

Known Limitations

Endpoints

Request Body

Response Body

Related Documentation