AgentGateway Guardrails

Overview

AgentGateway can call Fiddler’s guardrail webhook adapter before and after every LLM call, redacting or blocking personally identifiable information (PII) and secrets at the proxy layer. Guardrails run inline on the request path — they can rewrite or block the call — while AgentGateway tracing exports spans out-of-band for observability. They are independent; for a single config that runs both, see the combined example.

You want to…	Use	Page
Observe prompts, responses, tokens, and latency	`frontendPolicies.tracing`	AgentGateway Integration
Redact or block PII and secrets	`llm.models[].guardrails` webhook	This page

Guardrails require AgentGateway v1.4.0 or later (the first stable release whose webhook headers CEL config can route calls to Fiddler’s canonical, versioned endpoints), and are available in Fiddler 26.17 and later — the release that first contains Fiddler’s AgentGateway guardrail adapter. On earlier Fiddler releases the endpoints below return HTTP 404.

How It Works

Each gateway speaks its own wire format, so Fiddler exposes a dedicated adapter per gateway (/v3/guardrails/agentgateway/*, /v3/guardrails/kong, and /v3/guardrails/litellm/* — see each page’s own Endpoints/API Reference section). AgentGateway wraps each request or response in its own envelope and expects one of three actions back:

Action	Meaning	AgentGateway behavior
`pass`	No issues detected	Forwards the request/response unchanged
`mask`	Sensitive content redacted	Forwards the request/response with redacted content
`reject`	Request must not proceed	Returns an HTTP error to the client

Unlike the Kong integration, which blocks only, AgentGateway can redact in place (mask) so the sanitized call still reaches the model — with the streaming caveat in Failure Mode, Timeouts, and Streaming.

Prerequisites

Fiddler guardrails enabled on your deployment, on Fiddler 26.17 or later (the release that contains the AgentGateway guardrail adapter). If guardrails are not enabled, the adapter returns HTTP 403 “Guardrails is not enabled for this cluster” or HTTP 404 “not available on freemium deployments” — see Troubleshooting.
Your Fiddler instance URL and a Fiddler API key (organizational settings).
AgentGateway v1.4.0+ (see the callout above for the earlier-version constraint).
Network egress from AgentGateway to your Fiddler instance over HTTPS.
About ten minutes.

Quick Start

Configure AgentGateway

Add a named backend for the Fiddler guardrail webhook, then reference it from each model’s guardrails block. This uses AgentGateway’s llm.models configuration shape:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
backends:
  - name: fiddler-guardrail
    host: <your-fiddler-instance>:443
    policies:
      # Required for any HTTPS backend host — without it AgentGateway speaks
      # plain HTTP to the host:port above and the connection hangs.
      backendTLS: {}
      backendAuth:
        # AgentGateway injects the "Bearer" scheme itself — set the raw token.
        # Restrict this file's permissions and inject it at deploy time from a
        # secret manager; see the note below on environment expansion.
        key: "<your-fiddler-api-key>"

llm:
  models:
    - name: gpt-5-mini
      provider: openai
      params:
        model: gpt-5-mini
      guardrails:
        request:
          - webhook:
              target:
                backend: /fiddler-guardrail
              headers:
                # AgentGateway defaults webhook calls to the backend's root-level
                # /request and /response paths. Setting :path via a CEL string
                # literal routes to Fiddler's canonical, versioned endpoints.
                ":path": '"/v3/guardrails/agentgateway/request"'
        response:
          - webhook:
              target:
                backend: /fiddler-guardrail
              headers:
                ":path": '"/v3/guardrails/agentgateway/response"'

On standalone (non-Kubernetes) AgentGateway deployments, backend names with no namespace are stored with a leading slash — reference them as /fiddler-guardrail, not fiddler-guardrail (see agentgateway/agentgateway#2220). A named backend: reference (rather than an inline host: on the webhook target) is required to attach backendAuth/backendTLS policies to the guardrail call.AgentGateway supports $VAR expansion for credentials in its config; if it applies to backendAuth.key on your version you can set key: "$FIDDLER_API_KEY" instead of a literal token. Verify it against your deployment — Fiddler has not confirmed $VAR expansion for this specific field.

Start AgentGateway

agentgateway -f agentgateway_config.yaml

With no gateways block, the llm.models config implies a default gateway serving LLM traffic on port 4000.

Verify

Send a request containing a synthetic (correctly formatted but not real) secret:

curl -i http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"My API key is sk-ant-api03-abcdefghijklmnopqrstu"}]}'

With the default request guard, the secret is redacted in place — the model receives [REDACTED <LABEL>] instead of the secret, and the completion returns normally. To prove the redaction deterministically rather than inferring it from the completion, capture what AgentGateway forwarded to the provider (for example, with a local echo backend), because Fiddler produces no product-side record of a guardrail decision on this integration (see Data Handling). Set x-fiddler-pii-mode: block (or x-fiddler-secrets-mode: block) to reject instead of redact, which returns an HTTP error to the client. Use synthetic data only — never a real credential or real personal data.

What This Configuration Enables

The quick-start config sends no x-fiddler-guardrails header, so the adapter runs both of its checks on the request path:

Check	What it detects	Default action	Threshold
PII	Personally identifiable information	Redact (`mask`)	0.8
Secrets	Credentials and API keys	Redact (`mask`)	Any detection

One thing is important:

There is a 50,000-character scan cap. Under the default fail-open behavior, text over the cap is skipped and passes to the model unscanned; under fail-closed it is rejected — see Failure Mode, Timeouts, and Streaming.

What Gets Scanned

Check	Request guard	Response guard	Default action
PII	✓	✓	Redact
Secrets	✓	✓	Redact

Only free-text message content is scanned. AgentGateway’s webhook protocol sends a simplified message shape — {role, content} only — and strips tool_calls, name, and other OpenAI chat-completions fields before calling the webhook, so there is nothing beyond message text for Fiddler to scan or redact on this integration.

Check Behavior

Checks are configured server-side with the same defaults used across all Fiddler guardrail integrations — see Guardrails for the PII model and the secrets detection tutorial for secrets. Because AgentGateway’s wire body carries no per-request config field, those defaults are overridden per route via static HTTP headers in the webhook’s headers CEL config — the same mechanism used for the :path override in Step 1.

Header-Based Configuration

Header	Applies to	Value	Effect
`x-fiddler-guardrails`	All	Comma-separated check names — `pii`, `secrets`	Restricts which checks run. Case-insensitive and whitespace-tolerant. Absent, blank, or naming no recognized check at all runs every check with defaults.
`x-fiddler-pii-mode`	`pii`	`redact` (default) or `block`	Overrides the PII action.
`x-fiddler-pii-threshold`	`pii`	Float, e.g. `0.8`	Overrides the PII detection threshold. An unparseable value is ignored (falls back to the default), not rejected.
`x-fiddler-pii-entities`	`pii`	Comma-separated entity names	Switches PII detection out of preset mode to check only the named entity types — see the PII Detection tutorial.
`x-fiddler-secrets-mode`	`secrets`	`redact` (default) or `block`	Overrides the secrets action.
`x-fiddler-failure-mode`	All	`open` or `closed`	What happens when a check itself fails to complete (detector timeout or error) — default `open`. Separate from AgentGateway’s own `failureMode`, which governs an unreachable webhook — see Failure Mode.
`x-fiddler-timeout`	All	Seconds, e.g. `12`	Fiddler’s internal wall-clock budget for running checks — default 12s, capped at 60s.

guardrails:
  request:
    - webhook:
        target:
          backend: /fiddler-guardrail
        headers:
          ":path": '"/v3/guardrails/agentgateway/request"'
          "x-fiddler-guardrails": '"pii,secrets"'
          "x-fiddler-pii-mode": '"block"'

Each header value must be a CEL string literal — note the nested quotes ('"pii,secrets"'). A bare, unquoted value (for example "x-fiddler-pii-mode": "block") is parsed as an unresolvable CEL field reference; AgentGateway drops that header instead of raising an error, so the override is silently ignored.

Naming a subset silently disables the rest. x-fiddler-guardrails restricts checks to those named, so x-fiddler-guardrails: pii runs only PII and silently disables secrets. A header naming a check that is not pii or secrets (for example a typo) is dropped with a server-side warning; every check runs with defaults only when none of the requested names is recognized. A partial typo like pii,screts still runs pii and silently skips secrets. Per-check override headers take effect only when their check is present in x-fiddler-guardrails; but x-fiddler-failure-mode and x-fiddler-timeout are parsed before check selection, so they do take effect even when x-fiddler-guardrails is absent.

Action Mapping

AgentGateway’s webhook protocol uses serde untagged deserialization on the response body — there is no explicit "type" discriminator. The action shapes are distinguished by their JSON structure:

// pass (allow)
{"action": {}}

// reject (block) — body is a string
{"action": {"body": "Request blocked by guardrail.", "status_code": 403, "reason": "..."}}

// mask (redact), request guard — body is an object with messages
{"action": {"body": {"messages": [{"role": "user", "content": "my email is [REDACTED EMAIL_ADDRESS]"}]}, "reason": "PII/secrets redacted"}}

// mask (redact), response guard — body is an object with choices
{"action": {"body": {"choices": [{"message": {"role": "assistant", "content": "your key is [REDACTED SECRET]"}}]}, "reason": "PII/secrets redacted"}}

mask and reject are disambiguated by the type of body: an object ({"messages": [...]} or {"choices": [...]}) means mask; a string means reject. Redacted content is a labelled in-place substitution — [REDACTED <LABEL>], with PII / SECRET fallbacks — so surrounding text is preserved, not replaced wholesale.

Failure Mode, Timeouts, and Streaming

AgentGateway’s webhook timeout. AgentGateway applies a default 10-second wall-clock timeout to the guardrail webhook call (with_default_timeout in crates/agentgateway/src/llm/policy/mod.rs). Whether a policy such as policies.http.requestTimeout on the named backend raises this limit is unverified — test it against your AgentGateway version before relying on a longer budget. If your guardrail backend is slower than the effective timeout (for example a cold-starting inference worker), the call times out before the check completes. AgentGateway’s failureMode. The webhook policy defaults to failClosed: if the webhook is unreachable or errors (including a timeout), the request is rejected rather than allowed through unscanned. Set failureMode: failOpen to allow requests through instead — weigh this against your security posture.

guardrails:
  request:
    - webhook:
        target:
          backend: /fiddler-guardrail
        headers:
          ":path": '"/v3/guardrails/agentgateway/request"'
        failureMode: failOpen   # default: failClosed

Fiddler’s side. Fiddler’s adapter budget is 12 seconds by default (capped at 60s), configurable per route with x-fiddler-timeout. While AgentGateway’s default 10-second window applies, raising x-fiddler-timeout above 10s has no effect — the check must still complete within AgentGateway’s window (and whether that window can be raised is unverified, above). Fiddler’s own failure mode (x-fiddler-failure-mode) defaults to open. Streaming. Masking cannot redact content that has already been sent to the client, so on a streamed response the mask action does not protect the caller: detected PII or secrets can reach the client unredacted, with no error and no signal. AgentGateway’s upstream guardrail documentation recorded this streaming limitation (accessed 2026-08-14; AgentGateway’s latest docs are live-edited, so re-verify — see the maintenance note in the integration page). If you depend on the response guard for redaction, do not enable streaming, or treat the response guard as best-effort on streamed responses.

Data Handling

Fiddler’s AgentGateway guardrail adapter is stateless: it evaluates the text in-request and returns a decision. No prompt, no response, and no detection result is written to any Fiddler datastore. The only durable record is two content-free operational counters (Prometheus series) labelled by gateway, direction, and outcome. Unlike the Kong and LiteLLM integrations, an AgentGateway guardrail decision produces no span and no UI record in Fiddler — Fiddler’s AgentGateway mapper has no guardrail handling. Guardrail enforcement here is gateway-side only; to observe the LLM calls themselves, add AgentGateway tracing.

Troubleshooting

HTTP 403 or 404 from the guardrail webhook A 403 (“Guardrails is not enabled for this cluster”) or 404 (“not available on freemium deployments”) means Fiddler guardrails are not enabled on your deployment, or your deployment predates 26.17. Because AgentGateway defaults to failClosed, either response blocks your traffic — confirm your deployment’s entitlement and release with your Fiddler administrator. HTTP 401 from the guardrail webhook The backendAuth.key is missing or wrong. Set the raw Fiddler API token (AgentGateway adds the Bearer scheme itself). A check appears not to run x-fiddler-guardrails names a subset, so the omitted checks are disabled — for example, naming only pii disables secrets. Omit the header to run both. An override header appears to be ignored Its value is probably a bare CEL field reference rather than a string literal; AgentGateway drops it silently. Use nested quotes: '"block"', not "block". Also confirm the corresponding check is named in x-fiddler-guardrails. The webhook calls /request at the backend root instead of the versioned path The headers :path override requires AgentGateway v1.4.0+. On earlier versions the webhook falls back to /request and /response at the backend root. A slow backend times out AgentGateway’s default 10-second webhook window is shorter than Fiddler’s 12-second budget, so while that window applies, raising x-fiddler-timeout alone does not help — the check must complete within AgentGateway’s window. Whether the AgentGateway-side timeout can be raised is unverified (see Failure Mode, Timeouts, and Streaming).

Endpoints

POST /v3/guardrails/agentgateway/request
POST /v3/guardrails/agentgateway/response

Authentication: Authorization: Bearer <your-fiddler-api-key>, set via backendAuth.key on the AgentGateway backend (AgentGateway injects the Bearer scheme itself, so configure the raw token). Fiddler’s adapter returns HTTP 200 for every decision (pass, mask, reject) — the status_code inside a reject action tells AgentGateway what to return to the client (default 403). It returns a non-200 only for conditions outside the decision path: 400 (malformed payload), 401 (missing or invalid auth), 403 (guardrails not enabled), and 404 (freemium deployment). Under AgentGateway’s failClosed default, each of these blocks the client request.

Request Body

/request — the pre-LLM prompt guard:

{
  "body": {
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "hello world"}
    ]
  }
}

/response — the post-LLM response guard:

{
  "body": {
    "choices": [
      {"message": {"role": "assistant", "content": "Hello! How can I help?"}}
    ]
  }
}

Only content fields are scanned; AgentGateway’s simplified webhook message shape has no other fields to accept.

Response Body

See Action Mapping for the full shape of each variant. Summarized:

Action	HTTP status	Shape
`pass`	200	`{"action": {}}`
`mask`	200	`{"action": {"body": {"messages": [...]} \| {"choices": [...]}, "reason": "..."}}`
`reject`	200	`{"action": {"body": "<reason>", "status_code": 403, "reason": "..."}}`

Known Limitations

Limitation	Details
Masking no-ops on streamed responses	On a streamed response, redaction cannot protect content already sent — detected PII or secrets reach the client unredacted, with no error. See Failure Mode, Timeouts, and Streaming.
10-second webhook timeout	AgentGateway applies a default 10s timeout to the webhook call; whether a policy overrides it is unverified. A slower backend times out before the check completes.
Per-route, not per-request, configuration	Check selection and overrides are static headers in the webhook policy, so they apply to every request on that route — AgentGateway’s wire body carries no per-request config field.
Text content only	AgentGateway strips `tool_calls`, `name`, and other fields before calling the webhook, so Fiddler scans only message `content`.
No product-side decision record	Guardrail decisions produce no span or UI record in Fiddler — enforcement is gateway-side only.
AgentGateway v1.4.0+ required for canonical endpoints	Earlier versions cannot route webhook calls to a path other than the hardcoded `/request` / `/response`.

Next Steps

AgentGateway Integration — add LLM tracing, and the combined tracing + guardrails example.
Fiddler Guardrails — the PII and secrets checks behind the adapter.
Kong AI Gateway Guardrails — Fiddler guardrails via Kong (blocks rather than redacts).
LiteLLM Guardrails — Fiddler guardrails via the LiteLLM proxy.
AgentGateway webhook guardrails documentation — upstream reference (live-edited; verify against your version).

Getting Started

Concepts

Evaluate & Test

Reference

Overview

How It Works

Prerequisites

Quick Start

What This Configuration Enables

What Gets Scanned

Check Behavior

Header-Based Configuration

Action Mapping

Failure Mode, Timeouts, and Streaming

Data Handling

Troubleshooting

Endpoints

Request Body

Response Body

Known Limitations

Next Steps

​Overview

​How It Works

​Prerequisites

​Quick Start

​What This Configuration Enables

​What Gets Scanned

​Check Behavior

​Header-Based Configuration

​Action Mapping

​Failure Mode, Timeouts, and Streaming

​Data Handling

​Troubleshooting

​Endpoints

​Request Body

​Response Body

​Known Limitations

​Next Steps

Overview

How It Works

Prerequisites

Quick Start

What This Configuration Enables

What Gets Scanned

Check Behavior

Header-Based Configuration

Action Mapping

Failure Mode, Timeouts, and Streaming

Data Handling

Troubleshooting

Endpoints

Request Body

Response Body

Known Limitations

Next Steps