Kong AI Gateway Guardrails

Overview

Kong can enforce Fiddler Guardrails on prompts before they reach the model — blocking unsafe content, PII, and secrets at the gateway. This uses Kong’s ai-custom-guardrail plugin, which calls Fiddler’s Kong guardrail adapter (/v3/guardrails/kong) on each request and emits a connected Guardrail span in the same trace as the LLM span. This page covers guardrails only. For LLM tracing setup (the ai-proxy and opentelemetry plugins, session grouping, and span mapping), see the Kong AI Gateway Integration.

Guardrails require Kong Gateway Enterprise 3.14+ — ai-custom-guardrail is an Enterprise plugin. (Tracing alone works on OSS Kong 3.13+.) The guardrail is an additional plugin on the same route you configured for tracing.

How It Works

Pre-call input guarding (post-call also supported). Each prompt is checked before the LLM is called (guarding_mode: INPUT); safe prompts continue to the model, and unsafe ones are blocked with HTTP 400 before the model is called. The adapter also supports post-call output guarding via input_type: "response" — see Output guarding.
Block, not mask. The ai-custom-guardrail plugin can only allow or block — it cannot substitute a redacted body. So detected PII and secrets are blocked, not masked. (This differs from the LiteLLM integration, where PII can be redacted.)
Connected guardrail span. Each check emits a fiddler-guardrail span (fiddler.span.type: guardrail) in the same trace as the LLM span, so the guardrail decision appears next to the model call in Fiddler.

Prompt	Fiddler decision	Result
Safe	allow	Forwarded to the model — LLM span + Guardrail span in one trace
Unsafe (harmful / jailbreak)	block	HTTP 400; Guardrail span with the block reason; LLM never called
Contains PII or secrets	block	HTTP 400 (cannot be masked via Kong); Guardrail span with the block reason

Configuration

Add the ai-custom-guardrail plugin to the same openai-service route you configured for tracing, alongside ai-proxy and opentelemetry. It sends each prompt to Fiddler’s /v3/guardrails/kong adapter, blocks when Fiddler returns block: true, and runs a small Lua check function that emits the connected Guardrail span:

  - name: ai-custom-guardrail
    service: openai-service
    config:
      guarding_mode: INPUT
      text_source: concatenate_all_content
      ssl_verify: true
      request:
        url: "${FIDDLER_URL}/v3/guardrails/kong"
        headers:
          Authorization: "Bearer ${FIDDLER_API_KEY}"
          Content-Type: "application/json"
        body:
          input: "$(content)"
      response:
        block: "$(check.block)"
        block_message: "$(check.block_message)"
      functions:
        check: |
          return function(resp)
            local cjson = require("cjson.safe")
            local span = kong.tracing.start_span("fiddler-guardrail")
            span:set_attribute("fiddler.span.type", "guardrail")
            span:set_attribute("guardrail_name", "fiddler-safety-pii")
            span:set_attribute("guardrail_mode", "input")
            span:set_attribute("guardrail_blocked", (resp and resp.block) == true)
            -- Capture the prompt onto the guardrail span (works around the Kong
            -- bug that drops gen_ai.input.messages from the LLM span — see below).
            local ok, prompt = pcall(function()
              local req = cjson.decode(kong.request.get_raw_body())
              local last
              if req and type(req.messages) == "table" then
                for _, m in ipairs(req.messages) do
                  if type(m) == "table" and m.role == "user"
                      and type(m.content) == "string" then
                    last = m.content
                  end
                end
              end
              return last
            end)
            if ok and prompt then
              span:set_attribute("gen_ai.llm.input.user", prompt)
            end
            local decision = cjson.encode({
              block = (resp and resp.block) == true,
              action = (resp and resp.action) or "none",
            })
            if decision then
              span:set_attribute("gen_ai.llm.output", decision)
            end
            span:finish()
            return {
              block = (resp and resp.block) == true,
              block_message = (resp and resp.reason) or "Blocked by Fiddler guardrail",
            }
          end

The check Lua function requires KONG_UNTRUSTED_LUA=on (the same setting the session-grouping pre-function needs). Without it Kong refuses to load the inline Lua and the Guardrail span is never emitted.

Add this plugin to the same declarative Kong config that holds your ai-proxy and opentelemetry plugins (see the Kong AI Gateway Integration for the base setup).

The ${FIDDLER_URL} and ${FIDDLER_API_KEY} values are placeholders. Kong does not read environment variables from its declarative config, so replace each ${...} with your actual value before starting Kong — otherwise Kong fails to start. See Step 2 of the tracing setup.

Check Behavior

Kong extracts the text (the prompt, or the buffered model output) and posts it to the adapter, which runs the appropriate Fiddler checks and returns a block decision: Request

{ "input": "the text to evaluate", "input_type": "request" }

Field	Type	Required	Description
`input`	`string`	Yes	Text to evaluate — the prompt (pre-call) or the model output (post-call).
`input_type`	`"request"` \| `"response"`	No (default `"request"`)	`"request"` runs input checks (Safety + PII/secrets); `"response"` runs output checks (PII/secrets — Safety applies to input only). Set to `"response"` when the plugin is in `guarding_mode: OUTPUT`.

Response — allowed:

{ "block": false, "action": "none" }

Response — blocked (unsafe, PII, or secrets):

{ "block": true, "action": "block", "reason": "Sensitive information (PII or secrets) detected — blocking request (cannot be redacted via Kong)." }

Kong’s check function maps block → whether to reject the request and reason → the client-facing block_message. The underlying checks are the same Safety and PII checks described in Fiddler Guardrails, plus secrets detection.

Output Guarding

The Configuration above guards the input prompt (guarding_mode: INPUT, sending input_type: "request"). The Fiddler adapter also supports post-call output guarding — when it receives input_type: "response" it runs PII and secrets checks on the model output (Safety applies to input only, so unsafe-content classification is not re-run on the output). To guard the output, add a second ai-custom-guardrail plugin in guarding_mode: OUTPUT whose request body sets input_type: "response" — in output mode Kong’s $(content) resolves to the model’s response:

  - name: ai-custom-guardrail
    service: openai-service
    config:
      guarding_mode: OUTPUT
      text_source: concatenate_all_content
      ssl_verify: true
      request:
        url: "${FIDDLER_URL}/v3/guardrails/kong"
        headers:
          Authorization: "Bearer ${FIDDLER_API_KEY}"
          Content-Type: "application/json"
        body:
          input: "$(content)"
          input_type: "response"
      response:
        block: "$(resp.block)"
        block_message: "$(resp.reason)"

Here $(resp.block) and $(resp.reason) map directly to the adapter’s response fields, so a blocked output returns HTTP 400 to the caller with the block reason. To also emit a connected Guardrail span on the output path, add a functions.check block like the one in the input configuration. As with the input config, replace the ${FIDDLER_URL} and ${FIDDLER_API_KEY} placeholders with your actual values — Kong does not interpolate environment variables from its declarative config (see Step 2 of the tracing setup).

Verified on Kong Gateway Enterprise 3.14: in guarding_mode: OUTPUT, $(content) resolves to the model response and input_type: "response" is sent to the adapter, which runs PII/secrets checks on the output and blocks when detections are found (Safety is applied to input only). Re-verify after major Kong upgrades, since Gen AI OTel/guardrail behavior is still evolving.

Verifying Guardrails

Send a request through Kong’s OpenAI-compatible endpoint. A prompt containing PII or unsafe content is blocked at the gateway with HTTP 400; a benign prompt returns a normal completion:

# Blocked — contains PII (returns HTTP 400 with the block message):
curl -i http://localhost:8000/openai/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"My SSN is 123-45-6789 and email is john.doe@email.com"}]}'

# Allowed — benign prompt returns a normal completion:
curl -i http://localhost:8000/openai/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What are two healthy breakfast ideas?"}]}'

In Fiddler, allowed prompts show an LLM span and a Guardrail span in one trace; blocked prompts show a Guardrail span carrying the block reason (with no LLM span, since the model was never called).

Troubleshooting

Guardrail not blocking / no Guardrail span appearing The ai-custom-guardrail plugin is Kong Enterprise 3.14+ only — it is silently absent on OSS Kong. Confirm you are on Kong Enterprise 3.14+, that KONG_UNTRUSTED_LUA=on is set (required for the check function that emits the span), and that the ${FIDDLER_URL}/v3/guardrails/kong URL and Authorization header in the plugin config are correct. Check your Kong logs for errors calling the adapter. Prompt missing from the LLM span when guardrails are enabled Kong’s Gen AI OTel tracing is still Tech Preview. When an ai-custom-guardrail plugin shares the route, Kong 3.14.x drops gen_ai.input.messages from the LLM span and degrades gen_ai.output.messages to the raw response-body map. The check function above works around this by reading the original request with kong.request.get_raw_body() and stamping the prompt onto the Guardrail span as gen_ai.llm.input.user, so the prompt is still visible in the trace. Re-verify this behavior after Kong upgrades.

Known Limitations

Limitation	Details
Kong Enterprise 3.14+ required	The `ai-custom-guardrail` plugin is an Enterprise-only plugin. Tracing alone works on OSS Kong 3.13+.
Block, not mask	The plugin can only allow or block, so PII and secrets are blocked rather than redacted (unlike the LiteLLM integration).
Output guarding needs a second plugin	The example config guards input only. Post-call output guarding is supported but requires a separate `ai-custom-guardrail` plugin in `guarding_mode: OUTPUT` sending `input_type: "response"` — see Output guarding.

Kong AI Gateway Integration — LLM tracing setup for Kong (proxy, OTel export, session grouping)
Fiddler Guardrails — the Safety and PII checks behind the Kong guardrail adapter
LiteLLM Guardrails — Fiddler guardrails via the LiteLLM proxy gateway

Getting Started

Concepts

Evaluate & Test

Reference

Overview

How It Works

Configuration

Check Behavior

Output Guarding

Verifying Guardrails

Troubleshooting

Known Limitations

​Overview

​How It Works

​Configuration

​Check Behavior

​Output Guarding

​Verifying Guardrails

​Troubleshooting

​Known Limitations

​Related Documentation

Overview

How It Works

Configuration

Check Behavior

Output Guarding

Verifying Guardrails

Troubleshooting

Known Limitations

Related Documentation