Kong AI Gateway Guardrails

​

Overview

​

How It Works

​

Prerequisites

• Fiddler guardrails are enabled on your deployment. If they are not, the adapter returns HTTP 403 “Guardrails is not enabled for this cluster” or HTTP 404 “not available on freemium deployments” — see Troubleshooting. Contact your Fiddler administrator if you are unsure.
• Your Fiddler instance URL (for example https://your-instance.fiddler.ai) and a Fiddler API key (found under organizational settings).
• Kong Gateway 3.14+ with an AI Gateway Enterprise license (see the callout above).
• untrusted_lua = on on your Kong deployment — required for the inline check function (set it as KONG_UNTRUSTED_LUA=on or in kong.conf; it cannot be set in declarative config).
• A provider API key for the model Kong proxies (for example OPENAI_API_KEY).
• Network egress from Kong to your Fiddler instance over HTTPS.
• About ten minutes.

​

Quick Start

_format_version: "3.0"

services:
  - name: openai-service
    host: api.openai.com
    port: 443
    protocol: https
    routes:
      - name: openai-chat-route
        paths:
          - /openai
        strip_path: true

plugins:
  - name: ai-proxy
    service: openai-service
    config:
      route_type: llm/v1/chat
      auth:
        header_name: Authorization
        # Kong resolves {vault://env/...} references on referenceable fields.
        # Export the WHOLE header value: export OPENAI_KEY='Bearer sk-...'
        header_value: "{vault://env/OPENAI_KEY}"
      model:
        provider: openai
        name: gpt-5-mini

  - name: ai-custom-guardrail
    service: openai-service
    config:
      guarding_mode: INPUT
      text_source: concatenate_all_content
      ssl_verify: true
      request:
        # Replace ${FIDDLER_URL} and ${FIDDLER_API_KEY} with real values before
        # starting Kong — Kong does not read environment variables from
        # declarative config, and this field is not a Vault-referenceable field.
        url: "${FIDDLER_URL}/v3/guardrails/kong"
        headers:
          Authorization: "Bearer ${FIDDLER_API_KEY}"
          Content-Type: "application/json"
        body:
          input: "$(content)"
      response:
        block: "$(check.block)"
        block_message: "$(check.block_message)"
      functions:
        check: |
          return function(resp)
            local span = kong.tracing.start_span("fiddler-guardrail")
            span:set_attribute("fiddler.span.type", "guardrail")
            span:set_attribute("guardrail_name", "fiddler-pii")
            span:set_attribute("guardrail_mode", "input")
            span:set_attribute("guardrail_blocked", (resp and resp.block) == true)
            span:finish()
            return {
              block = (resp and resp.block) == true,
              block_message = (resp and resp.reason) or "Blocked by Fiddler guardrail",
            }
          end

# Blocked — contains synthetic PII (returns HTTP 400 with a block reason):
curl -i http://localhost:8000/openai/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"My SSN is 123-45-6789 and email is jane.doe@example.com"}]}'

# Allowed — a benign prompt returns a normal completion:
curl -i http://localhost:8000/openai/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"What are two healthy breakfast ideas?"}]}'

​

What This Configuration Enables

• Kong’s request contract carries no guardrail configuration. The adapter accepts only input and input_type, and silently ignores any other field — so a parameter copied from another gateway’s setup (for example LiteLLM’s additional_provider_specific_params) has no effect.

​

Check Behavior

{ "input": "the text to evaluate", "input_type": "request" }

{ "block": false, "action": "none" }

{ "block": true, "action": "block", "reason": "Sensitive information (PII or secrets) detected — blocking request (cannot be redacted via Kong)." }

• PII or secrets: “Sensitive information (PII or secrets) detected — blocking request (cannot be redacted via Kong).”
• A generic fallback: “Request blocked by guardrail.”

​

Output Guarding

  - name: ai-custom-guardrail
    service: openai-service
    config:
      guarding_mode: OUTPUT
      text_source: concatenate_all_content
      ssl_verify: true
      request:
        url: "${FIDDLER_URL}/v3/guardrails/kong"
        headers:
          Authorization: "Bearer ${FIDDLER_API_KEY}"
          Content-Type: "application/json"
        body:
          input: "$(content)"
          input_type: "response"
      response:
        block: "$(resp.block)"
        block_message: "$(resp.reason)"

​

Failure Behavior

• On a detector timeout or an inference error, the adapter returns block: false and the unscanned content reaches the model. There is no customer-side lever to make it fail closed.
• The adapter’s wall-clock budget is a fixed 12 seconds and is not adjustable from Kong.
• An unexpected error inside the adapter surfaces as HTTP 500 to Kong. What Kong does then is governed by the plugin’s own stop_on_error field, which defaults to true — so by default Kong stops processing the request on an adapter error rather than letting it through. Kong’s plugin also has its own connection timeout (default 10,000 ms), separate from Fiddler’s 12-second budget.

​

Data Handling

• Fiddler evaluates the text in-request and returns a decision. No prompt, no response, and no detection result is written to any Fiddler datastore, and Fiddler emits no span.
• The only durable record is two content-free operational counters (Prometheus series) labelled by gateway, direction, and outcome. There is no per-request guardrail decision log or dashboard for this integration.
• The Guardrail span you see in Fiddler is emitted by your own Kong check function and exported by your own opentelemetry plugin. Without KONG_UNTRUSTED_LUA=on and that check block, a guardrail call leaves no trace in Fiddler at all.

​

Troubleshooting

return function(resp)
  local cjson = require("cjson.safe")
  local span = kong.tracing.start_span("fiddler-guardrail")
  span:set_attribute("fiddler.span.type", "guardrail")
  span:set_attribute("guardrail_name", "fiddler-pii")
  span:set_attribute("guardrail_mode", "input")
  span:set_attribute("guardrail_blocked", (resp and resp.block) == true)
  -- Optional: copy the last user message onto the span. This stores prompt text
  -- in Fiddler — see the retention warning in Data Handling.
  local ok, prompt = pcall(function()
    local req = cjson.decode(kong.request.get_raw_body())
    local last
    if req and type(req.messages) == "table" then
      for _, m in ipairs(req.messages) do
        if type(m) == "table" and m.role == "user" and type(m.content) == "string" then
          last = m.content
        end
      end
    end
    return last
  end)
  if ok and prompt then
    span:set_attribute("gen_ai.llm.input.user", prompt)
  end
  span:finish()
  return {
    block = (resp and resp.block) == true,
    block_message = (resp and resp.reason) or "Blocked by Fiddler guardrail",
  }
end

​

Known Limitations

​

Next Steps

• Kong AI Gateway Integration — set up LLM tracing for Kong (proxy, OTel export, session grouping).
• Fiddler Guardrails — the PII and secrets checks behind the Kong guardrail adapter.
• LiteLLM Guardrails — Fiddler guardrails via the LiteLLM proxy, with redaction support.
• AgentGateway Guardrails — Fiddler guardrails via AgentGateway’s webhook protocol.