Portkey AI Gateway Guardrails

Overview

Portkey’s Bring Your Own Guardrails (BYOG) webhook allows Fiddler to evaluate prompts and responses flowing through Portkey and either allow, block, or redact PII and secrets — all without Fiddler sitting in the traffic path. Unlike Kong Guardrails (which can only block), Portkey supports transformedData — the adapter can return modified request or response JSON with PII/secrets redacted inline. This is closer to LiteLLM Guardrails behavior.

Feature	Details
PII detection and redaction	Social security numbers, credit cards, email addresses, phone numbers, and more
Secrets detection	API keys, tokens, passwords in prompts or responses
Inline redaction	PII/secrets are replaced with `[REDACTED <TYPE>]` in the request/response JSON via `transformedData`
Fail-secure	If redaction cannot be applied (malformed JSON body), the adapter blocks the request rather than leaking unredacted content
Before and after hooks	Scan prompts (`beforeRequestHook`) and/or responses (`afterRequestHook`) independently

Portkey’s BYOG webhook feature is available on all Portkey tiers (cloud and self-hosted) — unlike OTel trace export which is enterprise/self-hosted only (see Portkey Integration). The Fiddler guardrail adapter requires a Fiddler deployment with guardrails enabled. On self-hosted Portkey gateways, the Fiddler webhook URL must be allowlisted in TRUSTED_CUSTOM_HOSTS if it is on a private network.

Architecture

Prerequisites

A running Fiddler deployment with guardrails enabled
A Fiddler API key with guardrails read permission
A running Portkey instance (cloud or self-hosted)

Setup

Create a BYOG Guardrail in Portkey

In your Portkey configuration, create a guardrail with the BYOG webhook type:

{
  "type": "webhook",
  "webhookUrl": "https://<your-fiddler-instance>/v3/guardrails/portkey",
  "headers": {
    "Authorization": "Bearer <FIDDLER_API_KEY>"
  },
  "timeout": 30000
}

Field	Value
`webhookUrl`	Your Fiddler instance URL + `/v3/guardrails/portkey`
`headers.Authorization`	`Bearer <FIDDLER_API_KEY>` — a Fiddler API key with guardrails permission
`timeout`	Timeout in milliseconds (recommended: 30000 for first request, which may trigger model warm-up)

The FIDDLER_API_KEY is a secret — do not commit it to source control. Inject it from your secret manager.

Attach the Guardrail to Your Portkey Config

Attach the guardrail check to your Portkey virtual key or config. You can choose to scan:

beforeRequestHook — scans the user’s prompt before it reaches the LLM
afterRequestHook — scans the LLM’s response before it reaches the user
Both — scans both prompts and responses

Refer to Portkey’s guardrail documentation for configuration details.

Verify the Integration

Send a test request through Portkey with PII content to verify the guardrail is working:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.portkey.ai/v1",
    api_key="<your-portkey-api-key>",
    default_headers={
        "x-portkey-api-key": "<your-portkey-api-key>",
        "x-portkey-provider": "openai",
    },
)

# This should trigger PII redaction
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "My SSN is 123-45-6789"}],
)

If guardrails are working, the SSN will be replaced with [REDACTED SOCIAL_SECURITY_NUMBER] in the request that reaches the LLM.

Wire Contract

Request (Portkey → Fiddler)

Portkey POSTs the following JSON to /v3/guardrails/portkey:

Field	Type	Description
`request.json`	object	OpenAI-compliant request body
`request.text`	string	Last message/prompt content
`request.isStreamingRequest`	boolean	Whether the request uses streaming
`response.json`	object	OpenAI-compliant response body (empty for `beforeRequestHook`)
`response.text`	string	Last message content from the response
`response.statusCode`	integer or null	HTTP status from the LLM provider
`provider`	string	Provider slug (e.g. `"openai"`, `"anthropic"`)
`requestType`	string	`"chatComplete"`, `"complete"`, or `"embed"`
`eventType`	string	`"beforeRequestHook"` or `"afterRequestHook"`
`metadata`	object or null	Custom metadata attached to the request

Response (Fiddler → Portkey)

Field	Type	Description
`verdict`	boolean	`true` = allow (possibly with redactions), `false` = block
`transformedData`	object or null	Modified request/response JSON with redacted content (only present when PII/secrets are redacted)

Decision Mapping

Guardrail decision	Portkey response	Effect
NONE (benign)	`{verdict: true}`	Request/response passes through unchanged
REDACT (PII/secrets detected)	`{verdict: true, transformedData: {...}}`	Portkey uses the redacted JSON in place of the original
BLOCK (safety violation)	`{verdict: false}`	Portkey applies its configured guardrail action (block, retry, etc.)
REDACT (malformed JSON body)	`{verdict: false}`	Fail-secure — blocks to prevent PII leakage when redaction cannot be applied

Redaction Example

When PII is detected in a beforeRequestHook, the response includes modified request JSON:

{
  "verdict": true,
  "transformedData": {
    "request": {
      "json": {
        "model": "gpt-4o",
        "messages": [
          {
            "role": "user",
            "content": "My social security number is [REDACTED SOCIAL_SECURITY_NUMBER] and I need help filing taxes"
          }
        ]
      }
    }
  }
}

For afterRequestHook, the transformedData contains modified response JSON with both the json body and text field redacted.

Which Checks Run

The Portkey guardrail adapter runs three checks: Safety (prompt safety / jailbreak detection), PII, and secrets. Faithfulness — offered elsewhere in Fiddler Protect — is not available on the gateway path.

Data Handling

The guardrail adapter is stateless. Fiddler evaluates the text in-request and returns a decision; no prompt, response, or detection result is retained after the webhook call completes. The only durable artifacts are content-free operational counters (request count, latency). No customer data is stored, logged, or used for model training.

Troubleshooting

Webhook returns 503 or times out on first request The guardrails ML models may need to warm up on first request. Set the Portkey webhook timeout to at least 30 seconds (30000ms) for the initial call. Subsequent calls are typically under 2 seconds. Webhook returns 400 The request payload is invalid. Ensure eventType is either "beforeRequestHook" or "afterRequestHook". Check that the payload is valid JSON. Webhook returns 403 The Fiddler API key does not have guardrails permission, or guardrails are not enabled on the Fiddler deployment. PII not being redacted (verdict true, no transformedData) The PII detection model may not have detected PII in the text. The adapter only returns transformedData when PII or secrets are detected above the configured confidence threshold (default: 0.5 for PII in production deployments). Also verify the request.text or response.text field contains the content to scan — Portkey extracts these from the message body.

Next Steps

Portkey Integration — set up OTel observability with Portkey.
Kong Guardrails — guardrail enforcement on Kong.
LiteLLM Guardrails — guardrail enforcement on LiteLLM.
AgentGateway Guardrails — guardrail enforcement on AgentGateway.

Getting Started

Concepts

Evaluate & Test

Reference

Overview

Architecture

Prerequisites

Setup

Wire Contract

Request (Portkey → Fiddler)

Response (Fiddler → Portkey)

Decision Mapping

Redaction Example

Which Checks Run

Data Handling

Troubleshooting

Next Steps

​Overview

​Architecture

​Prerequisites

​Setup

​Wire Contract

​Request (Portkey → Fiddler)

​Response (Fiddler → Portkey)

​Decision Mapping

​Redaction Example

​Which Checks Run

​Data Handling

​Troubleshooting

​Next Steps

Overview

Architecture

Prerequisites

Setup

Wire Contract

Request (Portkey → Fiddler)

Response (Fiddler → Portkey)

Decision Mapping

Redaction Example

Which Checks Run

Data Handling

Troubleshooting

Next Steps