Portkey AI Gateway Integration

Overview

Portkey is an AI gateway that proxies requests to LLM providers (OpenAI, Anthropic, Google, Mistral, Cohere, Azure OpenAI, and more) with built-in caching, fallbacks, load balancing, and observability. Fiddler integrates with Portkey via its experimental “Complete Logs Export” feature, which pushes full LLM request/response logs as OTel spans to any OTLP-compatible endpoint.

Capability	Notes
Zero instrumentation	Point your app at Portkey — no Fiddler SDK or code changes needed
LLM span tracing	Token counts, model name, provider, latency, and full message content
Multi-provider support	OpenAI, Anthropic, Google, Mistral, Cohere, Azure OpenAI, and 200+ models via Portkey
OTel export	Portkey exports traces directly to Fiddler’s OTel Collector over HTTPS
Guardrails	Block or redact PII and secrets at the gateway via Portkey’s “Bring Your Own Guardrails” webhook — see Portkey Guardrails

Portkey’s “Complete Logs Export” OTel feature is labelled experimental and is available on self-hosted / enterprise Portkey deployments only. It is not available on Portkey’s free cloud tier. This page’s tracing setup is built on that feature.

Architecture

Portkey exposes an OpenAI-compatible API. Your application calls Portkey instead of the provider directly. Portkey handles routing, caching, and fallbacks, then exports OTel spans to Fiddler for observability.

Prerequisites

Fiddler account with a GenAI application already created
A running Portkey instance (self-hosted or enterprise — the OTel export feature is not available on the free cloud tier)
A valid LLM provider API key configured in Portkey
Your Fiddler API key (found under organizational settings) and application UUID (found under application settings)

Quick Start

Step 1: Configure Portkey’s OTel Export

Set the following environment variables on your Portkey deployment to enable OTel trace export to Fiddler:

EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED=true
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT=https://<your-fiddler-instance>/v1/traces
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <FIDDLER_API_KEY>,Fiddler-Application-Id=<FIDDLER_APP_ID>"

Variable	Value
`EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED`	`true` — enables Portkey’s OTel trace export
`EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT`	Your Fiddler instance URL with `/v1/traces` path
`EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS`	Comma-separated key=value pairs for auth and application routing

The FIDDLER_API_KEY is a secret — do not commit it to source control. Inject it at deploy time from your secret manager.

Step 2: Point Your Application at Portkey

No changes to your application code are needed if you already use Portkey. If you are adding Portkey for the first time:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.portkey.ai/v1",  # or your self-hosted Portkey URL
    api_key="<your-portkey-api-key>",
    default_headers={
        "x-portkey-api-key": "<your-portkey-api-key>",
        "x-portkey-provider": "openai",
    },
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)

Portkey is OpenAI-compatible, so any OpenAI SDK client works with no changes beyond the base_url and headers.

Step 3: Verify Traces Are Arriving

Open the Fiddler UI and navigate to your application’s Explorer. You should see LLM spans appearing within a few seconds of making your first LLM call through Portkey.

Span Type Mapping

Fiddler classifies Portkey spans based on the gen_ai.operation.name attribute. Portkey names its spans "{operation} {model}" (e.g. "chat gpt-4o"), and the mapper reads the gen_ai.operation.name attribute to determine the span type:

`gen_ai.operation.name`	Span name pattern	Fiddler treatment
`chat`	`"chat {model}"`	`llm` (forwarded)
`text_completion`	`"text_completion {model}"`	`llm` (forwarded)
`generate_content`	`"generate_content {model}"`	`llm` (forwarded)
`embeddings`	`"embeddings {model}"`	Dropped (not a recognized LLM operation)

Unlike Kong, Portkey emits a single span per LLM request — there is no multi-span hierarchy or infrastructure spans to filter.

Attribute Mapping

Portkey follows the OTel GenAI semantic conventions v1.40.0. Fiddler’s mapper normalizes these automatically:

Portkey attribute	Fiddler treatment
`gen_ai.provider.name`	Copied to `gen_ai.system` when absent; passed through unchanged
`gen_ai.request.model`, `gen_ai.response.model`	Passed through unchanged
`gen_ai.usage.input_tokens`	Passed through unchanged
`gen_ai.usage.output_tokens`	Passed through unchanged
`gen_ai.usage.total_tokens`	Computed as `input + output` when absent
`gen_ai.agent.name`	Defaulted to `<UNKNOWN_AGENT>` when absent
`gen_ai.conversation.id`	Defaulted to `trace_id.hex()` when absent
`gen_ai.system_instructions`	Parsed into `gen_ai.llm.input.system` (system prompt text)
`gen_ai.input.messages`	Parsed into `gen_ai.llm.input.user` (last user message text)
`gen_ai.output.messages`	Parsed into `gen_ai.llm.output` (assistant message text)

Portkey uses the semconv 1.40.0 structured message format with parts arrays (e.g. [{"role":"user","parts":[{"type":"text","content":"..."}]}]). The mapper handles both native OTLP array encoding and JSON string fallback.

Guardrails

Beyond tracing, Portkey can enforce Fiddler Guardrails on prompts and responses via the “Bring Your Own Guardrails” (BYOG) webhook. Unlike Kong (which can only block), Portkey supports transformedData — PII and secrets can be redacted in place rather than just blocked. See Portkey Guardrails for setup and the full configuration.

Troubleshooting

Traces not appearing in Fiddler Verify all three environment variables are set correctly on your Portkey deployment. Check that EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED is true and that the endpoint URL includes /v1/traces. The Fiddler-Application-Id header is required on the OTel export request. If missing, Fiddler skips those spans and logs a warning. Prompt and response content not showing Portkey includes full message content by default when the OTel export is enabled. If content is missing, check your Portkey configuration for any payload-logging restrictions. OTel export feature not available The “Complete Logs Export” feature is available on self-hosted and enterprise Portkey deployments only. It is not available on Portkey’s free cloud tier.

Known Limitations

Limitation	Details
Enterprise / self-hosted only	Portkey’s OTel export is not available on the free cloud tier
Experimental feature	Portkey labels the OTel export as experimental — behavior may change across Portkey releases
No session grouping	Portkey does not natively set `gen_ai.conversation.id` from a request header; Fiddler defaults it to `trace_id.hex()`, so each LLM call appears as a separate session
HTTP/JSON only	Portkey exports OTLP over HTTP/JSON — gRPC export is not supported

Next Steps

Portkey Guardrails — block or redact PII and secrets at the Portkey gateway.
Kong AI Gateway Integration — add Fiddler observability through Kong.
LiteLLM Integration — add Fiddler observability through LiteLLM.
AgentGateway Integration — add Fiddler observability through AgentGateway.
Portkey documentation — Portkey’s official docs.

Agentic AI & LLM Frameworks

Data Platforms & Pipelines

ML Platforms & Tools

Cloud Platforms & Deployment

Monitoring & Alerting

Overview

Architecture

Prerequisites

Quick Start

Step 1: Configure Portkey’s OTel Export

Step 2: Point Your Application at Portkey

Step 3: Verify Traces Are Arriving

Span Type Mapping

Attribute Mapping

Guardrails

Troubleshooting

Known Limitations

Next Steps

​Overview

​Architecture

​Prerequisites

​Quick Start

​Step 1: Configure Portkey’s OTel Export

​Step 2: Point Your Application at Portkey

​Step 3: Verify Traces Are Arriving

​Span Type Mapping

​Attribute Mapping

​Guardrails

​Troubleshooting

​Known Limitations

​Next Steps

Overview

Architecture

Prerequisites

Quick Start

Step 1: Configure Portkey’s OTel Export

Step 2: Point Your Application at Portkey

Step 3: Verify Traces Are Arriving

Span Type Mapping

Attribute Mapping

Guardrails

Troubleshooting

Known Limitations

Next Steps