> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fiddler.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Portkey AI Gateway Integration

> Integrate Portkey AI Gateway with Fiddler for zero-instrumentation LLM observability — no application code changes required.

## Overview

[Portkey](https://portkey.ai/) is an AI gateway that proxies requests to LLM providers (OpenAI, Anthropic, Google, Mistral, Cohere, Azure OpenAI, and more) with built-in caching, fallbacks, load balancing, and observability. Fiddler integrates with Portkey via its experimental "Complete Logs Export" feature, which pushes full LLM request/response logs as OTel spans to any OTLP-compatible endpoint.

| Capability | Notes |
| - | - |
| **Zero instrumentation** | Point your app at Portkey — no Fiddler SDK or code changes needed |
| **LLM span tracing** | Token counts, model name, provider, latency, and full message content |
| **Multi-provider support** | OpenAI, Anthropic, Google, Mistral, Cohere, Azure OpenAI, and 200+ models via Portkey |
| **OTel export** | Portkey exports traces directly to Fiddler's OTel Collector over HTTPS |
| **Guardrails** | Block or redact PII and secrets at the gateway via Portkey's "Bring Your Own Guardrails" webhook — see [Portkey Guardrails](/protection/portkey-guardrails) |

<Note>
  Portkey's "Complete Logs Export" OTel feature is labelled experimental and is available on **self-hosted / enterprise** Portkey deployments only. It is not available on Portkey's free cloud tier. This page's tracing setup is built on that feature.
</Note>

## Architecture

```mermaid theme={null}
flowchart TD
    A["Your app"] -->|standard OpenAI client| B["Portkey AI Gateway"]
    B -->|proxies to OpenAI / Anthropic / …| C["LLM Provider"]
    B -->|OTel spans over HTTPS| D["Fiddler\n(enrichment, monitoring, dashboards)"]
    B -->|BYOG webhook| E["Fiddler Guardrails\n(PII/secret detection)"]
```

Portkey exposes an OpenAI-compatible API. Your application calls Portkey instead of the provider directly. Portkey handles routing, caching, and fallbacks, then exports OTel spans to Fiddler for observability.

## Prerequisites

* Fiddler account with a GenAI application already created
* A running [Portkey](https://portkey.ai/) instance (self-hosted or enterprise — the OTel export feature is not available on the free cloud tier)
* A valid LLM provider API key configured in Portkey
* Your Fiddler API key (found under organizational settings) and application UUID (found under application settings)

## Quick Start

### Step 1: Configure Portkey's OTel Export

Set the following environment variables on your Portkey deployment to enable OTel trace export to Fiddler:

```bash theme={null}
EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED=true
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT=https://<your-fiddler-instance>/v1/traces
EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <FIDDLER_API_KEY>,Fiddler-Application-Id=<FIDDLER_APP_ID>"
```

| Variable | Value |
| - | - |
| `EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED` | `true` — enables Portkey's OTel trace export |
| `EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_ENDPOINT` | Your Fiddler instance URL with `/v1/traces` path |
| `EXPERIMENTAL_GEN_AI_OTEL_EXPORTER_OTLP_HEADERS` | Comma-separated key=value pairs for auth and application routing |

<Warning>
  The `FIDDLER_API_KEY` is a secret — do not commit it to source control. Inject it at deploy time from your secret manager.
</Warning>

### Step 2: Point Your Application at Portkey

No changes to your application code are needed if you already use Portkey. If you are adding Portkey for the first time:

```python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.portkey.ai/v1",  # or your self-hosted Portkey URL
    api_key="<your-portkey-api-key>",
    default_headers={
        "x-portkey-api-key": "<your-portkey-api-key>",
        "x-portkey-provider": "openai",
    },
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)
```

Portkey is OpenAI-compatible, so any OpenAI SDK client works with no changes beyond the `base_url` and headers.

### Step 3: Verify Traces Are Arriving

Open the Fiddler UI and navigate to your application's **Explorer**. You should see LLM spans appearing within a few seconds of making your first LLM call through Portkey.

## Span Type Mapping

Fiddler classifies Portkey spans based on the `gen_ai.operation.name` attribute. Portkey names its spans `"{operation} {model}"` (e.g. `"chat gpt-4o"`), and the mapper reads the `gen_ai.operation.name` attribute to determine the span type:

| `gen_ai.operation.name` | Span name pattern | Fiddler treatment |
| - | - | - |
| `chat` | `"chat {model}"` | `llm` (forwarded) |
| `text_completion` | `"text_completion {model}"` | `llm` (forwarded) |
| `generate_content` | `"generate_content {model}"` | `llm` (forwarded) |
| `embeddings` | `"embeddings {model}"` | Dropped (not a recognized LLM operation) |

Unlike Kong, Portkey emits a single span per LLM request — there is no multi-span hierarchy or infrastructure spans to filter.

## Attribute Mapping

Portkey follows the OTel GenAI semantic conventions v1.40.0. Fiddler's mapper normalizes these automatically:

| Portkey attribute | Fiddler treatment |
| - | - |
| `gen_ai.provider.name` | Copied to `gen_ai.system` when absent; passed through unchanged |
| `gen_ai.request.model`, `gen_ai.response.model` | Passed through unchanged |
| `gen_ai.usage.input_tokens` | Passed through unchanged |
| `gen_ai.usage.output_tokens` | Passed through unchanged |
| `gen_ai.usage.total_tokens` | Computed as `input + output` when absent |
| `gen_ai.agent.name` | Defaulted to `<UNKNOWN_AGENT>` when absent |
| `gen_ai.conversation.id` | Defaulted to `trace_id.hex()` when absent |
| `gen_ai.system_instructions` | Parsed into `gen_ai.llm.input.system` (system prompt text) |
| `gen_ai.input.messages` | Parsed into `gen_ai.llm.input.user` (last user message text) |
| `gen_ai.output.messages` | Parsed into `gen_ai.llm.output` (assistant message text) |

Portkey uses the semconv 1.40.0 structured message format with `parts` arrays (e.g. `[{"role":"user","parts":[{"type":"text","content":"..."}]}]`). The mapper handles both native OTLP array encoding and JSON string fallback.

## Guardrails

Beyond tracing, Portkey can enforce [Fiddler Guardrails](/protection/portkey-guardrails) on prompts and responses via the "Bring Your Own Guardrails" (BYOG) webhook. Unlike Kong (which can only block), Portkey supports `transformedData` — PII and secrets can be **redacted in place** rather than just blocked.

See [Portkey Guardrails](/protection/portkey-guardrails) for setup and the full configuration.

## Troubleshooting

**Traces not appearing in Fiddler**

Verify all three environment variables are set correctly on your Portkey deployment. Check that `EXPERIMENTAL_GEN_AI_OTEL_TRACES_ENABLED` is `true` and that the endpoint URL includes `/v1/traces`.

The `Fiddler-Application-Id` header is required on the OTel export request. If missing, Fiddler skips those spans and logs a warning.

**Prompt and response content not showing**

Portkey includes full message content by default when the OTel export is enabled. If content is missing, check your Portkey configuration for any payload-logging restrictions.

**OTel export feature not available**

The "Complete Logs Export" feature is available on self-hosted and enterprise Portkey deployments only. It is not available on Portkey's free cloud tier.

## Known Limitations

| Limitation | Details |
| - | - |
| **Enterprise / self-hosted only** | Portkey's OTel export is not available on the free cloud tier |
| **Experimental feature** | Portkey labels the OTel export as experimental — behavior may change across Portkey releases |
| **No session grouping** | Portkey does not natively set `gen_ai.conversation.id` from a request header; Fiddler defaults it to `trace_id.hex()`, so each LLM call appears as a separate session |
| **HTTP/JSON only** | Portkey exports OTLP over HTTP/JSON — gRPC export is not supported |

## Next Steps

* [Portkey Guardrails](/protection/portkey-guardrails) — block or redact PII and secrets at the Portkey gateway.
* [Kong AI Gateway Integration](/integrations/agentic-ai/kong-integration) — add Fiddler observability through Kong.
* [LiteLLM Integration](/integrations/agentic-ai/litellm-integration) — add Fiddler observability through LiteLLM.
* [AgentGateway Integration](/integrations/agentic-ai/agentgateway-integration) — add Fiddler observability through AgentGateway.
* [Portkey documentation](https://portkey.ai/docs) — Portkey's official docs.