> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fiddler.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Publishing Production Data

> Navigate our client guide to publishing production data. Learn how to provide event data to Fiddler, update it, and retrieve it efficiently.

### Publish Inference Events to Fiddler

After you onboard an ML model or LLM application as a Fiddler Model, you can publish inference events for analysis, performance monitoring, and reporting. There are two types of inference data:

* Pre-production data: Static datasets such as training or testing data that serve as Baseline references for comparison
* Production data: Time series data from live model inferences that Fiddler monitors against your baselines

#### Integration Methods

Fiddler offers two ways to publish inference data:

**Python Client Library**

Use the Python client for Python environments. Publish production and pre-production inference data with `Model.publish_stream()` for streaming events or `Model.publish_batch()` for batch uploads.

For more details, see the [Python client SDK](/sdk-api/python-client/connection) documentation.

**REST API**

Use the [REST API](/sdk-api/rest-api/alert-rules) for language-agnostic integration across any platform. Both production and pre-production inference data use a common interface.

### Publish Pre-Production Data

Publish pre-production data to Fiddler as a single dataset. You can add multiple baseline datasets to a model to create customized references for different metrics and alert rules.

Fiddler accepts pre-production data in these formats:

* Pandas DataFrame
* Parquet file
* CSV file

For detailed instructions, see the [Creating a Baseline](/developers/python-client-guides/publishing-production-data/creating-a-baseline-dataset) guide.

> **Note**:
>
> Pre-production datasets are immutable after publication. You can't update them or delete individual rows.

#### Upload a Static Pre-Production Baseline

```python theme={null}
dataset_file_path = 'path_to_your_data.parquet'
dataset_name = 'a_unique_identifying_name'

project = fdl.Project.from_name(name='your_project_name')
model = fdl.Model.from_name(name='your_model_name', project_id=project.id)

job = model.publish_batch(
    source=dataset_file_path,
    environment=fdl.EnvType.PRE_PRODUCTION,
    dataset_name=dataset_name,
)
# publish_batch() is asynchronous. Use the publish job's wait() method
# if synchronous behavior is desired.
# job.wait()
```

### Publish Production Data

Fiddler provides several methods for publishing and editing production inference data:

* Batch publishing (`Model.publish_batch()`): Send data in batches using pandas DataFrames, Parquet files, or CSV files
* Stream publishing (`Model.publish_stream()`): Send individual events or small batches in near real-time
* Update publishing: Modify previously published data
* Delete publishing: Remove published data when needed

Fiddler accepts production data in these formats:

* Pandas DataFrames
* Parquet files
* CSV files
* List of Python dictionaries (limited to stream and updates)

A list of dictionaries is an additional data format on top of the three common to pre-production and production data.

Choose the method that best fits your use case by reviewing the publishing guides below:

* [Publish batch events](/developers/python-client-guides/publishing-production-data/publishing-batches-of-events)
* [Stream live events](/developers/python-client-guides/publishing-production-data/streaming-live-events)
* [Update published events](/developers/python-client-guides/publishing-production-data/updating-events)
* [Delete events](/developers/python-client-guides/publishing-production-data/deleting-events)
* [Publish ranking events](/developers/python-client-guides/publishing-production-data/ranking-events)
* [Event deduplication](/developers/python-client-guides/publishing-production-data/event-deduplication)

#### Key Considerations

Here are some considerations to keep in mind as you onboard models and begin publishing production data to Fiddler.

**Inference Event Unique Identifier**

Fiddler requires a unique identifier on each event published should you later need to update ground truth labels and/or metadata columns.

* Define the unique identifier column name when onboarding a model: `Model.event_id_col`
* A unique index on the event id column is not enforced
* When [event deduplication](/developers/python-client-guides/publishing-production-data/event-deduplication) is not enabled, duplicate values are accepted and events sharing the same event id value will all be used in calculating metrics

**Event Deduplication**

When event deduplication is enabled, Fiddler uses the `event_id_col` value to detect and drop duplicate events across multiple layers — within a single batch, against recently stored events, and across concurrent publish calls. This prevents duplicate events from inflating aggregated metrics such as drift scores, event counts, and model performance indicators.

* Event deduplication is not enabled by default — contact Fiddler support to enable it
* Events without an event ID (null or missing) are never deduplicated
* Deduplication applies to insert operations only — update operations are not deduplicated

For full details on how each layer works, see [Event Deduplication](/developers/python-client-guides/publishing-production-data/event-deduplication).

**Inference Event Timestamp**

Fiddler requires a timestamp for each inference event which is used as the event occurrence timestamp in time-series monitoring charts and alert rule evaluation.

* Define the timestamp column name when onboarding a model: `Model.event_ts_col`
* If not defined, Fiddler will use the time of publication as the event occurrence timestamp
* Timestamps are stored and rendered in UTC
* Timestamps with timezone are accepted but will be converted to UTC
* Fiddler supports basic pandas timestamp formats by inferring from the data

### Data Retention Policy

Fiddler retains production inference event data for 90 days. Contact your Fiddler customer success representative if you need a different retention period.

#### Raw Event Data

* Retained for 90 days from publication date
* Automatically deleted after 90 days
* Policy applies globally

#### Pre-Calculated Metrics

* Standard metrics derived from raw data are retained indefinitely
* Dashboards and charts continue to display historical trends after raw data expires

#### Runtime features

* Custom metrics require raw event data and aren't available for data older than 90 days