Skip to main content
This capability is currently in private preview and is not yet available to all customers. Contact your Fiddler Customer Success Manager to request access.

​
Overview

The Executive Dashboard gives organization leadership a single, at-a-glance view of GenAI adoption, quality, and cost across every application — without stitching together per-application dashboards.
  • Top-line KPIs — Total Apps, Active Apps, Total Traces, Total Spans, Errors, Sessions, and Evaluations Run over a selectable 7-day, 14-day, or 30-day window.
  • Evaluator scores — Average Quality alongside Answer Relevance, Context Relevance, RAG Faithfulness, Response Faithfulness, Conciseness, Prompt Safety, and PII Detection, aggregated across all applications.
  • Usage and latency — Traffic and token-usage trends, Average Trace Latency, and P90 Span Latency.
  • Application Health Overview — a per-application table (traces, average latency, tokens, average quality) with drill-down links into each application.
  • Period-over-period change — each metric card shows the change versus the immediately preceding window of equal length.

​
How metrics are captured

Executive Dashboard metrics are not computed from raw traces at page load. Instead, Fiddler continuously pre-aggregates them into a dedicated metric store:
  • Hourly aggregation. Background jobs run every hour and summarize the most recently completed hour of traces, spans, and evaluator results into per-application metric values (counts, sums, averages, and latency percentiles). Coarser views — daily or weekly trends — are derived from these hourly values.
  • Metric history outlives raw data. The metric store is retained independently of your raw trace retention policy. When raw traces and spans are deleted at the end of their configured retention period, the aggregated metric history remains, so long-range trends and period-over-period comparisons keep working.
  • Freshness. Because aggregation runs hourly on completed hours, the most recent data point on the dashboard can lag real time by up to an hour. Evaluator scores that are produced asynchronously are incorporated by subsequent aggregation passes.
  • What is aggregated. Only numeric metric values are stored in the metric store — counts, token totals, latency statistics, and evaluator score aggregates per application and hour. No prompt, response, or other span content is copied into it.

​
Alert counts and the 30-day limit

Critical and warning alert counts are derived from Fiddler’s alert breach event log, not from raw traces. Each time an alert’s severity changes to critical or warning, one event is counted in the hour it was logged. An alert window that escalates from warning to critical is counted once as a warning and once as critical.
The alert breach event log is retained for 30 days by default, independent of your trace retention and of the metric store’s own retention. Alert counts therefore start accumulating when this metric is first collected and are not backfilled for earlier periods. A 30-day view, or the preceding window used for period-over-period change, may show incomplete alert counts until 30 days of history exist.

​
Cost and last activity

Cost is read from the cost attributes your instrumentation already emits, using Fiddler’s semantic attribute mapping, and is reported as total, input, and output cost. The values carry no currency in the source attributes, so Fiddler treats them as US dollars (USD). Not every SDK or model emits cost. Applications that do not report a cost attribute have no cost value rather than a cost of zero. The last-activity timestamp is the timestamp recorded on the application’s newest span, not the time the metric was aggregated. Because only completed hours are aggregated, it can lag real time by up to two hours.
Period-over-period change compares the selected window against the window of equal length immediately before it — a 30-day view reads 60 days of metric history. Deltas appear once enough history has accumulated in the metric store.