Fiddler Query Language

Overview

Custom Metrics and Segments are defined using the Fiddler Query Language (FQL), a flexible set of constants, operators, and functions which can accommodate a large variety of metrics.

Definitions

Term	Definition
Row-level function	A function which executes row-wise for a set of data. Returns a value for each row.
Aggregate function	A function which executes across rows. Returns a single value for a given set of rows.

FQL Rules

Column names can be referenced by name either with double quotes (“my_column”) or with no quotes (my_column).
Single quotes (’) are used to represent string values.

Data Types

FQL distinguishes between three data types:

Data type	Supported values	Examples	Supported Model Schema Data Types
Number	Any numeric value (integers and floats are both included)	`10` `2.34`	`DataType.INTEGER` `DataType.FLOAT`
Boolean	Only `true` and `false`	`true` `false`	`DataType.BOOLEAN`
String	Any value wrapped in single quotes (`'`)	`‘This is a string.'` `'200.0’`	`DataType.CATEGORY` `DataType.STRING`

Constants

Symbol	Description
`true`	Boolean constant for true expressions
`false`	Boolean constant for false expressions
`null`	Represents an absent or missing value. Use as a return value in conditional expressions — for example: `if(is_null(x), null, x * 2)`. To test whether a value is null, use `is_null()` or `is_not_null()`.

Operators

Symbol	Description	Syntax	Returns	Examples
`^`	Exponentiation	`Number ^ Number`	`Number`	`2.5 ^ 4` `(column1 - column2)^2`
`-`	Unary negation	`-Number`	`Number`	`-column1`
`*`	Multiplication	`Number * Number`	`Number`	`2 * 10` `2 * column1` `column1 * column2` `sum(column1) * 10`
`/`	Division	`Number / Number`	`Number`	`2 / 10` `2 / column1` `column1 / column2` `sum(column1) / 10`
`%`	Modulo	`Number % Number`	`Number`	`2 % 10` `2 % column1` `column1 % column2` `sum(column1) % 10`
`+`	Addition	`Number + Number`	`Number`	`2 + 2` `2 + column1` `column1 + column2` `average(column1) + 2`
`-`	Subtraction	`Number - Number`	`Number`	`2 - 2` `2 - column1` `column1 - column2` `average(column1) - 2`
`<`	Less than	`Number < Number`	`Boolean`	`10 < 20` `column1 < 10` `column1 < column2` `average(column2) < 5`
`<=`	Less than or equal to	`Number <= Number`	`Boolean`	`10 <= 20` `column1 <= 10` `column1 <= column2` `average(column2) <= 5`
`>`	Greater than	`Number > Number`	`Boolean`	`10 > 20` `column1 > 10` `column1 > column2` `average(column2) > 5`
`>=`	Greater than or equal to	`Number >= Number`	`Boolean`	`10 >= 20` `column1 >= 10` `column1 >= column2` `average(column2) >= 5`
`==`	Equals	`Number == Number`	`Boolean`	`10 == 20` `column1 == 10` `column1 == column2` `average(column2) == 5`
`!=`	Does not equal	`Number != Number`	`Boolean`	`10 != 20` `column1 != 10` `column1 != column2` `average(column2) != 5`
`not`	Logical NOT	`not Boolean`	`Boolean`	`not true` `not column1`
`and`	Logical AND	`Boolean and Boolean`	`Boolean`	`true and false` `column1 and column2`
`or`	Logical OR	`Boolean or Boolean`	`Boolean`	`true or false` `column1 or column2`

The comparison operators are not limited to numbers. Each requires its two operands to share the same data type — numbers, strings, or booleans — and rejects mixed-type comparisons. Equality (==, !=) and the ordering operators apply the same rule, so ordering is not limited to numeric operands (string comparisons are lexicographic, and true is greater than false). The null literal is not a valid comparison operand — use is_null() or is_not_null() instead.

Constant functions

Symbol	Description	Syntax	Returns	Examples
`e()`	Base of the natural logarithm	`e()`	`Number`	`e() == 2.718281828459045`
`pi()`	The ratio of a circle’s circumference to its diameter	`pi()`	`Number`	`pi() == 3.141592653589793`

Row-level functions

Row-level functions can be applied either to a single value or to a column/row expression (in which case they are mapped element-wise to each value in the column/row expression).

Symbol	Description	Syntax	Returns	Examples
`if(condition, value1, value2)`	Evaluates `condition` and returns `value1` if true, otherwise returns `value2`. `value1` and `value2` must have the same type.	`if(Boolean, Any, Any)`	`Any`	`if(false, ‘yes’, ‘no’) == ‘no’` `if(column1 == 1, ‘yes’, ‘no’)`
`length(x)`	Returns the length of string `x`.	`length(String)`	`Number`	`length('Hello world') == 11`
`to_string(x)`	Converts a value `x` to a string.	`to_string(Any)`	`String`	`to_string(42) == ‘42’` `to_string(true) == ‘true’`
`startswith(str, prefix)`	Returns `true` if `str` starts with `prefix`.	`startswith(String, String)`	`Boolean`	`startswith('abcde', 'abc')`
`substring(str, offset, length)`	Returns a substring of `str` of length `length` from offset `offset`. The first character has an offset of 1.	`substring(String, Number, Number)`	`String`	`substring('abcde', 2, 3) == 'bcd'`
`match(str, regex)`	Returns `true` if `str` matches the pattern `regex` in re2 regular syntax.	`match(String, String)`	`Boolean`	`match('abcde', 'a.c.*e')`
`is_null(x)`	Returns `true` if `x` is null, otherwise returns `false`. For a numeric `x`, non-finite values (`NaN` or infinity) also count as null.	`is_null(Any)`	`Boolean`	`is_null(“column1”)` `is_null(”) == false`
`is_not_null(x)`	Returns `true` if `x` is not null, otherwise returns `false`.	`is_not_null(Any)`	`Boolean`	`is_not_null(“column1”)` `is_not_null(”) == true`
`in(x, ...values)`	Returns `true` if `x` equals any of the listed values. Values must be literals of one type, matching the type of `x`. A null value for `x` returns null, which does not satisfy a filter.	`in(Any, Any, ...)`	`Boolean`	`in(country, 'US', 'CA')`
`not_in(x, ...values)`	Returns `true` if `x` equals none of the listed values. Same value rules as `in`. A null value for `x` returns null, which does not satisfy a filter.	`not_in(Any, Any, ...)`	`Boolean`	`not_in(country, 'US', 'CA')`
`abs(x)`	Returns the absolute value of number `x`.	`abs(Number)`	`Number`	`abs(-3) == 3`
`exp(x)`	Returns `e^x`, where `e` is the base of the natural logarithm.	`exp(Number)`	`Number`	`exp(1) == 2.718281828459045`
`log(x)`	Returns the natural logarithm (base `e`) of number `x`.	`log(Number)`	`Number`	`log(e) == 1`
`log2(x)`	Returns the binary logarithm (base `2`) of number `x`.	`log2(Number)`	`Number`	`log2(16) == 4`
`log10(x)`	Returns the common logarithm (base `10`) of number `x`.	`log10(Number)`	`Number`	`log10(1000) == 3`
`sqrt(x)`	Returns the positive square root of number `x`.	`sqrt(Number)`	`Number`	`sqrt(144) == 12`

Using in(...) and not_in(...)

Every value must be a literal — a quoted string, a number, or a boolean. Expressions are not allowed in the value list, so in(country, region) is rejected even when region is a valid column.
All values must share one data type, and that type must match the tested expression. in(country, 'US', 5) is rejected because the values are mixed.
At least one value is required. in(country) is a parse error.
A null value for x returns null. Both in(x, ...) and not_in(x, ...) return null when x is null, and a null result does not satisfy a filter — so a null row is excluded by both. To also match missing values, combine the expression with the or operator — not_in(country, 'US') or is_null(country).

Aggregate functions

Every Custom Metric must be wrapped in an aggregate function or be a combination of aggregate functions.

Symbol	Description	Syntax	Returns	Examples
`sum(x)`	Returns the sum of a numeric column or row expression `x`.	`sum(Number)`	`Number`	`sum(column1 + column2)`
`average(x)`	Returns the arithmetic mean/average value of a numeric column or row expression `x`.	`average(Number)`	`Number`	`average(2 * column1)`
`count(x)`	Returns the number of non-null rows of a column or row expression `x`.	`count(Any)`	`Number`	`count(column1)`
`min(x)`	Returns the minimum value of a numeric column or row expression `x`.	`min(Number)`	`Number`	`min(column1)`
`max(x)`	Returns the maximum value of a numeric column or row expression `x`.	`max(Number)`	`Number`	`max(column1)`
`quantile(x, level)`	Returns the quantile (percentile) of a numeric column or row expression `x` at the specified `level`. The `level` must be a constant between 0.0 and 1.0 (defaults to 0.5 if not specified).	`quantile(Number, level=Number)`	`Number`	`quantile(Output, level=0.5)` `quantile(latency_ms, level=0.95)` `quantile(response_time, level=0.99)`
`greatest(...aggs)`	Returns the maximum value across one or more numerical aggregates.	`greatest(Number, ...)`	`Number`	`greatest(sum(col1), sum(col2), sum(col3))`
`least(...aggs)`	Returns the minimum value across one or more numerical aggregates.	`least(Number, ...)`	`Number`	`least(sum(col1), sum(col2), sum(col3))`

min(x) / max(x) vs least(...) / greatest(...): min(x) and max(x) aggregate a single row-level expression across rows (e.g., min(column1) returns the smallest value of column1 across all rows in the time window). least(...) and greatest(...) compare multiple aggregate results and return the smallest or largest among them (e.g., least(sum(col1), sum(col2)) compares two already-computed sums).

Built-in metric functions are available for ML models only (classification, regression, and ranking tasks). They are not supported in custom metrics for agentic or GenAI applications. For agentic applications, use the attribute() function with aggregate functions instead — see Custom Metrics for Agentic Applications.

Built-in metric functions

Symbol	Description	Syntax	Returns	Examples
`jsd(column, baseline)`	The Jensen-Shannon distance of column `column` with respect to baseline `baseline`.	`jsd(Any, String)`	`Number`	`jsd(column1, 'my_baseline')`
`psi(column, baseline)`	The population stability index of column `column` with respect to baseline `baseline`.	`psi(Any, String)`	`Number`	`psi(column1, 'my_baseline')`
`null_violation_count(column)`	Number of rows with null values in column `column`.	`null_violation_count(Any)`	`Number`	`null_violation_count(column1)`
`range_violation_count(column)`	Number of rows with out-of-range values in column `column`.	`range_violation_count(Any)`	`Number`	`range_violation_count(column1)`
`type_violation_count(column)`	Number of rows with invalid data types in column `column`.	`type_violation_count(Any)`	`Number`	`type_violation_count(column1)`
`any_violation_count(column)`	Number of rows with at least one Data Integrity violation in `column`.	`any_violation_count(Any)`	`Number`	`any_violation_count(column1)`
`null_violation_percentage(column)`	Percentage of rows with null values in column `column`, out of total rows.	`null_violation_percentage(Any)`	`Number`	`null_violation_percentage(column1)`
`range_violation_percentage(column)`	Percentage of rows with out-of-range values in column `column`, out of total rows. Not supported for string or timestamp columns.	`range_violation_percentage(Any)`	`Number`	`range_violation_percentage(column1)`
`type_violation_percentage(column)`	Percentage of rows with invalid data types in column `column`, out of total rows. Not supported for string or timestamp columns.	`type_violation_percentage(Any)`	`Number`	`type_violation_percentage(column1)`
`any_violation_percentage(column)`	Percentage of rows with at least one Data Integrity violation in `column`, out of total rows.	`any_violation_percentage(Any)`	`Number`	`any_violation_percentage(column1)`
`traffic()`	Total row count. Includes null rows.	`traffic()`	`Number`	`traffic()`
`tp(class)`	True positive boolean state. Available for binary classification and multiclass classification models. For multiclass, `class` is used to specify the positive class.	`tp(class=Optional[String])`	`Boolean`	`tp()` `tp(class=‘class1’)`
`tn(class)`	True negative boolean state. Available for binary classification and multiclass classification models. For multiclass, `class` is used to specify the positive class.	`tn(class=Optional[String])`	`Boolean`	`tn()` `tn(class=‘class1’)`
`fp(class)`	False positive boolean state. Available for binary classification and multiclass classification models. For multiclass, `class` is used to specify the positive class.	`fp(class=Optional[String])`	`Boolean`	`fp()` `fp(class=‘class1’)`
`fn(class)`	False negative boolean state. Available for binary classification and multiclass classification models. For multiclass, `class` is used to specify the positive class.	`fn(class=Optional[String])`	`Boolean`	`fn()` `fn(class=‘class1’)`
`precision(target, threshold)`	Precision between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`precision(target=Optional[Any], threshold=Optional[Number])`	`Number`	`precision()` `precision(target=column1)` `precision(threshold=0.5)` `precision(target=column1, threshold=0.5)`
`recall(target, threshold)`	Recall between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`recall(target=Optional[Any], threshold=Optional[Number])`	`Number`	`recall()` `recall(target=column1)` `recall(threshold=0.5)` `recall(target=column1, threshold=0.5)`
`f1_score(target, threshold)`	F1 score between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`f1_score(target=Optional[Any], threshold=Optional[Number])`	`Number`	`f1_score()` `f1_score(target=column1)` `f1_score(threshold=0.5)` `f1_score(target=column1, threshold=0.5)`
`fpr(target, threshold)`	False positive rate between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`fpr(target=Optional[Any], threshold=Optional[Number])`	`Number`	`fpr()` `fpr(target=column1)` `fpr(threshold=0.5)` `fpr(target=column1, threshold=0.5)`
`auroc(target)`	Area under the ROC curve between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`auroc(target=Optional[Any])`	`Number`	`auroc()` `auroc(target=column1)`
`data_count(target)`	Total count of scored, labeled predictions. Available for binary classification model tasks. Unlike `traffic()`, this counts only rows that have both a prediction and a label. If `target` is specified, it will be used in place of the default target column.	`data_count(target=Optional[Any])`	`Number`	`data_count()` `data_count(target=column1)`
`geometric_mean(target, threshold)`	Geometric mean score between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`geometric_mean(target=Optional[Any], threshold=Optional[Number])`	`Number`	`geometric_mean()` `geometric_mean(target=column1)` `geometric_mean(threshold=0.5)` `geometric_mean(target=column1, threshold=0.5)`
`expected_calibration_error(target)`	Expected calibration error between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`expected_calibration_error(target=Optional[Any])`	`Number`	`expected_calibration_error()` `expected_calibration_error(target=column1)`
`log_loss(target)`	Log loss (binary cross entropy) between target and output. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`log_loss(target=Optional[Any])`	`Number`	`log_loss()` `log_loss(target=column1)`
`calibrated_threshold(target)`	Optimal threshold value for a high TPR and a low FPR. Available for binary classification model tasks. If `target` is specified, it will be used in place of the default target column.	`calibrated_threshold(target=Optional[Any])`	`Number`	`calibrated_threshold()` `calibrated_threshold(target=column1)`
`accuracy(target, threshold)`	Accuracy score between target and outputs. Available for multiclass classification model tasks. If `target` is specified, it will be used in place of the default target column.	`accuracy(target=Optional[Any], threshold=Optional[Number])`	`Number`	`accuracy()` `accuracy(target=column1)` `accuracy(threshold=0.5)` `accuracy(target=column1, threshold=0.5)`
`log_loss(target)`	Log loss score between target and outputs. Available for multiclass classification model tasks. If `target` is specified, it will be used in place of the default target column.	`log_loss(target=Optional[Any])`	`Number`	`log_loss()` `log_loss(target=column1)`
`r2(target)`	R-squared score between target and output. Available for regression model tasks. If `target` is specified, it will be used in place of the default target column.	`r2(target=Optional[Any])`	`Number`	`r2()` `r2(target=column1)`
`mse(target)`	Mean squared error between target and output. Available for regression model tasks. If `target` is specified, it will be used in place of the default target column.	`mse(target=Optional[Any])`	`Number`	`mse()` `mse(target=column1)`
`mae(target)`	Mean absolute error between target and output. Available for regression model tasks. If `target` is specified, it will be used in place of the default target column.	`mae(target=Optional[Any])`	`Number`	`mae()` `mae(target=column1)`
`mape(target)`	Mean absolute percentage error between target and output. Available for regression model tasks. If `target` is specified, it will be used in place of the default target column.	`mape(target=Optional[Any])`	`Number`	`mape()` `mape(target=column1)`
`wmape(target)`	Weighted mean absolute percentage error between target and output. Available for regression model tasks. If `target` is specified, it will be used in place of the default target column.	`wmape(target=Optional[Any])`	`Number`	`wmape()` `wmape(target=column1)`
`map(target)`	Mean average precision score. Available for ranking model tasks. If `target` is specified, it will be used in place of the default target column.	`map(target=Optional[Any])`	`Number`	`map()` `map(target=column1)`
`ndcg_mean(target)`	Mean normalized discounted cumulative gain score. Available for ranking model tasks. If `target` is specified, it will be used in place of the default target column.	`ndcg_mean(target=Optional[Any])`	`Number`	`ndcg_mean()` `ndcg_mean(target=column1)`
`query_count(target)`	Count of ranking queries. Available for ranking model tasks. If `target` is specified, it will be used in place of the default target column.	`query_count(target=Optional[Any])`	`Number`	`query_count()` `query_count(target=column1)`
`gini(actual, predicted)`	Gini coefficient derived from the Lorenz curve. Measures how well a model’s predicted scores rank actual values. Available for ML custom metrics (all model task types, including regression). Both `actual` and `predicted` are required keyword arguments of type `Number`. To compute the normalized Gini coefficient, divide by the ideal Gini: `gini(actual=Target, predicted=Score) / gini(actual=Target, predicted=Target)`. For binary classification, the normalized Gini can also be derived from AUC: `2 * auroc() - 1`.	`gini(actual=Number, predicted=Number)`	`Number`	`gini(actual=Target, predicted=Score)`

Getting Started

Concepts

Evaluate & Test

Reference

Overview

Definitions

FQL Rules

Data Types

Constants

Operators

Constant functions

Row-level functions

Aggregate functions

Built-in metric functions

​Overview

​Definitions

​FQL Rules

​Data Types

​Constants

​Operators

​Constant functions

​Row-level functions

​Aggregate functions

​Built-in metric functions

Overview

Definitions

FQL Rules

Data Types

Constants

Operators

Constant functions

Row-level functions

Aggregate functions

Built-in metric functions