Skip to main content
Evaluator to detect secrets and credentials in text using Fiddler Centor Models. The FTLSecretDetection evaluator scans text for API keys, tokens, and credentials using ~42 known credential formats and Shannon entropy analysis. This is a CPU-only pipeline — deterministic, low-latency, and requiring no GPU. Key Features:
  • Pattern-based detection: ~42 known credential formats across LLM providers, cloud platforms, source control, messaging, and developer tools (entropy-detected secrets are labeled Possible Secret)
  • Entropy analysis: Catches unknown or custom secrets that exceed entropy thresholds
  • Fast: CPU-only, sub-millisecond per token — no inference overhead
Use Cases:
  • Secret leakage detection: Identify credentials in LLM prompts or responses
  • Compliance auditing: Scan text datasets for inadvertently captured credentials
  • Data sanitization: Locate and redact secrets in datasets before training or fine-tuning
Scoring Logic: Unlike probability-based evaluators, FTLSecretDetection returns one Score per detected secret:
  • No secrets detected: Returns an empty list
  • Secrets detected: Returns one Score per detection, with name set to the secret type label and value set to 1.0
To retrieve character-level positions for redaction, use the REST API directly — see Secret Detection tutorial.

​
Parameters

  • text (str) – The text to scan for secrets and credentials.
  • score_name_prefix (str | None)
  • score_fn_kwargs_mapping (ScoreFnKwargsMappingType | None)

​
Returns

A list of Score objects, one per detected secret:
  • name: The secret type label (e.g., "Anthropic API Key", "AWS Access Key ID")
  • evaluator_name: "FTLSecretDetection"
  • value: 1.0 for each detection (binary — present or absent)

​
Raises

ValueError – If the text is empty or None.

​
Example

FTLSecretDetection uses regex patterns and entropy thresholds — not an ML model. This means it has no false-negative rate for known credential formats (pattern match is exact), but may produce occasional false positives on high-entropy non-secret strings (e.g. UUIDs, git hashes, and base64-encoded data are explicitly excluded via allowlist).

​
name = ‘ftl_secret_detection’

​
score()

Scan a text string for secrets and credentials.

​
Parameters

​
str
required
The text to scan for secrets and credentials.

​
Returns

A list of Score objects, one per detected secret. Empty list if no secrets found.