Skip to main content
Evaluator to detect secrets and credentials in text using Fiddler Centor Models. The FTLSecretDetection evaluator scans text for API keys, tokens, and credentials using ~42 known credential formats and Shannon entropy analysis. This is a CPU-only pipeline — deterministic, low-latency, and requiring no GPU. Key Features:
  • Pattern-based detection: ~42 known credential formats across LLM providers, cloud platforms, source control, messaging, and developer tools (entropy-detected secrets are labeled Possible Secret)
  • Entropy analysis: Catches unknown or custom secrets that exceed entropy thresholds
  • Fast: CPU-only, sub-millisecond per token — no inference overhead
Use Cases:
  • Secret leakage detection: Identify credentials in LLM prompts or responses
  • Compliance auditing: Scan text datasets for inadvertently captured credentials
  • Data sanitization: Locate and redact secrets in datasets before training or fine-tuning
Scoring Logic: Unlike probability-based evaluators, FTLSecretDetection returns one Score per detected secret:
  • No secrets detected: Returns an empty list
  • Secrets detected: Returns one Score per detection, with name set to the secret type label and value set to 1.0
To retrieve character-level positions for redaction, use the REST API directly — see Secret Detection tutorial.

​
Parameters

  • text (str) – The text to scan for secrets and credentials.
  • score_name_prefix (str | None)
  • score_fn_kwargs_mapping (ScoreFnKwargsMappingType | None)

​
Returns

A list of Score objects, one per detected secret:
  • name: The secret type label (e.g., "Anthropic API Key", "AWS Access Key ID")
  • evaluator_name: "FTLSecretDetection"
  • value: 1.0 for each detection (binary — present or absent)

​
Raises

ValueError – If the text is empty or None.

​
Example

from fiddler_evals.evaluators import FTLSecretDetection

evaluator = FTLSecretDetection()

# Clean text — no secrets
scores = evaluator.score("What is the weather like today?")
print(f"Secrets found: {len(scores)}")
# Secrets found: 0

# Text containing an API key
scores = evaluator.score(
    "My Anthropic key is sk-ant-api03-abcdefghijklmnopqrstu"
)
for score in scores:
    print(f"Detected: {score.name} (value={score.value})")
# Detected: Anthropic API Key (value=1.0)

# Check whether any secrets were found
has_secrets = len(scores) > 0
secret_types = [score.name for score in scores]
print(f"Secret types found: {secret_types}")
# Secret types found: ['Anthropic API Key']
FTLSecretDetection uses regex patterns and entropy thresholds — not an ML model. This means it has no false-negative rate for known credential formats (pattern match is exact), but may produce occasional false positives on high-entropy non-secret strings (e.g. UUIDs, git hashes, and base64-encoded data are explicitly excluded via allowlist).

​
name = ‘ftl_secret_detection’

​
score()

Scan a text string for secrets and credentials.

​
Parameters

​
text
str
required
The text to scan for secrets and credentials.

​
Returns

A list of Score objects, one per detected secret. Empty list if no secrets found.