AI Security

​

Overview

As GenAI applications handle increasingly sensitive data and interact directly with users, security becomes a pressing concern. A single security breach, whether it’s a successful jailbreak attempt, leaked PII, or harmful content reaching users, can undermine trust and expose organizations to significant risk. This cookbook provides a framework for using Fiddler’s evaluators and guardrails to protect your AI applications from threats, including jailbreak attempts, harmful content generation, PII leakage, and policy violations.

---

​

Understanding AI Security Risks

AI security encompasses multiple threat vectors: Prompt-based attacks:

• Jailbreaking: Attempts to bypass safety restrictions and make the model behave in unintended ways
• Prompt injection: Malicious instructions embedded in user inputs to manipulate model behavior
• Roleplaying exploitation: Using fictional scenarios to elicit restricted information or harmful content

Content safety risks:

• Harmful content generation: Producing content that could cause psychological, physical, or social harm
• Illegal content: Generating content that violates laws or regulations
• Unethical outputs: Responses that violate ethical guidelines or corporate policies

Data privacy risks:

• PII leakage in responses: Model outputs that inadvertently expose personally identifiable information
• PII in prompts: Users submitting sensitive personal data that must be detected and protected

​

Out-of-the-Box Security Evaluators

Fiddler provides pre-built scoring mechanisms called evaluators that assess AI systems across multiple risk dimensions. Learn more: Enrichments

​

Custom LLM-as-a-Judge for Security

While Fiddler’s out-of-the-box evaluators cover common security risks, you may have organization-specific security policies that require custom evaluation. LLM-as-a-Judge evaluators allow you to encode your unique security guidelines into automated checks. Use LLM-as-a-Judge when you need to:

• Enforce company-specific content policies beyond standard safety categories
• Detect violations of industry-specific regulations (healthcare, finance, legal)
• Flag content that conflicts with your organization’s ethical guidelines
• Identify security risks unique to your application domain

Example applications:

• A healthcare AI that must never provide medical diagnoses (even when users request them)
• A financial AI that must refuse to give personalized investment advice
• A customer service AI that must escalate certain sensitive topics to human agents
• An educational AI that must not provide answers to homework assignments

By creating custom prompt templates that define your specific security requirements, you can automatically flag violations and enforce your policies at scale. Learn more: Prompt Specs Quick Start

​

Recommended Security Evaluators

​

1. Prompt Safety

What it detects: Evaluates the safety of text (prompts and responses) across multiple risk dimensions. Additional dimensions available: hateful, harassing, racist, sexist, violent, sexual How to use:

• Apply to both user prompts (inputs) and AI responses (outputs)
• Set severity thresholds based on your risk tolerance
• Track trends over time to identify emerging attack patterns

Value: Acts as a comprehensive safety net, catching both malicious user attempts and problematic model outputs before they cause harm.

​

2. PII Detection

What it detects: Identifies personally identifiable information in both user prompts and AI responses. Where to apply: Input Detection (User Prompts):

• Purpose: Identify when users are submitting sensitive personal information
• Actions:
  • Warn users not to share PII
  • Redact PII before processing
  • Log incidents for security review
• Example: User asks “Can you analyze my credit report? My SSN is 123-45-6789…”

Output Detection (AI Responses):

• Purpose: Catch when the model inadvertently includes PII in responses
• Actions:
  • Block response from reaching the user
  • Regenerate without PII
  • Flag for investigation (How did the model access this PII?)
• Example: Model trained on customer service logs accidentally includes someone’s phone number in a response

Value: Prevents privacy violations, ensures compliance with GDPR/CCPA/HIPAA, and protects users from identity theft.

---

​

Guardrails vs. Post-Production Observability

Fiddler supports two complementary approaches to AI security: real-time guardrails and post-production observability. Understanding when to use each is critical for building secure AI systems.

​

Real-Time Guardrails

What they are: Security checks that evaluate and potentially block AI inputs or outputs before they are processed or delivered to users. How they work:

1. User submits a prompt → Guardrail evaluates for safety/PII
2. If violation detected → Request is blocked or modified
3. If safe → Request proceeds to model
4. Model generates response → Guardrail evaluates output
5. If violation detected → Response is blocked or regenerated
6. If safe → Response delivered to user

When to use real-time guardrails: Tradeoffs:

• Latency: Adds processing time to each request (typically 100-500ms)
• False positives: May occasionally block legitimate requests
• Cost: Requires additional compute for real-time evaluation

Learn more: Guardrails Quick Start

​

Post-Production Observability

What it is: Continuous monitoring and analysis of AI behavior after requests have been processed, using historical data to identify patterns, trends, and emerging threats. How it works:

1. AI processes requests normally (no blocking)
2. All prompts and responses are logged to Fiddler
3. Security evaluators run asynchronously on logged data
4. Dashboards show trends, patterns, and anomalies
5. Alerts trigger when thresholds are exceeded
6. Teams investigate and respond to issues

When to use post-production observability: Tradeoffs:

• No prevention: Issues are detected after they occur
• Requires follow-up: Teams must act on insights
• Best for learning: Ideal for understanding threats and improving defenses

Learn more: Enrichments

​

Using Both Approaches Together

The most secure AI systems combine real-time guardrails with post-production observability: Real-time guardrails provide:

• Immediate protection for users
• Prevention of high-severity incidents
• Compliance with “must prevent” requirements

Post-production observability provides:

• Insights to improve guardrails
• Detection of sophisticated attacks that evade guardrails
• Trend analysis for proactive security

Example workflow:

---

​

How These Evaluators Can Help

​

1. Prevent Security Incidents Before They Occur

Real-time guardrails act as a security perimeter, blocking malicious inputs and harmful outputs before they reach users. This prevents:

• Reputational damage from AI generating offensive content
• Legal liability from privacy violations
• User harm from dangerous or misleading information

​

2. Detect and Respond to Emerging Threats

Post-production observability helps you identify:

• New attack vectors: Novel jailbreak techniques not caught by existing rules
• Systematic weaknesses: Topics or phrasings where the model consistently fails safety checks
• Coordinated attacks: Patterns suggesting organized attempts to compromise your AI

​

3. Maintain Compliance and Auditability

For regulated industries, security monitoring provides:

• Audit trails demonstrating proactive security measures
• Compliance evidence for GDPR, CCPA, HIPAA, and other regulations
• Incident documentation showing how you detected and responded to threats
• Risk assessment data to support security reviews and certifications

​

4. Build Trust with Users

Transparent security practices signal to users that you take their safety and privacy seriously:

• Publish security metrics and response times
• Communicate how you protect user data
• Demonstrate continuous improvement in safety measures

​

5. Optimize Security vs. User Experience

By analyzing false positives in observability dashboards, you can:

• Tune guardrails to reduce unnecessary blocking
• Identify legitimate use cases that trigger safety flags
• Balance security rigor with user experience

---

​

Get Started

Ready to secure your AI applications? Here’s how to begin: For step-by-step tutorials:

• Guardrails setup: Guardrails Quick Start
• Enrichments & observability: Enrichments
• Custom LLM-as-a-Judge: Prompt Specs Quick Start

Security is not a one-time configuration, it’s an ongoing practice. By combining real-time guardrails with continuous observability, you can protect users, maintain compliance, and build AI systems worthy of trust.