Get Started in <10 Minutes
Welcome to Fiddler! Choose your integration path based on what you want to accomplish. Each quick start gets you up and running in 10-20 minutes.
🎯 What Do You Want to Do?
🤖 Monitor AI Agents & Multi-Step Workflows
Best for: Applications using LangGraph, Strands, or custom agentic frameworks
Your AI agents make complex decisions across multiple steps. Monitor the complete workflow from initial reasoning to final response.
Choose your framework:
What you'll monitor:
Agent decision-making and tool selection
Multi-step reasoning chains
LLM calls with inputs/outputs
Tool usage and external API calls
Error propagation and recovery
💬 Monitor Simple LLM Applications
Best for: Single-shot LLM inference, chatbots, simple RAG systems
You're using LLMs for straightforward tasks like Q&A, content generation, or basic chat interfaces.
Quick Start: Simple LLM Monitoring → ⏱️ 15 min
What you'll monitor:
LLM prompts and completions
Token usage and costs
Response latency
Quality metrics (toxicity, PII, sentiment)
Embedding visualizations
📊 Monitor Traditional ML Models
Best for: Scikit-learn, XGBoost, TensorFlow, PyTorch models in production
You have traditional ML models (classification, regression, ranking) deployed and need to track their performance.
Quick Start: Simple ML Monitoring → ⏱️ 15 min
What you'll monitor:
Model performance (accuracy, precision, recall)
Data drift and distribution shifts
Feature importance
Prediction analytics
Custom business metrics
🧪 Evaluate & Test LLM Applications
Best for: Pre-deployment testing, A/B testing, regression testing
You want to systematically evaluate LLM quality before deployment or compare different prompts/models.
Quick Start: LLM Evaluation → ⏱️ 20 min
What you'll evaluate:
Response accuracy and relevance
Semantic similarity
Custom domain-specific metrics
Safety and bias
RAG-specific metrics (faithfulness, context relevance)
🛡️ Add Safety Guardrails
Best for: Protecting LLM applications from harmful content, PII leaks, hallucinations
You need real-time protection to prevent your LLM from generating harmful, sensitive, or incorrect content.
Quick Start: Guardrails → ⏱️ 15 min
What you'll protect against:
Harmful and toxic content
PII leaks (emails, SSNs, credit cards)
Hallucinations and unsupported claims
Jailbreak attempts
Content policy violations
🤔 Not Sure Where to Start?
If you're building with AI agents:
Start with Agentic Monitoring - it covers everything you need for multi-step workflows.
If you're using LLMs for simple tasks:
Start with Simple LLM Monitoring - perfect for chat, Q&A, and content generation.
If you have traditional ML models:
Start with Simple ML Monitoring - track performance and drift for any ML model.
If you want to test before deploying:
Start with LLM Evaluation - build confidence with systematic testing.
If you need to protect your users:
Start with Guardrails - add safety checks in minutes.
🚀 Quick Comparison
📚 After Your Quick Start
Once you've completed a quick start:
Explore the UI - View your dashboards, metrics, and insights
Set Up Alerts - Get notified when issues occur
Customize Metrics - Add domain-specific monitoring
Read Advanced Guides - Deep dive into specific features
Join the Community - Get help and share best practices
💡 Pro Tips
Start Simple: Pick one quick start, complete it fully, then expand
Use Notebooks: Most quick starts have Colab notebooks for hands-on learning
Test Data First: Use sample data before connecting production systems
Monitor + Evaluate: Combine monitoring with evaluation for comprehensive coverage
Layer Guardrails: Add safety checks on both inputs and outputs
Need Help?
Documentation: Browse our complete documentation
Getting Started Guides: Read conceptual overviews for Agentic, LLM, ML, Evaluations, or Guardrails
Support: Contact your Fiddler team or [email protected]
Community: Join our Slack community (ask your Fiddler contact for an invite)
Ready to get started? Pick a quick start above and you'll be monitoring or protecting your AI applications in under 20 minutes! 🚀