🎯 What Do You Want to Do?
🤖 Monitor AI Agents & Multi-Step Workflows
Best for: Applications using LangGraph, Strands, or custom agentic frameworks Your AI agents make complex decisions across multiple steps. Monitor the complete workflow from initial reasoning to final response. Choose your framework:| Framework | Integration | Time | Quick Start |
|---|---|---|---|
| LangGraph / LangChain | Auto-instrumentation | 10 min | LangGraph SDK → |
| Strands Agents | Native integration | 10 min | Strands Agents SDK → |
| Custom / Other | OpenTelemetry | 15 min | OpenTelemetry → |
- Agent decision-making and tool selection
- Multi-step reasoning chains
- LLM calls with inputs/outputs
- Tool usage and external API calls
- Error propagation and recovery
💬 Monitor Simple LLM Applications
Best for: Single-shot LLM inference, chatbots, simple RAG systems You’re using LLMs for straightforward tasks like Q&A, content generation, or basic chat interfaces. Quick Start: Simple LLM Monitoring → ⏱️ 10 min What you’ll monitor:- LLM prompts and completions
- Token usage and costs
- Response latency
- Quality metrics (toxicity, PII, sentiment)
- Embedding visualizations
📊 Monitor Traditional ML Models
Best for: Scikit-learn, XGBoost, TensorFlow, PyTorch models in production You have traditional ML models (classification, regression, ranking) deployed and need to track their performance. Quick Start: Simple ML Monitoring → ⏱️ 10 min What you’ll monitor:- Model performance (accuracy, precision, recall)
- Data drift and distribution shifts
- Feature importance
- Prediction analytics
- Custom business metrics
🧪 Evaluate & Test LLM Applications
Best for: Pre-deployment testing, A/B testing, regression testing You want to systematically evaluate LLM quality before deployment or compare different prompts/models. Quick Start: Experiments → ⏱️ 15 min What you’ll evaluate:- Response accuracy and relevance
- Semantic similarity
- Custom domain-specific metrics
- Safety and bias
- RAG-specific metrics (faithfulness, context relevance)
🛡️ Add Safety Guardrails
Best for: Protecting LLM applications from harmful content, PII leaks, and hallucinations You need real-time protection to prevent your LLM from generating harmful, sensitive, or incorrect content. Quick Start: Guardrails → ⏱️ 10 min What you’ll protect against:- Harmful and toxic content
- PII leaks (emails, SSNs, credit cards)
- Hallucinations and unsupported claims
- Jailbreak attempts
- Content policy violations
🤔 Not Sure Where to Start?
If you’re building with AI agents:
Start with Agentic Observability - it covers everything you need for multi-step workflows.If you’re using LLMs for simple tasks:
Start with Simple LLM Monitoring - perfect for chat, Q&A, and content generation.If you have traditional ML models:
Start with Simple ML Monitoring - track performance and drift for any ML model.If you want to test before deploying:
Start with Experiments - build confidence with systematic testing.If you need to protect your users:
Start with Guardrails - add safety checks in minutes.🚀 Quick Comparison
| Use Case | Monitoring Type | Best Quick Start | Time |
|---|---|---|---|
| Multi-agent systems, complex workflows | Agentic | LangGraph / Strands / OTeL | 10 min |
| Simple chatbots, Q&A, content generation | LLM | Simple LLM Monitoring | 15 min |
| Classification, regression, ranking models | ML | Simple ML Monitoring | 10 min |
| Pre-deployment testing, A/B testing | Experiments | Experiments | 15 min |
| Safety, PII protection, hallucination prevention | Guardrails | Guardrails | 10 min |
📚 After Your Quick Start
Once you’ve completed a quick start:- Explore the UI - View your dashboards, metrics, and insights
- Set Up Alerts - Get notified when issues occur
- Customize Metrics - Add domain-specific monitoring
- Read Advanced Guides - Deep dive into specific features
- Join the Community - Get help and share best practices
💡 Pro Tips
- Start Simple: Pick one quick start, complete it fully, then expand
- Use Notebooks: Most quick starts have Colab notebooks for hands-on learning
- Test Data First: Use sample data before connecting production systems
- Monitor + Evaluate: Combine monitoring with evaluation for comprehensive coverage
- Layer Guardrails: Add safety checks on both inputs and outputs
Need Help?
- Documentation: Browse our complete documentation
- Getting Started Guides: Read conceptual overviews for Agentic, LLM, ML, Experiments, or Guardrails
- Support: Contact your Fiddler team or <support@fiddler.ai>