Guides
In-depth guides for getting the most out of ThinkHive’s features.
ThinkEval — Evaluation FrameworkCases & FixesRAG EvaluationHallucination DetectionDrift MonitoringShadow TestingMulti-Agent TracingAPI Key ManagementWebhooksPII & ComplianceBilling & CreditsSelf-HostingDeploymentHuman ReviewNondeterminism TestingBusiness Metrics & ROITranscript AnalysisGuardrail Policies
Feature Overview
Quality Evaluation
ThinkHive provides comprehensive quality evaluation for AI agents:
| Feature | Description | Use Case |
|---|---|---|
| ThinkEval | Configurable evaluation suites with graders and judges | Systematic quality measurement |
| RAG Evaluation | Measure retrieval and generation quality | RAG pipelines, knowledge bases |
| Hallucination Detection | Identify fabricated information | Customer-facing agents |
| Drift Monitoring | Track quality changes over time | Production monitoring |
Improvement Workflow
ThinkHive helps you systematically improve your agents:
| Feature | Description | Use Case |
|---|---|---|
| Case Clustering | Group similar failures | Prioritization |
| Fix Generation | AI-suggested improvements | Accelerated fixes |
| Shadow Testing | Validate before deploy | Risk reduction |
Platform & Operations
Manage your ThinkHive integration:
| Feature | Description | Use Case |
|---|---|---|
| API Keys | Key creation, rotation, and security | Secure integrations |
| Webhooks | Real-time event notifications | Automation, alerting |
| PII & Compliance | PII redaction, GDPR, HIPAA | Privacy and regulatory compliance |
| Billing & Credits | Usage tracking and credit packs | Cost management |
Guardrails & Safety
Protect your AI agents in real time:
| Feature | Description | Use Case |
|---|---|---|
| Guardrail Policies | Configurable scanning policies | Content safety |
| Human Review | Review queue with calibration | Quality assurance |
| Nondeterminism Testing | Pass@k reliability analysis | Eval confidence |
| Transcript Analysis | Pattern detection in conversations | Risk monitoring |
Business Intelligence
Connect AI performance to business outcomes:
| Feature | Description | Use Case |
|---|---|---|
| Business Metrics & ROI | Revenue impact and correlation | Justifying AI investment |
| ROI Analytics | Per-agent ROI breakdown | Cost optimization |
| Customer Context | Link traces to customer data | VIP monitoring |
| Calibration | Confidence accuracy | Risk assessment |
Recommended Learning Path
-
Start with Basics
-
Integrate Your Agent
-
Evaluate Quality
-
Investigate & Fix
-
Guard & Review
-
Monitor Production
-
Business & Compliance