Explainability & Analysis

ThinkHive provides a multi-layered analysis stack for understanding AI agent behavior. From lightweight rule-based checks to deep LLM-powered investigations, you can choose the right level of analysis for each use case.

API	Version	Purpose
Explainer	v1	Core trace analysis, failure clustering, counterfactuals
Analyzer	v2	Cost-aware analysis with estimation, batch processing
Deep Dive	v2	Multi-step root cause investigations
Error Analysis	v2	Error categorization and trend detection
Transcript Patterns	v1	Conversation pattern mining and risk assessment

Analysis Tiers

Some analysis endpoints, such as Explainer and Analyzer, expose a tier setting that controls analysis depth and credit cost. Deep Dive, Error Analysis, and Transcript Patterns use endpoint-specific options instead.

Tier	Credits	Latency	Features
`rule_based`	0 (free)	~100ms	Pattern matching, regex checks, basic metrics
`fast_llm`	0.5	~2s	Quick LLM pass for quality scoring and summarization
`full_llm`	1	~5s	Comprehensive analysis with RAG evaluation, hallucination detection
`deep`	2	~15s	Full analysis plus counterfactuals, root cause, and business impact

Credits: Analysis consumes credits based on the selected tier. Check your balance at GET /api/credits/balance. Rule-based analysis is always free and does not require credits.

Explainer API (v1)

The Explainer API provides core trace analysis capabilities including quality scoring, RAG evaluation, hallucination detection, failure clustering, and counterfactual reasoning.

Analyze Traces

POST /api/v1/explainer/analyze

Analyze one or more traces with configurable depth and evaluation modules.

Request Body:

{
  "traces": [
    {
      "traceId": "tr_abc123",
      "userMessage": "What is the refund policy for enterprise plans?",
      "agentResponse": "Enterprise plans have a 30-day refund policy. Contact your account manager to initiate a refund request within the first 30 days of your billing cycle.",
      "outcome": "success",
      "retrievedDocuments": [
        {
          "content": "Enterprise Refund Policy: All enterprise subscriptions are eligible for a full refund within 30 calendar days of the billing date. Refund requests must be submitted through your designated account manager.",
          "source": "knowledge-base/billing/enterprise.md",
          "score": 0.94
        }
      ],
      "metadata": {
        "model": "gpt-4o",
        "agentId": "agent_support_01",
        "customerId": "cust_ent_500",
        "sessionId": "sess_82jf3k"
      }
    }
  ],
  "options": {
    "tier": "full_llm",
    "includeRagEvaluation": true,
    "includeHallucinationDetection": true,
    "includeBusinessImpact": true,
    "includeCounterfactuals": false
  }
}

Request Fields:

Field	Type	Required	Description
`traces`	array	Yes	Array of trace objects to analyze
`traces[].traceId`	string	No	Existing trace ID; if omitted, a new ID is generated
`traces[].userMessage`	string	Yes	The user’s input message
`traces[].agentResponse`	string	Yes	The agent’s response
`traces[].outcome`	string	No	`success`, `failure`, or `unknown`
`traces[].retrievedDocuments`	array	No	RAG context documents for grounding evaluation
`traces[].metadata`	object	No	Arbitrary metadata (model, customer, session, etc.)
`options.tier`	string	No	Analysis tier: `rule_based`, `fast_llm`, `full_llm`, `deep` (default: `full_llm`)
`options.includeRagEvaluation`	boolean	No	Enable RAG grounding metrics (default: `true`)
`options.includeHallucinationDetection`	boolean	No	Enable hallucination scanning (default: `true`)
`options.includeBusinessImpact`	boolean	No	Enable business impact scoring (default: `false`)
`options.includeCounterfactuals`	boolean	No	Enable counterfactual analysis (default: `false`, requires `deep` tier)

Response:

{
  "success": true,
  "data": {
    "results": [
      {
        "traceId": "tr_abc123",
        "analysis": {
          "overallScore": 0.91,
          "outcome": "success",
          "ragEvaluation": {
            "groundedness": 0.95,
            "faithfulness": 0.92,
            "citationAccuracy": 0.88,
            "contextRelevance": 0.93,
            "answerRelevance": 0.90
          },
          "hallucinationReport": {
            "detected": false,
            "hallucinations": [],
            "confidence": 0.96
          },
          "businessImpact": {
            "severity": "low",
            "revenueRisk": 0,
            "customerSatisfactionImpact": 0.05,
            "category": "billing_inquiry"
          },
          "insights": [
            "Response is well-grounded in the enterprise refund policy document",
            "Correctly identifies the 30-day refund window and account manager process",
            "Consider adding a direct link to the refund request form"
          ]
        },
        "creditsUsed": 1
      }
    ],
    "summary": {
      "totalAnalyzed": 1,
      "avgScore": 0.91,
      "totalCreditsUsed": 1
    }
  }
}

Get Stored Explanation

GET /api/v1/explainer/trace/:traceId

Retrieve a previously generated explanation for a specific trace.

Path Parameters:

Parameter	Type	Description
`traceId`	string	The trace ID to retrieve

Response:

{
  "success": true,
  "data": {
    "traceId": "tr_abc123",
    "explanation": {
      "summary": "User inquired about enterprise refund policy. Agent provided accurate information grounded in the knowledge base, correctly citing the 30-day window and account manager requirement.",
      "qualityScore": 0.91,
      "tier": "full_llm",
      "ragMetrics": {
        "groundedness": 0.95,
        "faithfulness": 0.92,
        "citationAccuracy": 0.88,
        "contextRelevance": 0.93
      },
      "hallucinationReport": {
        "detected": false,
        "hallucinations": [],
        "confidence": 0.96
      },
      "generatedAt": "2025-03-10T14:30:00Z",
      "expiresAt": "2025-06-10T14:30:00Z"
    }
  }
}

List Explanations

GET /api/v1/explainer/list

List stored explanations with filtering and pagination.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`outcome`	string	—	Filter by outcome: `success`, `failure`, `unknown`
`minScore`	number	—	Minimum quality score (0.0 - 1.0)
`maxScore`	number	—	Maximum quality score (0.0 - 1.0)
`startDate`	string	—	ISO 8601 start date
`endDate`	string	—	ISO 8601 end date
`limit`	number	20	Items per page (max: 100)
`offset`	number	0	Pagination offset

Response:

{
  "success": true,
  "data": [
    {
      "traceId": "tr_abc123",
      "agentId": "agent_support_01",
      "outcome": "success",
      "qualityScore": 0.91,
      "tier": "full_llm",
      "userMessage": "What is the refund policy for enterprise plans?",
      "agentResponse": "Enterprise plans have a 30-day refund policy...",
      "hallucinationDetected": false,
      "createdAt": "2025-03-10T14:30:00Z"
    },
    {
      "traceId": "tr_def456",
      "agentId": "agent_support_01",
      "outcome": "failure",
      "qualityScore": 0.35,
      "tier": "full_llm",
      "userMessage": "Can I upgrade mid-cycle?",
      "agentResponse": "Yes, upgrades are prorated automatically...",
      "hallucinationDetected": true,
      "createdAt": "2025-03-10T13:15:00Z"
    }
  ],
  "pagination": {
    "total": 1284,
    "limit": 20,
    "offset": 0,
    "hasMore": true
  }
}

Failure Clusters

GET /api/v1/explainer/clusters/:agentId

Get automatically clustered failure patterns for an agent. Clusters group similar failures together to surface systemic issues.

Path Parameters:

Parameter	Type	Description
`agentId`	string	The agent ID to get clusters for

Query Parameters:

Parameter	Type	Default	Description
`startDate`	string	30 days ago	ISO 8601 start date
`endDate`	string	now	ISO 8601 end date
`minClusterSize`	number	3	Minimum traces per cluster
`status`	string	—	Filter by status: `open`, `resolved`, `investigating`

Response:

{
  "success": true,
  "data": {
    "clusters": [
      {
        "id": "cluster_auth_001",
        "name": "Authentication Guidance Errors",
        "description": "Agent provides outdated MFA setup instructions, missing the new authenticator app requirement introduced in v3.2",
        "traceCount": 47,
        "severity": "high",
        "status": "open",
        "firstSeen": "2025-03-01T08:00:00Z",
        "lastSeen": "2025-03-10T12:30:00Z",
        "commonPatterns": [
          "Missing authenticator app step in MFA setup",
          "References deprecated SMS-based verification",
          "Incorrect support contact for security issues"
        ],
        "sampleTraceIds": ["tr_f01", "tr_f02", "tr_f03"],
        "impactEstimate": {
          "affectedCustomers": 34,
          "estimatedRevenueLoss": 2400,
          "issueId": "issue_auth_001"
        }
      },
      {
        "id": "cluster_billing_002",
        "name": "Proration Calculation Errors",
        "description": "Agent hallucinates proration amounts when customers ask about mid-cycle plan changes",
        "traceCount": 23,
        "severity": "critical",
        "status": "investigating",
        "firstSeen": "2025-03-05T10:00:00Z",
        "lastSeen": "2025-03-10T11:45:00Z",
        "commonPatterns": [
          "Incorrect daily rate calculation",
          "Missing enterprise discount in proration"
        ],
        "sampleTraceIds": ["tr_f10", "tr_f11", "tr_f12"],
        "impactEstimate": {
          "affectedCustomers": 18,
          "estimatedRevenueLoss": 8500,
          "issueId": "issue_billing_002"
        }
      }
    ],
    "totalClusters": 5,
    "totalFailures": 150
  }
}

Cluster Details

GET /api/v1/explainer/clusters/:agentId/:clusterId

Get detailed information about a specific failure cluster, including all associated traces and suggested remediation.

Path Parameters:

Parameter	Type	Description
`agentId`	string	The agent ID
`clusterId`	string	The cluster ID

Response:

{
  "success": true,
  "data": {
    "cluster": {
      "id": "cluster_auth_001",
      "name": "Authentication Guidance Errors",
      "description": "Agent provides outdated MFA setup instructions, missing the new authenticator app requirement introduced in v3.2",
      "status": "open",
      "severity": "high",
      "traceCount": 47,
      "rootCause": "Knowledge base contains outdated MFA documentation from v3.0. The v3.2 update added authenticator app requirement but the corresponding KB article was not updated.",
      "suggestedFix": {
        "type": "knowledge_update",
        "description": "Update the MFA setup guide (knowledge-base/security/mfa-setup.md) to include the authenticator app requirement from v3.2. Remove references to SMS-based verification which is now deprecated.",
        "priority": "high",
        "estimatedImpact": "Would resolve ~47 failures per week",
        "affectedDocuments": [
          "knowledge-base/security/mfa-setup.md",
          "knowledge-base/security/2fa-overview.md"
        ]
      },
      "traces": [
        {
          "id": "tr_f01",
          "userMessage": "How do I set up two-factor authentication?",
          "agentResponse": "To enable 2FA, go to Settings > Security and click 'Enable SMS Verification'...",
          "failureReason": "Recommended deprecated SMS verification instead of authenticator app",
          "qualityScore": 0.25,
          "timestamp": "2025-03-10T12:30:00Z"
        },
        {
          "id": "tr_f02",
          "userMessage": "I need to add MFA to my account",
          "agentResponse": "You can enable MFA by navigating to your security settings and entering your phone number...",
          "failureReason": "Missing authenticator app setup step, directed to phone-based flow",
          "qualityScore": 0.30,
          "timestamp": "2025-03-10T11:15:00Z"
        }
      ]
    }
  }
}

Batch Analysis

POST /api/v1/explainer/generate-all/:agentId

Trigger analysis for all unanalyzed traces belonging to an agent. Runs asynchronously and returns a summary of the batch job.

Path Parameters:

Parameter	Type	Description
`agentId`	string	The agent ID to batch analyze

Query Parameters:

Parameter	Type	Default	Description
`limit`	number	50	Maximum traces to analyze (max: 500)
`tier`	string	`fast_llm`	Analysis tier to use
`outcomeFilter`	string	—	Only analyze traces with this outcome

Response:

{
  "success": true,
  "data": {
    "jobId": "batch_expl_001",
    "status": "processing",
    "queued": 45,
    "skipped": 5,
    "estimatedCredits": 22.5,
    "estimatedDuration": "~90s",
    "remainingCredits": 977.5
  }
}

⚠️

Batch analysis runs in the background. Use the jobId to poll for completion. Large batches may take several minutes depending on the selected tier.

Counterfactual Analysis

GET /api/v1/explainer/counterfactual/:traceId

Generate counterfactual explanations for a trace, answering “what would need to change for this trace to have a different outcome?”

Path Parameters:

Parameter	Type	Description
`traceId`	string	The trace ID to analyze

Response:

{
  "success": true,
  "data": {
    "traceId": "tr_f01",
    "originalOutcome": "failure",
    "originalScore": 0.25,
    "counterfactuals": [
      {
        "id": "cf_001",
        "change": "Update knowledge base to include authenticator app setup instructions from v3.2",
        "expectedOutcome": "success",
        "expectedScore": 0.90,
        "confidence": 0.88,
        "impact": "Agent would correctly guide users through the authenticator app flow instead of the deprecated SMS path",
        "changeType": "knowledge_update",
        "effort": "low"
      },
      {
        "id": "cf_002",
        "change": "Add a guardrail to detect references to deprecated SMS verification and block the response",
        "expectedOutcome": "partial_success",
        "expectedScore": 0.70,
        "confidence": 0.75,
        "impact": "Would prevent incorrect information but may result in a less helpful response until KB is updated",
        "changeType": "guardrail_addition",
        "effort": "medium"
      }
    ],
    "creditsUsed": 2
  }
}

Search Explanations

POST /api/v1/explainer/search

Semantic search across stored explanations using natural language queries.

Request Body:

{
  "query": "hallucinations in billing responses",
  "agentId": "agent_support_01",
  "filters": {
    "outcome": "failure",
    "minScore": 0.0,
    "maxScore": 0.5,
    "startDate": "2025-03-01T00:00:00Z",
    "endDate": "2025-03-10T23:59:59Z"
  },
  "limit": 10
}

Request Fields:

Field	Type	Required	Description
`query`	string	Yes	Natural language search query
`agentId`	string	No	Filter by agent ID
`filters.outcome`	string	No	Filter by outcome
`filters.minScore`	number	No	Minimum quality score
`filters.maxScore`	number	No	Maximum quality score
`filters.startDate`	string	No	ISO 8601 start date
`filters.endDate`	string	No	ISO 8601 end date
`limit`	number	No	Max results (default: 10, max: 50)

Response:

{
  "success": true,
  "data": {
    "results": [
      {
        "traceId": "tr_def456",
        "relevanceScore": 0.93,
        "qualityScore": 0.35,
        "outcome": "failure",
        "userMessage": "Can I upgrade mid-cycle?",
        "agentResponse": "Yes, upgrades are prorated automatically at a daily rate...",
        "matchReason": "Hallucination detected in billing proration calculation",
        "hallucinationReport": {
          "detected": true,
          "hallucinations": [
            {
              "claim": "prorated at a daily rate of 1/30th",
              "source": "none",
              "severity": "high",
              "explanation": "No source document specifies daily rate calculation method"
            }
          ]
        },
        "createdAt": "2025-03-10T13:15:00Z"
      }
    ],
    "totalResults": 23,
    "searchCreditsUsed": 0
  }
}

Search is powered by vector similarity and does not consume analysis credits.

Error Signatures

GET /api/v1/explainer/signatures

Retrieve unique error signatures across all analyzed traces. Signatures represent distinct failure patterns that have been deduplicated and fingerprinted.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`severity`	string	—	Filter by severity: `low`, `medium`, `high`, `critical`
`startDate`	string	30 days ago	ISO 8601 start date
`endDate`	string	now	ISO 8601 end date
`limit`	number	20	Items per page (max: 100)
`offset`	number	0	Pagination offset

Response:

{
  "success": true,
  "data": {
    "signatures": [
      {
        "id": "sig_mfa_001",
        "fingerprint": "a3f8c2e1b4d5",
        "name": "Deprecated SMS verification reference",
        "description": "Agent references SMS-based 2FA which was deprecated in v3.2",
        "severity": "high",
        "occurrences": 47,
        "firstSeen": "2025-03-01T08:00:00Z",
        "lastSeen": "2025-03-10T12:30:00Z",
        "affectedAgents": ["agent_support_01", "agent_onboarding_02"],
        "clusterId": "cluster_auth_001",
        "sampleTraceId": "tr_f01"
      },
      {
        "id": "sig_prorate_002",
        "fingerprint": "b7d9e3f2a1c6",
        "name": "Hallucinated proration amounts",
        "description": "Agent fabricates proration calculations not supported by any source document",
        "severity": "critical",
        "occurrences": 23,
        "firstSeen": "2025-03-05T10:00:00Z",
        "lastSeen": "2025-03-10T11:45:00Z",
        "affectedAgents": ["agent_support_01"],
        "clusterId": "cluster_billing_002",
        "sampleTraceId": "tr_f10"
      }
    ],
    "totalSignatures": 12,
    "pagination": {
      "limit": 20,
      "offset": 0,
      "hasMore": false
    }
  }
}

Analyzer API (v2)

The v2 Analyzer API adds cost estimation, structured result storage, and improved batch processing over the v1 Explainer.

Analyze with Cost Estimation

POST /api/v2/analyzer/analyze

Analyze traces with upfront cost estimation. The response includes the estimated credit cost before analysis begins, allowing you to confirm or abort.

Request Body:

{
  "traces": [
    {
      "traceId": "tr_xyz789",
      "userMessage": "How do I configure SSO with Okta?",
      "agentResponse": "To configure SSO with Okta, navigate to Admin > Identity Providers > Add Provider. Select 'Okta' from the list and enter your Okta domain URL...",
      "outcome": "success",
      "retrievedDocuments": [
        {
          "content": "SSO Configuration Guide: Navigate to Admin > Identity Providers...",
          "source": "docs/sso/okta-integration.md",
          "score": 0.91
        }
      ]
    }
  ],
  "config": {
    "tier": "full_llm",
    "modules": ["rag_evaluation", "hallucination_detection", "business_impact"],
    "dryRun": false
  }
}

Config Fields:

Field	Type	Required	Description
`tier`	string	No	Analysis tier (default: `full_llm`)
`modules`	array	No	Specific modules to enable: `rag_evaluation`, `hallucination_detection`, `business_impact`, `counterfactuals`
`dryRun`	boolean	No	If `true`, returns cost estimate without running analysis (default: `false`)

{
  "success": true,
  "data": {
    "id": "analysis_v2_001",
    "status": "completed",
    "costEstimate": {
      "totalCredits": 1,
      "breakdown": {
        "baseAnalysis": 0.5,
        "ragEvaluation": 0.2,
        "hallucinationDetection": 0.2,
        "businessImpact": 0.1
      }
    },
    "results": [
      {
        "traceId": "tr_xyz789",
        "overallScore": 0.88,
        "modules": {
          "ragEvaluation": {
            "groundedness": 0.92,
            "faithfulness": 0.90,
            "contextRelevance": 0.85,
            "answerRelevance": 0.87
          },
          "hallucinationDetection": {
            "detected": false,
            "confidence": 0.94,
            "hallucinations": []
          },
          "businessImpact": {
            "severity": "low",
            "category": "technical_support",
            "revenueRisk": 0,
            "customerSatisfactionImpact": 0.02
          }
        },
        "insights": [
          "Response accurately follows the SSO configuration guide",
          "Missing mention of required admin permissions for SSO setup"
        ]
      }
    ],
    "creditsUsed": 1,
    "remainingCredits": 976.5
  }
}

Get Analysis Results

GET /api/v2/analyzer/results/:id

Retrieve results of a previously completed analysis.

Path Parameters:

Parameter	Type	Description
`id`	string	The analysis ID returned from the analyze endpoint

Response:

{
  "success": true,
  "data": {
    "id": "analysis_v2_001",
    "status": "completed",
    "tier": "full_llm",
    "traceCount": 1,
    "createdAt": "2025-03-10T15:00:00Z",
    "completedAt": "2025-03-10T15:00:05Z",
    "results": [
      {
        "traceId": "tr_xyz789",
        "overallScore": 0.88,
        "modules": {
          "ragEvaluation": {
            "groundedness": 0.92,
            "faithfulness": 0.90,
            "contextRelevance": 0.85,
            "answerRelevance": 0.87
          },
          "hallucinationDetection": {
            "detected": false,
            "confidence": 0.94
          },
          "businessImpact": {
            "severity": "low",
            "category": "technical_support"
          }
        }
      }
    ],
    "creditsUsed": 1
  }
}

Analysis History

GET /api/v2/analyzer/history

List past analysis runs with summary statistics.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`tier`	string	—	Filter by analysis tier
`status`	string	—	Filter by status: `completed`, `failed`, `processing`
`startDate`	string	30 days ago	ISO 8601 start date
`endDate`	string	now	ISO 8601 end date
`limit`	number	20	Items per page (max: 100)
`offset`	number	0	Pagination offset

Response:

{
  "success": true,
  "data": {
    "analyses": [
      {
        "id": "analysis_v2_001",
        "tier": "full_llm",
        "status": "completed",
        "traceCount": 1,
        "avgScore": 0.88,
        "creditsUsed": 1,
        "createdAt": "2025-03-10T15:00:00Z",
        "completedAt": "2025-03-10T15:00:05Z"
      },
      {
        "id": "analysis_v2_002",
        "tier": "deep",
        "status": "completed",
        "traceCount": 10,
        "avgScore": 0.62,
        "creditsUsed": 20,
        "createdAt": "2025-03-10T14:00:00Z",
        "completedAt": "2025-03-10T14:02:30Z"
      }
    ],
    "pagination": {
      "total": 84,
      "limit": 20,
      "offset": 0,
      "hasMore": true
    },
    "summary": {
      "totalAnalyses": 84,
      "totalCreditsUsed": 156.5,
      "avgScore": 0.74
    }
  }
}

Batch Analysis

POST /api/v2/analyzer/batch

Submit a batch of traces for background analysis. Returns immediately with a batch ID for polling.

Request Body:

{
  "agentId": "agent_support_01",
  "traceIds": [
    "tr_001", "tr_002", "tr_003", "tr_004", "tr_005",
    "tr_006", "tr_007", "tr_008", "tr_009", "tr_010"
  ],
  "config": {
    "tier": "fast_llm",
    "modules": ["rag_evaluation", "hallucination_detection"],
    "priority": "normal",
    "callbackUrl": "https://your-app.com/webhooks/analysis-complete"
  }
}

Request Fields:

Field	Type	Required	Description
`agentId`	string	Yes	Agent these traces belong to
`traceIds`	array	Yes	Array of trace IDs to analyze (max: 500)
`config.tier`	string	No	Analysis tier (default: `fast_llm`)
`config.modules`	array	No	Analysis modules to run
`config.priority`	string	No	`low`, `normal`, or `high` (default: `normal`)
`config.callbackUrl`	string	No	Webhook URL to notify on completion

Response:

{
  "success": true,
  "data": {
    "batchId": "batch_v2_001",
    "status": "queued",
    "totalTraces": 10,
    "estimatedCredits": 5.0,
    "estimatedDuration": "~20s",
    "progress": {
      "completed": 0,
      "failed": 0,
      "pending": 10
    },
    "createdAt": "2025-03-10T15:30:00Z",
    "pollUrl": "/api/v2/analyzer/results/batch_v2_001"
  }
}

When a callbackUrl is provided, ThinkHive sends a POST request to that URL when the batch completes. The payload includes the batchId, status, and summary statistics. See Webhooks for payload format.

Deep Dive API

The Deep Dive API performs multi-step, investigative analysis on traces and failure patterns. It goes beyond single-pass analysis to recursively explore root causes, test hypotheses, and produce actionable remediation plans.

Start Investigation

POST /api/v2/deep-dive/investigate

Launch a deep investigation into a trace, cluster, or error pattern.

Request Body:

{
  "target": {
    "type": "trace",
    "id": "tr_f01"
  },
  "scope": {
    "includeRelatedTraces": true,
    "maxRelatedTraces": 20,
    "timeWindow": "7d",
    "includeClusterContext": true
  },
  "analysis": {
    "rootCauseDepth": 3,
    "generateHypotheses": true,
    "testHypotheses": true,
    "generateRemediationPlan": true
  }
}

Request Fields:

Field	Type	Required	Description
`target.type`	string	Yes	Investigation target: `trace`, `cluster`, or `signature`
`target.id`	string	Yes	ID of the trace, cluster, or signature to investigate
`scope.includeRelatedTraces`	boolean	No	Pull in similar traces for context (default: `true`)
`scope.maxRelatedTraces`	number	No	Max related traces to include (default: 20, max: 100)
`scope.timeWindow`	string	No	Time window for related traces: `1d`, `7d`, `30d` (default: `7d`)
`scope.includeClusterContext`	boolean	No	Include parent cluster data if available (default: `true`)
`analysis.rootCauseDepth`	number	No	Levels of causal analysis to perform (default: 3, max: 5)
`analysis.generateHypotheses`	boolean	No	Generate failure hypotheses (default: `true`)
`analysis.testHypotheses`	boolean	No	Test hypotheses against related traces (default: `true`)
`analysis.generateRemediationPlan`	boolean	No	Produce actionable fix plan (default: `true`)

Response:

{
  "success": true,
  "data": {
    "investigationId": "inv_001",
    "status": "completed",
    "target": {
      "type": "trace",
      "id": "tr_f01"
    },
    "relatedTracesAnalyzed": 15,
    "rootCauseAnalysis": {
      "primaryCause": "Outdated knowledge base article for MFA setup",
      "causalChain": [
        {
          "level": 1,
          "cause": "Agent retrieved outdated MFA documentation (v3.0)",
          "evidence": "Retrieved document references SMS verification, deprecated since v3.2",
          "confidence": 0.92
        },
        {
          "level": 2,
          "cause": "Knowledge base update process did not flag security-related articles for review after v3.2 release",
          "evidence": "Article last modified 2025-01-10, v3.2 released 2025-02-15",
          "confidence": 0.85
        },
        {
          "level": 3,
          "cause": "No automated staleness detection for security-critical documentation",
          "evidence": "No content freshness checks configured for the security/ document path",
          "confidence": 0.72
        }
      ]
    },
    "hypotheses": [
      {
        "id": "hyp_001",
        "statement": "The MFA failure pattern is caused by a stale knowledge base article",
        "tested": true,
        "result": "confirmed",
        "evidence": "12 of 15 related traces retrieved the same outdated document",
        "confidence": 0.91
      },
      {
        "id": "hyp_002",
        "statement": "The agent's system prompt lacks instructions to verify document recency",
        "tested": true,
        "result": "partially_confirmed",
        "evidence": "System prompt mentions 'use the most recent information' but does not instruct the agent to check document dates",
        "confidence": 0.68
      }
    ],
    "remediationPlan": {
      "priority": "high",
      "steps": [
        {
          "order": 1,
          "action": "Update knowledge-base/security/mfa-setup.md with v3.2 authenticator app instructions",
          "effort": "low",
          "estimatedImpact": "Resolves ~47 failures/week",
          "owner": "documentation_team"
        },
        {
          "order": 2,
          "action": "Add document freshness guardrail for security/ path with 30-day staleness threshold",
          "effort": "medium",
          "estimatedImpact": "Prevents future stale documentation issues in security articles",
          "owner": "platform_team"
        },
        {
          "order": 3,
          "action": "Add system prompt instruction to cross-reference document modification dates for security topics",
          "effort": "low",
          "estimatedImpact": "Provides defense-in-depth against stale content",
          "owner": "agent_team"
        }
      ]
    },
    "creditsUsed": 8,
    "duration": "12.4s",
    "completedAt": "2025-03-10T16:00:12Z"
  }
}

Get Investigation Results

GET /api/v2/deep-dive/results/:id

Retrieve results of a completed or in-progress investigation.

Path Parameters:

Parameter	Type	Description
`id`	string	The investigation ID

Response:

{
  "success": true,
  "data": {
    "investigationId": "inv_001",
    "status": "completed",
    "target": {
      "type": "trace",
      "id": "tr_f01"
    },
    "rootCauseAnalysis": {
      "primaryCause": "Outdated knowledge base article for MFA setup",
      "causalChain": [
        {
          "level": 1,
          "cause": "Agent retrieved outdated MFA documentation (v3.0)",
          "confidence": 0.92
        }
      ]
    },
    "hypotheses": [
      {
        "id": "hyp_001",
        "statement": "The MFA failure pattern is caused by a stale knowledge base article",
        "result": "confirmed",
        "confidence": 0.91
      }
    ],
    "remediationPlan": {
      "priority": "high",
      "steps": [
        {
          "order": 1,
          "action": "Update knowledge-base/security/mfa-setup.md with v3.2 authenticator app instructions",
          "effort": "low"
        }
      ]
    },
    "creditsUsed": 8,
    "completedAt": "2025-03-10T16:00:12Z"
  }
}

Get Suggestions

GET /api/v2/deep-dive/suggestions

Get AI-generated suggestions for traces or clusters worth investigating, based on recent failure patterns and anomaly detection.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter suggestions for a specific agent
`limit`	number	5	Max suggestions to return (max: 20)
`minConfidence`	number	0.7	Minimum confidence threshold for suggestions

Response:

{
  "success": true,
  "data": {
    "suggestions": [
      {
        "id": "sug_001",
        "type": "cluster",
        "targetId": "cluster_billing_002",
        "title": "Investigate proration calculation errors",
        "reason": "23 failures in the last 5 days with increasing frequency. Severity is critical and no investigation has been started.",
        "confidence": 0.94,
        "estimatedImpact": {
          "affectedTraces": 23,
          "estimatedRevenueLoss": 8500,
          "trend": "increasing"
        },
        "suggestedTier": "deep",
        "estimatedCredits": 10
      },
      {
        "id": "sug_002",
        "type": "signature",
        "targetId": "sig_timeout_003",
        "title": "New error pattern: API timeout in order lookup",
        "reason": "New signature detected 2 hours ago with 8 occurrences. May indicate an upstream service degradation.",
        "confidence": 0.82,
        "estimatedImpact": {
          "affectedTraces": 8,
          "estimatedRevenueLoss": 0,
          "trend": "new"
        },
        "suggestedTier": "full_llm",
        "estimatedCredits": 4
      }
    ]
  }
}

Error Analysis API

The Error Analysis API categorizes errors across your agents, tracks trends over time, and surfaces emerging patterns before they become widespread issues.

Categorize Errors

POST /api/v2/error-analysis/categorize

Categorize errors from a set of traces into a taxonomy. Can use an existing category set or generate new categories from the data.

Request Body:

{
  "agentId": "agent_support_01",
  "traceIds": ["tr_f01", "tr_f02", "tr_f10", "tr_f11", "tr_f20"],
  "options": {
    "useExistingCategories": true,
    "generateNewCategories": true,
    "minCategorySize": 2
  }
}

Request Fields:

Field	Type	Required	Description
`agentId`	string	Yes	Agent ID to scope categorization
`traceIds`	array	No	Specific trace IDs to categorize; if omitted, categorizes recent failures
`options.useExistingCategories`	boolean	No	Map errors to existing categories (default: `true`)
`options.generateNewCategories`	boolean	No	Allow creation of new categories from data (default: `true`)
`options.minCategorySize`	number	No	Minimum errors to form a new category (default: 2)

Response:

{
  "success": true,
  "data": {
    "categorized": 5,
    "categories": [
      {
        "id": "cat_stale_kb",
        "name": "Stale Knowledge Base",
        "description": "Errors caused by outdated or missing documentation in the knowledge base",
        "errorCount": 2,
        "traceIds": ["tr_f01", "tr_f02"],
        "severity": "high",
        "isNew": false
      },
      {
        "id": "cat_hallucination",
        "name": "Hallucinated Data",
        "description": "Agent fabricated specific data points (numbers, dates, calculations) not present in any source",
        "errorCount": 2,
        "traceIds": ["tr_f10", "tr_f11"],
        "severity": "critical",
        "isNew": false
      },
      {
        "id": "cat_scope_violation",
        "name": "Scope Violation",
        "description": "Agent answered questions outside its designated domain without escalating",
        "errorCount": 1,
        "traceIds": ["tr_f20"],
        "severity": "medium",
        "isNew": true
      }
    ],
    "uncategorized": 0,
    "creditsUsed": 1
  }
}

Get Categories

GET /api/v2/error-analysis/categories

List all error categories with aggregate statistics.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`severity`	string	—	Filter by severity: `low`, `medium`, `high`, `critical`
`sortBy`	string	`errorCount`	Sort by: `errorCount`, `severity`, `lastSeen`, `name`
`order`	string	`desc`	Sort order: `asc` or `desc`
`limit`	number	20	Items per page (max: 100)
`offset`	number	0	Pagination offset

Response:

{
  "success": true,
  "data": {
    "categories": [
      {
        "id": "cat_hallucination",
        "name": "Hallucinated Data",
        "description": "Agent fabricated specific data points not present in any source",
        "severity": "critical",
        "errorCount": 45,
        "affectedAgents": ["agent_support_01", "agent_sales_03"],
        "firstSeen": "2025-02-15T10:00:00Z",
        "lastSeen": "2025-03-10T11:45:00Z",
        "trend": "stable",
        "weekOverWeekChange": -0.02
      },
      {
        "id": "cat_stale_kb",
        "name": "Stale Knowledge Base",
        "description": "Errors caused by outdated or missing documentation",
        "severity": "high",
        "errorCount": 38,
        "affectedAgents": ["agent_support_01"],
        "firstSeen": "2025-03-01T08:00:00Z",
        "lastSeen": "2025-03-10T12:30:00Z",
        "trend": "increasing",
        "weekOverWeekChange": 0.15
      },
      {
        "id": "cat_scope_violation",
        "name": "Scope Violation",
        "description": "Agent answered outside its designated domain",
        "severity": "medium",
        "errorCount": 12,
        "affectedAgents": ["agent_support_01"],
        "firstSeen": "2025-03-08T14:00:00Z",
        "lastSeen": "2025-03-10T09:00:00Z",
        "trend": "new",
        "weekOverWeekChange": null
      }
    ],
    "pagination": {
      "total": 8,
      "limit": 20,
      "offset": 0,
      "hasMore": false
    }
  }
}

Error Trends

GET /api/v2/error-analysis/trends

Get time-series error trend data for visualization and alerting.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`categoryId`	string	—	Filter by error category ID
`granularity`	string	`day`	Time bucket: `hour`, `day`, `week`
`startDate`	string	30 days ago	ISO 8601 start date
`endDate`	string	now	ISO 8601 end date

Response:

{
  "success": true,
  "data": {
    "trends": [
      {
        "period": "2025-03-08",
        "totalErrors": 18,
        "byCategory": {
          "cat_hallucination": 8,
          "cat_stale_kb": 7,
          "cat_scope_violation": 3
        },
        "bySeverity": {
          "critical": 8,
          "high": 7,
          "medium": 3,
          "low": 0
        }
      },
      {
        "period": "2025-03-09",
        "totalErrors": 22,
        "byCategory": {
          "cat_hallucination": 9,
          "cat_stale_kb": 10,
          "cat_scope_violation": 3
        },
        "bySeverity": {
          "critical": 9,
          "high": 10,
          "medium": 3,
          "low": 0
        }
      },
      {
        "period": "2025-03-10",
        "totalErrors": 15,
        "byCategory": {
          "cat_hallucination": 6,
          "cat_stale_kb": 5,
          "cat_scope_violation": 4
        },
        "bySeverity": {
          "critical": 6,
          "high": 5,
          "medium": 4,
          "low": 0
        }
      }
    ],
    "summary": {
      "totalErrors": 55,
      "avgErrorsPerDay": 18.3,
      "trendDirection": "increasing",
      "topCategory": "cat_hallucination",
      "alertThresholdBreached": false
    }
  }
}

Transcript Patterns API

The Transcript Patterns API analyzes multi-turn conversations to detect behavioral patterns, communication risks, and interaction quality signals.

Analyze Patterns

POST /api/transcript-patterns/analyze

Analyze a conversation transcript for behavioral patterns, tone shifts, and interaction quality.

Request Body:

{
  "traceId": "tr_conv_001",
  "transcript": [
    {
      "role": "user",
      "content": "I need to cancel my subscription immediately",
      "timestamp": "2025-03-10T14:00:00Z"
    },
    {
      "role": "agent",
      "content": "I understand you'd like to cancel. Before I process that, could I ask what prompted this decision? We may be able to address your concerns.",
      "timestamp": "2025-03-10T14:00:03Z"
    },
    {
      "role": "user",
      "content": "The product doesn't work. I've been trying to get help for 3 days and nobody responds.",
      "timestamp": "2025-03-10T14:00:15Z"
    },
    {
      "role": "agent",
      "content": "I sincerely apologize for the delayed support response. That's not the experience we want for our customers. Let me look into your recent support tickets right away. Can you share your account email so I can pull up your history?",
      "timestamp": "2025-03-10T14:00:18Z"
    }
  ],
  "options": {
    "detectPatterns": true,
    "assessRisk": true,
    "evaluateTone": true,
    "checkCompliance": true
  }
}

Request Fields:

Field	Type	Required	Description
`traceId`	string	No	Link analysis to an existing trace
`transcript`	array	Yes	Array of conversation turns
`transcript[].role`	string	Yes	`user`, `agent`, or `system`
`transcript[].content`	string	Yes	Message content
`transcript[].timestamp`	string	No	ISO 8601 timestamp
`options.detectPatterns`	boolean	No	Detect conversation patterns (default: `true`)
`options.assessRisk`	boolean	No	Run risk assessment (default: `true`)
`options.evaluateTone`	boolean	No	Analyze tone and sentiment shifts (default: `true`)
`options.checkCompliance`	boolean	No	Check for compliance violations (default: `false`)

Response:

{
  "success": true,
  "data": {
    "traceId": "tr_conv_001",
    "patterns": [
      {
        "id": "pat_retention_001",
        "name": "Retention Attempt",
        "description": "Agent proactively attempted retention before processing cancellation",
        "category": "retention",
        "sentiment": "positive",
        "turns": [1],
        "quality": "good"
      },
      {
        "id": "pat_empathy_002",
        "name": "Empathetic Acknowledgment",
        "description": "Agent acknowledged customer frustration and apologized for delayed support",
        "category": "empathy",
        "sentiment": "positive",
        "turns": [3],
        "quality": "excellent"
      },
      {
        "id": "pat_escalation_003",
        "name": "Frustration Escalation",
        "description": "Customer expressed increasing frustration with support response times",
        "category": "escalation",
        "sentiment": "negative",
        "turns": [2],
        "quality": "neutral"
      }
    ],
    "toneAnalysis": {
      "userSentiment": {
        "overall": "negative",
        "trajectory": "stable_negative",
        "turns": [
          { "turn": 0, "sentiment": "negative", "intensity": 0.7 },
          { "turn": 2, "sentiment": "negative", "intensity": 0.8 }
        ]
      },
      "agentSentiment": {
        "overall": "empathetic",
        "trajectory": "appropriately_responsive",
        "turns": [
          { "turn": 1, "sentiment": "professional", "intensity": 0.6 },
          { "turn": 3, "sentiment": "empathetic", "intensity": 0.8 }
        ]
      }
    },
    "riskAssessment": {
      "overallRisk": "medium",
      "churnRisk": 0.72,
      "escalationRisk": 0.45,
      "complianceRisk": 0.05,
      "factors": [
        {
          "factor": "Delayed support response (3 days)",
          "risk": "high",
          "recommendation": "Implement SLA monitoring with escalation triggers"
        },
        {
          "factor": "Active cancellation request",
          "risk": "high",
          "recommendation": "Route to retention specialist with authority to offer concessions"
        }
      ]
    },
    "qualityScore": 0.78,
    "creditsUsed": 1
  }
}

Pattern Categories

GET /api/transcript-patterns/categories

List all detected pattern categories with frequency and quality statistics.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`startDate`	string	30 days ago	ISO 8601 start date
`endDate`	string	now	ISO 8601 end date

Response:

{
  "success": true,
  "data": {
    "categories": [
      {
        "id": "retention",
        "name": "Retention Patterns",
        "description": "Patterns related to customer retention attempts",
        "patternCount": 156,
        "avgQuality": 0.72,
        "topPatterns": [
          { "name": "Retention Attempt", "count": 89, "avgQuality": 0.75 },
          { "name": "Discount Offer", "count": 42, "avgQuality": 0.68 },
          { "name": "Feature Highlight", "count": 25, "avgQuality": 0.71 }
        ]
      },
      {
        "id": "empathy",
        "name": "Empathy Patterns",
        "description": "Patterns showing empathetic agent responses",
        "patternCount": 234,
        "avgQuality": 0.81,
        "topPatterns": [
          { "name": "Empathetic Acknowledgment", "count": 145, "avgQuality": 0.84 },
          { "name": "Frustration Validation", "count": 89, "avgQuality": 0.76 }
        ]
      },
      {
        "id": "escalation",
        "name": "Escalation Patterns",
        "description": "Patterns indicating conversation escalation",
        "patternCount": 98,
        "avgQuality": 0.45,
        "topPatterns": [
          { "name": "Frustration Escalation", "count": 62, "avgQuality": 0.40 },
          { "name": "Repeated Requests", "count": 36, "avgQuality": 0.52 }
        ]
      },
      {
        "id": "compliance",
        "name": "Compliance Patterns",
        "description": "Patterns related to regulatory or policy compliance",
        "patternCount": 15,
        "avgQuality": 0.88,
        "topPatterns": [
          { "name": "Data Collection Disclosure", "count": 10, "avgQuality": 0.90 },
          { "name": "Terms Reference", "count": 5, "avgQuality": 0.84 }
        ]
      }
    ]
  }
}

Risk Assessment

GET /api/transcript-patterns/risks

Get aggregated risk assessments across conversations, identifying high-risk interaction patterns and agents.

Query Parameters:

Parameter	Type	Default	Description
`agentId`	string	—	Filter by agent ID
`riskLevel`	string	—	Filter by risk level: `low`, `medium`, `high`, `critical`
`riskType`	string	—	Filter by type: `churn`, `escalation`, `compliance`, `reputation`
`startDate`	string	30 days ago	ISO 8601 start date
`endDate`	string	now	ISO 8601 end date
`limit`	number	20	Items per page (max: 100)
`offset`	number	0	Pagination offset

Response:

{
  "success": true,
  "data": {
    "risks": [
      {
        "id": "risk_001",
        "traceId": "tr_conv_050",
        "agentId": "agent_support_01",
        "riskLevel": "high",
        "riskScore": 0.85,
        "riskTypes": ["churn", "escalation"],
        "summary": "Enterprise customer threatened cancellation after agent provided incorrect billing information. Customer has been waiting 5 days for resolution.",
        "factors": [
          {
            "type": "churn",
            "score": 0.88,
            "detail": "Enterprise customer ($50K ARR) with active cancellation language"
          },
          {
            "type": "escalation",
            "score": 0.82,
            "detail": "Unresolved issue persisting for 5 days, customer sentiment deteriorating"
          }
        ],
        "recommendedActions": [
          "Escalate to customer success manager immediately",
          "Prepare corrected billing statement",
          "Schedule executive check-in within 24 hours"
        ],
        "detectedAt": "2025-03-10T16:30:00Z"
      }
    ],
    "summary": {
      "totalRisks": 34,
      "byLevel": {
        "critical": 2,
        "high": 8,
        "medium": 15,
        "low": 9
      },
      "byType": {
        "churn": 12,
        "escalation": 18,
        "compliance": 3,
        "reputation": 1
      },
      "avgRiskScore": 0.52
    },
    "pagination": {
      "total": 34,
      "limit": 20,
      "offset": 0,
      "hasMore": true
    }
  }
}

Error Handling

All analysis endpoints return standard error responses:

{
  "success": false,
  "error": {
    "code": "INSUFFICIENT_CREDITS",
    "message": "Not enough credits to perform this analysis. Required: 10, available: 3.5",
    "required": 10,
    "available": 3.5
  }
}

Common Error Codes:

Code	HTTP Status	Description
`INSUFFICIENT_CREDITS`	402	Not enough credits for the requested analysis tier
`TRACE_NOT_FOUND`	404	The specified trace ID does not exist
`ANALYSIS_IN_PROGRESS`	409	An analysis is already running for this target
`RATE_LIMITED`	429	Too many analysis requests; retry after the specified interval
`TIER_NOT_AVAILABLE`	403	The requested tier is not available on your plan
`BATCH_TOO_LARGE`	400	Batch size exceeds the maximum allowed (500 traces)

Next Steps

Traces & Spans — Ingest the traces you want to analyze
Guardrails — Prevent issues before they reach users
Issues & Fixes — Manage and remediate identified problems
Webhooks & Notifications — Get notified when analysis completes

Evaluation & Grading Business Metrics & ROI