Transcript Analysis Guide

Analyze conversation transcripts to detect escalation indicators, PII exposure, sentiment shifts, and other risk patterns.

Why Transcript Analysis?

Individual evaluation scores tell you whether a single response was good. Transcript analysis looks at the entire conversation to find patterns that only emerge across multiple turns:

Escalation spirals where the user becomes increasingly frustrated
PII leakage where sensitive information appears in AI responses
Sentiment degradation where conversation quality drops over time
Repetition loops where the AI gives the same unhelpful answer repeatedly

⚠️

Transcript analysis is especially critical for compliance-sensitive industries. PII exposure in AI conversations can trigger GDPR, HIPAA, or CCPA violations.

Analyzing Transcripts

Run pattern detection

import { transcriptPatterns } from '@thinkhive/sdk';
 
const analysis = await transcriptPatterns.analyze({
  traceId: 'trace_abc',
  patterns: ['escalation', 'pii', 'sentiment', 'repetition'],
});
 
console.log(analysis);
// {
//   traceId: 'trace_abc',
//   turnCount: 8,
//   patterns: {
//     escalation: {
//       detected: true,
//       severity: 'high',
//       triggerTurn: 4,
//       indicators: ['user expressed frustration', 'requested human agent', 'used caps']
//     },
//     pii: {
//       detected: true,
//       exposures: [
//         { type: 'email', turn: 2, source: 'user', text: 'j***@example.com' },
//         { type: 'phone', turn: 5, source: 'agent', text: '(555) ***-**42' }
//       ]
//     },
//     sentiment: {
//       trajectory: [0.6, 0.5, 0.3, 0.1, -0.2, -0.4, -0.3, -0.5],
//       trend: 'declining',
//       shiftPoints: [{ turn: 4, delta: -0.3, trigger: 'unresolved issue' }]
//     },
//     repetition: {
//       detected: true,
//       loops: [{ turns: [3, 5, 7], similarity: 0.92 }]
//     }
//   },
//   riskScore: 0.78,
//   riskLevel: 'high'
// }

from thinkhive import transcript_patterns
 
analysis = transcript_patterns.analyze(
    trace_id="trace_abc",
    patterns=["escalation", "pii", "sentiment", "repetition"],
)
 
print(analysis["patterns"]["escalation"])
# {
#   "detected": True,
#   "severity": "high",
#   "trigger_turn": 4,
#   "indicators": ["user expressed frustration", "requested human agent"]
# }

curl -X POST https://app.thinkhive.ai/api/transcript-patterns/analyze \
  -H "Authorization: Bearer $THINKHIVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "traceId": "trace_abc",
    "patterns": ["escalation", "pii", "sentiment", "repetition"]
  }'

Detect escalation indicators

Escalation detection identifies signals that a conversation is going off-track.

const escalation = await transcriptPatterns.detectEscalation({
  traceId: 'trace_abc',
  indicators: {
    frustrationKeywords: true,     // "this is useless", "not helpful"
    capsUsage: true,               // Excessive uppercase
    requestForHuman: true,         // "let me talk to a person"
    repeatedQuestions: true,       // User asks the same thing multiple times
    negativeEmoticons: true,       // Angry emojis or punctuation
  },
});
 
console.log(escalation);
// {
//   escalated: true,
//   severity: 'high',
//   turnByTurnRisk: [0.1, 0.2, 0.3, 0.6, 0.8, 0.9, 0.85, 0.95],
//   triggerTurn: 4,
//   triggerMessage: 'This is the third time I have asked the same question!',
//   recommendation: 'Route to human agent at turn 4 based on escalation signals.'
// }

Identify PII exposure

const piiReport = await transcriptPatterns.detectPII({
  traceId: 'trace_abc',
  entityTypes: ['email', 'phone', 'ssn', 'credit_card', 'address', 'name'],
});
 
console.log(piiReport);
// {
//   totalExposures: 3,
//   exposures: [
//     { type: 'email', turn: 2, source: 'user', redacted: 'j***@example.com' },
//     { type: 'phone', turn: 5, source: 'agent', redacted: '(555) ***-**42' },
//     { type: 'name', turn: 6, source: 'agent', redacted: 'J*** D**' },
//   ],
//   agentExposures: 2, // PII leaked BY the agent (more critical)
//   complianceRisk: 'high',
//   recommendation: 'Agent exposed user PII in turns 5 and 6. Review guardrail policies.'
// }

🚫

Agent-side PII exposure (where the AI reveals user PII in its response) is a critical compliance risk. Set up alerts for any agent-side PII detection.

Track sentiment shifts

const sentiment = await transcriptPatterns.analyzeSentiment({
  traceId: 'trace_abc',
});
 
console.log(sentiment);
// {
//   trajectory: [
//     { turn: 1, score: 0.6, label: 'positive' },
//     { turn: 2, score: 0.5, label: 'neutral' },
//     { turn: 3, score: 0.3, label: 'neutral' },
//     { turn: 4, score: 0.1, label: 'neutral' },
//     { turn: 5, score: -0.2, label: 'negative' },
//     { turn: 6, score: -0.4, label: 'negative' },
//   ],
//   overallTrend: 'declining',
//   significantShifts: [
//     { fromTurn: 3, toTurn: 5, delta: -0.5, cause: 'unresolved_issue' }
//   ],
//   averageSentiment: 0.13,
// }

Risk Assessment

Get a comprehensive risk score for a conversation.

const risk = await transcriptPatterns.assessRisk({
  traceId: 'trace_abc',
});
 
console.log(risk);
// {
//   overallRisk: 0.78,
//   riskLevel: 'high',
//   factors: [
//     { factor: 'escalation', weight: 0.35, score: 0.9 },
//     { factor: 'pii_exposure', weight: 0.30, score: 0.8 },
//     { factor: 'sentiment_decline', weight: 0.20, score: 0.7 },
//     { factor: 'repetition', weight: 0.15, score: 0.5 },
//   ],
//   actions: [
//     'Review guardrail policies for PII filtering',
//     'Add escalation detection trigger at frustration threshold 0.6',
//     'Investigate root cause of repetition loop in turns 3-7'
//   ]
// }

Batch Analysis

Analyze patterns across many conversations to find systemic issues.

const batchResults = await transcriptPatterns.batchAnalyze({
  agentId: 'agent_123',
  period: '7d',
  patterns: ['escalation', 'pii', 'sentiment'],
});
 
console.log(batchResults.summary);
// {
//   totalConversations: 3420,
//   escalationRate: 0.12,         // 12% of conversations escalated
//   piiExposureRate: 0.03,        // 3% had PII exposure
//   avgSentimentTrend: 'stable',
//   topEscalationTopics: [
//     { topic: 'billing disputes', count: 89 },
//     { topic: 'account access', count: 67 },
//     { topic: 'refund requests', count: 45 },
//   ],
//   recommendation: 'Focus on billing dispute handling -- highest escalation rate.'
// }

Setting Up Alerts

// Alert on high-risk conversations
await transcriptPatterns.createAlert({
  agentId: 'agent_123',
  conditions: {
    riskLevel: 'high',
    piiExposure: { source: 'agent', any: true },
    escalation: { severity: 'high' },
  },
  actions: {
    webhook: 'https://your-app.com/alerts',
    slackChannel: '#ai-incidents',
  },
});

Best Practices

Pattern Priority by Industry

Industry	Priority Patterns	Why
Healthcare	PII, compliance, escalation	HIPAA requirements
Finance	PII, sentiment, escalation	Regulatory risk
E-commerce	Sentiment, escalation, repetition	Customer retention
SaaS	Escalation, repetition, sentiment	Churn prevention

Enable PII detection on all conversations in regulated industries — do not sample
Set escalation thresholds based on your domain — support conversations naturally have more frustration than sales
Track sentiment trends weekly to catch gradual quality degradation
Investigate repetition loops — they often indicate missing knowledge base content
Combine transcript analysis with guardrails for real-time protection

Next Steps

Guardrail Policies — Real-time content protection
Compliance — PII handling and compliance settings
API Reference — Full API documentation

Business Metrics & ROI Guardrail Policies