Troubleshooting

Solutions to common issues with ThinkHive.

Installation Issues

npm install fails

# Clear npm cache
npm cache clean --force
rm -rf node_modules package-lock.json
npm install

Python package conflicts

# Use virtual environment
python -m venv venv
source venv/bin/activate
pip install thinkhive

Authentication Issues

Invalid API Key

⚠️

Check these first:

  • API key is not expired
  • Key has correct tier permissions
  • No extra whitespace in the key
// Verify key format
if (!apiKey.startsWith('th_')) {
  console.error('Invalid API key format');
}

Session Expired

For browser apps, redirect to login:

if (error.code === 'SESSION_EXPIRED') {
  window.location.href = '/login';
}

Tracing Issues

Traces Not Appearing

  1. Verify SDK initialization
import { isInitialized } from '@thinkhive/sdk';
console.log('SDK initialized:', isInitialized());
  1. Check network connectivity
curl https://app.thinkhive.ai/api/health
  1. Enable debug mode
init({ debug: true });

Missing Spans

Ensure async operations complete:

// Good - awaits completion
await traceLLM({ ... }, async () => {
  return await llmCall();
});
 
// Bad - doesn't await
traceLLM({ ... }, () => {
  llmCall(); // Not awaited
});

API Errors

Rate Limited (429)

Wait and retry with backoff:

async function withRetry(fn, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (e) {
      if (e.status === 429 && i < maxRetries - 1) {
        await sleep(Math.pow(2, i) * 1000);
        continue;
      }
      throw e;
    }
  }
}

Validation Error (400)

Check request payload matches schema:

// Required fields
const trace = {
  agentId: 'required',
  spans: [],  // Must be array
};

MCP Server Issues

Server Not Starting

# Check environment
echo $THINKHIVE_API_KEY
 
# Test with debug
DEBUG=* npx @thinkhive/mcp-explainer

Claude Code Not Seeing Tools

  1. Restart Claude Code
  2. Check ~/.claude/settings.json syntax
  3. View logs: ~/.claude/logs/

Evaluation Issues

Evaluation Suite Returns No Results

  • Verify the traces you selected exist and have spans
  • Check that the evaluation criteria match the trace content
  • For LLM judges, ensure you have sufficient credits — see Billing & Credits

Low Evaluation Scores

  • Review the evaluation criteria — the prompt may be too strict
  • Check that the traces contain the expected content (input, output, context)
  • Use deterministic graders first to rule out data issues

Jury Mode Disagreement

If jury judges produce wildly different scores:

  • Review the evaluation prompt for ambiguity
  • Consider adjusting judge weights
  • Add calibration examples to the evaluation prompt

Webhook Issues

Webhooks Not Delivered

  1. Check endpoint is reachable — your endpoint must return 200 OK within 30 seconds
  2. Verify HTTPS — HTTP endpoints are not supported
  3. Check delivery history in Settings → Webhooks → Delivery History
  4. Circuit breaker — if your endpoint failed 5+ times, delivery is paused. Wait 30 minutes or re-enable in settings

Duplicate Webhook Events

Webhooks may be delivered more than once due to retries. Use the event id field to deduplicate:

const processedEvents = new Set();
 
app.post('/webhooks/thinkhive', (req, res) => {
  if (processedEvents.has(req.body.id)) {
    return res.status(200).json({ received: true }); // Already processed
  }
  processedEvents.add(req.body.id);
  // Process event...
  res.status(200).json({ received: true });
});

Case & Fix Issues

Cases Not Auto-Creating

  • Verify you have enough traces with failures (default threshold: 3 similar failures)
  • Check that the agent has traces with error states or low quality scores
  • Auto-case creation requires trace analysis to complete first

Shadow Test Failures

  • Ensure the fix configuration is valid
  • Check that the evaluation suite used for validation has appropriate criteria
  • Review the shadow test logs for specific error messages

Self-Hosting Issues

Database Connection Errors

# Test connection
psql $DATABASE_URL -c "SELECT 1"
 
# Check SSL requirements
# Neon requires: ?sslmode=require
# Local PostgreSQL: ?sslmode=disable

Container Won’t Start

# Check logs
docker logs thinkhive
 
# Common issues:
# - Missing required env vars (DATABASE_URL, SESSION_SECRET, API_KEY_SECRET)
# - Database not reachable
# - Port conflict (5001 already in use)

Health Check Failing

# Liveness (basic)
curl http://localhost:5001/health/live
 
# Readiness (includes DB)
curl http://localhost:5001/health/ready
 
# If readiness fails, check DATABASE_URL and database connectivity

Getting Help