Webhooks

Webhooks deliver real-time notifications to your application when events occur in ThinkHive — such as new cases, evaluation completions, or quality alerts.

Overview

ThinkHive’s webhook system includes:

Automatic retries with exponential backoff
Circuit breaker protection to prevent overwhelming failing endpoints
Event filtering to receive only the events you care about
Signature verification for secure delivery

Configuring Webhooks

Navigate to Settings

Go to Settings → Webhooks in the ThinkHive dashboard.

Add a webhook endpoint

Provide the HTTPS URL where you want to receive events.

⚠️

Webhook endpoints must use HTTPS. HTTP endpoints are not supported for security reasons.

Select events

Choose which events to subscribe to:

Event	Description
`case.created`	New case created from failure cluster
`case.updated`	Case status or severity changed
`case.resolved`	Case marked as resolved
`evaluation.completed`	Evaluation suite run finished
`evaluation.failed`	Evaluation suite run failed
`fix.proposed`	New fix generated for a case
`shadow_test.completed`	Shadow test finished
`alert.quality_degradation`	Quality dropped below threshold
`alert.drift_detected`	Drift detected in agent behavior
`trace.ingested`	New trace received (high volume)

Save and test

Click Test Webhook to send a test event to your endpoint.

Webhook Payload

All webhook payloads follow this structure:

{
  "id": "evt_abc123",
  "type": "case.created",
  "timestamp": "2025-01-15T10:30:00Z",
  "data": {
    "id": "case_def456",
    "title": "Incomplete MFA instructions",
    "severity": "high",
    "traceCount": 12
  }
}

Example Payloads

case.created

{
  "id": "evt_001",
  "type": "case.created",
  "timestamp": "2025-01-15T10:30:00Z",
  "data": {
    "id": "case_abc",
    "title": "Agent fails to handle refund requests",
    "severity": "high",
    "traceCount": 23,
    "pattern": "Missing refund policy context in retrieval"
  }
}

evaluation.completed

{
  "id": "evt_002",
  "type": "evaluation.completed",
  "timestamp": "2025-01-15T11:00:00Z",
  "data": {
    "suiteId": "suite_xyz",
    "runId": "eval_789",
    "totalTraces": 50,
    "averageScore": 4.2,
    "passRate": 0.88
  }
}

alert.quality_degradation

{
  "id": "evt_003",
  "type": "alert.quality_degradation",
  "timestamp": "2025-01-15T12:00:00Z",
  "data": {
    "agentId": "agent_abc",
    "metric": "groundedness",
    "previousValue": 0.92,
    "currentValue": 0.71,
    "threshold": 0.80,
    "window": "1h"
  }
}

Verifying Webhook Signatures

ThinkHive signs webhook payloads using HMAC-SHA256. Always verify signatures to ensure the request is authentic.

import crypto from 'crypto';
 
function verifyWebhookSignature(
  payload: string,
  signature: string,
  secret: string
): boolean {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');
 
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expected)
  );
}
 
// Express middleware
app.post('/webhooks/thinkhive', (req, res) => {
  const signature = req.headers['x-thinkhive-signature'];
  const isValid = verifyWebhookSignature(
    JSON.stringify(req.body),
    signature,
    process.env.THINKHIVE_WEBHOOK_SECRET
  );
 
  if (!isValid) {
    return res.status(401).json({ error: 'Invalid signature' });
  }
 
  // Process the event
  const event = req.body;
  switch (event.type) {
    case 'case.created':
      handleNewCase(event.data);
      break;
    case 'evaluation.completed':
      handleEvaluationResult(event.data);
      break;
  }
 
  res.status(200).json({ received: true });
});

Retry Policy

If your endpoint returns a non-2xx status code, ThinkHive retries with exponential backoff:

Attempt	Delay
1st retry	30 seconds
2nd retry	2 minutes
3rd retry	10 minutes
4th retry	1 hour
5th retry	6 hours

After 5 failed attempts, the webhook is marked as failed and you receive an email notification. Failed webhooks can be retried manually from the dashboard.

Circuit Breaker

If your endpoint fails consistently, the circuit breaker activates:

Open: After 5 consecutive failures, delivery is paused
Half-Open: After 30 minutes, a test delivery is attempted
Closed: If the test succeeds, normal delivery resumes

Best Practices

Respond quickly — return 200 OK as soon as you receive the event, then process asynchronously
Verify signatures — always validate the X-ThinkHive-Signature header
Handle duplicates — use the event id field to deduplicate, as retries may deliver the same event twice
Use HTTPS — webhook endpoints must use TLS encryption
Monitor delivery — check the webhook delivery log in Settings → Webhooks → Delivery History

Managing Webhooks via API

# List webhooks
curl "https://app.thinkhive.ai/api/v1/explainer/webhooks" \
  -H "Authorization: Bearer thk_your_api_key"
 
# Create a webhook
curl -X POST "https://app.thinkhive.ai/api/v1/explainer/webhooks" \
  -H "Authorization: Bearer thk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-app.com/webhooks/thinkhive",
    "events": ["case.created", "evaluation.completed"],
    "active": true
  }'
 
# Delete a webhook
curl -X DELETE "https://app.thinkhive.ai/api/v1/explainer/webhooks/wh_abc123" \
  -H "Authorization: Bearer thk_your_api_key"

Next Steps

API Reference: Webhooks & Notifications — Full webhook endpoint documentation
Cases & Fixes — Understand cases that trigger webhook events
ThinkEval — Evaluation system that triggers webhook events

API Key Management PII & Compliance