Business Metrics & ROI API

Track business outcomes tied to your AI agents, measure return on investment, and evaluate output quality with automated scoring.

All endpoints require authentication via API key. See Authentication for details.

Quick Start

import { init, businessMetrics, roiAnalytics, qualityMetrics } from '@thinkhive/sdk';
 
init({ apiKey: 'th_your_api_key' });
 
// Record a business metric
await businessMetrics.record({
  name: 'tickets_resolved',
  value: 42,
  agentId: 'agent_abc123',
  metadata: { channel: 'chat' }
});
 
// Get ROI summary
const roi = await roiAnalytics.getSummary({ period: '30d' });
console.log(roi.totalSavings);  // 12500.00
 
// Evaluate RAG quality
const quality = await qualityMetrics.evaluateRAG({ traceId: 'trace_xyz' });
console.log(quality.scores.faithfulness);  // 0.94

import thinkhive
 
thinkhive.init(api_key='th_your_api_key')
 
# Record a business metric
thinkhive.business_metrics.record(
    name='tickets_resolved',
    value=42,
    agent_id='agent_abc123',
    metadata={'channel': 'chat'}
)
 
# Get ROI summary
roi = thinkhive.roi_analytics.get_summary(period='30d')
print(roi.total_savings)  # 12500.00
 
# Evaluate RAG quality
quality = thinkhive.quality_metrics.evaluate_rag(trace_id='trace_xyz')
print(quality.scores.faithfulness)  # 0.94

# Record a business metric
curl -X POST https://app.thinkhive.ai/api/metrics \
  -H "Authorization: Bearer th_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "tickets_resolved",
    "value": 42,
    "agentId": "agent_abc123",
    "metadata": { "channel": "chat" }
  }'

Business Metrics

Record and query business-level metrics associated with your AI agents.

Record Metric

POST /api/metrics

Request Body:

{
  "name": "tickets_resolved",
  "value": 42,
  "agentId": "agent_abc123",
  "timestamp": "2025-03-10T14:00:00Z",
  "metadata": {
    "channel": "chat",
    "region": "us-east"
  }
}

Field	Type	Required	Description
`name`	string	Yes	Metric name (e.g., `tickets_resolved`, `revenue`, `csat_score`)
`value`	number	Yes	Metric value
`agentId`	string	No	Associate metric with a specific agent
`timestamp`	string	No	ISO 8601 timestamp (defaults to now)
`metadata`	object	No	Arbitrary key-value pairs for filtering

Response:

{
  "success": true,
  "data": {
    "id": "metric_abc123",
    "name": "tickets_resolved",
    "value": 42,
    "agentId": "agent_abc123",
    "timestamp": "2025-03-10T14:00:00Z",
    "metadata": {
      "channel": "chat",
      "region": "us-east"
    },
    "createdAt": "2025-03-10T14:00:01Z"
  }
}

SDK Usage:

const metric = await businessMetrics.record({
  name: 'tickets_resolved',
  value: 42,
  agentId: 'agent_abc123',
  metadata: { channel: 'chat' }
});

Get Current Metrics

Retrieve the latest value for each metric.

GET /api/metrics

Query Parameters:

Parameter	Type	Description
`agentId`	string	Filter by agent
`name`	string	Filter by metric name
`page`	number	Page number (default: 1)
`limit`	number	Items per page (default: 20)

Response:

{
  "success": true,
  "data": [
    {
      "id": "metric_abc123",
      "name": "tickets_resolved",
      "value": 42,
      "agentId": "agent_abc123",
      "timestamp": "2025-03-10T14:00:00Z"
    },
    {
      "id": "metric_def456",
      "name": "csat_score",
      "value": 4.6,
      "agentId": "agent_abc123",
      "timestamp": "2025-03-10T14:00:00Z"
    }
  ],
  "pagination": {
    "page": 1,
    "limit": 20,
    "total": 2,
    "hasMore": false
  }
}

SDK Usage:

const metrics = await businessMetrics.getCurrent({
  agentId: 'agent_abc123'
});

Get Metric History

Retrieve historical values for a metric over time.

GET /api/metrics/history

Query Parameters:

Parameter	Type	Description
`name`	string	Required. Metric name
`agentId`	string	Filter by agent
`from`	string	Start date (ISO 8601)
`to`	string	End date (ISO 8601)
`interval`	string	Aggregation interval: `hour`, `day`, `week`, `month` (default: `day`)

Response:

{
  "success": true,
  "data": {
    "name": "tickets_resolved",
    "agentId": "agent_abc123",
    "interval": "day",
    "dataPoints": [
      { "timestamp": "2025-03-08T00:00:00Z", "value": 38, "count": 12 },
      { "timestamp": "2025-03-09T00:00:00Z", "value": 45, "count": 15 },
      { "timestamp": "2025-03-10T00:00:00Z", "value": 42, "count": 14 }
    ]
  }
}

SDK Usage:

const history = await businessMetrics.getHistory({
  name: 'tickets_resolved',
  agentId: 'agent_abc123',
  from: '2025-03-01',
  to: '2025-03-10',
  interval: 'day'
});

Delete Metric

DELETE /api/metrics/:id

Path Parameters:

Parameter	Type	Description
`id`	string	Metric ID

Response:

{
  "success": true,
  "message": "Metric deleted"
}

ROI Analytics

Measure the financial impact of your AI agents with automated cost-savings calculations, trend analysis, and predictive modeling.

Get ROI Summary

GET /api/v1/roi-analytics/summary

Query Parameters:

Parameter	Type	Description
`period`	string	Time period: `7d`, `30d`, `90d`, `ytd`, `all` (default: `30d`)
`agentId`	string	Filter to a specific agent

Response:

{
  "success": true,
  "data": {
    "period": "30d",
    "totalSavings": 12500.00,
    "totalCost": 3200.00,
    "netROI": 9300.00,
    "roiPercentage": 290.6,
    "ticketsDeflected": 1850,
    "avgCostPerResolution": 1.73,
    "humanCostPerResolution": 8.50,
    "agentCount": 3,
    "breakdown": {
      "laborSavings": 10200.00,
      "efficiencyGains": 1800.00,
      "errorReduction": 500.00
    }
  }
}

SDK Usage:

const summary = await roiAnalytics.getSummary({
  period: '30d',
  agentId: 'agent_abc123'
});
 
console.log(`Net ROI: $${summary.netROI}`);
console.log(`ROI %: ${summary.roiPercentage}%`);

summary = thinkhive.roi_analytics.get_summary(
    period='30d',
    agent_id='agent_abc123'
)
 
print(f"Net ROI: ${summary.net_roi}")
print(f"ROI %: {summary.roi_percentage}%")

curl https://app.thinkhive.ai/api/v1/roi-analytics/summary?period=30d \
  -H "Authorization: Bearer th_your_api_key"

Per-Agent ROI

GET /api/v1/roi-analytics/by-agent/:agentId

Path Parameters:

Parameter	Type	Description
`agentId`	string	Agent ID

Query Parameters:

Parameter	Type	Description
`period`	string	Time period: `7d`, `30d`, `90d`, `ytd`, `all` (default: `30d`)

Response:

{
  "success": true,
  "data": {
    "agentId": "agent_abc123",
    "agentName": "Customer Support Agent",
    "period": "30d",
    "savings": 4500.00,
    "cost": 1100.00,
    "netROI": 3400.00,
    "roiPercentage": 309.1,
    "interactions": 620,
    "successRate": 0.87,
    "avgHandlingTime": 12.4,
    "humanHandlingTime": 480.0,
    "timeSavedHours": 289.7
  }
}

Configure ROI Settings

Set cost parameters used for ROI calculations.

POST /api/v1/roi-analytics/configure

Request Body:

{
  "humanCostPerHour": 35.00,
  "avgHumanHandlingMinutes": 8,
  "aiCostModel": "per-token",
  "currency": "USD",
  "overheadMultiplier": 1.3,
  "agentOverrides": [
    {
      "agentId": "agent_abc123",
      "humanCostPerHour": 45.00,
      "avgHumanHandlingMinutes": 12
    }
  ]
}

Field	Type	Required	Description
`humanCostPerHour`	number	No	Fully loaded hourly cost of a human agent (default: 25.00)
`avgHumanHandlingMinutes`	number	No	Average minutes a human takes per interaction (default: 6)
`aiCostModel`	string	No	Cost model: `per-token`, `per-request`, `flat-rate`
`currency`	string	No	Currency code (default: `USD`)
`overheadMultiplier`	number	No	Multiplier for overhead costs (default: 1.0)
`agentOverrides`	array	No	Per-agent cost overrides

Response:

{
  "success": true,
  "data": {
    "configId": "roi_config_001",
    "updatedAt": "2025-03-10T14:00:00Z"
  }
}

SDK Usage:

await roiAnalytics.configure({
  humanCostPerHour: 35.00,
  avgHumanHandlingMinutes: 8,
  currency: 'USD'
});

Predicted Impact

Forecast future ROI based on current trends and planned changes.

GET /api/v1/roi-analytics/predicted-impact

Query Parameters:

Parameter	Type	Description
`horizon`	string	Prediction horizon: `30d`, `90d`, `6m`, `1y` (default: `90d`)
`agentId`	string	Filter to a specific agent
`scenario`	string	Scenario: `baseline`, `optimistic`, `conservative` (default: `baseline`)

Response:

{
  "success": true,
  "data": {
    "horizon": "90d",
    "scenario": "baseline",
    "predictedSavings": 38000.00,
    "predictedCost": 9600.00,
    "predictedNetROI": 28400.00,
    "confidence": 0.82,
    "projections": [
      { "month": "2025-04", "savings": 12500, "cost": 3200 },
      { "month": "2025-05", "savings": 13000, "cost": 3200 },
      { "month": "2025-06", "savings": 12500, "cost": 3200 }
    ],
    "assumptions": [
      "Volume growth rate: 5% month-over-month",
      "Agent success rate stable at 87%"
    ]
  }
}

SDK Usage:

const impact = await roiAnalytics.getPredictedImpact({
  horizon: '90d',
  scenario: 'optimistic'
});
 
console.log(`Predicted savings: $${impact.predictedSavings}`);
console.log(`Confidence: ${(impact.confidence * 100).toFixed(0)}%`);

Correlation Analysis

Analyze correlations between business metrics and AI agent performance.

GET /api/v1/roi-analytics/correlation

Query Parameters:

Parameter	Type	Description
`metricX`	string	Required. First metric name (e.g., `csat_score`)
`metricY`	string	Required. Second metric name (e.g., `tickets_resolved`)
`period`	string	Time period (default: `30d`)
`agentId`	string	Filter to a specific agent

Response:

{
  "success": true,
  "data": {
    "metricX": "csat_score",
    "metricY": "tickets_resolved",
    "correlation": 0.73,
    "pValue": 0.002,
    "significance": "strong",
    "dataPoints": 30,
    "summary": "Strong positive correlation between CSAT score and ticket resolution volume."
  }
}

SDK Usage:

const corr = await roiAnalytics.getCorrelation({
  metricX: 'csat_score',
  metricY: 'tickets_resolved',
  period: '30d'
});
 
console.log(`Correlation: ${corr.correlation} (${corr.significance})`);

ROI Trends

GET /api/v1/roi-analytics/trends

Query Parameters:

Parameter	Type	Description
`period`	string	Time period: `7d`, `30d`, `90d`, `ytd` (default: `30d`)
`interval`	string	Data granularity: `day`, `week`, `month` (default: `day`)
`agentId`	string	Filter to a specific agent

Response:

{
  "success": true,
  "data": {
    "period": "30d",
    "interval": "week",
    "trend": "improving",
    "changePercent": 12.3,
    "dataPoints": [
      {
        "timestamp": "2025-03-03T00:00:00Z",
        "savings": 2800,
        "cost": 780,
        "netROI": 2020,
        "interactions": 420
      },
      {
        "timestamp": "2025-03-10T00:00:00Z",
        "savings": 3100,
        "cost": 800,
        "netROI": 2300,
        "interactions": 465
      }
    ]
  }
}

Quality Metrics

Evaluate the quality of AI agent outputs with RAG scoring, hallucination detection, and batch evaluation.

RAG Quality Scores

Retrieve retrieval-augmented generation quality scores for a specific trace.

GET /api/v1/quality/rag-scores/:traceId

Path Parameters:

Parameter	Type	Description
`traceId`	string	Trace ID to evaluate

Response:

{
  "success": true,
  "data": {
    "traceId": "trace_xyz",
    "scores": {
      "faithfulness": 0.94,
      "relevance": 0.88,
      "contextPrecision": 0.91,
      "contextRecall": 0.85,
      "answerCorrectness": 0.90
    },
    "retrievedChunks": 5,
    "usedChunks": 3,
    "details": [
      {
        "claim": "The return policy allows 30-day returns.",
        "supported": true,
        "sourceChunk": "chunk_002",
        "confidence": 0.96
      },
      {
        "claim": "Free shipping is included on all orders.",
        "supported": false,
        "sourceChunk": null,
        "confidence": 0.12
      }
    ]
  }
}

SDK Usage:

const scores = await qualityMetrics.evaluateRAG({
  traceId: 'trace_xyz'
});
 
console.log(`Faithfulness: ${scores.scores.faithfulness}`);
console.log(`Unsupported claims: ${scores.details.filter(d => !d.supported).length}`);

scores = thinkhive.quality_metrics.evaluate_rag(
    trace_id='trace_xyz'
)
 
print(f"Faithfulness: {scores.scores.faithfulness}")
unsupported = [d for d in scores.details if not d.supported]
print(f"Unsupported claims: {len(unsupported)}")

curl https://app.thinkhive.ai/api/v1/quality/rag-scores/trace_xyz \
  -H "Authorization: Bearer th_your_api_key"

Detect Hallucinations

Analyze agent output for hallucinated or unsupported claims.

POST /api/v1/quality/hallucination-detection

Request Body:

{
  "output": "Our product supports 15 languages and has a 99.9% uptime guarantee.",
  "context": [
    "The product supports English, Spanish, French, German, and Japanese.",
    "Our SLA guarantees 99.5% uptime."
  ],
  "agentId": "agent_abc123",
  "traceId": "trace_xyz"
}

Field	Type	Required	Description
`output`	string	Yes	The agent output to analyze
`context`	string[]	Yes	Source documents or context the agent had access to
`agentId`	string	No	Agent that generated the output
`traceId`	string	No	Associated trace ID

Response:

{
  "success": true,
  "data": {
    "hallucinated": true,
    "score": 0.35,
    "claims": [
      {
        "text": "Our product supports 15 languages",
        "verdict": "hallucinated",
        "confidence": 0.92,
        "reason": "Context mentions only 5 languages (English, Spanish, French, German, Japanese)."
      },
      {
        "text": "has a 99.9% uptime guarantee",
        "verdict": "hallucinated",
        "confidence": 0.88,
        "reason": "SLA specifies 99.5% uptime, not 99.9%."
      }
    ],
    "summary": "2 of 2 claims are hallucinated. The output exaggerates language support and uptime guarantees."
  }
}

SDK Usage:

const result = await qualityMetrics.detectHallucinations({
  output: 'Our product supports 15 languages...',
  context: ['The product supports English, Spanish...'],
  agentId: 'agent_abc123'
});
 
if (result.hallucinated) {
  console.log(`Found ${result.claims.length} hallucinated claims`);
}

Evaluate Batch

Run quality evaluation across multiple traces at once.

POST /api/v1/quality/evaluate-batch

Request Body:

{
  "traceIds": ["trace_001", "trace_002", "trace_003"],
  "metrics": ["faithfulness", "relevance", "contextPrecision"],
  "agentId": "agent_abc123"
}

Field	Type	Required	Description
`traceIds`	string[]	Yes	List of trace IDs to evaluate (max 100)
`metrics`	string[]	No	Metrics to compute (default: all)
`agentId`	string	No	Filter traces by agent

Response:

{
  "success": true,
  "data": {
    "batchId": "batch_eval_001",
    "status": "completed",
    "total": 3,
    "summary": {
      "avgFaithfulness": 0.91,
      "avgRelevance": 0.85,
      "avgContextPrecision": 0.88,
      "passRate": 0.67
    },
    "results": [
      {
        "traceId": "trace_001",
        "scores": { "faithfulness": 0.95, "relevance": 0.90, "contextPrecision": 0.92 },
        "passed": true
      },
      {
        "traceId": "trace_002",
        "scores": { "faithfulness": 0.88, "relevance": 0.72, "contextPrecision": 0.80 },
        "passed": false
      },
      {
        "traceId": "trace_003",
        "scores": { "faithfulness": 0.90, "relevance": 0.93, "contextPrecision": 0.91 },
        "passed": true
      }
    ]
  }
}

SDK Usage:

const batch = await qualityMetrics.evaluateBatch({
  traceIds: ['trace_001', 'trace_002', 'trace_003'],
  metrics: ['faithfulness', 'relevance']
});
 
console.log(`Pass rate: ${(batch.summary.passRate * 100).toFixed(0)}%`);
console.log(`Avg faithfulness: ${batch.summary.avgFaithfulness}`);

Quality Summary

Get an aggregated quality overview across all agents or a specific agent.

GET /api/v1/quality/summary

Query Parameters:

Parameter	Type	Description
`agentId`	string	Filter to a specific agent
`period`	string	Time period: `7d`, `30d`, `90d` (default: `30d`)

Response:

{
  "success": true,
  "data": {
    "period": "30d",
    "totalEvaluations": 1250,
    "avgScores": {
      "faithfulness": 0.91,
      "relevance": 0.87,
      "contextPrecision": 0.89,
      "contextRecall": 0.83,
      "answerCorrectness": 0.88
    },
    "passRate": 0.82,
    "hallucinationRate": 0.08,
    "topIssues": [
      {
        "type": "low_relevance",
        "count": 45,
        "description": "Agent responses not fully addressing user queries"
      },
      {
        "type": "hallucination",
        "count": 28,
        "description": "Unsupported claims in product pricing responses"
      }
    ]
  }
}

Quality Trends

Track quality metrics over time.

GET /api/v1/quality/trends

Query Parameters:

Parameter	Type	Description
`period`	string	Time period: `7d`, `30d`, `90d` (default: `30d`)
`interval`	string	Data granularity: `day`, `week` (default: `day`)
`metric`	string	Specific metric to track (default: all)
`agentId`	string	Filter to a specific agent

Response:

{
  "success": true,
  "data": {
    "period": "30d",
    "interval": "week",
    "trends": {
      "faithfulness": [
        { "timestamp": "2025-03-03T00:00:00Z", "value": 0.89, "evaluations": 310 },
        { "timestamp": "2025-03-10T00:00:00Z", "value": 0.91, "evaluations": 325 }
      ],
      "relevance": [
        { "timestamp": "2025-03-03T00:00:00Z", "value": 0.84, "evaluations": 310 },
        { "timestamp": "2025-03-10T00:00:00Z", "value": 0.87, "evaluations": 325 }
      ]
    },
    "overallTrend": "improving",
    "changePercent": 2.8
  }
}

Error Responses

All endpoints return standard error responses:

{
  "success": false,
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Field 'name' is required",
    "details": {}
  }
}

Status Code	Error Code	Description
400	`VALIDATION_ERROR`	Invalid request body or query parameters
401	`UNAUTHORIZED`	Missing or invalid API key
403	`FORBIDDEN`	Insufficient permissions
404	`NOT_FOUND`	Resource not found
429	`RATE_LIMITED`	Too many requests (see rate limits)

Next Steps

Agents API - Manage your AI agents
Traces API - Query interaction traces
Evaluation Guide - Set up automated evaluations
RAG Evaluation Guide - Deep dive into RAG quality scoring
Hallucination Detection Guide - Detect and prevent hallucinations

Explainability & Analysis Webhooks & Notifications