SLO Manager

This tracks Service Level Objectives. It stores SLO definitions in PostgreSQL, evaluates them against live Prometheus queries, and computes error budgets and burn rates.

Core Concepts

SLI (Service Level Indicator) — a Prometheus query expression that returns a value between 0 and 1 (e.g. 1 - (rate(http_requests_total{status=~"5.."}[5m]) / rate(http_requests_total[5m]))).

SLO target — the required minimum SLI over the rolling window (e.g. 0.999 = 99.9%).

Error budget — the allowed fraction of bad events: 1 - target. Expressed as remaining percentage in the UI.

Burn rate — how fast error budget is being consumed: (1 - SLI) / (1 - target). A burn rate of 1.0 means the budget is being consumed exactly at target rate. Values above 2.0 indicate the budget will be exhausted before the window ends.

Evaluation Flow

On each GET /api/v1/slos/{id}/status call:

1. Loads the SLO definition from PostgreSQL
2. Executes the prom_query field against the configured Prometheus endpoint
3. Computes error_budget = (totalBudget - consumed) / totalBudget
4. Computes burn_rate = consumed / totalBudget
5. Sets breaching = current_sli < target
6. Returns the SLOStatus struct with all computed fields

SLO Types

Type	Description
availability	Fraction of successful requests
latency	Fraction of requests under a latency threshold
error_rate	Inverse error rate
throughput	Minimum request rate

REST API

Method	Path	Description
GET	/api/v1/slos	List all SLOs
POST	/api/v1/slos	Create SLO (name, service, type, target, prom_query, window_days)
GET	/api/v1/slos/summary	Summary: total, healthy, at_risk, breaching, with status for each SLO
GET	/api/v1/slos/{id}	SLO definition
GET	/api/v1/slos/{id}/status	Live evaluation (SLI, error budget, burn rate)
GET	/healthz	Health check

Example SLO Definition

{
  "name": "payments-api availability",
  "service": "payments-api",
  "namespace": "production",
  "type": "availability",
  "target": 0.999,
  "prom_query": "sum(rate(http_requests_total{app=\"payments-api\",status!~\"5..\"}[5m])) / sum(rate(http_requests_total{app=\"payments-api\"}[5m]))",
  "window_days": 30
}

Environment Variables

Variable	Default	Description
DATABASE_URL	—	PostgreSQL connection (optional — SLOs can be evaluated without persistence)
PROMETHEUS_URL	http://localhost:9090	Prometheus endpoint
PORT	8100	HTTP port