| Routing |
priority chains, cross-provider fallback, capability aliases (cheap, fast, long-context, vision), weighted canary splits |
| Money |
cache-aware token accounting, itemised cost, idempotent retries, lifetime/daily/monthly budgets, max_tokens clamping, model allowlists, CSV statements |
| Safety |
per-provider circuit breaker with Retry-After, request size limits, per-key concurrency caps, parameter-fidelity 422s |
| Operations |
Flyway migrations, readiness gating, shutdown drain, provider probe/health, admin audit log, webhook alerts, Prometheus metrics, OpenTelemetry tracing, ECS JSON logs, OpenAPI |
| Features |
exact-match response cache (streaming included), embeddings, opt-in redacted prompt logging, nightly provider canary |
| Scale and tenancy |
Valkey-backed shared RPM limiter, database budget reservations, projects, scoped admin tokens |
| Assurance |
Jazzer fuzzing of the request parser, property tests for translation invariants |
| Post-0.1.0 audit |
all twelve fixes (#38–#51): unpriced routes disabled, generic 500s with a reference id, chunked-body limits, admin auth limiter, master-key rotation, projectId filters, tag-on-main guard |
| Runnability |
tools/demo.sh (60-second tour against the mock), docs/WHY-PATH.md |
| Routing and billing (0.2) |
adaptive routing strategies (#63), operator-defined aliases (#68), project rate and concurrency limits (#65), project budget alerts (#69), POST /v1/tokens/count (#66), idempotent embeddings (#67) |
| Product (0.3) |
semantic cache tier (#71), guardrails: PII redaction and moderation (#70), end-user attribution and budgets (#64), token counting (#66), idempotent embeddings (#67) |
| Reach |
Azure OpenAI (#73) |
| Kubernetes |
deploy/helm/path (#72): drain-aware grace period enforced at render time, HPA, PDB, optional Postgres and Valkey |