Problem
No performance benchmarks or SLA targets exist. We can't promise customers any latency guarantees.
Current state:
TraceBuilder tracks duration_ms but nothing consumes it
- No vitest benchmarks
- No CI gates for performance regression
- No p95/p99 measurements
Risk
- Can't set SLA for Pro/Enterprise tiers
- Performance regressions go unnoticed
- Customers experience unpredictable latency
Solution
Add comprehensive benchmarking:
// bench/evaluation.bench.ts
import { bench, describe } from 'vitest';
describe('evaluation latency', () => {
bench('small plan (10 resources)', async () => { ... });
bench('medium plan (100 resources)', async () => { ... });
bench('large plan (1000 resources)', async () => { ... });
bench('with state lookups (mocked)', async () => { ... });
});
Tasks
SLA Targets (Proposed)
| Scenario |
p50 |
p95 |
p99 |
| Small plan (≤10 resources) |
< 50ms |
< 100ms |
< 200ms |
| Medium plan (≤100 resources) |
< 200ms |
< 500ms |
< 1s |
| Large plan (≤1000 resources) |
< 1s |
< 3s |
< 5s |
| With live state (per resource) |
+50ms |
+100ms |
+200ms |
Priority: P1 — Required for Pro tier launch
Problem
No performance benchmarks or SLA targets exist. We can't promise customers any latency guarantees.
Current state:
TraceBuildertracksduration_msbut nothing consumes itRisk
Solution
Add comprehensive benchmarking:
Tasks
vitest benchconfiguration/metricsendpointSLA Targets (Proposed)
Priority: P1 — Required for Pro tier launch