The reason Kubernetes earns its place in this project.
Worker load is a function of BullMQ queue depth, not of HTTP request rate.
A standard CPU-based HPA is the wrong signal: a worker blocked on network I/O
against slow feeds shows low CPU while the backlog grows.
Scope
Experiment to run
Use benchmark:stages:mvp (100 / 1000 / 5000 / 10000 feeds) as the load source
and capture, for each stage:
- queue depth over time
- replica count over time
- time to drain the backlog
- end-to-end latency percentiles from feed publication to alert
Definition of done
A chart showing replicas tracking queue depth, with the backlog draining, and
the CPU-HPA comparison that justifies the choice.
The reason Kubernetes earns its place in this project.
Worker load is a function of BullMQ queue depth, not of HTTP request rate.
A standard CPU-based HPA is the wrong signal: a worker blocked on network I/O
against slow feeds shows low CPU while the backlog grows.
Scope
ScaledObjectfor theworkerDeployment using the Redis scaler againstthe BullMQ waiting list
minReplicaCount,maxReplicaCount, target queue length per replica,cooldown and polling interval
in-flight job
Experiment to run
Use
benchmark:stages:mvp(100 / 1000 / 5000 / 10000 feeds) as the load sourceand capture, for each stage:
Definition of done
A chart showing replicas tracking queue depth, with the backlog draining, and
the CPU-HPA comparison that justifies the choice.