Skip to content

Autoscale the worker on BullMQ queue depth with KEDA #19

Description

@FullFran

The reason Kubernetes earns its place in this project.

Worker load is a function of BullMQ queue depth, not of HTTP request rate.
A standard CPU-based HPA is the wrong signal: a worker blocked on network I/O
against slow feeds shows low CPU while the backlog grows.

Scope

  • Install KEDA in the cluster
  • ScaledObject for the worker Deployment using the Redis scaler against
    the BullMQ waiting list
  • Tune minReplicaCount, maxReplicaCount, target queue length per replica,
    cooldown and polling interval
  • Verify scale-to-zero when the queue is empty, and that a job arriving wakes it
  • Confirm graceful shutdown (see probes issue) so scale-down never drops an
    in-flight job
  • Compare against a plain CPU-based HPA and record why it is worse here

Experiment to run

Use benchmark:stages:mvp (100 / 1000 / 5000 / 10000 feeds) as the load source
and capture, for each stage:

  • queue depth over time
  • replica count over time
  • time to drain the backlog
  • end-to-end latency percentiles from feed publication to alert

Definition of done

A chart showing replicas tracking queue depth, with the backlog draining, and
the CPU-HPA comparison that justifies the choice.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestkubernetesKubernetes deployment and orchestration

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions