Spec revision: 2.0.0 — aligned to the Bymax Lib Standard and to the current BullMQ API Last updated: 2026-06-23 Status: Draft for implementation Type: Public npm package (
@bymax-one/nest-queue) Target engine: BullMQ^5.16.0(validated against5.79.1, the current release at the time of writing) Origin: Consolidates and supersedes an internal hand-rolled BullMQ helper (~154 LoC) used in a production NestJS app, re-designed as a public, fully-typed library.
- Alignment with the Bymax Lib Standard
- Vision and Value Proposition
- Architecture
- Package Structure
- Configuration API
- Main Service
- Workers
- Flows
- Job Schedulers (repeatable / cron jobs)
- Metrics and Health Check
- Delivery Semantics and Shutdown Strategy
- Integration with
@bymax-one/nest-cache - Error Code Catalog
- What is NOT in the package
- Dependencies
- Implementation Phases
- Known Limitations
- Example Integration
- Testing Strategy and Quality Gates
- CI/CD and Release Engineering
- Versioning and Migration Policy
- Comparison with
@nestjs/bullmq
This spec follows the cross-portfolio Bymax public-library standard (the same one applied to @bymax-one/nest-logger, @bymax-one/nest-cache, and @bymax-one/nest-notification). The following are normative invariants for this package; where the body of the document and these invariants ever disagree, these win.
- Dynamic module via
ConfigurableModuleBuilder. The module is built on@nestjs/common'sConfigurableModuleBuilder, exposingforRoot()/forRootAsync(). Global scope is opted-in throughisGlobalmapped toDynamicModule.globalviasetExtras— not a hand-written@Global()decorator. - Quality floor. 100% line/branch coverage on every implemented file (enforced by
jest.coverage.config.tsthresholds set to100/100/100/100); Stryker mutation testing with{ high: 99, low: 95, break: 95 }, targeting 100%, run as a pre-release gate (not on every commit). - Bundle budget measured in KiB brotli by
scripts/check-size.mjs(not "KB gzipped"). - Recurring jobs use the current BullMQ Job Schedulers API (
upsertJobScheduler/removeJobScheduler/getJobSchedulers). The legacy "repeatable jobs" API (addRepeatable/removeRepeatable/getRepeatableJobs) is deprecated and removed in BullMQ v6 and must not appear in the public surface. - Observability and deduplication are first-class, not consumer responsibilities: the module accepts a BullMQ
telemetryinstance (OpenTelemetry viabullmq-otel) and exposes BullMQ nativededuplicationoptions throughenqueue. - English everywhere — code, comments, JSDoc, identifiers, commit messages, and every file under
docs/. - Repo-as-config files are mandatory deliverables and appear in the package tree:
SECURITY.md,CLAUDE.md,AGENTS.md,commitlint.config.cjs, and the four Copilot review files (.github/copilot-instructions.md,.github/instructions/code.instructions.md,.github/instructions/tests.instructions.md,.github/agents/agent-code-reviewer.agent.md). - Planning docs are English and kept in sync with this spec:
development_plan.mdholds the phased plan + status dashboard, and detailed executable tasks live one-file-per-phase underdocs/tasks/phase-NN-<slug>.md(the rust-auth layout). Each phase file carries a JIRA-style task index, acceptance criteria, exact file paths, and a self-contained agent prompt per task. - CI hard gates: lint + typecheck + 100% coverage, CodeQL, OpenSSF Scorecard ≥ 7.0, OSV-Scanner, TruffleHog OSS (secret scan), and npm publish with provenance (OIDC trusted publishing). GitHub Actions are pinned by commit SHA.
- A dogfood example app (
nest-queue-example) consumes the published package end-to-end before each release.
@bymax-one/nest-queue is a NestJS wrapper over BullMQ that standardizes the use of job queues across applications in the Bymax ecosystem. The package encapsulates queue creation, worker registration, flows (hierarchical jobs), Job Schedulers (recurring / cron jobs), and the Redis connection lifecycle in a dynamic module (forRoot / forRootAsync) with end-to-end strong typing.
The lib does not reimplement BullMQ — it orchestrates BullMQ inside the NestJS container:
- Resolves the Redis connection in dual mode (brings its own connection from
@bymax-one/nest-cacheor opens its own connection from config), applying the correctmaxRetriesPerRequestpolicy per connection role (see §2.3). - Applies consistent
defaultJobOptions(exponential retry, completed/failed job retention, priority). - Exposes a central
QueueServicewith typed helpers (enqueue<T>,enqueueBulk<T>,upsertJobScheduler,getJob,getJobs,getMetrics), including native BullMQ deduplication options onenqueue. - Offers the
@Processor()decorator and programmatic API to register workers, with safe defaults (concurrency,rate limiting) and both worker-local (@OnWorkerEvent, fullJob) and global (@OnQueueEvent) event hooks. - Supports Flow Producers (BullMQ Flows) for job graphs with parent/child dependencies.
- Accepts an optional OpenTelemetry
telemetryinstance so spans propagate fromenqueuethrough the handler. - Manages graceful shutdown (worker close with bounded drain, queue close, disconnect) under explicit at-least-once semantics.
In any NestJS architecture that uses BullMQ "by hand", there are five recurring frictions that this lib resolves:
- Redis connection dedicated to BullMQ: BullMQ's worker connection must set
maxRetriesPerRequest: nullbecause workers issue long-lived blocking commands (BRPOPLPUSH,BZPOPMIN,BLMOVE); BullMQ throws atWorkerconstruction otherwise. The plain Queue connection, by contrast, should keep ioredis' default retry count so a transient Redis outage makesenqueuefail fast instead of hanging. Getting this per-role policy right by hand — and not mixing it with the app's cache client — is error-prone. - Ad-hoc
defaultJobOptions: without a central place, each team defines retry, backoff, and retention inconsistently. - Workers without explicit
concurrency: BullMQ's default of 1 is silently serial. Without a pattern likeDEFAULT_WORKER_CONCURRENCY, nobody notices. - Weak job typing: most integrations fall into
Job<any, any>, losing compile-time checks. - Disordered shutdown: queues and workers are left open in
onModuleDestroy, holding the process onSIGTERM.
The lib resolves each of these frictions with sensible defaults and a typed API.
- NestJS applications that need asynchronous processing queues (email sending, report generation, AI processing, webhooks, etc.).
- Multi-tenant SaaS apps that need per-tenant prefix isolation (
prefix: 'tenant:foo:bullmq'). - Standalone workers running in separate containers consuming the same queues (the lib registers workers identically in any NestJS application).
| Aspect | Detail |
|---|---|
| Registry | Public npm (@bymax-one/nest-queue) |
| Cost | Zero — open source package |
| License | MIT |
| Runtime | Node.js 24+ |
| Framework | NestJS 11+ (server) |
| Engine | BullMQ ^5.16.0 (Job Schedulers API) |
| Subpaths | . (server), ./shared |
| Size | ≤ 18 KiB brotli (check-size.mjs target) |
- Configuration over convention — everything configurable via
forRoot()/forRootAsync(), with sensible defaults. - Dependency inversion over connection — the lib accepts a
Redis(ioredis) injected by another module (mode A) or opens its own connection from config (mode B). Never reads environment variables directly. - Zero opinion on persistence — BullMQ uses Redis as storage; the lib does not persist anything else. History, dashboards, and auditing are the consumer's responsibility.
- End-to-end typing —
enqueue<TData, TResult>(),Job<TData, TResult>, workers typed via decorator generics. - Safe defaults — retry with exponential backoff (3 attempts), minimum explicit
concurrencyof 2, retention of 24h for completed and 7d for failed. - No deep barrel re-exports — direct exports in
src/server/index.tsto ensure tree-shaking. - English in code, English in documentation — aligned with the
@bymax-one/nest-*portfolio standard. - Current API only — recurring jobs use BullMQ Job Schedulers (
upsertJobScheduler); deduplication and OpenTelemetry are first-class passthroughs, never re-implemented and never deferred to the consumer. The public surface never exposes a BullMQ method that BullMQ itself has deprecated.
The package organizes its surface into three layers with distinct activation levels:
| Component | Responsibility |
|---|---|
| BymaxQueueModule | Dynamic module with forRoot() / forRootAsync() |
| QueueService | Factory and cache of Queue<T>, helpers enqueue/getJob/getJobs/getMetrics |
| WorkerRegistry | Centralized registry of workers (programmatic and decorator-based) |
| ConnectionResolver | Resolves the Redis in mode A (injected) or mode B (own) |
| QueueLifecycle | onModuleDestroy orchestrates drain + close of queues and workers |
| Component | Activation | Responsibility |
|---|---|---|
| FlowService | flows: { enabled: true } |
Wrapper over FlowProducer for hierarchical jobs |
| MetricsService | metrics: { enabled: true } |
getJobCounts() collection + light cache for health checks |
| Decorator | Purpose |
|---|---|
| @Processor(queue) | Marks a class as a processor for a specific queue |
| @Process(jobName?) | Marks a method as a job handler (filters by optional jobName) |
| @OnWorkerEvent() | Worker-local listener — handler receives the full Job (completed, failed, progress, active) |
| @OnQueueEvent() | Global (cross-instance) listener via QueueEvents — receives jobId + serialized payload |
Principle: when
flowsormetricsare not configured, their providers are not registered in the NestJS container, avoiding overhead and unnecessary dependencies. The same applies toQueueEvents: a global-events connection is opened only when at least one@OnQueueEventis registered for a queue (worker-local@OnWorkerEventlisteners need no extra connection).
@bymax-one/nest-queue uses the NestJS Dynamic Module pattern. The lib is not a separate service — it runs inside each NestJS application as an imported module. The host application controls:
- The Redis connection (mode A: injects; mode B: lib opens)
- The global job defaults (
defaultJobOptions) - The Redis key prefix (
prefix) - Which workers are registered (via decorator or programmatically)
┌──────────────────────────────────────────────────┐
│ Host Application (NestJS) │
│ │
│ ┌────────────────────────────────────────────┐ │
│ │ @bymax-one/nest-queue module │ │
│ │ │ │
│ │ QueueService ←→ WorkerRegistry ←→ Redis │ │
│ │ ↕ ↕ ↕ │ │
│ │ FlowService @Processor() BullMQ │ │
│ │ ↕ ↕ │ │
│ │ MetricsService QueueLifecycle │ │
│ └────────┬───────────────────────────────────┘ │
│ │ │
│ ┌──────▼──────┐ (mode A) │
│ │ ioredis │ ←── injected by │
│ │ (shared) │ @bymax-one/nest-cache │
│ └─────────────┘ │
│ │
│ OR (mode B) │
│ │
│ ┌─────────────┐ │
│ │ ioredis │ ←── opened by the lib from │
│ │ (own) │ connection: { ... } │
│ └─────────────┘ │
└──────────────────────────────────────────────────┘
The lib accepts two mutually exclusive modes of providing the Redis connection to BullMQ. The choice is made by the presence of one of the two keys in BymaxQueueModuleOptions:
The application injects an already-configured Redis (typically coming from @bymax-one/nest-cache.getClientForQueue()). The lib does not create nor close that connection — it only uses it.
BymaxQueueModule.forRootAsync({
imports: [BymaxCacheModule],
inject: [BYMAX_CACHE_QUEUE_REDIS],
useFactory: (queueRedis: Redis) => ({
connection: { client: queueRedis },
defaultJobOptions: { attempts: 3 }
})
})The lib receives connection parameters (URL or RedisOptions object) and internally creates a dedicated ioredis. The connection is closed in onModuleDestroy.
BymaxQueueModule.forRoot({
connection: {
url: process.env.REDIS_URL,
// or:
host: 'localhost',
port: 6379,
db: 1,
// Queue connection keeps default retries; the lib duplicates this with
// maxRetriesPerRequest:null for worker/QueueEvents connections (see §2.3)
},
defaultJobOptions: { attempts: 3 }
})Important — per-role retry policy. The lib does not blindly force
maxRetriesPerRequest: nullon every connection. It applies the BullMQ-recommended policy per connection role:
- Queue / FlowProducer connection — keeps the default retry count so that
enqueuefails fast during a Redis outage rather than blocking the caller indefinitely.- Worker / QueueEvents connection — must be
maxRetriesPerRequest: null. The lib obtains these viaclient.duplicate({ maxRetriesPerRequest: null }), so the override is applied even when the shared client (mode A) was created with a different value.In mode A,
onModuleInitinspects the injected client: if a worker is going to be registered and the duplicated worker connection cannot be coerced tonull(e.g. a wrapper that ignores the override), the lib fails fast withqueue.connection_requires_null_retriesinstead of letting BullMQ crash later atWorkerconstruction. See §2.3.
BullMQ requires separate connections for blocking consumers: a Worker (and each QueueEvents) holds a connection busy on long-lived blocking commands (BRPOPLPUSH, BZPOPMIN, BLMOVE), so it cannot share the same socket as the non-blocking Queue. The lib implements the following rules:
| Component | Connection used (mode A) | Connection used (mode B) | maxRetriesPerRequest |
|---|---|---|---|
Queue |
Received client (shared across all Queues) | Main client opened by the lib | default (fail-fast) |
FlowProducer |
Received client | Main client opened by the lib | default (fail-fast) |
Worker |
client.duplicate({ maxRetriesPerRequest: null }) (one per worker) |
client.duplicate({ maxRetriesPerRequest: null }) (one per worker) |
null (blocking) |
QueueEvents |
client.duplicate({ maxRetriesPerRequest: null }) (one per queue) |
client.duplicate({ maxRetriesPerRequest: null }) (one per queue) |
null (blocking) |
Connection duplication — and the per-role maxRetriesPerRequest override — is internally managed by ConnectionResolver. The consumer does not need to think about it.
- The application calls
BymaxQueueModule.forRootAsync({ ... }). - The module resolves the options via factory or ConfigService.
ConnectionResolverdecides the mode (A or B) by inspectingoptions.connection.- In mode A, it validates the received
Redisis usable for the Queue role (status === 'ready' | 'connecting'). It does not requiremaxRetriesPerRequest === nullon the received client — the Queue role keeps default retries and worker connections are duplicated withnull(see §2.3). It fails fast only if a worker connection cannot be coerced tonull. - In mode B, it opens
ioredisfor the Queue role and waits for thereadyevent (with a fixed 10s connect timeout, after which it fails fast withqueue.connection_timeout). - Providers are conditionally registered:
FlowServiceonly ifflows.enabled;MetricsServiceonly ifmetrics.enabled. If atelemetryinstance is configured, it is passed to everyQueue/Workerconstructed. - The module applies
discover()(via NestJSDiscoveryService) to find classes annotated with@Processor()and register their workers. - The module is ready to receive
enqueue()calls and process jobs.
Application calls enqueue<TData, TResult>(queueName, jobName, data)
│
▼
QueueService.getOrCreateQueue(queueName)
│
▼
Queue<TData, TResult>.add(jobName, data, { ...defaultJobOptions, ...opts })
│
▼ (BullMQ persists in Redis)
│
▼ (Registered worker pulls the job)
WorkerRegistry → Processor class → @Process method
│
▼
Method returns TResult (or throws)
│
▼
BullMQ marks as completed/failed and emits events
│
▼ (optional)
QueueEvents listener (@OnQueueEvent) runs side-effect
The package is organized in 2 subpaths with distinct responsibilities:
@bymax-one/nest-queue/
├── package.json
├── tsconfig.json
├── tsconfig.build.json
├── tsconfig.server.json
├── tsconfig.e2e.json
├── tsconfig.jest.json
├── tsup.config.ts
├── src/
│ ├── server/ # NestJS backend
│ │ ├── index.ts # Barrel export (server)
│ │ ├── bymax-queue.module.ts # Root dynamic module
│ │ ├── bymax-queue.constants.ts # Injection tokens (Symbol)
│ │ ├── interfaces/
│ │ │ ├── queue-module-options.interface.ts
│ │ │ ├── queue-connection.interface.ts
│ │ │ ├── queue-job-data.interface.ts
│ │ │ ├── worker-options.interface.ts
│ │ │ └── processor-metadata.interface.ts
│ │ ├── config/
│ │ │ ├── default-options.ts # DEFAULT_JOB_OPTIONS, DEFAULT_WORKER_CONCURRENCY
│ │ │ └── resolved-options.ts # mergeOptions(user, defaults)
│ │ ├── services/
│ │ │ ├── queue.service.ts # Public — enqueue, getJob, getMetrics
│ │ │ ├── worker-registry.service.ts # Internal — register/unregister workers
│ │ │ ├── flow.service.ts # Opt-in — FlowProducer wrapper
│ │ │ ├── metrics.service.ts # Opt-in — getJobCounts cache
│ │ │ └── connection-resolver.service.ts # Internal — mode A/B resolution
│ │ ├── decorators/
│ │ │ ├── processor.decorator.ts # @Processor(queueName)
│ │ │ ├── process.decorator.ts # @Process(jobName?)
│ │ │ ├── on-worker-event.decorator.ts # @OnWorkerEvent('completed' | ...) — full Job
│ │ │ └── on-queue-event.decorator.ts # @OnQueueEvent('completed' | ...) — global
│ │ ├── lifecycle/
│ │ │ └── queue-lifecycle.service.ts # onModuleDestroy: drain + close
│ │ ├── errors/
│ │ │ ├── queue-error-codes.ts # QUEUE_ERROR_MESSAGES + QUEUE_ERROR_STATUS (code→HTTP); re-exports QUEUE_ERROR_CODES from shared
│ │ │ └── queue-exception.ts # QueueException
│ │ └── utils/
│ │ ├── duplicate-connection.ts # ioredis duplicate({ maxRetriesPerRequest: null }) for blocking conns
│ │ └── validate-connection.ts # assert duplicated worker conn resolves to maxRetriesPerRequest === null
│ │
│ └── shared/ # Types and constants (zero deps)
│ ├── index.ts
│ ├── types/
│ │ ├── job-status.types.ts # JobStatus union
│ │ ├── queue-metrics.types.ts # QueueMetrics interface
│ │ └── job-scheduler-options.types.ts # JobSchedulerRepeatOptions
│ └── constants/
│ ├── job-status.ts # JOB_STATUS constants
│ └── error-codes.ts # QUEUE_ERROR_CODES
│
├── test/ # E2E tests with testcontainers
│ └── queue.e2e-spec.ts
├── scripts/
│ └── check-size.mjs # bundle budget gate (KiB brotli)
├── .github/
│ ├── workflows/ # ci.yml, codeql.yml, release.yml, scorecard.yml, osv-scanner.yml
│ ├── copilot-instructions.md # repo-wide Copilot review config
│ ├── instructions/
│ │ ├── code.instructions.md
│ │ └── tests.instructions.md
│ └── agents/
│ └── agent-code-reviewer.agent.md
├── docs/
│ ├── technical_specification.md
│ ├── development_plan.md # phased plan + status dashboard
│ ├── mutation_testing_plan.md
│ ├── mutation_testing_results.md
│ └── tasks/ # one file per phase (executable agent prompts)
│ ├── phase-01-foundation.md
│ ├── phase-02-workers.md
│ ├── phase-03-flows-schedulers-metrics.md
│ ├── phase-04-async-shutdown-e2e.md
│ └── phase-05-release.md
├── eslint.config.mjs
├── jest.config.ts
├── jest.coverage.config.ts # thresholds 100/100/100/100
├── jest.e2e.config.ts
├── jest.stryker.config.ts
├── stryker.config.json # break 95, high 99, low 95
├── commitlint.config.cjs # Conventional Commits
├── SECURITY.md
├── CLAUDE.md
├── AGENTS.md
├── README.md
├── LICENSE
└── CHANGELOG.md
The package uses the exports field of package.json to expose two entry points with automatic tree-shaking:
| Subpath | Entry point | Description | Dependencies |
|---|---|---|---|
. (server) |
dist/server/index.js |
NestJS module, services, decorators, FlowProducer | NestJS 11, bullmq, ioredis |
./shared |
dist/shared/index.js |
Types and constants (job status, error codes, metrics) | Zero |
{
"exports": {
".": {
"types": "./dist/server/index.d.ts",
"import": "./dist/server/index.mjs",
"require": "./dist/server/index.cjs"
},
"./shared": {
"types": "./dist/shared/index.d.ts",
"import": "./dist/shared/index.mjs",
"require": "./dist/shared/index.cjs"
}
}
}Server (@bymax-one/nest-queue):
// Main module
export { BymaxQueueModule } from './bymax-queue.module'
// Injection tokens (Symbol)
export {
BYMAX_QUEUE_OPTIONS,
BYMAX_QUEUE_REDIS_CLIENT,
BYMAX_QUEUE_CONNECTION_MODE
} from './bymax-queue.constants'
// Public services
export { QueueService } from './services/queue.service'
export { FlowService } from './services/flow.service'
export { MetricsService } from './services/metrics.service'
export { WorkerRegistry } from './services/worker-registry.service' // advanced — dynamic workers
// Decorators
export { Processor } from './decorators/processor.decorator'
export { Process } from './decorators/process.decorator'
export { OnWorkerEvent } from './decorators/on-worker-event.decorator'
export { OnQueueEvent } from './decorators/on-queue-event.decorator'
// Interfaces (types)
export type {
BymaxQueueModuleOptions,
BymaxQueueModuleAsyncOptions,
QueueConnectionConfig,
WorkerOptions,
ProcessorMetadata
} from './interfaces'
// Errors and constants
export { QueueException } from './errors/queue-exception'
// QUEUE_ERROR_CODES is defined once in ./shared (zero-dep) and re-exported by the
// server barrel; QUEUE_ERROR_MESSAGES (the human-readable map) lives with the codes.
export { QUEUE_ERROR_CODES, QUEUE_ERROR_MESSAGES } from './errors/queue-error-codes'
export { DEFAULT_WORKER_CONCURRENCY, DEFAULT_JOB_OPTIONS } from './config/default-options'
// Re-exports BullMQ types for convenience (we do not duplicate the lib)
export type {
Job, JobsOptions, Queue, Worker, FlowProducer, FlowJob, JobNode,
JobSchedulerJson, Telemetry, SandboxedJob
} from 'bullmq'Shared (@bymax-one/nest-queue/shared):
// Types (zero deps)
export type { JobStatus, QueueMetrics, JobSchedulerRepeatOptions } from './types'
// Constants (zero deps) — QUEUE_ERROR_CODES is the single source of truth,
// re-exported by the server barrel above.
export { JOB_STATUS, QUEUE_ERROR_CODES } from './constants'Public vs internal API:
QueueService,FlowService(if enabled), andMetricsService(if enabled) are the primary public services.WorkerRegistryis also exported, but as an advanced surface — itsregister/registerSandboxed/unregister/listmethods exist for dynamic (e.g. multi-tenant) worker creation (§6.6); most apps use the@Processordecorator instead.ConnectionResolverandQueueLifecycleare internal and must not be accessed.
This is the main interface that controls all module behavior. The host application provides these options when registering the module.
import type { JobsOptions, QueueOptions, Telemetry } from 'bullmq'
import type { Redis, RedisOptions } from 'ioredis'
export interface BymaxQueueModuleOptions {
/**
* Register the module globally. When true, the module's providers
* (QueueService, FlowService, MetricsService) are available app-wide without
* re-importing. Mapped to DynamicModule.global by ConfigurableModuleBuilder.setExtras.
* Default: true (QueueService is intended as an app-wide singleton).
*/
isGlobal?: boolean
/**
* Redis connection configuration. REQUIRED.
*
* Mutually exclusive between Mode A (bring your own client) and Mode B (lib opens its own).
*/
connection: QueueConnectionConfig
/**
* Default options applied to every job enqueued through QueueService.
*
* Per-job overrides are passed in the third argument of enqueue().
*
* Default:
* attempts: 3,
* backoff: { type: 'exponential', delay: 2000 },
* removeOnComplete: { age: 24 * 3600, count: 1000 },
* removeOnFail: { age: 7 * 24 * 3600, count: 5000 }
*/
defaultJobOptions?: JobsOptions
/**
* Optional prefix for all Redis keys.
* Useful for multi-tenant isolation (e.g. 'tenant:foo:bullmq').
* Default: 'bull' (BullMQ default)
*/
prefix?: string
/**
* Default options applied to every Queue created by QueueService.
* Merged with `defaultJobOptions` and `prefix`.
*/
queueOptions?: Partial<Omit<QueueOptions, 'connection' | 'defaultJobOptions' | 'prefix'>>
/**
* Flow Producer configuration. Opt-in.
*/
flows?: {
/** Enables FlowService registration. Default: false */
enabled?: boolean
}
/**
* Metrics collection configuration. Opt-in.
*/
metrics?: {
/** Enables MetricsService registration. Default: false */
enabled?: boolean
/** Cache TTL in milliseconds for getJobCounts(). Default: 5000 */
cacheTtlMs?: number
}
/**
* OpenTelemetry instrumentation. Opt-in.
*
* Pass a BullMQ `Telemetry` implementation — typically `new BullMQOtel(...)`
* from the `bullmq-otel` package. When provided, it is attached to every Queue
* and Worker the lib creates, so trace context propagates from enqueue() into
* the job handler (and on to any child jobs in a flow).
*
* `bullmq-otel` is an OPTIONAL peer dependency — only install it if you set this.
*/
telemetry?: Telemetry
/**
* Lifecycle behavior on application shutdown.
*/
shutdown?: {
/**
* Max time (ms) to wait for in-flight jobs to finish before forcing close.
* Default: 30_000 (30s)
*/
drainTimeoutMs?: number
/**
* If true, the lib calls Queue.drain() during shutdown (removes all waiting/delayed jobs).
* DANGER: only enable in dev/test — production should let jobs survive the restart.
* Default: false
*/
drainOnShutdown?: boolean
}
}
/**
* Connection configuration — Mode A or Mode B.
*
* Mode A: client is provided externally (typically from @bymax-one/nest-cache).
* Mode B: lib opens its own ioredis using url or options.
*/
export type QueueConnectionConfig =
| { client: Redis; ownsConnection?: false } // Mode A
| { url: string; options?: Partial<RedisOptions> } // Mode B (url)
| { options: RedisOptions } // Mode B (options)For asynchronous configuration (factory that depends on other modules):
import type { ModuleMetadata, Type } from '@nestjs/common'
export interface BymaxQueueOptionsFactory {
createQueueOptions(): Promise<BymaxQueueModuleOptions> | BymaxQueueModuleOptions
}
export interface BymaxQueueModuleAsyncOptions extends Pick<ModuleMetadata, 'imports'> {
/** Factory function that returns options */
useFactory?: (...args: unknown[]) => Promise<BymaxQueueModuleOptions> | BymaxQueueModuleOptions
/** Class implementing BymaxQueueOptionsFactory */
useClass?: Type<BymaxQueueOptionsFactory>
/** Existing provider implementing BymaxQueueOptionsFactory */
useExisting?: Type<BymaxQueueOptionsFactory>
/** Providers to inject into the factory */
inject?: Array<Type<unknown> | string | symbol>
}The module is built on NestJS' ConfigurableModuleBuilder, which generates a typed ConfigurableModuleClass exposing register/registerAsync (re-exported here as forRoot/forRootAsync for portfolio consistency). Global scope is opted-in through the isGlobal option, mapped to DynamicModule.global via setExtras — there is no hand-written @Global() decorator.
import { ConfigurableModuleBuilder } from '@nestjs/common'
export const {
ConfigurableModuleClass,
MODULE_OPTIONS_TOKEN: BYMAX_QUEUE_OPTIONS,
OPTIONS_TYPE,
ASYNC_OPTIONS_TYPE
} = new ConfigurableModuleBuilder<BymaxQueueModuleOptions>({ moduleName: 'BymaxQueue' })
.setClassMethodName('forRoot')
.setExtras(
{ isGlobal: true },
(definition, extras) => ({ ...definition, global: extras.isGlobal })
)
.build()
export class BymaxQueueModule extends ConfigurableModuleClass {
/**
* Synchronous registration. Use when options are static and known at module load time.
* Provided by ConfigurableModuleClass; declared here for documentation only.
*/
static forRoot(options: typeof OPTIONS_TYPE): DynamicModule
/**
* Asynchronous registration. Use when options depend on other modules
* (ConfigService, BymaxCacheModule, etc.).
*/
static forRootAsync(options: typeof ASYNC_OPTIONS_TYPE): DynamicModule
}No
forFeaturestub. Earlier drafts exposed a no-opforFeature(). Shipping a documented public method that does nothing is a credibility risk, so it is removed.QueueServiceis an app-wide singleton (isGlobal: trueby default) and reaches every queue by name; per-queue injection tokens (@InjectQueue('email')) are tracked as a future, additive feature and will be introduced as a realforFeatureonly when they do something.
The lib applies the following defaults when fields are not informed:
export const DEFAULT_WORKER_CONCURRENCY = 2 as const
export const DEFAULT_JOB_OPTIONS = {
attempts: 3,
backoff: { type: 'exponential', delay: 2000 },
removeOnComplete: {
age: 24 * 3600, // 24 hours
count: 1000 // last 1000 completed jobs
},
removeOnFail: {
age: 7 * 24 * 3600, // 7 days
count: 5000 // cap retained failed jobs to bound Redis memory under high failure rates
}
} as const satisfies JobsOptionsMemory note:
removeOnFailkeeps failed jobs for debugging, but an unboundedage-only policy lets a failure storm grow Redis memory without limit. Thecountcap is the backstop. Tune both per workload (see §16.6 on backpressure and payload size).
Decision:
DEFAULT_WORKER_CONCURRENCY = 2mirrors a value validated in a production NestJS workload. Workers must always pass this value (or a lower justified one) to the decorator/factory, and tune it by workload type (§6.7).
QueueService is the central interaction point with queues. It:
- Keeps an internal cache (
Map<string, Queue>) of created queues — one queue per name. - Applies
defaultJobOptionsandprefixfrom the configuration to every new queue. - Exposes typed helpers for enqueue, lookup, and metrics.
import { Injectable } from '@nestjs/common'
import { Job, JobsOptions, Queue, QueueOptions } from 'bullmq'
import type { Redis } from 'ioredis'
@Injectable()
export class QueueService {
private readonly queues = new Map<string, Queue>()
constructor(
private readonly redis: Redis,
private readonly options: ResolvedQueueOptions
) {}
// see methods below
}All methods exposed by QueueService are generic in TData (job payload) and optionally TResult (handler return).
// Domain types (defined by consumer)
interface SendEmailJobData {
to: string
templateId: string
variables: Record<string, string>
}
interface SendEmailJobResult {
messageId: string
acceptedAt: string
}
// Enqueue with full type inference
const job = await queueService.enqueue<SendEmailJobData, SendEmailJobResult>(
'email', // queueName
'send-welcome', // jobName
{ to: 'user@x.com', templateId: 'welcome', variables: {} }, // data — typed!
{ priority: 5 } // options
)
// job: Job<SendEmailJobData, SendEmailJobResult, 'send-welcome'>Returns the existing Queue or creates a new one with the default configuration.
/**
* Returns the existing Queue for `queueName` or creates a new one.
*
* Subsequent calls with the same name return the cached instance, sharing
* the same Redis connection and default options. Per-queue overrides via
* the second argument are merged on top of the module defaults.
*
* @param queueName - Unique queue name (used as Redis key prefix)
* @param overrides - Optional QueueOptions overrides (excluding connection/prefix)
* @returns Typed BullMQ Queue
*/
getOrCreateQueue<TData = unknown, TResult = unknown>(
queueName: string,
overrides?: Partial<Omit<QueueOptions, 'connection' | 'prefix'>>
): Queue<TData, TResult>Adds a single job to a queue.
/**
* Adds a job to the queue. Creates the queue lazily if it doesn't exist.
*
* @param queueName - Target queue
* @param jobName - Job identifier used by @Process to dispatch
* @param data - Typed payload (TData)
* @param options - Per-job overrides: priority, delay, removeOnComplete,
* jobId (idempotent insert), and `deduplication` (see 5.4.1).
* @returns The created Job
*/
async enqueue<TData = unknown, TResult = unknown>(
queueName: string,
jobName: string,
data: TData,
options?: JobsOptions
): Promise<Job<TData, TResult, string>>Two distinct, complementary mechanisms — both native to BullMQ and surfaced through options (no custom code in this lib):
-
jobId(idempotent insert). If you setoptions.jobId, adding a second job with the same id is a no-op while the first still exists. Best for "exactly one job for this entity id" (e.g.jobId: \welcome:${userId}``). -
deduplication(windowed deduplication). BullMQ'sdeduplicationoption collapses repeated enqueues of the same logical work within a window. Modes:Mode Options Behavior Simple { id }Deduplicate until the in-flight job completes/fails. Throttle { id, ttl }Ignore duplicates for ttlms.Debounce { id, ttl, extend: true, replace: true }Keep only the latest data; each duplicate resets the TTL (use with delay).Keep-last-if-active { id, keepLastIfActive: true }While a job is processing, store the latest data and auto-create one follow-up job when it finishes. // Throttle: at most one index-rebuild per search term every 5s await queue.enqueue('search', 'reindex', { term }, { deduplication: { id: `reindex:${term}`, ttl: 5_000 } })
This replaces the previous draft's "global deduplication is the consumer's job" stance — it is now a first-class, documented feature. The deduplication key is independent of
jobId; inspect/clear it with the nativegetDeduplicationJobId/removeDeduplicationKey.
Adds multiple jobs in batch — a single Redis roundtrip.
interface BulkJob<TData> {
name: string
data: TData
options?: JobsOptions
}
/**
* Enqueues multiple jobs in a single Redis roundtrip.
*
* Useful for fan-out scenarios (e.g. send 100 emails) where the cost
* of 100 round trips is significant.
*
* The batch is bounded by MAX_BULK_SIZE (default 1000) to guard against a
* self-inflicted DoS / Redis-memory spike; exceeding it throws
* `queue.bulk_enqueue_failed` before anything is enqueued. Split larger
* fan-outs into chunks.
*
* @param queueName - Target queue
* @param jobs - Array of job descriptors (length <= MAX_BULK_SIZE)
* @returns Array of created Jobs in the same order
*/
async enqueueBulk<TData = unknown, TResult = unknown>(
queueName: string,
jobs: ReadonlyArray<BulkJob<TData>>
): Promise<Array<Job<TData, TResult, string>>>Creates or updates a Job Scheduler — the current BullMQ API for recurring (cron / interval) jobs. Full details in §8.
/**
* Creates or updates a Job Scheduler (recurring job). Idempotent by `schedulerId`:
* calling it again with the same id atomically updates the schedule and the job
* template (BullMQ performs an `override: true` upsert). This supersedes the
* deprecated `addRepeatable` API, which BullMQ removes in v6.
*
* @param queueName - Target queue
* @param schedulerId - Stable, unique scheduler identifier (the upsert key)
* @param repeat - Schedule: cron `pattern` OR `every` (ms), plus tz/limit/start/end
* @param template - Optional job template ({ name, data, opts }) applied to every run.
* `name` defaults to `schedulerId`; `data` defaults to `{}`.
* @returns The first scheduled (delayed) Job, or undefined
*/
async upsertJobScheduler<TData = unknown, TResult = unknown>(
queueName: string,
schedulerId: string,
repeat: JobSchedulerRepeatOptions,
template?: { name?: string; data?: TData; opts?: JobsOptions }
): Promise<Job<TData, TResult, string> | undefined>/**
* Fetches a job by id. Returns null if not found.
*/
async getJob<TData = unknown, TResult = unknown>(
queueName: string,
jobId: string
): Promise<Job<TData, TResult, string> | null>
/**
* Fetches jobs by status with pagination.
*
* @param queueName - Target queue
* @param status - Job status filter (waiting | active | completed | failed | delayed | paused)
* @param start - Starting index (inclusive). Default: 0
* @param end - Ending index (inclusive). Default: 50
*/
async getJobs<TData = unknown, TResult = unknown>(
queueName: string,
status: JobStatus,
start?: number,
end?: number
): Promise<Array<Job<TData, TResult, string>>>Returns a snapshot of job counters. Detailed implementation in §9.
/**
* Returns job counts grouped by status for a queue.
*
* If metrics caching is enabled (options.metrics.enabled), this method uses
* the MetricsService cache (TTL = metrics.cacheTtlMs).
*/
async getMetrics(queueName: string): Promise<QueueMetrics>Return shape:
export interface QueueMetrics {
queue: string
counts: {
waiting: number
active: number
completed: number
failed: number
delayed: number
paused: number
}
collectedAt: string // ISO timestamp
}async pauseQueue(queueName: string): Promise<void>
async resumeQueue(queueName: string): Promise<void>
/**
* Removes up to `limit` jobs older than `gracePeriodMs` in a given status.
* Thin wrapper over BullMQ `Queue.clean(grace, limit, type?)` — the argument
* order mirrors BullMQ exactly: `limit` is required and precedes the status,
* and `0` means "no limit". Returns the ids of the removed jobs.
*/
async cleanQueue(
queueName: string,
gracePeriodMs: number,
limit: number,
status?: 'completed' | 'failed' | 'delayed' | 'wait' | 'active' | 'paused'
): Promise<string[]>Workers can be registered in two ways, both supported simultaneously:
- Via the
@Processor(queueName)decorator — recommended. The lib discovers annotated classes inonModuleInitvia NestJSDiscoveryServiceand creates the workers automatically. - Via the programmatic API (
WorkerRegistry.register()) — for dynamic cases (creating workers at runtime based on external config).
import { Processor, Process, OnQueueEvent } from '@bymax-one/nest-queue'
@Processor('email', { concurrency: 5, limiter: { max: 10, duration: 1000 } })
export class EmailProcessor {
constructor(private readonly mailer: MailerService) {}
// Handler for all jobs in this queue (no jobName filter)
@Process()
async handleAny(job: Job<SendEmailJobData, SendEmailJobResult>): Promise<SendEmailJobResult> {
const result = await this.mailer.send(job.data)
return { messageId: result.id, acceptedAt: new Date().toISOString() }
}
// OR: handler filtered by specific jobName
@Process('send-welcome')
async handleWelcome(job: Job<SendEmailJobData>): Promise<SendEmailJobResult> {
// ...
}
// Worker-local event listeners — receive the FULL Job (job.data, timings,
// attemptsMade), processed in this worker's process. Best for logging,
// progress, and per-job side-effects.
@OnWorkerEvent('completed')
onCompleted(job: Job<SendEmailJobData, SendEmailJobResult>): void {
// job.data, job.returnvalue, job.finishedOn, job.attemptsMade all available
}
@OnWorkerEvent('failed')
onFailed(job: Job<SendEmailJobData> | undefined, error: Error): void {
// job may be undefined if it failed before the worker fetched it
}
}/**
* Marks a class as a queue processor.
*
* The class is instantiated by the NestJS DI container, so it can inject
* any provider available in the host application's module graph.
*
* @param queueName - Queue this processor consumes from
* @param workerOptions - Worker configuration (concurrency, limiter, etc.)
*/
export function Processor(queueName: string, workerOptions?: WorkerOptions): ClassDecoratorexport interface WorkerOptions {
/**
* Maximum number of concurrent jobs processed by this worker.
* Default: DEFAULT_WORKER_CONCURRENCY (2)
*/
concurrency?: number
/**
* Rate limiter — max N jobs per duration ms.
* Example: { max: 10, duration: 1000 } = 10 jobs/sec max.
*/
limiter?: {
max: number
duration: number
}
// NOTE: there is intentionally no `sandboxed` boolean here. BullMQ sandboxed
// processors are created by passing a FILE PATH (not a class) to the Worker
// constructor, so they run in a separate process/thread and CANNOT use NestJS
// dependency injection. A boolean toggle on a DI-managed @Processor class is
// therefore impossible. Sandboxed CPU-bound work has its own registration
// path — see §6.8.
/**
* Auto-run the worker on registration. Default: true.
* Set to false to manually call worker.run() later.
*/
autorun?: boolean
/**
* Lock duration for active jobs. Default: 30_000 (30s).
* Increase if your handler takes longer than 30s.
*/
lockDuration?: number
/**
* Stalled job check interval. Default: 30_000 (30s).
*/
stalledInterval?: number
}/**
* Marks a method as a job handler within a @Processor class.
*
* If jobName is omitted, the handler receives all jobs of the queue.
* If jobName is provided, only jobs with matching name are dispatched here.
*
* Multiple @Process methods are allowed in the same class. Dispatch order:
* 1. Method with matching jobName (most specific)
* 2. Method without jobName (fallback)
*
* @param jobName - Optional job name filter
*/
export function Process(jobName?: string): MethodDecoratorBullMQ exposes events at two levels, and this lib surfaces both. Choosing the right one matters: the worker-local listener hands you the full Job; the global one does not.
// Worker-local events — fired by THIS worker's process. Handler receives the
// full Job object (job.data, job.returnvalue, job.attemptsMade, timings).
type WorkerEventName = 'completed' | 'failed' | 'progress' | 'active' | 'stalled' | 'closing' | 'closed' | 'error'
/**
* Marks a method as a worker-local event listener. No extra Redis connection
* is required — these fire on the Worker the @Processor already owns.
* Signatures mirror BullMQ Worker events:
* completed(job) failed(job | undefined, err)
* progress(job, progress) active(job) stalled(jobId)
*/
export function OnWorkerEvent(eventName: WorkerEventName): MethodDecorator
// Global events — fired across ALL instances via a dedicated QueueEvents
// connection. Handler receives only ids + serialized strings, NOT the Job.
type QueueEventName = 'completed' | 'failed' | 'active' | 'progress' | 'waiting' | 'delayed' | 'stalled' | 'paused' | 'resumed' | 'cleaned' | 'drained'
/**
* Marks a method as a global queue-event listener. The lib lazily creates ONE
* QueueEvents instance per queue (on a duplicated, maxRetriesPerRequest:null
* connection) only when at least one @OnQueueEvent is registered for it.
* Signatures mirror BullMQ QueueEvents — e.g.:
* completed({ jobId, returnvalue }) failed({ jobId, failedReason })
* Note: `returnvalue`/`failedReason` are STRINGS (serialized), and the key is
* lowercase `returnvalue`. To get the Job, call queueService.getJob(jobId)
* (which may be null if removeOnComplete already evicted it).
*/
export function OnQueueEvent(eventName: QueueEventName): MethodDecorator| Use case | Decorator |
|---|---|
Log the completed/failed job with its data, timings, attempts |
@OnWorkerEvent (no round-trip, full Job) |
Report/observe job progress updates |
@OnWorkerEvent('progress') |
| React to events from jobs processed on other instances | @OnQueueEvent |
| Cross-instance dashboards / drained-queue signals | @OnQueueEvent |
Migration note: apps coming from
@nestjs/bullmqthat used@OnWorkerEvent('completed')(job => …)map 1:1 onto this lib's@OnWorkerEvent— they do not need to be rewritten asQueueEventslisteners, and they keep access tojob.data.
Long-running handlers report progress with the native job.updateProgress() (a number or a JSON object). The matching @OnWorkerEvent('progress') listener receives (job, progress):
@Process('generate-report')
async handle(job: Job<ReportJobData, ReportResult>): Promise<ReportResult> {
await job.updateProgress(10)
const data = await this.fetch(job.data.reportId)
await job.updateProgress({ stage: 'render', pct: 60 })
return this.render(data)
}
@OnWorkerEvent('progress')
onProgress(job: Job<ReportJobData>, progress: number | object): void {
// push to a websocket, etc.
}To register workers outside the decorator pattern (e.g., create 10 identical workers with different names from a config):
@Injectable()
export class DynamicWorkerBootstrap implements OnModuleInit {
constructor(private readonly workers: WorkerRegistry) {}
async onModuleInit(): Promise<void> {
for (const tenant of activeTenants) {
await this.workers.register({
queueName: `email:${tenant.id}`,
handler: async job => this.processEmail(tenant.id, job.data),
options: { concurrency: tenant.tier === 'premium' ? 10 : 2 }
})
}
}
}export interface ProgrammaticWorkerConfig<TData = unknown, TResult = unknown> {
queueName: string
handler: (job: Job<TData, TResult>) => Promise<TResult>
options?: WorkerOptions
}
@Injectable()
export class WorkerRegistry {
async register<TData, TResult>(
config: ProgrammaticWorkerConfig<TData, TResult>
): Promise<Worker<TData, TResult>>
/** Registers a file-based, out-of-process (sandboxed) worker — no NestJS DI (§6.8). */
async registerSandboxed(config: {
queueName: string
processorFile: string | URL
options?: WorkerOptions & { useWorkerThreads?: boolean }
}): Promise<Worker>
async unregister(queueName: string): Promise<void>
list(): readonly string[]
}The lib makes concurrency an explicit, deliberate choice for every worker registered via decorator. The motivation: BullMQ's silent default of 1 is serial and caused throughput surprises in practice. When a @Processor is registered without explicit workerOptions.concurrency, the lib logs a warning and falls back to DEFAULT_WORKER_CONCURRENCY (it does not hard-fail) — so a missing value is always surfaced, never silently serial.
Documented recommendation — tune by workload type, not a magic number:
- I/O-bound handlers (HTTP calls, DB, email): concurrency lets the worker overlap waits, so higher values improve throughput. Start at
2, raise while watching the downstream system's limits, and pair withlimiterwhen the downstream is rate-limited.- CPU-bound handlers (image/PDF/crypto/AI): in-process concurrency does not add parallelism — all jobs share one event loop and merely interleave. Keep concurrency low and move the work to a sandboxed processor (§6.8) or scale horizontally with more worker instances.
DEFAULT_WORKER_CONCURRENCY = 2is a safe, non-serial starting point — not an inherently optimal value.
For CPU-bound work that must not block the event loop, BullMQ runs the processor in a separate process or worker thread. This is not a flag on a @Processor class — a sandboxed processor lives in its own file and receives no NestJS DI:
// cpu-heavy.processor.ts — standalone, NO NestJS DI available here
import type { SandboxedJob } from 'bullmq'
export default async function (job: SandboxedJob): Promise<unknown> {
// pure, dependency-light work (resize image, render PDF, hash, …)
return heavyCompute(job.data)
}// register it programmatically by file path
await workerRegistry.registerSandboxed({
queueName: 'thumbnails',
processorFile: new URL('./cpu-heavy.processor.js', import.meta.url),
options: { concurrency: 4, useWorkerThreads: false } // process by default; threads opt-in
})Constraints (documented so consumers are not surprised):
- The processor file cannot inject NestJS providers — pass everything it needs via
job.dataor environment. - The file must be a built
.js/.cjs/.mjsartifact reachable at runtime (account for it intsup/dist). - Communication is over IPC, so payloads must be serializable and are subject to the same size guidance as job data (§16).
Flows allow creating trees of jobs with parent-child relationships: a parent job only becomes processable once all its children have completed. Useful for:
- Processing pipelines (extract → transform → load).
- Fan-out + fan-in (generate 100 thumbnails in parallel + assemble the end PDF).
- AI workflows (embedding of N documents + end LLM call).
Child failure does NOT fail the parent by default. In BullMQ, if a child exhausts its retries and fails, the parent simply stays in the
waiting-childrenstate — it neither completes nor fails. To make a child's failure propagate up and fail its parent, setfailParentOnFailure: trueon that child (see §7.2). Plan for unrecovered child failures explicitly, or the parent can wait indefinitely.
The lib exposes this feature via FlowService, registered only if options.flows.enabled === true.
import type { FlowJob, FlowProducer, JobNode } from 'bullmq'
@Injectable()
export class FlowService {
/**
* Adds a flow (a tree of jobs with parent-child relations).
*
* The root job becomes processable only after every descendant has completed.
* By BullMQ default a failed (retry-exhausted) child does NOT fail the root —
* the parent stays in `waiting-children` until the child eventually completes.
* To make a child's failure fail its parent, set `failParentOnFailure: true`
* on that child in the flow definition (it propagates upward if ancestors set
* it too). Use `ignoreDependencyOnFailure: true` to let a parent proceed
* despite a failed child.
*
* @param flow - Flow tree definition
* @returns The root JobNode (which contains the created Job and children)
*/
async add<TData = unknown>(flow: FlowJob): Promise<JobNode<TData>>
/**
* Bulk add multiple flows in a single roundtrip.
*/
async addBulk(flows: ReadonlyArray<FlowJob>): Promise<Array<JobNode<unknown>>>
/**
* Returns the underlying FlowProducer for advanced use cases.
* Prefer the helpers above when possible.
*/
getProducer(): FlowProducer
}await flowService.add({
name: 'generate-pdf',
queueName: 'pdf',
data: { reportId: '123' },
children: [
{ name: 'fetch-data', queueName: 'data', data: { reportId: '123' } },
{ name: 'render-charts', queueName: 'render', data: { reportId: '123' } },
{
name: 'render-tables',
queueName: 'render',
data: { reportId: '123' },
children: [
{ name: 'fetch-table-1', queueName: 'data', data: { tableId: 't1' } },
{ name: 'fetch-table-2', queueName: 'data', data: { tableId: 't2' } }
]
}
]
})- Created once in
onModuleInit(ifflows.enabled). - Uses the main connection (mode A or B).
- Closed in
onModuleDestroybefore the queues.
Why "Job Schedulers" and not "Repeatable Jobs". BullMQ introduced the Job Schedulers API in
5.16.0and deprecated the legacy "repeatable jobs" surface (addRepeatable,getRepeatableJobs,removeRepeatable,removeRepeatableByKey), which is removed in BullMQ v6. A public library must not expose a method that the upstream engine is deleting, so this package builds exclusively onupsertJobScheduler/removeJobScheduler/getJobSchedulers.
A scheduler fires on one of two schedule kinds:
| Form | When to use | Example |
|---|---|---|
| Cron | Human times (every day at 03:00) | pattern: '0 3 * * *' |
| Every (ms) | Fixed intervals (every 5 min) | every: 5 * 60 * 1000 |
BullMQ parses cron with cron-parser, which accepts both 5-field and 6-field (seconds) patterns — e.g. '*/30 * * * * *' runs every 30 seconds. The first run of a newly created scheduler fires immediately (for both cron and interval) — this is BullMQ's current default, so no flag is needed. (BullMQ's legacy immediately option was deprecated in 5.19.0 and is therefore intentionally not exposed here, per §0 invariant #4.) Interval schedulers accept an offset to phase-shift the cadence.
export type JobSchedulerRepeatOptions =
| {
/** Crontab expression (5-field, or 6-field with seconds). */
pattern: string
/** IANA timezone (e.g. 'America/Sao_Paulo'). Default: UTC */
tz?: string
/** Optional cap on the number of runs */
limit?: number
/** Start time (epoch ms or ISO). */
startDate?: number | string
/** Stop time (epoch ms or ISO). Must be in the future or BullMQ throws. */
endDate?: number | string
}
| {
/** Interval in milliseconds between runs */
every: number
/** Optional cap on the number of runs */
limit?: number
/** Phase offset (ms) for interval schedulers */
offset?: number
startDate?: number | string
endDate?: number | string
}This is a thin, validated projection of BullMQ's
RepeatOptions(minus the internalkey). The lib does not invent its own scheduling semantics.
// Cron — every day at 03:00 São Paulo time
await queueService.upsertJobScheduler<CleanupJobData>(
'maintenance', // queueName
'nightly-cleanup', // schedulerId (the stable upsert key)
{ pattern: '0 3 * * *', tz: 'America/Sao_Paulo' },
{ name: 'cleanup', data: { mode: 'soft' } }
)
// Interval — every 5 minutes
await queueService.upsertJobScheduler<HeartbeatJobData>(
'monitoring',
'api-heartbeat',
{ every: 5 * 60 * 1000 },
{ name: 'heartbeat', data: { service: 'api' } }
)upsertJobScheduler is idempotent by schedulerId — BullMQ performs an atomic override: true upsert. Calling it again with the same id updates the schedule and the job template in place (no duplicate scheduler), which is exactly the behavior you want when the host application re-registers schedulers on every boot.
/** Removes a scheduler by id. Returns true if one was removed. */
async removeJobScheduler(queueName: string, schedulerId: string): Promise<boolean>
/** Lists schedulers for a queue (paginated), for inspection/health. */
async getJobSchedulers(
queueName: string,
start?: number,
end?: number,
asc?: boolean
): Promise<JobSchedulerJson[]>The lib validates repeat before delegating to BullMQ and throws queue.invalid_repeat_options (400) when:
- both
patternandeveryare present, or neither is; everyis not a positive integer;patternfails to parse — the lib delegates cron parsing to BullMQ (which usescron-parserinternally) and rethrows any parse failure asqueue.invalid_repeat_options. It never ships a hand-rolled cron regex (incorrect for 6-field patterns and a ReDoS risk) and adds no direct dependency to do this;endDateis in the past (BullMQ itself rejects this).
Register schedulers in an OnApplicationBootstrap hook of the host application, not inside a job handler. Because upsertJobScheduler is idempotent by id, running this on every boot is safe and keeps the scheduler definition in code (auditable, reviewable) rather than as hidden Redis state.
The lib exposes basic counters sufficient for health checks and simple dashboards. It does not provide history, alerting, or SLA tracking — that is the consumer's responsibility (see §13).
export interface QueueMetrics {
/** Queue name */
queue: string
/** Job counts grouped by status (instantaneous snapshot) */
counts: {
waiting: number
active: number
completed: number
failed: number
delayed: number
paused: number
}
/** ISO timestamp of the moment counts were sampled */
collectedAt: string
}Default implementation — calls Queue.getJobCounts() directly on Redis:
async getMetrics(queueName: string): Promise<QueueMetrics> {
const queue = this.getOrCreateQueue(queueName)
const counts = await queue.getJobCounts(
'waiting', 'active', 'completed', 'failed', 'delayed', 'paused'
)
return {
queue: queueName,
counts,
collectedAt: new Date().toISOString()
}
}When options.metrics.enabled === true, the lib registers a MetricsService that applies in-memory caching (configurable TTL, default 5s). Useful when the /health endpoint is called every second by a load balancer and we want to avoid 6 roundtrips/s to Redis.
@Injectable()
export class MetricsService {
/**
* Returns cached metrics for `queueName`. Cache TTL = options.metrics.cacheTtlMs.
*/
async get(queueName: string): Promise<QueueMetrics>
/**
* Returns metrics for all known queues (those created via QueueService).
*/
async getAll(): Promise<readonly QueueMetrics[]>
/**
* Forces cache invalidation. Useful in tests.
*/
invalidate(queueName?: string): void
}The lib does not implement a HealthIndicator from @nestjs/terminus — that would introduce an unnecessary peer dep. Instead, we document the pattern:
// In the host application
@Injectable()
export class QueueHealthIndicator extends HealthIndicator {
constructor(private readonly metrics: MetricsService) { super() }
async isHealthy(key: string): Promise<HealthIndicatorResult> {
const all = await this.metrics.getAll()
const totalActive = all.reduce((sum, m) => sum + m.counts.active, 0)
const isHealthy = totalActive < SOME_THRESHOLD
return this.getStatus(key, isHealthy, { active: totalActive })
}
}This is the single most important contract for any queue and is stated up front: BullMQ (and therefore this lib) guarantees at-least-once delivery, not exactly-once. A job whose worker crashes, is killed, or exceeds its lock is recovered as stalled and re-run by another worker. Consequences for consumers:
- Handlers must be idempotent. Re-running a handler with the same
job.id/job.datamust be safe (use an idempotency key on writes, upserts instead of inserts, or an "already-processed" marker keyed byjob.id). - Use
jobIdand/ordeduplication(§5.4.1) to collapse duplicate producers, but understand that this does not change the at-least-once guarantee on the consumer side. - Tune
lockDuration(§6.3) to comfortably exceed your handler's worst-case runtime so healthy long jobs are never falsely treated as stalled.
The lib documents this prominently rather than implying a stronger guarantee it cannot deliver.
BullMQ workers continuously block on Redis (BRPOPLPUSH / BZPOPMIN / BLMOVE) waiting for work. If the process receives SIGTERM (deploy, scale-down) and simply terminates, two problems occur:
- In-flight job loses its lock, is marked
stalledafterlockDuration, and is re-executed elsewhere — an avoidable at-least-once duplication. - Open workers and queues keep Redis connections open, delaying
exit(0)(Kubernetes may escalate toSIGKILL).
The lib resolves this via QueueLifecycle.onModuleDestroy(), which executes the protocol below.
worker.close() (non-force) already waits for the currently active jobs to finish and then resolves; it takes no timeout argument. To bound that wait we race it against a timer and escalate to worker.close(true) (force) on expiry. The protocol:
1. Receive SIGTERM (Nest triggers onModuleDestroy)
2. For each registered Worker, concurrently:
a. Promise.race([ worker.close(), timeout(options.shutdown.drainTimeoutMs) ])
- worker.close() stops fetching new jobs and waits for active ones to finish
b. If the timeout wins, call worker.close(true) to force-close
- in-flight jobs lose their lock and become `stalled` (will be retried);
emit `queue.shutdown_timeout_exceeded` so the operator knows
3. For each registered QueueEvents: queueEvents.close()
4. If options.shutdown.drainOnShutdown === true (DEV/TEST ONLY):
a. For each Queue: queue.drain() — removes waiting + delayed jobs
5. For each Queue: queue.close()
6. Connection teardown:
- Mode B (lib owns the connection): redis.quit() on the main client and on
every duplicated worker/QueueEvents connection
- Mode A (injected client): close only the DUPLICATED worker/QueueEvents
connections the lib created; never touch the consumer's shared client
7. Log shutdown metrics (total time, forced workers, drained jobs)
- Minimized duplication for in-flight jobs: if the worker finishes within
drainTimeoutMs, the job iscompletedbeforeclose()resolves and is never re-run. On timeout, the job becomesstalledand is retried (at-least-once — by design, and surfaced viaqueue.shutdown_timeout_exceeded). - No-loss for waiting jobs: unless
drainOnShutdown: true, queues keepwaiting/delayedjobs intact for the next deploy. - Clean Redis connection close: the lib closes exactly the connections it owns, preventing
Connection closederror logs and leaked sockets — and never closes a connection the consumer still uses (Mode A).
BymaxQueueModule.forRoot({
// ...
shutdown: {
drainTimeoutMs: 30_000, // default
drainOnShutdown: false // default — production-safe
}
})Configure terminationGracePeriodSeconds: 60 on the deployment to allow margin for drainTimeoutMs: 30000.
The canonical integration between @bymax-one/nest-cache and @bymax-one/nest-queue uses mode A (bring your own connection). The reason: nest-cache already exposes a Redis client dedicated to BullMQ, and nest-queue applies the per-role retry policy automatically (it uses that client for the Queue role and duplicates it with maxRetriesPerRequest: null for the blocking worker/QueueEvents connections — see §2.3), so you do not configure connection retries yourself. For the cleanest fail-fast behavior, configure nest-cache's queue client with default retries; the lib still forces null where BullMQ requires it.
import { Module } from '@nestjs/common'
import { BymaxCacheModule, BYMAX_CACHE_QUEUE_REDIS } from '@bymax-one/nest-cache'
import { BymaxQueueModule } from '@bymax-one/nest-queue'
import type { Redis } from 'ioredis'
@Module({
imports: [
// Cache module exposes a Redis client tagged for BullMQ usage
BymaxCacheModule.forRoot({
url: process.env.REDIS_URL,
queueClient: { enabled: true }
}),
// Queue module consumes the cache module's Redis client (Mode A)
BymaxQueueModule.forRootAsync({
imports: [BymaxCacheModule],
inject: [BYMAX_CACHE_QUEUE_REDIS],
useFactory: (queueRedis: Redis) => ({
connection: { client: queueRedis },
defaultJobOptions: {
attempts: 5,
backoff: { type: 'exponential', delay: 5000 }
},
flows: { enabled: true },
metrics: { enabled: true, cacheTtlMs: 3000 }
})
})
]
})
export class AppModule {}- Worker in a separate container: if you run the workers in a dedicated pod (without nest-cache), use mode B with the Redis URL directly.
- Multi-tenant with distinct Redis instances: each tenant has its own Redis — you cannot share a generic connection.
The injected client is used as-is for the Queue role — the lib never mutates the consumer's client, so the Queue connection inherits whatever maxRetriesPerRequest the consumer configured (use the default for fail-fast on a Redis outage; if the shared client is already null, the Queue inherits null and simply loses that fast-fail, while remaining fully functional). For workers and QueueEvents, the lib always creates client.duplicate({ maxRetriesPerRequest: null }), so the blocking-connection requirement is satisfied without mutating the shared client and without breaking the consumer's cache operations.
The lib does not "warn and continue" on a non-null shared client — that would be unsafe, because the duplicated worker connection inherits the parent's options and BullMQ throws at Worker construction if it is not null. Instead, onModuleInit verifies the duplicated worker connection resolves to maxRetriesPerRequest === null; if a custom client wrapper prevents the override from taking effect, the lib fails fast with a clear error rather than crashing later:
[BymaxQueueModule] FATAL: worker connection could not be set to maxRetriesPerRequest=null
(resolved 20). BullMQ requires null for blocking worker commands. Provide a client whose
.duplicate({ maxRetriesPerRequest: null }) is honored, or use Mode B.
→ throws QueueException(queue.connection_requires_null_retries)
import { HttpException, HttpStatus } from '@nestjs/common'
import { QUEUE_ERROR_MESSAGES, QUEUE_ERROR_STATUS } from './queue-error-codes'
/**
* Standardized exception for queue operations.
* All exceptions follow the same response format. The HTTP status is derived
* from the code via QUEUE_ERROR_STATUS (the §12.2 mapping); the optional
* `statusCode` argument overrides it only when explicitly passed.
*/
export class QueueException extends HttpException {
constructor(
code: string,
statusCode: HttpStatus = QUEUE_ERROR_STATUS[code] ?? HttpStatus.INTERNAL_SERVER_ERROR,
details?: Record<string, unknown>
) {
super(
{
error: {
code,
message: QUEUE_ERROR_MESSAGES[code] ?? 'Queue error',
details: details ?? null
}
},
statusCode
)
}
}| Code | HTTP | Message | Context |
|---|---|---|---|
queue.connection_invalid |
500 | Invalid Redis connection configuration | Mode B with malformed URL, or mode A with client in end/close status |
queue.connection_requires_null_retries |
500 | Worker connection must have maxRetriesPerRequest=null | Mode A: the duplicated worker/QueueEvents connection could not be coerced to null (e.g. a client wrapper that ignores the duplicate() override) — fail-fast at init |
queue.connection_timeout |
500 | Redis connection timeout | Mode B: did not reach ready within the timeout (default 10s) |
queue.queue_not_found |
404 | Queue not found | Attempt to get metrics/jobs from an unregistered queue |
queue.job_not_found |
404 | Job not found | getJob() with a non-existent id |
queue.invalid_job_data |
400 | Invalid job data | enqueue() with data that fails schema validation (if the consumer enables it) |
queue.invalid_repeat_options |
400 | Invalid repeat options | upsertJobScheduler() with both/neither of pattern/every, every <= 0, an unparseable cron, or a past endDate |
queue.duplicate_processor |
500 | Multiple @Processor decorators target the same queue | Two classes annotated with @Processor('foo') — only one per queue is allowed |
queue.shutdown_timeout_exceeded |
500 | Shutdown timeout exceeded | Workers did not finish within drainTimeoutMs — log warning, force close |
queue.bulk_enqueue_failed |
500 | Bulk enqueue failed | enqueueBulk() failed — thrown with the per-job errors in details.errors |
queue.worker_registration_failed |
500 | Failed to register worker | Error instantiating BullMQ Worker (connection problem, invalid option) |
queue.invalid_options |
500 | Invalid module options | forRoot() with connection absent or ambiguous (client + URL at the same time) |
export const QUEUE_ERROR_CODES = {
CONNECTION_INVALID: 'queue.connection_invalid',
CONNECTION_REQUIRES_NULL_RETRIES: 'queue.connection_requires_null_retries',
CONNECTION_TIMEOUT: 'queue.connection_timeout',
QUEUE_NOT_FOUND: 'queue.queue_not_found',
JOB_NOT_FOUND: 'queue.job_not_found',
INVALID_JOB_DATA: 'queue.invalid_job_data',
INVALID_REPEAT_OPTIONS: 'queue.invalid_repeat_options',
DUPLICATE_PROCESSOR: 'queue.duplicate_processor',
SHUTDOWN_TIMEOUT_EXCEEDED: 'queue.shutdown_timeout_exceeded',
BULK_ENQUEUE_FAILED: 'queue.bulk_enqueue_failed',
WORKER_REGISTRATION_FAILED: 'queue.worker_registration_failed',
INVALID_OPTIONS: 'queue.invalid_options'
} as const{
"error": {
"code": "queue.connection_requires_null_retries",
"message": "Worker connection must have maxRetriesPerRequest=null",
"details": {
"actualValue": 20,
"expectedValue": null
}
}
}- Framework-neutral code, HTTP-friendly shape.
QueueException extends HttpExceptionfor ergonomics in HTTP request contexts, but it always carries a stable, transport-independentcode. Queue work frequently runs outside an HTTP request (workers,OnApplicationBootstrap); consumers catching aQueueExceptionthere should branch oncode, not on the HTTP status. (A framework-neutral base error with an HTTP mapping layer is tracked as a possible future refinement.) - No secrets in
details.detailsmust never contain a raw connection string, password, orjob.data. Connection identifiers are masked (redis://default:***@host:6379) before they reach a message, log, ordetailspayload (see §18/Appendix B).actualValue/expectedValueabove are scalar config values, never credentials. - No payload leakage. Validation errors reference field names, never echo
job.datavalues, so PII in payloads is not surfaced through exceptions.
The lib was designed with clear boundaries. The items below are the responsibility of the consuming application:
| Item | Reason | Where to implement |
|---|---|---|
| BullBoard UI (visual dashboard) | UI is a separate product (bull-board is a heavy peer dep). The lib exposes Queue objects for the consumer to plug BullBoard at will. |
Host application mounts @bull-board/express or @bull-board/nestjs |
| Alerting (PagerDuty, Slack, etc.) | Threshold decisions are product- and SLA-specific | @OnQueueEvent('failed') in the consumer + tool integration |
| SLA tracking (job age, percentile latency) | Requires historical storage (Prometheus, ClickHouse) — out of scope | Consumer publishes metrics via prom-client or similar |
| Job audit log | Retention of completed jobs is managed by BullMQ; long-term auditing requires another storage | @OnQueueEvent('completed') hook + consumer table |
| Rate limiting per tenant (cross-queue) | Worker limiter is per queue; per-tenant limits require external coordination |
Consumer implements via Redis tokens |
| Migration from old queues (BullMQ v3 → v5) | Can be destructive — we do not automate it | Consumer follows the official BullMQ migration guide |
| In-transit payload encryption | Redis TLS is ioredis client configuration | Pass tls: {} in connection.options |
| Job persistence beyond removeOn* | Long retention requires external archiving | @OnWorkerEvent('completed') hook → S3/database |
These dependencies must be installed in the host application that uses the server subpath. The package does not include them — it expects them to already exist.
| Package | Version | Reason |
|---|---|---|
@nestjs/common |
^11.0.0 |
Core framework — decorators, exceptions, providers, ConfigurableModuleBuilder |
@nestjs/core |
^11.0.0 |
Core framework — module system, DI container, DiscoveryService |
bullmq |
^5.16.0 |
Queue engine. 5.16.0 is the floor because it introduced the Job Schedulers API (§8). See §14.5 |
ioredis |
^5.0.0 |
Redis client (always present, even in mode A where the consumer passes) |
reflect-metadata |
^0.2.0 |
Metadata reflection for the decorators |
The package has no direct dependencies ("dependencies": {}). All functionality is delivered via peer deps + own code + Node.js builtins. This keeps the install footprint minimal and avoids version conflicts in the host application.
| Package | Version | Needed when |
|---|---|---|
bullmq-otel |
^1.0.0 |
Only if you set options.telemetry for OpenTelemetry tracing (§4, §16.5) |
bullmq-otel is declared in peerDependenciesMeta as optional: true, so installs do not warn when telemetry is unused. The ./shared subpath has zero peer deps (only TS types).
| Subpath | Peer Dependencies |
|---|---|
. (server) |
@nestjs/common ^11, @nestjs/core ^11, bullmq ^5.16, ioredis ^5, reflect-metadata ^0.2; optional bullmq-otel ^1 |
./shared |
None |
At the time of writing, the current BullMQ release is 5.79.1. The lib's floor is ^5.16.0 because that is where the Job Schedulers API (the only recurring-jobs API this lib exposes, §8) landed.
| BullMQ | Status | Decision |
|---|---|---|
| v5 (≥ 5.16) | Current stable. Every API the lib uses (Queue, Worker, FlowProducer, JobsOptions, getJobCounts, upsertJobScheduler, deduplication, Telemetry) is GA. |
✅ Official support. |
| v6 | Removes the legacy repeatable-jobs API. Because this lib already uses Job Schedulers (not addRepeatable), it is forward-compatible by design. Promote to `^5.16.0 |
Why this matters: the earlier draft built on
addRepeatable/removeRepeatable, which v6 deletes — that would have forced a breaking change on the first v6 bump. Standardizing on Job Schedulers removes that landmine.
The peer dep in the initial package.json is:
"peerDependencies": {
"bullmq": "^5.16.0"
}This decision is reviewed in every minor release of the lib.
{
"name": "@bymax-one/nest-queue",
"version": "0.1.0",
"description": "NestJS dynamic module wrapping BullMQ — typed jobs, flows, job schedulers, deduplication, OpenTelemetry, graceful shutdown",
"author": "Bymax One <support@bymax.one>",
"license": "MIT",
"homepage": "https://github.com/bymaxone/nest-queue#readme",
"repository": { "type": "git", "url": "https://github.com/bymaxone/nest-queue.git" },
"bugs": { "url": "https://github.com/bymaxone/nest-queue/issues" },
"type": "module",
"sideEffects": false,
"files": ["dist", "LICENSE", "README.md", "CHANGELOG.md"],
"exports": {
".": {
"types": "./dist/server/index.d.ts",
"import": "./dist/server/index.mjs",
"require": "./dist/server/index.cjs"
},
"./shared": {
"types": "./dist/shared/index.d.ts",
"import": "./dist/shared/index.mjs",
"require": "./dist/shared/index.cjs"
}
},
"scripts": {
"build": "pnpm clean && tsup",
"lint": "eslint src",
"lint:fix": "eslint src --fix",
"test": "jest",
"test:cov": "jest --coverage",
"test:e2e": "jest --config jest.e2e.config.ts",
"test:cov:all": "jest --config jest.coverage.config.ts --coverage",
"mutation": "stryker run",
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.server.json",
"size": "node scripts/check-size.mjs",
"clean": "rm -rf dist coverage",
"prepublishOnly": "pnpm clean && pnpm typecheck && pnpm lint && pnpm test:cov:all && pnpm build",
"release": "pnpm publish --provenance"
},
"peerDependencies": {
"@nestjs/common": "^11.0.0",
"@nestjs/core": "^11.0.0",
"bullmq": "^5.16.0",
"ioredis": "^5.0.0",
"reflect-metadata": "^0.2.0",
"bullmq-otel": "^1.0.0"
},
"peerDependenciesMeta": {
"bullmq-otel": { "optional": true }
},
"devDependencies": {
"@nestjs/common": "^11.0.0",
"@nestjs/core": "^11.0.0",
"@nestjs/testing": "^11.0.0",
"@stryker-mutator/core": "^9.0.0",
"@stryker-mutator/jest-runner": "^9.0.0",
"@testcontainers/redis": "^11.0.0",
"@types/jest": "^30.0.0",
"bullmq": "^5.79.1",
"bullmq-otel": "^1.3.1",
"ioredis": "^5.11.1",
"jest": "^30.0.0",
"ts-jest": "^29.0.0",
"tsup": "^8.0.0",
"typescript": "^5.6.0"
},
"keywords": [
"nestjs", "bullmq", "queue", "redis", "worker", "flow",
"cron", "job-scheduler", "scheduler", "deduplication",
"opentelemetry", "otel", "graceful-shutdown", "job"
],
"packageManager": "pnpm@11.0.0",
"engines": { "node": ">=24.0.0" },
"publishConfig": {
"access": "public",
"provenance": true,
"registry": "https://registry.npmjs.org/"
}
}
packageManagermust match the repo's pinned pnpm (the portfolio is on pnpm 11);devDependenciesversions are illustrative — pin exact resolved versions in the realpackage.json.
Testing strategy: unit tests are written alongside each phase (TDD), not accumulated in Phase 4. Each phase must reach 100% line/branch coverage on the files it implements (enforced by
jest.coverage.config.tsthresholds100/100/100/100). Stryker mutation testing (break 95, target 100%) runs as a pre-release gate, not per commit. See §18.
| Phase | Complexity | Focus | Deliverables |
|---|---|---|---|
| 1 | MEDIUM | Module + QueueService | Scaffold, dynamic module (ConfigurableModuleBuilder), ConnectionResolver, basic QueueService (enqueue/getJob/getMetrics) + tests |
| 2 | MEDIUM | Workers (decorator + programmatic) | @Processor, @Process, @OnWorkerEvent, @OnQueueEvent, WorkerRegistry (+ registerSandboxed) + tests |
| 3 | MEDIUM | Flows + Job Schedulers + Deduplication/Telemetry | FlowService, upsertJobScheduler/removeJobScheduler/getJobSchedulers, deduplication options, telemetry passthrough, MetricsService + tests |
| 4 | HIGH | forRootAsync + shutdown + E2E + release | forRootAsync, graceful-shutdown protocol, E2E tests with real Redis (testcontainers), mutation baseline, docs/README, release prep |
Phase mapping to the execution plan. This section describes 4 logical phases; the execution plan (
docs/development_plan.md) and the per-phase task files (docs/tasks/) keep the same scope but split this Phase 4 into plan Phase 4 (forRootAsync + graceful shutdown + E2E + mutation baseline) and plan Phase 5 (release/publish). 4 spec phases ≡ 5 plan phases — not a contradiction.
Execution by AI agents — there is no estimate in human days. Relative complexity per phase is indicated above; fine-grained granularity per sub-step lives in
docs/development_plan.md(Appendix B — Complexity Matrix).
Goal: Create the base structure of the package, the dynamic module, and the main QueueService with support for the two connection modes.
Deliverables:
-
Project scaffold
package.jsonwith peer deps (NestJS 11, bullmq^5.16, ioredis 5; optionalbullmq-otel)tsconfig.json,tsconfig.build.json,tsconfig.server.json,tsconfig.e2e.json,tsconfig.jest.jsontsup.config.tswith 2 entries (server,shared)- Directory structure in
src/ eslint.config.mjs,jest.config.ts,jest.coverage.config.ts(100/100/100/100),commitlint.config.cjs,.prettierrcscripts/check-size.mjs(budget ≤ 18 KiB brotli)
-
Base interfaces
queue-module-options.interface.ts—BymaxQueueModuleOptions,BymaxQueueModuleAsyncOptionsqueue-connection.interface.ts—QueueConnectionConfig(union of mode A and B)worker-options.interface.ts—WorkerOptionsqueue-job-data.interface.ts— utility types
-
Constants and configuration
bymax-queue.constants.ts—BYMAX_QUEUE_OPTIONS,BYMAX_QUEUE_REDIS_CLIENT,BYMAX_QUEUE_CONNECTION_MODE(Symbol)config/default-options.ts—DEFAULT_JOB_OPTIONS,DEFAULT_WORKER_CONCURRENCYconfig/resolved-options.ts— merge defaults with user options
-
ConnectionResolver- Detects mode A (presence of
connection.client) or mode B (presence ofconnection.urlorconnection.options) - Per-role policy: Queue/FlowProducer connection keeps default retries; worker/QueueEvents connections obtained via
duplicate({ maxRetriesPerRequest: null }) - Mode A: validates
client.status; fails fast (queue.connection_requires_null_retries) if a worker connection cannot be coerced tonull - Mode B: opens
iorediswith timeout - Exposes
getQueueClient(),duplicateForBlocking(), andisOwned()(to know what it must close)
- Detects mode A (presence of
-
QueueService— core methodsgetOrCreateQueue<TData, TResult>()enqueue<TData, TResult>()enqueueBulk<TData, TResult>()getJob<TData, TResult>()getJobs<TData, TResult>()getMetrics()(version without cache)pauseQueue(),resumeQueue(),cleanQueue()
-
Dynamic module
bymax-queue.module.ts— built onConfigurableModuleBuilder;isGlobalmapped toDynamicModule.globalviasetExtras(no manual@Global(), noforFeaturestub). The synchronousforRoot()lands in this phase; the asynchronousforRootAsync()is implemented in plan Phase 4 (it depends on the flows/metrics providers resolved first — seedevelopment_plan.md§5.1 and the Appendix A dependency graph)- Conditional registration of FlowService and MetricsService (stubs for now, activated in Phase 3)
-
Errors
errors/queue-error-codes.ts— all constants from §12.3errors/queue-exception.ts—QueueExceptionclass
-
Unit tests (Phase 1)
ConnectionResolver: valid mode A, invalid mode A (maxRetries != null), mode B with URL, mode B with options, timeoutQueueService: each method with mockedQueue(ioredis-mock for Redis)BymaxQueueModule:forRootandforRootAsync(validate registered providers)- Coverage: 100% line/branch on implemented files
Goal: Implement worker registration via decorator and programmatic API.
Deliverables:
-
Decorators
decorators/processor.decorator.ts—@Processor(queueName, workerOptions?)decorators/process.decorator.ts—@Process(jobName?)decorators/on-worker-event.decorator.ts—@OnWorkerEvent(eventName)(worker-local, fullJob)decorators/on-queue-event.decorator.ts—@OnQueueEvent(eventName)(globalQueueEvents)- Metadata stored via
SetMetadata(exclusive Symbol keys)
-
WorkerRegistryregister<TData, TResult>(config)— creates BullMQWorker, performsduplicate()on the connectionunregister(queueName)— closes the workerlist()— returns registered names
-
Discovery service integration
- In
onModuleInit, uses NestJSDiscoveryServiceto find classes annotated with@Processor - For each class: instantiates via DI container, extracts metadata from each
@Processmethod, registers the worker - Wires
@OnWorkerEventlisteners directly to theWorker(fullJob, no extra connection) - Wires
@OnQueueEventlisteners to the corresponding (lazy)QueueEvents
- In
3b. Sandboxed worker registration
WorkerRegistry.registerSandboxed({ queueName, processorFile, options })for file-based, out-of-process processors (no DI) — see §6.8
-
Lazy
QueueEvents- Creates
QueueEventsonly when there is at least one@OnQueueEventfor that queue - Uses
connection.duplicate()(one connection per listener)
- Creates
-
Worker options validation
- Warning if
concurrencyis missing → usesDEFAULT_WORKER_CONCURRENCY - Error if
concurrency < 1 - Error if
limiter.max < 1orlimiter.duration < 1
- Warning if
-
Unit tests (Phase 2)
- Decorators: verify metadata via
Reflect.getMetadata WorkerRegistry: register, list, unregister, registerSandboxed- Discovery: class with
@Processor+@Processis discovered and wired up @OnWorkerEventhandler receives the fullJob;@OnQueueEventreceives fired global events- Coverage: 100% line/branch on implemented files
- Decorators: verify metadata via
Goal: Implement opt-in features (Flows, Metrics, Telemetry) plus Job Schedulers and deduplication.
Deliverables:
-
FlowService- Constructor creates
FlowProducerwith the main connection - Methods
add(),addBulk(),getProducer() - Registered only if
options.flows.enabled === true
- Constructor creates
-
Job Schedulers in
QueueServiceupsertJobScheduler<TData>()— wrapper overQueue.upsertJobScheduler()(idempotent byschedulerId)removeJobScheduler()— wrapper overQueue.removeJobScheduler()getJobSchedulers()— wrapper overQueue.getJobSchedulers()- Validation: exactly one of
pattern/every;every > 0;endDatein the future; cronpatternparse delegated to BullMQ (cron-parser), parse failure rethrown asqueue.invalid_repeat_options— never a hand-rolled regex, no direct dependency
-
Deduplication + telemetry
- Surface BullMQ
deduplicationoptions onenqueue(Simple/Throttle/Debounce/Keep-last-if-active) - Attach
options.telemetryto everyQueue/Workerwhen provided
- Surface BullMQ
-
MetricsService- In-memory cache (
Map<string, { metrics: QueueMetrics; expiresAt: number }>) get(queueName),getAll(),invalidate()- Registered only if
options.metrics.enabled === true
- In-memory cache (
-
Integration in
QueueService.getMetrics()- If
MetricsServiceis available (registered), delegates - Otherwise, fetches directly via
Queue.getJobCounts()
- If
-
Unit tests (Phase 3)
FlowService: add with 3-level tree, addBulkupsertJobScheduler: valid cron pattern (5- and 6-field), invalid cron (rejects), validevery, idempotent re-upsert,removeJobScheduler, pastendDaterejected- Deduplication: each mode collapses duplicates as documented
MetricsService: cache hit/miss, invalidate, TTL expiry- Coverage: 100% line/branch on implemented files
Goal: Validate the package end-to-end, ensure graceful shutdown, and prepare for release.
Deliverables:
-
QueueLifecycleonModuleDestroy()implements the protocol from §10.2- Structured logs (count of drained jobs, total time)
- Respects
drainTimeoutMsanddrainOnShutdown
-
E2E tests (with testcontainers)
- Spin up real Redis via
@testcontainers/redis - Scenario 1: enqueue + process + verify result
- Scenario 2: graceful shutdown (worker finishes in-flight job)
- Scenario 3: flow with 3 levels, all complete
- Scenario 4: Job Scheduler (
upsertJobScheduler) fires twice in 10s; re-upsert does not duplicate - Scenario 5: failure → exponential retry → eventual success
- Scenario 6: deduplication collapses N rapid enqueues to one processed job
- Scenario 7: Mode-A worker connection is coerced to
maxRetriesPerRequest:nulland the Queue connection keeps defaults
- Spin up real Redis via
-
Documentation
- JSDoc on all public exports
README.mdwith badges, quick start, examples, subpath tableCHANGELOG.mdwith0.1.0entrySECURITY.md,CLAUDE.md,AGENTS.md
-
Polish
- Review of barrel exports (
src/server/index.ts) - Options validation at init (clear error messages)
scripts/check-size.mjs— verifies thatdist/server/index.mjsis within the ≤ 18 KiB brotli budget- Final lint + typecheck + coverage report
- Review of barrel exports (
-
Release prep
- Tag
v0.1.0 - Trigger
release.ymlworkflow (publish with--provenance) - Verify the npm page
- Tag
| Limitation | Impact | Alternative |
|---|---|---|
| NestJS only | Does not work with plain Express, standalone Fastify, or other frameworks | BullMQ directly + your own factory |
| Node.js only | No support for Deno, Bun, or other runtimes | Not planned |
| Limitation | Impact | Alternative |
|---|---|---|
BullMQ ^5.16 (0.1.x) |
Floor is 5.16 (Job Schedulers); v6 enabled once e2e passes (already forward-compatible) |
Track the lib's 1.x release for the ^6 promotion |
| No support for legacy Bull (without MQ) | Apps still using bull v4 or earlier need to migrate first |
Official BullMQ migration guide |
| No support for Redis Cluster sharding | BullMQ has experimental support — the lib does not exercise those paths | Manually validate in cluster environments |
| Limitation | Impact | Alternative |
|---|---|---|
| Mode A does not control connection health | If the consumer closes the connection silently, jobs fail silently | Document the contract; consumer keeps the connection alive |
| Mode B opens one connection per app instance | In deployments with 10 instances, there are 10+ connections | Configure connectionName for visibility in Redis |
| No advanced connection pooling | BullMQ does not use a pool — one connection per Worker/QueueEvents | No alternative — inherent to BullMQ |
| Limitation | Impact | Alternative |
|---|---|---|
| No job versioning | If the payload changes incompatibly, old jobs break | Version via jobName (process-v2) |
| No native job tags / labels | BullMQ does not support query by tag | Include tags inside data and filter manually |
| No built-in dead-letter queue helper | A job that exhausts attempts goes to failed, not to a separate DLQ |
Detect attemptsMade >= attempts in @OnWorkerEvent('failed') and re-enqueue to a *-dlq queue (pattern in §16.6) |
Cron-with-seconds (6-field patterns) is supported — see §8.1. Native windowed deduplication is supported — see §5.4.1. Neither is a limitation.
OpenTelemetry tracing is supported via the optional telemetry option (§4) — spans propagate from enqueue through the handler when you pass a bullmq-otel instance. The remaining boundary is metrics export:
| Boundary | Impact | Where to implement |
|---|---|---|
| No Prometheus metrics export | The lib exposes counters via getMetrics() but does not publish them |
Consumer reads getMetrics() / getAll() and publishes via prom-client |
| No SLA/percentile history | Counters are instantaneous snapshots, not time-series | Consumer ships metrics to Prometheus/ClickHouse |
These are documented patterns, not lib features — but a professional consumer must handle them:
- Backpressure / unbounded growth.
removeOnComplete/removeOnFailbound retained finished jobs, but nothing bounds the waiting set if producers outpace workers. MonitorgetMetrics().counts.waiting; when it crosses a threshold, throttle producers orpauseQueue(). Treat a growingwaitingdepth as the primary saturation signal. - Payload size. Job data is stored in Redis and round-tripped as JSON; large payloads bloat memory and slow every operation. Keep payloads small (rule of thumb: < ~30 KB) and use the claim-check pattern — store the blob in S3/DB and enqueue only a reference.
enqueueBulkarrays should be bounded (the lib caps batch size; see §16/Appendix B). - Dead-letter queue. Move exhausted jobs explicitly:
@OnWorkerEvent('failed') async onFailed(job: Job | undefined, err: Error): Promise<void> { if (job && job.attemptsMade >= (job.opts.attempts ?? 1)) { await this.queue.enqueue('emails-dlq', job.name, job.data, { // preserve context for forensics jobId: `dlq:${job.id}` }) } }
// app.module.ts
import { Module } from '@nestjs/common'
import { BymaxCacheModule, BYMAX_CACHE_QUEUE_REDIS } from '@bymax-one/nest-cache'
import { BymaxQueueModule } from '@bymax-one/nest-queue'
import { BullMQOtel } from 'bullmq-otel'
import type { Redis } from 'ioredis'
import { EmailProcessor } from './email.processor'
import { ReportProcessor } from './report.processor'
@Module({
imports: [
BymaxCacheModule.forRoot({
url: process.env.REDIS_URL ?? 'redis://localhost:6379',
queueClient: { enabled: true }
}),
BymaxQueueModule.forRootAsync({
imports: [BymaxCacheModule],
inject: [BYMAX_CACHE_QUEUE_REDIS],
useFactory: (queueRedis: Redis) => ({
connection: { client: queueRedis }, // Mode A — worker conns duplicated with maxRetriesPerRequest:null
prefix: `app:${process.env.NODE_ENV}`,
defaultJobOptions: {
attempts: 5,
backoff: { type: 'exponential', delay: 2000 }
},
flows: { enabled: true },
metrics: { enabled: true, cacheTtlMs: 3000 },
telemetry: new BullMQOtel('app-queue'), // OpenTelemetry spans (positional form works on the ^1 floor)
shutdown: { drainTimeoutMs: 45_000 }
})
})
],
providers: [EmailProcessor, ReportProcessor]
})
export class AppModule {}BymaxQueueModule.forRoot({
connection: {
url: process.env.REDIS_URL ?? 'redis://localhost:6379',
options: {
db: 2,
tls: { rejectUnauthorized: true }
}
},
defaultJobOptions: { attempts: 3 }
})// shared/jobs/send-email.job.ts
export interface SendEmailJobData {
readonly to: string
readonly templateId: 'welcome' | 'reset-password' | 'invoice'
readonly variables: Readonly<Record<string, string>>
}
export interface SendEmailJobResult {
readonly messageId: string
readonly acceptedAt: string
}
export const EMAIL_QUEUE = 'email' as const
export const SEND_EMAIL_JOB = 'send-email' as const// services/notifications.service.ts
import { Injectable } from '@nestjs/common'
import { QueueService } from '@bymax-one/nest-queue'
import { EMAIL_QUEUE, SEND_EMAIL_JOB, SendEmailJobData, SendEmailJobResult } from '../shared/jobs/send-email.job'
@Injectable()
export class NotificationsService {
constructor(private readonly queue: QueueService) {}
async sendWelcomeEmail(userId: string, email: string): Promise<void> {
await this.queue.enqueue<SendEmailJobData, SendEmailJobResult>(
EMAIL_QUEUE,
SEND_EMAIL_JOB,
{
to: email,
templateId: 'welcome',
variables: { userId }
},
{
priority: 10,
jobId: `welcome:${userId}` // deduplication
}
)
}
}// processors/email.processor.ts
import { Injectable } from '@nestjs/common'
import { Processor, Process, OnWorkerEvent } from '@bymax-one/nest-queue'
import type { Job } from 'bullmq'
import { MailerService } from '../mailer.service'
import {
EMAIL_QUEUE,
SendEmailJobData,
SendEmailJobResult
} from '../shared/jobs/send-email.job'
@Injectable()
@Processor(EMAIL_QUEUE, {
concurrency: 5,
limiter: { max: 10, duration: 1000 } // 10 msg/s max (provider limit)
})
export class EmailProcessor {
constructor(private readonly mailer: MailerService) {}
@Process()
async handle(job: Job<SendEmailJobData, SendEmailJobResult>): Promise<SendEmailJobResult> {
const { to, templateId, variables } = job.data
const result = await this.mailer.send({ to, templateId, variables })
return {
messageId: result.id,
acceptedAt: new Date().toISOString()
}
}
@OnWorkerEvent('failed')
onFailed(job: Job<SendEmailJobData> | undefined, error: Error): void {
// Full Job in hand: forward job.data + error to alerting / structured log.
// Move to a DLQ if attempts are exhausted (see §16.6).
}
}// services/pdf-pipeline.service.ts
import { Injectable } from '@nestjs/common'
import { FlowService } from '@bymax-one/nest-queue'
@Injectable()
export class PdfPipelineService {
constructor(private readonly flow: FlowService) {}
async generateReport(reportId: string): Promise<string> {
const result = await this.flow.add({
name: 'compose-pdf',
queueName: 'pdf',
data: { reportId },
children: [
{
name: 'fetch-data',
queueName: 'data',
data: { reportId }
},
{
name: 'render-charts',
queueName: 'render',
data: { reportId },
children: [
{ name: 'render-chart-1', queueName: 'render', data: { chartId: 'c1' } },
{ name: 'render-chart-2', queueName: 'render', data: { chartId: 'c2' } }
]
}
]
})
return result.job.id ?? ''
}
}// bootstrap/schedulers.bootstrap.ts
import { Injectable, OnApplicationBootstrap } from '@nestjs/common'
import { QueueService } from '@bymax-one/nest-queue'
@Injectable()
export class SchedulersBootstrap implements OnApplicationBootstrap {
constructor(private readonly queue: QueueService) {}
async onApplicationBootstrap(): Promise<void> {
// Daily cleanup at 03:00 São Paulo — idempotent by schedulerId, safe on every boot
await this.queue.upsertJobScheduler(
'maintenance',
'cleanup-soft-deleted', // schedulerId (stable upsert key)
{ pattern: '0 3 * * *', tz: 'America/Sao_Paulo' },
{ name: 'cleanup', data: { batchSize: 1000 } }
)
// Heartbeat every 5 minutes
await this.queue.upsertJobScheduler(
'monitoring',
'api-heartbeat',
{ every: 5 * 60 * 1000 },
{ name: 'heartbeat', data: { service: 'api', region: 'sa-east-1' } }
)
}
}// controllers/health.controller.ts
import { Controller, Get } from '@nestjs/common'
import { MetricsService } from '@bymax-one/nest-queue'
@Controller('health')
export class HealthController {
constructor(private readonly metrics: MetricsService) {}
@Get('queues')
async queues(): Promise<{ healthy: boolean; queues: readonly object[] }> {
const all = await this.metrics.getAll()
const stuck = all.filter(m => m.counts.active > 100 || m.counts.failed > 1000)
return {
healthy: stuck.length === 0,
queues: all
}
}
}// bootstrap/dynamic-workers.bootstrap.ts
import { Injectable, OnModuleInit } from '@nestjs/common'
import { WorkerRegistry } from '@bymax-one/nest-queue' // advanced API — for dynamic worker creation
@Injectable()
export class DynamicWorkersBootstrap implements OnModuleInit {
constructor(
private readonly workers: WorkerRegistry,
private readonly tenants: TenantsService
) {}
async onModuleInit(): Promise<void> {
const activeTenants = await this.tenants.findActive()
for (const tenant of activeTenants) {
await this.workers.register({
queueName: `email:${tenant.id}`,
handler: async job => {
// tenant-isolated processing
return { messageId: 'x', acceptedAt: new Date().toISOString() }
},
options: {
concurrency: tenant.tier === 'premium' ? 10 : 2,
limiter: { max: tenant.emailRateLimit, duration: 1000 }
}
})
}
}
}| Layer | Tooling | Scope | Gate |
|---|---|---|---|
| Unit | Jest + ts-jest | Every service, resolver, decorator, validator with mocked BullMQ/Redis | 100% line/branch on implemented files (jest.coverage.config.ts → 100/100/100/100) |
| E2E | Jest + @testcontainers/redis |
Real Redis: enqueue→process, schedulers, flows, deduplication, retry, graceful shutdown, connection-role policy | Must pass in CI; run on a dedicated config (jest.e2e.config.ts) |
| Mutation | Stryker (jest.stryker.config.ts) |
Kills surviving mutants in core logic | stryker.config.json thresholds { high: 99, low: 95, break: 95 }, target 100% — pre-release gate, not per-commit |
- Coverage thresholds are configured to fail the build below 100% on implemented files; new code ships with its tests in the same phase (TDD).
- Mutation testing is expensive, so it runs in the release pipeline (and on demand), gating publication on
break 95. - Memory safety: Jest is capped (
maxWorkers: '50%') and recursive workspace scripts are serialized, so a local library dependency is not duplicated across many workers.
Enqueue + process + typed result; bounded graceful shutdown (finish in-flight, force-close on timeout → stalled); 3-level flow completion; upsertJobScheduler fires repeatedly and re-upsert is idempotent; deduplication collapses duplicates; exponential retry → eventual success; Mode-A worker connection coerced to maxRetriesPerRequest:null while the Queue connection keeps defaults.
| Workflow | Trigger | Responsibility |
|---|---|---|
ci.yml |
PR + push | Install (frozen lockfile) → typecheck → lint → unit (100% coverage) → e2e (Testcontainers) → check-size.mjs budget |
codeql.yml |
push to main + weekly | Static analysis (TS/JS) |
scorecard.yml |
push to main + weekly | OpenSSF Scorecard (target ≥ 7.0) |
osv-scanner.yml |
PR + weekly | Dependency vulnerability scan (OSV) |
release.yml |
tag v* |
typecheck → lint → coverage → mutation gate → build → pnpm publish --provenance (OIDC trusted publishing) |
- All third-party actions pinned by commit SHA; workflows use least-privilege
permissions:. - Secret scanning via TruffleHog OSS in CI.
- Conventional Commits enforced by
commitlint.config.cjs; the changelog is generated from commit history. - Provenance attestation on every published artifact; npm publish token is 2FA/granular-scoped.
SECURITY.md (disclosure policy), CLAUDE.md + AGENTS.md (agent guidance), and the four Copilot review files (.github/copilot-instructions.md, instructions/code.instructions.md, instructions/tests.instructions.md, agents/agent-code-reviewer.agent.md).
nest-queue-example consumes the published tarball end-to-end (Mode A with @bymax-one/nest-cache, a @Processor, a Job Scheduler, a flow, and a /health queue endpoint) and runs in CI before a release is finalized.
- SemVer, strictly. Pre-1.0 (
0.x) may break minor; from1.0.0, breaking changes bump major. - The public surface is exactly what
src/server/index.tsandsrc/shared/index.tsexport. Re-exported BullMQ types are passthroughs and follow BullMQ. - Deprecation policy: a public symbol is marked
@deprecatedfor at least one minor before removal in the next major, with a documented replacement.
- Floor
bullmq ^5.16.0(Job Schedulers). Promotion to^5.16.0 || ^6.0.0happens once the e2e suite is green on v6; because the lib already uses Job Schedulers (not the removed repeatable API) and no other removed v6 API, no public-API break is expected.
- From a hand-rolled BullMQ/
@nestjs/bullmqsetup:@Processor+WorkerHost.process()maps to@Processor+@Process();@OnWorkerEvent/@OnQueueEventmap 1:1 (worker-local handlers keep fullJobaccess — see §6.5).@InjectQueue('q')producers route throughQueueService.getOrCreateQueue('q')/enqueue('q', …). - From the legacy repeatable API: replace
queue.add(name, data, { repeat })/addRepeatablewithupsertJobScheduler(queueName, schedulerId, repeat, { name, data }); replaceremoveRepeatable*withremoveJobScheduler(queueName, schedulerId).
The official @nestjs/bullmq is excellent and this lib does not replace it lightly — it targets the opinionated, multi-app Bymax use case. Where they overlap, the differentiators are explicit:
| Concern | @nestjs/bullmq (official) |
@bymax-one/nest-queue |
|---|---|---|
| Module setup | BullModule.forRoot() + registerQueue() per queue |
Single forRoot()/forRootAsync(); queues created on demand by name (no per-queue registration) |
| Connection | You wire ioredis + maxRetriesPerRequest yourself |
Dual mode (BYO client from @bymax-one/nest-cache, or lib-owned); correct per-role retry policy applied automatically |
| Job defaults | Per registerQueue |
Centralized, opinionated defaultJobOptions (retry/backoff/retention) with per-queue/per-job overrides |
| Concurrency | Default 1 (silent) |
Explicit concurrency enforced + warning; workload-type guidance |
| Producer API | @InjectQueue + raw queue.add |
Typed enqueue<TData, TResult>(), enqueueBulk, upsertJobScheduler, with native deduplication surfaced |
| Shutdown | Framework closes workers | Explicit bounded-drain protocol with force-close + stalled accounting + connection-ownership rules |
| Observability | Manual | telemetry passthrough (OpenTelemetry) + MetricsService |
| Decorators | @Processor/WorkerHost.process, @OnWorkerEvent, @OnQueueEvent |
@Processor/@Process(jobName?), @OnWorkerEvent, @OnQueueEvent (method-level dispatch) |
Choose this lib when you run several NestJS apps that should share one opinionated queue setup, want the @bymax-one/nest-cache connection-sharing story, and value enforced defaults + a tested shutdown protocol. Choose @nestjs/bullmq when you want the thinnest possible official binding and prefer to wire connection/defaults yourself.
The lib needs to work both in apps that already use @bymax-one/nest-cache (common case in the bymax-one portfolio) and in apps that don't want to pull in the entire cache lib just to run workers. Mode A optimizes for sharing; mode B optimizes for simplicity.
The BullMQ default of 1 is silently serial, which surprised teams in practice. 2 is a safe, non-serial starting point that has been used successfully in a production NestJS workload. It is not claimed to be optimal — §6.7 documents that the right value depends on workload type (I/O-bound vs CPU-bound) and that CPU-bound work belongs in a sandboxed processor (§6.8). The lib warns when concurrency is omitted and falls back to DEFAULT_WORKER_CONCURRENCY, so a missing value is always surfaced rather than silently serial.
Decorator covers 95% of cases (static, known at compile time). Programmatic API covers the remaining 5% (multi-tenancy with per-tenant workers). Keeping both does not cost significant complexity — they share 100% of the Worker creation logic via WorkerRegistry.
BullBoard is a product with its own release cadence, with heavy peer deps (@bull-board/express, @bull-board/api). Coupling to the lib forces versions and freezes choices. Since QueueService.getOrCreateQueue returns the native BullMQ Queue, the consumer plugs in BullBoard in one line whenever they want.
The module is global by default (isGlobal: true, mapped via ConfigurableModuleBuilder.setExtras) — QueueService is a singleton accessible in any module of the app, reaching every queue by name. A forFeature that did nothing would be a documented no-op, which is a credibility risk in a public API, so it is not shipped. If per-queue DI tokens (@InjectQueue('email')) are added later, they will arrive as a real forFeature(['email']) that actually registers those tokens — an additive, non-breaking change.
Runtime / data
-
maxRetriesPerRequest: nullapplied only to worker/QueueEvents connections; Queue connection keeps default retries (§2.3) - Options validated in
onModuleInit(early, clear errors; fail-fast on un-coercible worker connection) - No
job.datais logged by default — the lib never logs payloads; the consumer opts in explicitly (privacy/PII invariant) - Connection strings are masked (
redis://default:***@host:port) before any log, exception message, orQueueException.details - Job data is treated as opaque — never deep-merged into objects (prototype-pollution guard against
__proto__/constructorkeys) - Cron patterns validated by BullMQ's own
cron-parser(the lib rethrows parse failures asqueue.invalid_repeat_options), never a hand-rolled regex (avoids ReDoS and 6-field bugs) and no direct dependency -
enqueueBulk/addBulkarray length and per-job payload size are bounded (self-DoS / Redis-memory guard); large blobs use the claim-check pattern (§16.6) - Redis transport uses TLS in production (
connection.options.tls); connection config comes from secrets, never literals - Redis AUTH required and a least-privilege ACL user recommended for the queue connection; Mode-B URL is operator-trusted only (no SSRF from untrusted config)
Process / release (supply chain)
- Graceful shutdown tested in e2e with a real
SIGTERM(drain + force-close path) - Mutation score: Stryker
break 95(high 99, low 95), targeting 100%, as a pre-release gate - 100% line/branch coverage on implemented files
- OSV-Scanner clean;
pnpm auditclean on recommended peer deps - TruffleHog OSS secret scan clean
- OpenSSF Scorecard ≥ 7.0; GitHub Actions pinned by commit SHA
- Committed lockfile enforced; npm publish with provenance (OIDC trusted publishing); publish token 2FA-protected
-
SECURITY.mdpresent with a disclosure policy
Next step: after this spec is approved, expand §15 into
development_plan.md(phased dashboard) and one file per phase underdocs/tasks/.