You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
service-job: type: 'once' schedules on DbJobAdapter get no leader election either — same routing limb as #13686, deliberately left out of its scope #13918
Found while implementing #13686 (interval leader election). Filed rather than fixed there: #13686's scope, its field evidence and #2219's declared capability are all about cron/interval, and widening the routing to a third schedule type is a behaviour change the reporter's cluster cannot confirm.
The gap
DbJobAdapter.schedule() decides which adapter owns a scheduled fire, and only the adapter it picks decides whether that fire is leader-elected:
type: 'cron' → this.cron (CronJobAdapter) — fires through runScheduled(), which takes lock.acquire('job:' + name, { waitMs: 0 }) and skips when a peer holds it ✓
type: 'once' → this.inner (IntervalJobAdapter) — a bare setTimeout, no lock anywhere in that file ✗
So on a multi-replica deployment a one-shot job registered with { type: 'once', at } runs once per replica, not once per cluster. Same failure shape as #13686 and, for a one-shot, arguably a worse one: there is no later tick during which a de-duplication marker could win, so every replica's copy lands in the same short window.
The fix is already sitting there
CronJobAdapter.schedule() handles type: 'once' itself (else if (schedule.type === 'once' && schedule.at)) and arms it with setTimeout(() => { void this.runScheduled(name); }, delay) — the same leader-elected fire path its cron and interval limbs use. So the repair is the one #13686 applied to the interval limb, one branch over: delegate once to this.cron when one is assembled, and register it on inner via IntervalJobAdapter.register() (stores without arming a timer) so trigger() / replay() / getExecutions() / listJobs() are unaffected.
What needs deciding before someone writes that
Not mechanical, which is why this is an issue and not a follow-up commit:
Crash semantics differ for a one-shot. For interval, a leader that dies mid-fire costs one tick and the next tick re-elects. For once there is no next tick: the lease expires with the work undone and nothing re-arms it, so "exactly once per cluster" becomes "at most once per cluster". That may be fine, or it may want the lock to be released only on success — a decision, not a detail.
Repro sketch
Same instrument as packages/services/service-job/src/db-job-adapter.interval-leader.test.ts: two DbJobAdapter stacks over one fake engine and one shared lock, a { type: 'once', at: <now + tick> } registration on each, one advanceTimersByTimeAsync. Today that executes the handler twice and writes two sys_job_run rows.
Found while implementing #13686 (interval leader election). Filed rather than fixed there: #13686's scope, its field evidence and #2219's declared capability are all about
cron/interval, and widening the routing to a third schedule type is a behaviour change the reporter's cluster cannot confirm.The gap
DbJobAdapter.schedule()decides which adapter owns a scheduled fire, and only the adapter it picks decides whether that fire is leader-elected:type: 'cron'→this.cron(CronJobAdapter) — fires throughrunScheduled(), which takeslock.acquire('job:' + name, { waitMs: 0 })and skips when a peer holds it ✓type: 'interval'→this.cronas of service-job: DbJobAdapter 的 interval 型调度在多副本下无 leader-election(cron 型有)—— #2219 声明的 interval 半边缺失,竞态实锤重复执行 #13686 ✓type: 'once'→this.inner(IntervalJobAdapter) — a baresetTimeout, no lock anywhere in that file ✗So on a multi-replica deployment a one-shot job registered with
{ type: 'once', at }runs once per replica, not once per cluster. Same failure shape as #13686 and, for a one-shot, arguably a worse one: there is no later tick during which a de-duplication marker could win, so every replica's copy lands in the same short window.The fix is already sitting there
CronJobAdapter.schedule()handlestype: 'once'itself (else if (schedule.type === 'once' && schedule.at)) and arms it withsetTimeout(() => { void this.runScheduled(name); }, delay)— the same leader-elected fire path its cron and interval limbs use. So the repair is the one #13686 applied to the interval limb, one branch over: delegateoncetothis.cronwhen one is assembled, and register it oninnerviaIntervalJobAdapter.register()(stores without arming a timer) sotrigger()/replay()/getExecutions()/listJobs()are unaffected.What needs deciding before someone writes that
Not mechanical, which is why this is an issue and not a follow-up commit:
type: 'once'against a multi-replica assembly was not measured — worth measuring before spending the change, per the startup-scope-discipline axis.oncethere is no next tick: the lease expires with the work undone and nothing re-arms it, so "exactly once per cluster" becomes "at most once per cluster". That may be fine, or it may want the lock to be released only on success — a decision, not a detail.Repro sketch
Same instrument as
packages/services/service-job/src/db-job-adapter.interval-leader.test.ts: twoDbJobAdapterstacks over one fake engine and one shared lock, a{ type: 'once', at: <now + tick> }registration on each, oneadvanceTimersByTimeAsync. Today that executes the handler twice and writes twosys_job_runrows.Landing site:
packages/services/service-job/src/db-job-adapter.ts(schedule()).