Repository navigation
Conversation
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: b399e0db-7004-448d-b501-748f6fe96457) |
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fanout could persist a checkpoint past events that their owning instance never read. In the 2026-09-30 13:03 incident, a batch landed after the workers had read but before the master read. Because the master counted only the instances that checkpointed, it advanced past the batch.
ReadTracker(wrappers/fanout-read-tracker.ts) records the furthest eid each instance can safely claim, including when it owned no events. Each instance's response includes that position, so the master takes the min across all instances.reduceCheckpointsthrows if an instance didn't report a queue that another instance reported. SetstrictCheckpoints: falseto turn this off.__cron.ts(falls back to now if missing or in the future). Workers get it, plus the starting checkpoints, from the master.reduceCheckpointsusedcheckpointData.read[queue]where it meantcheckpointData[queue].test/lib.streams.utest.tsnow restoresDate.nowafter itscounteranddevnulltests. Before, it left the clock stubbed to1000, which broke the new fanout tests whenever they ran after it.Test plan
fanout-read-tracker.utest,fanout-reduce-guard.utest,fanout-read-position.utest(replays the 13:03 incident through the real wrapper)*.utest.jssuite passes locally: 336 passing, 0 failing🤖 Generated with Claude Code
Note
High Risk
Changes core fanout checkpoint semantics and master/worker coordination; incorrect behavior could skip or duplicate queue events in production.
Overview
Fixes fanout persisting a checkpoint past events that partition owners never read (e.g. the 2026-09-30 13:03 incident when a batch landed after workers read but before the master).
Read position voting: Each instance now tracks how far it read via
ReadTracker, including unowned events it only filtered past. Worker/master responses usereportCheckpoints()so the master still takes the min across all instances, not only instances that calledcheckpoint.Guards and read window:
reduceCheckpointsthrows when an instance is silent for a queue another instance reported (strictCheckpoints: falseopts out). The shared max eid comes fromgetMaxEid(__cron.ts, with fallbacks), and workers receivemaxeidplus starting checkpoints from the master. A merge typo (checkpointData.read[queue]→checkpointData[queue]) is fixed.Tests: New unit/integration tests replay the incident; stream tests restore
Date.now/console.logaftercounteranddevnullso later fanout tests are not affected.Reviewed by Cursor Bugbot for commit 20b0399. Bugbot is set up for automated code reviews on this repo. Configure here.