Skip to content

fix(raft): queue each committed index once per Ready window - #348

Merged
farhan-syah merged 1 commit into
mainfrom
fix/p2-duplicate-index
Sep 23, 2026
Merged

farhan-syah merged 1 commit into
mainfrom
fix/p2-duplicate-index

Conversation

@EnRaiha

@EnRaiha EnRaiha commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

The P2 review left one unexplained shape: a Raft batch carrying the same committed index twice. The applier's delivery guard (each committed index handed off at most once, watermark claimed in one critical section) is the boundary — but the producer-side cause was unknown, and an append-shaped write gains a row if that guard ever slips.

Root cause found: collect_committed_entries resumed from last_applied alone. It runs on every commit-index advance — a follower's AppendEntries, a single voter's propose — while the loop drains Ready and advances last_applied later. Two advances inside one window queued the same committed range twice, putting one index twice into a single batch.

Part of #165 (P2 — cluster consensus safety).

What

  • collect_committed_entries resumes past the furthest index already queued into Ready (max(last_applied, queued_through)), so a second advance in the window queues only what is new. No new state, O(1).
  • apply_group_commits records the first committed index that is not greater than its predecessor (first_non_increasing_committed_index, zero-allocation windows(2)) with a warn — a producer-side regression is observable, and the applier guard stays the boundary.

Validation

  • Red proof: the new raft test fails on the base producer (repeated indices in one batch) and passes with the fix.
  • cargo test -p nodedb-raft --lib node:: — 86 passed.
  • cargo test -p nodedb-cluster --lib apply_committed — 1 passed (detector).
  • cargo check -p nodedb-cluster --all-targets — clean; repository preflight passes.

Notes

  • A cross-tick re-queue while the async data-plane apply lags remains possible by design (a fresh Ready starts empty). That is a repeated delivery across batches, covered by the applier's watermark guard, not a duplicate inside one batch.

Copilot AI lite review requested due to automatic review settings September 19, 2026 07:46
@EnRaiha EnRaiha added the area:cluster-raft Raft, replication, consensus safety label Sep 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@farhan-syah farhan-syah left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root cause verified independently: collect_committed_entries is the only producer of ready.committed_entries, and the loop calls advance_applied only after apply_group_commits, so two commit advances between drains re-queue the range. Fix at the producer, no new state. Correct layer.

Claims verified: red proof reproduces on the base producer ([2, 2, 3]), nodedb-raft node:: 86 pass, nodedb-cluster apply_committed 1 pass, clippy --all-targets -D warnings clean on both crates, fmt clean.

Two inline notes, neither blocks. After the change, amend the commit or squash: one commit on this branch.

Comment thread nodedb-cluster/src/raft_loop/tick/apply_committed.rs Outdated
Comment thread nodedb-cluster/src/raft_loop/tick/apply_committed.rs Outdated
collect_committed_entries resumed from last_applied alone. It runs on every
commit-index advance — a follower's AppendEntries, a single voter's propose —
while the loop drains Ready and advances last_applied later, so two advances
inside one window queued the same committed range twice and the applier
received one committed index twice in a single batch. An append-shaped write
gains a row from that repeat; the applier's delivery guard is the boundary,
not the reason it was invisible.

The collect now resumes past the furthest index already queued into Ready.
A tick apply path also records the first committed index that is not greater
than its predecessor, so a producer-side regression is observable instead of
silent.
@EnRaiha
EnRaiha force-pushed the fix/p2-duplicate-index branch from c1ac606 to 218b1bb Compare September 21, 2026 15:26
@EnRaiha
EnRaiha requested a review from farhan-syah September 21, 2026 15:26
@farhan-syah
farhan-syah merged commit bd534ea into main Sep 23, 2026
5 checks passed
@farhan-syah
farhan-syah deleted the fix/p2-duplicate-index branch September 23, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:cluster-raft Raft, replication, consensus safety

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants