docs(benchmark): record durability write-path campaign results (§7.3)#244
Merged
Conversation
-#242) Add BENCHMARK.md section 7.3 documenting the 2026-07-08 durability write-path campaign and update the tuning docs with the measured ratios. BENCHMARK.md: - section 7.3: full vs-Redis matrix (GCE c3-standard-8, Redis 7.0.15, --shards 2, 3 alternated reps, provenance-probed) -- appendfsync always P16 SET 0.12x -> 0.91x, everysec P16 SET 1.32x WIN, everysec P1 SET 0.80x -> 0.99x parity, pub/sub fan-out delivery 438 msg/s -> 5.09M msg/s (0 drops, 1.04x Redis). - per-PR root-cause table with the strace diagnostics that drove each fix (149,718 -> 96 futex calls for the park-free poll; 127,812 write(2) per 8s -> one write per batch for the coalesced write). - durability-invariant note (fsync-before-ack preserved; crash-matrix 3/3) and same-instance A/B reproduction steps (section 15). - executive summary + header updated. docs/production-guide.md, docs/benchmarks.md, docs/configuration.md: - appendfsync tuning guidance updated with the measured ratios and a plain-language explanation of group commit, coalesced batch writes, and the park-free writer poll -- all automatic, durability unchanged. Docs-only; no code paths touched. author: Tin Dang
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
📝 WalkthroughWalkthroughDocumentation-only update recording a 2026-07-08 durability write-path benchmark campaign. BENCHMARK.md gains updated headline metrics, an Executive Summary refresh, a new §7.3 section, and harness snippets. CHANGELOG.md, docs/benchmarks.md, docs/configuration.md, and docs/production-guide.md are updated with corresponding appendfsync durability guidance and write-path internals. ChangesDurability Benchmark Documentation
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/benchmarks.md`:
- Around line 109-110: The benchmark wording overstates the futex-wake rate in
the performance summary. Update the futex-storm description in the benchmarks
text to reflect the measured rate from the data (~19K/sec over the 8 s run,
based on 149,718 futex calls) and keep the rest of the AOF-writer/pipeline
batching explanation unchanged; use the existing benchmark narrative around the
AOF-writer poll and everysec behavior to locate the sentence.
In `@docs/production-guide.md`:
- Around line 318-324: The futex-rate wording in the Park-free writer poll
explanation is overstated and should match the benchmark figure already used
elsewhere. Update the numeric claim in the relevant prose near the AOF writer
polling discussion to reflect about 149,718 futex calls over 8 seconds, i.e.
roughly 19K/sec, and keep the rest of the explanation intact. Use the existing
"Park-free writer poll (`everysec`/`no`)" section in docs/production-guide.md as
the anchor for the edit.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: bd8a7de5-0c65-4cae-8135-a4f61a2184fa
📒 Files selected for processing (5)
BENCHMARK.mdCHANGELOG.mddocs/benchmarks.mddocs/configuration.mddocs/production-guide.md
Comment on lines
+109
to
+110
| The wins come from per-batch group commit (one fsync per pipeline batch, not per command), a single coalesced `write_all` per batch, a park-free AOF-writer poll that removes a ~150K/sec futex-wake storm on the shard thread under `everysec`, and coalesced pub/sub delivery writes. Durability is unchanged (`always` = RPO 0, `everysec` = RPO ≤ 1s), verified by the SIGKILL crash-recovery matrix. Full detail: `BENCHMARK.md` §7.3. | ||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Fix the futex-rate wording.
The benchmark data is 149,718 futex calls over an 8 s run, so this is ~19K/sec, not ~150K/sec. The current phrasing overstates the overhead by about 8×.
♻️ Suggested text fix
- a park-free AOF-writer poll that removes a ~150K/sec futex-wake storm on the shard thread under `everysec`,
+ a park-free AOF-writer poll that removes a ~19K/sec futex-wake storm on the shard thread under `everysec`,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| The wins come from per-batch group commit (one fsync per pipeline batch, not per command), a single coalesced `write_all` per batch, a park-free AOF-writer poll that removes a ~150K/sec futex-wake storm on the shard thread under `everysec`, and coalesced pub/sub delivery writes. Durability is unchanged (`always` = RPO 0, `everysec` = RPO ≤ 1s), verified by the SIGKILL crash-recovery matrix. Full detail: `BENCHMARK.md` §7.3. | |
| The wins come from per-batch group commit (one fsync per pipeline batch, not per command), a single coalesced `write_all` per batch, a park-free AOF-writer poll that removes a ~19K/sec futex-wake storm on the shard thread under `everysec`, and coalesced pub/sub delivery writes. Durability is unchanged (`always` = RPO 0, `everysec` = RPO ≤ 1s), verified by the SIGKILL crash-recovery matrix. Full detail: `BENCHMARK.md` §7.3. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/benchmarks.md` around lines 109 - 110, The benchmark wording overstates
the futex-wake rate in the performance summary. Update the futex-storm
description in the benchmarks text to reflect the measured rate from the data
(~19K/sec over the 8 s run, based on 149,718 futex calls) and keep the rest of
the AOF-writer/pipeline batching explanation unchanged; use the existing
benchmark narrative around the AOF-writer poll and everysec behavior to locate
the sentence.
Comment on lines
+318
to
+324
| - **Park-free writer poll (`everysec`/`no`).** The AOF writer thread does not park | ||
| in a blocking channel receive; it polls. This matters because a parked receiver | ||
| forces every shard thread to issue a futex wake on each write — at non-pipelined | ||
| `everysec` load that was ~150k wakes/sec of pure overhead on the hot path. | ||
| Polling keeps producer enqueues as plain userspace atomics, restoring P1 parity | ||
| with Redis. Under `always` the writer still parks (the client is already blocked | ||
| on the fsync ack, so receive latency there is client-visible RTT anyway). |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Fix the futex-rate wording here too.
This repeats the same benchmark figure as docs/benchmarks.md: 149,718 futex calls in an 8 s run, so the rate is ~19K/sec, not ~150K/sec. The current text inflates the non-pipelined overhead and weakens the explanation.
♻️ Suggested text fix
- a parked receiver forces every shard thread to issue a futex wake on each write — at non-pipelined
- `everysec` load that was ~150k wakes/sec of pure overhead on the hot path.
+ a parked receiver forces every shard thread to issue a futex wake on each write — at non-pipelined
+ `everysec` load that was ~19k wakes/sec (149,718 total over an 8 s run) of pure overhead on the hot path.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| - **Park-free writer poll (`everysec`/`no`).** The AOF writer thread does not park | |
| in a blocking channel receive; it polls. This matters because a parked receiver | |
| forces every shard thread to issue a futex wake on each write — at non-pipelined | |
| `everysec` load that was ~150k wakes/sec of pure overhead on the hot path. | |
| Polling keeps producer enqueues as plain userspace atomics, restoring P1 parity | |
| with Redis. Under `always` the writer still parks (the client is already blocked | |
| on the fsync ack, so receive latency there is client-visible RTT anyway). | |
| - **Park-free writer poll (`everysec`/`no`).** The AOF writer thread does not park | |
| in a blocking channel receive; it polls. This matters because a parked receiver | |
| forces every shard thread to issue a futex wake on each write — at non-pipelined | |
| `everysec` load that was ~19k wakes/sec (149,718 total over an 8 s run) of pure overhead on the hot path. | |
| Polling keeps producer enqueues as plain userspace atomics, restoring P1 parity | |
| with Redis. Under `always` the writer still parks (the client is already blocked | |
| on the fsync ack, so receive latency there is client-visible RTT anyway). |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/production-guide.md` around lines 318 - 324, The futex-rate wording in
the Park-free writer poll explanation is overstated and should match the
benchmark figure already used elsewhere. Update the numeric claim in the
relevant prose near the AOF writer polling discussion to reflect about 149,718
futex calls over 8 seconds, i.e. roughly 19K/sec, and keep the rest of the
explanation intact. Use the existing "Park-free writer poll (`everysec`/`no`)"
section in docs/production-guide.md as the anchor for the edit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the 2026-07-08 durability write-path campaign (PRs #238–#242) in
BENCHMARK.md§7.3 and updates the AOF tuning docs with the measured ratios. Docs-only — no code paths touched.What's recorded
BENCHMARK.md§7.3 — full vs-Redis matrix (GCE c3-standard-8, Redis 7.0.15,--shards 2, 3 alternated reps, provenance-probed):alwaysSET p16everysecSET p16everysecSET p1Plus: per-PR root-cause table with the strace diagnostics that drove each fix (149,718 → 96 futex calls; 127,812 → one-write-per-batch), the durability-invariant note (fsync-before-ack preserved, crash-matrix 3/3), and same-instance A/B reproduction steps.
Tuning docs (
production-guide.md,benchmarks.md,configuration.md) get the measured ratios and a plain-language explanation of group commit / coalesced writes / park-free writer poll — all automatic, durability unchanged.CHANGELOG updated.
Summary by CodeRabbit
appendfsyncmodes and updated performance notes.