Add confirm_lsn to replay a load-test backlog from the start - #129
Open
lukashes wants to merge 1 commit into
Open
Add confirm_lsn to replay a load-test backlog from the start#129lukashes wants to merge 1 commit into
lukashes wants to merge 1 commit into
Conversation
With confirm_lsn=false the slot is never advanced: feedback keeps flowing as a keepalive but always confirms the first LSN, so PostgreSQL keeps the WAL and every restart re-reads the same backlog. Generate load once, then rerun make start-outboxx as many times as needed. Default stays true, where advancing the slot is what makes delivery at-least-once. The load stand enables it and builds with -Dlog_level=info to keep the run quiet.
Contributor
📊 Benchmark ResultsCurrent run is the minimum over 3 passes, compared against the base branch (
Summary: ➡️ 11 neutral · ⚪ 2 ignored (sub-μs) Thresholds: <1μs ignore · 1–20μs 15% · 20–50μs 10% · ≥50μs 5%. Measured on a shared CI runner — treat small deltas as noise. Informational only; this check never fails the build. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Load tests want to generate a backlog once and then measure the reader against the same data repeatedly. Today the first
make start-outboxxconsumes the backlog and confirms the LSN, so the slot advances and the WAL is recycled; a second run reads nothing.Solution
confirm_lsnto[source.postgres], defaulttrue. Whenfalse,sendFeedbackkeeps sending (so the keepalive and liveness path are unchanged) but always confirms the first LSN seen, so the slot never advances and PostgreSQL keeps the WAL. Every restart then re-reads the same backlog from the start.mainwires the flag and warns at startup when it is off (the mode is not durable and must not ship to production).confirm_lsn = false, and its image builds with-Dlog_level=infoto keep the run quiet.Workflow:
make loadonce, thenmake start-outboxxas many times as you like.Verification
New integration test drives a real slot: with
confirm_lsn = false, after reading a multi-row batch and callingsendFeedback(last_lsn), the slot'sconfirmed_flush_lsnsettles on the first LSN, below the batch's last.