GXS: faster message loading, meta caches warmed up by a background scan - #359
Open
jolavillette wants to merge 4 commits into
Open
GXS: faster message loading, meta caches warmed up by a background scan#359jolavillette wants to merge 4 commits into
jolavillette wants to merge 4 commits into
Conversation
jolavillette
force-pushed
the
perf/gxs-meta-single-scan
branch
4 times, most recently
from
August 10, 2026 07:21
75f1d85 to
52ab2eb
Compare
…ry per group
Reading the meta of a whole group runs SELECT ... WHERE grpId=?, which
INDEX_MESSAGES_GRPID serves with one row lookup per message. Since the
payload blob lives in the same row, those lookups are scattered over the
whole file: warming up the cache of N groups costs N passes of random
I/O over a database that is hundreds of megabytes.
When more than one group still needs a cold full read, read the meta of
every message in a single sequential scan instead and fill every
per-group cache from it. The file is then read in physical order, and
the cost no longer grows with the number of groups.
Measured on a synthetic database of the same shape and size as a real
gxsforums_db (235 MB, 23 KB rows, 20 groups), cold cache:
20 per-group queries 29449 ms
one sequential scan 2146 ms 13.7x
and the scan does not get more expensive as groups are added, where the
per-group path grows linearly with them. On a node subscribed to
hundreds of forums this is the difference between tens of seconds of
startup and a fixed couple of seconds.
Nothing else changes: same columns, same cache contents, same values
returned. Callers and public API are untouched, and no database schema
or format is modified.
Trade-off: the scan fills the cache for groups that were not requested
yet. That is the same memory the cache reaches as soon as those groups
are browsed, but it is reached up front rather than progressively.
The scan reports itself through the existing opt-in profiler, so the
gain is verifiable on a real profile rather than taken on trust:
GXS-PROF loadAllMsgMetaInOneScan db=gxsforums_db groups=571 metas=48213 in 2100ms
Stacked on the channel loading branch: both reshape the same function,
and this one reuses the profiler introduced there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ed slices The warm-up scan was triggered synchronously inside retrieveGxsMsgMetaData by the second cold whole-group request, and ran under mDbMutex in one go. Cold page cache, it was measured at up to 57 s on a real gxsforums_db (235 MB): the caller -- possibly asking for a handful of metas from one group -- and every other reader of the service froze for that long at startup. Keep the trigger and the sequential scan, but run it on a dedicated thread in slices of 4096 rows by increasing rowid, taking mDbMutex only for the duration of one slice so readers interleave. Until the scan completes, cold groups keep being served by the indexed per-group query. Messages stored while the scan runs are cached by storeMessage() itself, so rowid reuse after deletions cannot leave a hole. The thread is joined in the destructor before the DB is closed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One line per database (rows, slices, duration) so the background warm-up can be observed and validated from the logs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fixed 4096-row slice held mDbMutex for ~10 s on a cold large-row database (23492 forum metas warmed in 6 slices of ~10 s each), stalling single-group readers for that long -- the very stall the background scan exists to avoid. Start at 256 rows and rescale each slice towards a 250 ms target, clamped to [64, 4096] rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jolavillette
force-pushed
the
perf/gxs-meta-single-scan
branch
from
August 12, 2026 10:38
52ab2eb to
2eb3604
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Performance work on the GXS message loading path, measured on a real profile (gxsforums_db 235 MB, gxschannels_db 638 MB, SQLCipher):
🤖 Generated with Claude Code