Repository navigation
Name the store's own text-search statements in the PostgreSQL log (#9236) - #9240
Open
erikdarlingdata wants to merge 2 commits into
Open
erikdarlingdata wants to merge 2 commits into
erikdarlingdata wants to merge 2 commits into
Conversation
erikdarlingdata
marked this pull request as ready for review
November 21, 2026 07:59
|
Review notes, all non-blocking:
|
There was a problem hiding this comment.
Verdict: LGTM
Claims
- VERIFIED —
RedactStoredStatementwithholds a statement carryingESCAPE '\'—Darling/Darling.Tests/PgLogEventsPipelineTests.cs:948assertsnullfor exactly... LIKE 'a\_%' ESCAPE '\' ...(an odd backslash run before a quote). - VERIFIED — PostgreSQL's default LIKE/ILIKE escape is the backslash, so results are the same — every changed pattern is a bound parameter built by an escaper that writes
\(SchemaHistoryReader.cs:356,380,385,DarlingMcpFindQueriesByTextTools.cs:589-590,CollectedDatabases.SearchPattern,AlertContextSerializer.BuildDedupKeyLikePattern). The one literal,'%\%%'(PgSettingScrub.cs:238), has a backslash before%, not before a quote. Live tests that would fail if the escape were lost:PercentAndUnderscore_MatchOnlyThemselves,TheRoute_Search_MatchesPercentUnderscoreAndBackslashOnlyAsThemselves,FindSchemaObjects_NamePatterns_AreLiteral_NotLikeWildcards. - VERIFIED — The redactor is unchanged —
PerformanceMonitor.Collectors/PgLogTextRedactor.csis not ingh pr diff --name-only. - VERIFIED — find_queries_by_text: six statements (Query Store, plan cache, all-servers, the two span statements, matched digests) — six hunks at
DarlingMcpFindQueriesByTextTools.cs:172,226,275,308,361,499, insideQueryStoreSql,PlanCacheSql,PlanCacheAllServersSql,PlanCacheAllServersSpanSql,PlanCacheAllServersLookupSpanSqlandMatchedDigestsSql. NoILIKEremains with the clause in that file. - VERIFIED — SchemaHistoryReader: the text-search select list and filter, and the name prefix / name contains filters —
SchemaHistoryReader.cs:361-365, 380, 385, 408-411. - VERIFIED — CollectedDatabases database-name search —
CollectedDatabases.cs:63. - VERIFIED — PgAlertHistoryStore dedup-key read —
PgAlertHistoryStore.cs:506. - VERIFIED — PgSettingScrub: the candidate read (three clauses), and
CandidateSqlmadeinternal—PgSettingScrub.cs:212, 238. - VERIFIED — The comments were updated, and
ServerDatabasesEndpointLiveTestsnow looks for the new spelling —CollectedDatabases.cs:58,PgSettingScrub.cs:187-188,ServerDatabasesEndpointLiveTests.cs:490,497. - VERIFIED — The sites left alone are shared with DuckDB or are DuckDB-only, and a PostgreSQL statement built from them is still withheld —
AttentionWindowSql.cs:133,145(used byDarlingAlertReadAdapter.cs:927-990),LongRunningQueryExclusions.cs:324,331,346(used byDarlingPgSessionStatesReader),DatabaseExclusionFilter.cs:64,122, andDuckDbAlertHistoryStore.cs:191,261,336. None of these builders takes a dialect parameter. - VERIFIED — The new test has 12 cases covering the find_queries constants,
AllServersJoinSql,NamesSearchLimitedSqlandCandidateSql—PgStatementsReadableInTheLogTests.cs:28-39. - VERIFIED — No behaviour test reaches the
SchemaHistoryReaderandPgAlertHistoryStorestatements without a database — both build their SQL inside async methods that take a connection (SchemaHistoryReader.cs:349,PgAlertHistoryStore.cs:495). - UNVERIFIED — Revert proof: 9 of 12 fail on dev's text, 12 of 12 pass — needs a test run. The count of 9 matches the tree (8 find_queries cases plus
NamesSearchLimitedSqlhad the clause on dev). The note "every one that reads those two files" is imprecise: the two probe cases read that file and pass on revert (see the PR comment). - UNVERIFIED —
pre-push.shguard totals (Darling 4980 / Lite 1242, 0 failed) and a build with 0 warnings — needs a build and a test run. - UNVERIFIED — The PostgreSQL 18.7 rig runs of the named live classes (117 + 211 tests) are green — needs the rig.
Riskiest lines
Darling/PerformanceMonitor.Darling.Storage/PgAlertHistoryStore.cs:506— the dedup LIKE now depends on the default escape for the\_/\%/\\thatBuildDedupKeyLikePatternwrites. Without an escape, the positive match on an underscored key would miss and the Tier-0 cooldown seed would be lost — covered byHoldEarnedByDeliveryTests.TheTier0Seed_StillSeesAMutedUndeliveredRow_WithItsDedupKey(live; the keyorders_dbis escaped toorders\_db, so the match fails if the escape is lost)Darling/PerformanceMonitor.Darling.Service/Mcp/DarlingMcpFindQueriesByTextTools.cs:172— Query Store text match with escaped%/_— covered byFindQueriesByTextTests.PercentAndUnderscore_MatchOnlyThemselvesDarling/PerformanceMonitor.Darling.Storage/SchemaHistoryReader.cs:380—lower(o.object_name) LIKE lower($n)with no clause;lower()keeps the backslash — covered byDarlingMcpSchemaToolsLiveTests.FindSchemaObjects_NamePatterns_AreLiteral_NotLikeWildcards
Traps near this diff
- "Lite and Darling ship in parity (CONTRIBUTING's two-store rule): a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding, not a nit" — does not apply. The fix is for PostgreSQL's log redactor, which Lite never feeds. The DuckDB twin keeps
ESCAPE '\\'correctly atLite/Services/DuckDbAlertHistoryStore.cs:191,261,336, because DuckDB has no default escape. - "A roster-based census does not see the file it does not roster" — does not apply as a defect. The new theory is a roster (
PgStatementsReadableInTheLogTests.cs:28-39) that missesSchemaHistoryReaderandPgAlertHistoryStore. The PR body names both omissions and says why, and the shared builders still withheld are listed as left alone on purpose. - "Ruling (#5833): tests are load-bearing. … never answer a trap with a test that reads source text or asserts how code is spelled" — does not apply. The new test runs the redactor on the statements and asserts its output, which is behaviour. The edit at
ServerDatabasesEndpointLiveTests.cs:497loosens an existing SQL-spelling pin and adds none. - "Every MCP filter is part of the query, not a post-filter on the page" — does not apply. The
ILIKEstays ahead ofORDER BY/LIMITin every changed statement (DarlingMcpFindQueriesByTextTools.cs:172-174,CollectedDatabases.cs:63-65). - "A diff that touches every line of a file is a line-ending conversion until proven otherwise" — does not apply. Every hunk is a few lines (
gh pr diff 9240: +78/-26).
Divergence (blind read vs PR body)
None of substance. The body's account matches the diff on every site. My blind read found the Darling PostgreSQL statements still built with ESCAPE '\' (AttentionWindowSql, LongRunningQueryExclusions, DatabaseExclusionFilter), and the body discloses those as deliberately left alone. One small mismatch: the revert-proof parenthetical "every one that reads those two files" overstates it, because the two probe cases read that file and never carried the clause. There is also a doc the body does not mention: AlertContext.cs:789 still says to pair the pattern with ESCAPE '\'. Both are in the PR comment as non-blocking notes.
Reviewer checklist
- Run the PostgreSQL live classes on a rig (
FindQueriesByTextTests,ServerDatabasesEndpointLiveTests,DarlingMcpSchemaToolsLiveTests,HoldEarnedByDeliveryTests,PgSettingScrubLiveTests) and confirm that metacharacter searches still match only themselves. - On a live store, force a find_queries_by_text timeout and confirm the store log shows the statement text instead of "statement withheld".
- Open a follow-up issue for a dialect flag on
AttentionWindowSql/LongRunningQueryExclusions/DatabaseExclusionFilter, so their PostgreSQL statements stop being withheld.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claude posting for Erik Darling.
Fixes #9236
PgLogTextRedactor.RedactStoredStatementwithholds any statement that carriesESCAPE '\'. PostgreSQL's default LIKE/ILIKE escape character is already the backslash, so the clause is dropped from the statements that run only on PostgreSQL. Results are the same, and the log can now show the statement. The redactor is unchanged.Sites changed (PostgreSQL only)
DarlingMcpFindQueriesByTextTools.cs: six statements, not five (Query Store, plan cache, all-servers, the two span statements, the matched-digests read).SchemaHistoryReader.cs: the text-search select list and filter, and the name prefix / name contains filters.CollectedDatabases.cs: the database-name search.PgAlertHistoryStore.cs: the dedup-key read.PgSettingScrub.cs: the candidate read (three clauses). ItsCandidateSqlfield is nowinternalso a test can reach it.CollectedDatabases,PgSettingScrub). One existing test,ServerDatabasesEndpointLiveTests, looked for the old spelling and now looks for the new one.Sites left alone
LongRunningQueryExclusions.BuildSqlPredicates: no dialect parameter. Lite's DuckDB wait-stats read and the PostgreSQL reads both call it, and DuckDB has no default escape character.AttentionWindowSql: the same builders serve Lite (DuckDB) and the Darling PostgreSQL reads, with no dialect parameter.DatabaseExclusionFilter: no dialect parameter. It writes for SQL Server and Lite collectors and for the PostgreSQL target provider.DuckDbAlertHistoryStore(three statements): DuckDB, never in a PostgreSQL log.///comment inDarlingPgSessionStatesReader.csdescribesBuildSqlPredicates, which is unchanged.Giving these builders a dialect flag is a separate change. Until then, a PostgreSQL statement built from them is still withheld in the log.
Test
New
PgStatementsReadableInTheLogTestspasses the statement the product sends throughRedactStoredStatementand asserts text comes back: the find_queries_by_text constants andAllServersJoinSql,CollectedDatabases.NamesSearchLimitedSqlandPgSettingScrub.CandidateSql. It has 12 cases.Revert proof: with
DarlingMcpFindQueriesByTextTools.csandCollectedDatabases.csput back to dev's text, 9 of 12 cases fail (every one that reads those two files). With the fix, 12 of 12 pass.No behaviour test reaches the
SchemaHistoryReaderandPgAlertHistoryStorestatements, because both are assembled inside a method that needs a database. Their live classes ran and pass.Gate and classes
pre-push.sh: GATE STOP (the selection is the whole suite). Guard lines:[guard-darling] Total: 4980, Failed: 0and[guard-lite] Total: 1242, Failed: 0. Build has 0 warnings.*FindQueriesByText*,QueryTextTrigramIndexTrackerTests,QueryStoreTextSeenLiveTests(117 tests);DarlingMcpSchemaToolsLiveTests,CollectedDatabasesSqlTests,ServerDatabasesEndpointLiveTests,*ExcludedDatabases*,NameOrderWithoutCaseLiveTests,*PgSettingScrub*,AlertResolutionHistoryLiveTests,OpenEngineAlertLiveTests,PgStatementTextScrubLiveTests,PgStatementsReadableInTheLogTests(211 tests).origin/dev(no overlap with these files) the project builds with 0 warnings and the new class passes.Release note: Store log entries for a timed-out or slow find_queries_by_text call, and a few other store reads, now show the statement instead of "statement withheld".