Skip to content

Leave the monitor's own statement and dropped databases out of Query Store regressions (#6654) - #6665

Merged
erikdarlingdata merged 5 commits into
devfrom
fix/6654-regressions-marker-dropped
Oct 24, 2026
Merged

erikdarlingdata merged 5 commits into
devfrom
fix/6654-regressions-marker-dropped

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Claude posting for Erik Darling.

Closes #6654.

Why

get_query_store_regressions listed the monitor's own statement and queries from a database the server has dropped. On Darling the collector's own-statement drop (QueryStoreCollector.SelfQueryMarker, QueryStoreCollector.cs:1792) never fires: it tests row.QueryText, and FetchQueryTextSeparately (on for Darling) nulls that column, so the statement's stats rows are stored and its text lands in query_store_text with no marker check. Every reader that resolves text from query_store_text can therefore list it. The regressions read had no text filter and no current-database filter, and dropped databases also filled the row cap. Round 1 (review 5636588372) widened the fix to every ranked Query Store reader (below).

What changes

  • QueryStoreRegressionBaseline.SuffixSql (shared by the MCP tool and the Viewer grid): the qualified stage, before the rank and the cap, now drops (a) a candidate whose resolved text (the same keyed query_store_text lookup the projection makes, with the MAX(query_text) sample as the fallback) contains SelfQueryMarker, and (b) a candidate in an excluded database (new @@X token, a text[] parameter: $10 in the MCP read, $9 in the Viewer read, bound with DatabaseFilter.ExcludeParameter). Bind takes the new token.
  • MCP tool: with no database chosen it narrows with CurrentDatabaseScope.NarrowAsync (same as the top-procedures tool), and reports databases_left_out and databases_left_out_note on the payload and on the empty answers (McpHelpers.WithDatabasesLeftOut). A named database is read as asked.
  • Viewer grid read: leaves the dropped databases out when no database is selected (the grid names them in a banner, see Round 1 notes).
  • Lite twin: get_query_store_regressions gets the same dropped-database exclusion (GetDroppedDatabasesAsync, BuildDbExcludeClause in qualified) and the same databases_left_out fields.
  • Test helpers that run the shared SQL by hand bind the new parameter.

Test plan

  • New live test QueryStoreRegressionsFailureGroupsLiveTests.TheMonitorsOwnStatementAndADroppedDatabase_AreNotListed_AndTheDroppedOneIsNamed: marker row (21) and dropped-database row (30) absent with no database chosen; 20 stays; databases_left_out = [qsGone]; naming the dropped database lists 30; naming the live database lists only 20. Revert-proved: with both SQL predicates neutered it fails with Expected [20], Actual [20, 21, 30].
  • New Lite test QueryStoreRegressionsToolTests.ADroppedDatabasesRegression_IsLeftOut_AndNamed_UntilTheDatabaseIsNamed. Revert-proved (exclusion not passed): Expected [20], Actual [20, 30].
  • Darling (rig on a throwaway PostgreSQL): *QueryStoreRegressions*, *DroppedDatabaseScope*, *ViewerQueriesTests*, *Census*, *Ratchet*: Total 563, Failed 0.
  • Lite: *QueryStoreRegressions*, *Census*, *Ratchet*: Total 41, Failed 0.
  • Guard the first test plan missed: CurrentDatabaseScopeLoggerThreadingTests (Stage=Guard). It FAILED on the Viewer's new NarrowAsync call (no logger) and passes after the fix below; the whole Stage=Guard trait passes (Total 4453, Failed 0).
  • Round 1 run on a throwaway PostgreSQL after merging dev: CurrentDatabaseScopeLoggerThreadingTests, *QueryStoreRegressions*, *DroppedDatabaseScope*, *ViewerQueriesTests*, *DatabaseFilter*, *WebPartialWindowBehaviourTests*, *WebDataStartNoteTests*, *ServerTabEmptyStatesAndHeadersTests*, *ViewerQueryGridsDataStartLiveTests*, *WebQueryStoreGridGroupsTests*, *Census*, *Ratchet*: Total 885, Failed 0. After a second merge of dev (no conflicts) the non-live classes and the Guard trait were re-run: Total 92 and 4453, Failed 0.
  • Full suites not run (CI decides; no pre-push.sh per the brief).

Round 1 notes

Merged dev. The one conflict, ViewerDataService.QueryStoreRegressions.cs, was dev's #6686 purge-edge margin against this PR's left-out databases in the table read; both are kept (the margin code, and excludedDatabases passed to AddQueryStoreRegressionsParameters).

  1. Guard run. CurrentDatabaseScopeLoggerThreadingTests failed on the Viewer call. Fixed the way it asks: the call passes a logger. The Viewer has no host logger, so a small ViewerStoreLogger (new file) forwards warnings to ViewerLogger.Warn; a failed dropped-database read now shows in the Viewer log instead of vanishing. The test's class summary said the Viewer never calls CurrentDatabaseScope directly; reworded.
  2. Stale doc comments in DatabaseFilter.cs, Excluding and the two ExcludeClause references (and the matching one in CurrentDatabaseScope.cs): the regressions read (MCP tool and Viewer grid) is named as a consumer.
  3. One definition of the predicate. New DatabaseFilter.ExcludeClause(string column, string parameter); the int overload calls it; QueryStoreRegressionBaseline uses it with ExcludeToken instead of a hand-written copy. The pin DroppedDatabaseScopeLiveTests (" AND ($6::text[] IS NULL OR database_name IS NULL OR database_name <> ALL($6))") still passes. The only change in the shared statement's text: the second @@X no longer repeats the ::text[] cast.
  4. Left-out databases are now said on both grids.
    • Viewer: GetQueryStoreRegressionsWithScopeAsync returns the rows with the left-out names (GetQueryStoreRegressionsAsync wraps it, so existing callers are unchanged). A new banner QueryStoreRegressionsLeftOutBanner sits under the "Showing since" banner (a grid row was added; the data grid is now row 3). Text from DatabasesLeftOutBannerText: "Left out of this grid: Gone. The server's newest database snapshot no longer lists it (dropped or detached since an earlier snapshot did). Pick the database in the filter to read its history." Same rule as DatabasesLeftOut: names sorted and distinct, at most 20 named with "(N in all; the first 20 are listed)", same reason. The plain-English checker reports it clean.
    • Web: the Query Store Regressions table did not read the payload's databases_left_out_note at all, on either a populated or an empty answer. It now names the key as its noteKey (populated answer), and panels.js draws it on an empty answer through a new leftOutStrip (only for a grid that names the key, so no other panel changes).
    • Tests: TheLeftOutBanner_NamesTheDatabases_InTheWordsOfDatabasesLeftOut (text for none, blank, one, several, more than 20); ALeftOutNote_IsDrawnOnAnEmptyAnswer_OnlyForAGridThatNamesIt (the shipped util.js under Node, via the existing harness); the live test now also reads the Viewer path (rows and names with no database chosen; naming the dropped database reads it as asked and names nothing). Two existing text pins were updated for the changed spelling (the data grid's row number; the tab's read call name; the empty-branch mount line).
  5. Marker subquery in the WHERE: not changed, see below.

Round 1 review (changes requested)

Review 5636588372. Per finding:

  1. Root cause and the other readers (the Divergence section). The collector's drop is dead on Darling (see Why). Fixed on the READ side with ONE predicate, QueryStoreSelfStatement (new, Darling/PerformanceMonitor.Darling.Storage): the marker against the text resolved from query_store_text, falling back to the inline sample where a read has one. The regressions read (QueryStoreRegressionBaseline) uses it too. A person or agent no longer sees the monitor's own statement in any ranked Query Store list. The collector is not changed (see Considered, not added). The stale comments are fixed: QueryStoreRegressionBaseline.cs, QueryStoreCollector.cs (the drop site now says it does not fire on the separate-fetch path), and the DarlingSelfQueryMarkerTests summary.
  2. Historical anchor. ViewerDataService.QueryStoreRegressions.cs now passes asOfUtc: literalEndUtc (the custom range's end; null for a preset, where the end is now), as both MCP twins anchor at the window end. Test: in TheMonitorsOwnStatementAndADroppedDatabase_AreNotListed_AndTheDroppedOneIsNamed, a custom range ending 2 hours ago, before the database was dropped, still lists the dropped database's query 30 and names nothing. I did not run it against the old code; with no anchor the dropped set is read as of now and query 30 would be left out.
  3. Fixture shape. The same test now seeds query 22 the way a live Darling store has it: NULL inline text on the stats rows, the marker only in query_store_text. Query 21 (inline sample) stays: it covers the fallback arm.
  4. CA1068. TryGetQueryStoreRegressionsFromTableAsync takes excludedDatabases before CancellationToken (it was added after it); three test call sites updated. The build shows no CA1068 for this file.

Readers covered (all hide the monitor's own statement):

Reader Where
get_query_store_top (raw, interval table and daily routes) DarlingDataReader.QueryStoreTopTail: tested on the already-resolved lateral text, before the LIMIT
Viewer Query Store grid ViewerDataService.QueryStore.cs QueryStoreTopSuffix: the same
Viewer current-vs-baseline comparison QueryStoreComparisonSql: tested on the resolved text of the final rows, AFTER the top-100 cut, so a self statement in the top 100 costs the grid one row (the cut is per period by executions; a correlated lookup per window row would cost more than it saves)
aborted and failed queries (DarlingQueryStoreAbortsReader) in page, before its cap; one NOT EXISTS probe per candidate (only queries with an aborted or failed run reach it); no inline fallback, because rows from before the separate fetch were already filtered by the collector
PLAN_REGRESSION fact (PgFactCollector.QueryPerf.cs) added to the existing #6548 NOT EXISTS, before its LIMIT 100
regressed-queries drill-down, table and daily routes (PgDrillDownCollector.Queries.cs) the same, before LIMIT 5 (two sites)
investigation evidence regression_targets (InvestigationEvidenceCapture.Families.cs) the same, before the row cap
regressions (MCP, Viewer, web) already done in round 0; now through the shared predicate

Readers left alone (fetch ONE query by id, no ranking): DarlingMcpPlanForcePrecheckTools (plan force precheck), DarlingReproScript (repro script), DarlingWorker ResolveStoredQueryStoreForActualPlanSql (actual-plan resolver). The web Query Store page has no read of its own: it calls the MCP readers above. Readers over the plan cache (query_stats, WAITFOR filters in ViewerDataService.QueryStats.cs and the like) are not Query Store.

Tests. New QueryStoreSelfStatementReadersLiveTests (production shape: NULL inline text, marker only in query_store_text, one normal query that stays) covers the MCP top, the Viewer grid, the aborts read and the Viewer comparison. The marker case was added to the existing live tests of the PLAN_REGRESSION fact (PlanRegressionBuiltDaysFallbackLiveTests), the drill-down on both routes (PlanRegressionDrillDownDailyParityLiveTests) and the evidence targets (InvestigationEvidenceFamiliesLiveTests). Revert proof: with the marker constant changed to a word that matches nothing, 5 of these classes' tests fail (the new one, the failure-groups test, and the three above); restored, they pass.

Spelling pin deleted. QueryStoreTopDailyReadLiveTests.TheRawAndTableStatements_AreByteIdenticalToTheirPreSplitText was a SHA-256 of the top statements' text, re-taken on each past change; its rule is spelling, and a correct change breaks it, so it and its helper are removed (the tail-sharing test beside it stays).

Ran (throwaway PostgreSQL from pg-runtime.zip, stopped after): *QueryStoreRegressions*, *DroppedDatabaseScope*, *ViewerQueries*, *QueryStoreTop*, CurrentDatabaseScopeLoggerThreadingTests, the new class, *Census*, *Ratchet*, DarlingSelfQueryMarkerTests, ViewerQueryStoreRegressionsDataStartTests, QueryStoreAbortsLiveTests, ViewerTextLookupWindowBoundLiveTests, DatabaseFilterTests, *PlanRegression*, *InvestigationEvidence*, *TopQueriesDatabaseFilter*, *DarlingMcpDataToolsTests*, *QueryStoreIntervalWide*, *TopFill*: Total 1048, Failed 1 (the hash pin above, deleted), Skipped 13. After the fixes: QueryStoreTopDailyReadLiveTests + DarlingSelfQueryMarkerTests Total 25, Failed 0; the whole Stage=Guard trait Total 4453, Failed 0; QueryStoreRegressionsFailureGroupsLiveTests Total 8, Failed 0 after the last edit. Not run: the full suite and Lite (the only Lite-shared change is a comment in QueryStoreCollector.cs); pre-push.sh; a cost check (EXPLAIN) of the new probes on a large store.
Merged dev (326ff65). One conflict, in InvestigationEvidenceFamiliesLiveTests.cs, ARegressionOfSqlServersOwnCleanup_IsLeftOut_SoTheApplicationQueryLeads: dev (#6660) changed query 512's text to the bracketed form sys.[change_tracking_1228027148], and this PR added query 513 (the monitor's own statement) to the same fixture loop. Kept both: the PR's 513 rows and loop, with 512 using dev's bracketed text. Build is 0 errors (the 2 warnings are in files this PR does not touch and match dev). Ran InvestigationEvidenceFamiliesLiveTests, *QueryStoreSelf*, *QueryStoreRegressions*, *QueryStoreTop*, every *Census* and *Ratchet* class on a throwaway PostgreSQL (stopped after): Total 667, Failed 0, Skipped 9 (the two live classes: 24 run, 0 skipped).

Considered, not added

CHANGELOG entry

SECTION: Fixed
ENTRY:

@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 24, 2026 02:51
queries would otherwise fill the 50-row cap. A database the user picked is read as asked; an unknown set leaves out nothing. */
var excludedDatabases = databaseNames is { Count: > 0 }
? null
: (await CurrentDatabaseScope.NarrowAsync(_dataSource, serverId, DatabaseFilter.All, windowStartUtc: baselineStartUtc, cancellationToken: cancellationToken)).ExcludedNames;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call passes no logger, and an existing guard fails on exactly that.

CurrentDatabaseScopeLoggerThreadingTests.EveryNonTestCaller_OfACurrentDatabaseScopeLookup_PassesALogger walks every *.cs under Darling/ except Darling.Tests/, matches CurrentDatabaseScope\.(…|NarrowAsync|…)\(, takes the text from the match to the next ;, and reports any call expression that does not contain logger (Darling.Tests/CurrentDatabaseScopeLoggerThreadingTests.cs:36-58). The expression here is

CurrentDatabaseScope.NarrowAsync(_dataSource, serverId, DatabaseFilter.All, windowStartUtc: baselineStartUtc, cancellationToken: cancellationToken)).ExcludedNames

— no logger, so this file becomes an offender and the guard goes red. Its own summary is the invariant this breaks: "The Viewer reaches the same readers with no host logger; it calls only the readers, whose logger parameter is optional, never CurrentDatabaseScope directly." (:27-28)

That is not only a test problem. ReadDroppedAsync fails open and reports the failure only through the logger (CurrentDatabaseScope.cs:157-172, whose comment is explicit: "a SQL or ordinal mistake in it turns all of item 4 off, so each failure is logged once per occurrence at Warning"). With no logger, a broken lookup silently turns the grid's exclusion off with nothing written anywhere — while the Lite twin does log it (LocalDataService.CurrentDatabases.cs:152-155, AppLogger.Warn). The viewer has ViewerLogger.Warn for exactly this shape of best-effort step (ViewerServerTab.Queries.cs:446-450); thread an ILogger through, or give the viewer the same once-per-occurrence warn the other two SKU paths have.

Comment on lines +212 to +213
var excludedDatabases = databaseNames is { Count: > 0 }
? null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No historical anchor: a custom range that ended before the drop loses the database.

Both MCP twins anchor the dropped set at the window's end when the read is historical — DarlingMcpQueryStoreRegressionTools.cs:99 (asOfUtc: string.IsNullOrWhiteSpace(as_of) ? null : windowEnd) and McpQueryTools.cs:632. This call passes windowStartUtc only, so the anchor falls through to DateTime.UtcNow (CurrentDatabaseScope.cs:134).

CurrentDatabaseScope's header states the rule the omission breaks: "Historical reads. With an as_of the set is the newest snapshot at or before it, with the horizon measured from it (#5633 review M2), so a window that ended before a drop still sees the database." (CurrentDatabaseScope.cs:42-43)

Failure: the toolbar is on a custom range of 1–7 Nov (IsCustomRange, ViewerServerTab.Queries.cs:470) and SalesDb was dropped on 20 Nov. ReadDroppedAsync anchored at now sees SalesDb's last snapshot older than the newest, puts it in the dropped set, and the grid drops every SalesDb regression from a window in which the database was alive and collected — where get_query_store_regressions with as_of=2025-11-07 keeps them. The comment two lines up claims parity with that tool ("as get_query_store_regressions and the top-procedures ranking do"), and this is the one respect in which it diverges. asOfUtc: endUtc is a no-op for the presets (whose end is now) and fixes the custom ranges.

Comment on lines +577 to +584
/* #6654: the monitor's own statements are dropped at collection (QueryStoreCollector.SelfQueryMarker), but the text this read
resolves from query_store_text can still carry the marker, so a candidate whose text does is left out before the cap too.
The same keyed lookup the output projection makes, with the MAX(query_text) sample as its fallback. */
AND strpos(COALESCE((SELECT x.query_sql_text
FROM query_store_text AS x
WHERE x.server_id = $1
AND x.database_name = f.database_name
AND x.query_id = f.query_id), f.query_text_sample, ''), '{{QueryStoreCollector.SelfQueryMarker}}') = 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The premise this comment (and the PR body) rests on does not hold on Darling, and the hole it leaves is wider than this read.

"the monitor's own statements are dropped at collection (QueryStoreCollector.SelfQueryMarker)"

They are not, on Darling. The client-side drop tests row.QueryText (QueryStoreCollector.cs:1792), and row.QueryText is read from the query_sql_text ordinal — which the shipping SELECT nulls whenever FetchQueryTextSeparately is on:

string queryTextCol = context.FetchQueryTextSeparately
    ? "query_sql_text = CONVERT(nvarchar(1), NULL),"
    : "query_sql_text = qst.query_sql_text,";

(QueryStoreCollector.cs:768-770, and that flag is true for Darling — DarlingCollectorRunner.cs:2783.)

So on Darling the marker filter at collection is a no-op: row.QueryText is always null, the self query's stats rows are stored, and the text lands separately through QueryStoreTextWriter.WriteAsync, whose only per-text gate is the sensitive-statement scrub (QueryStoreTextWriter.cs:93-110) — no marker check. That is the real root cause, and it is why query_store_text carries the marker in the first place.

Two consequences worth deciding on before this merges:

  1. The PR body's "so get_query_store_top (which reads stats rows) never shows them" is wrong for Darling. get_query_store_top resolves text from the same query_store_text lookup (DarlingDataReader.cs:2776-2783) and filters only t.query_text NOT LIKE 'WAITFOR%' (:2804). The monitor's own statement is still listed there, and in the Queries grids that share that tail. The collector's own comment asserts the opposite — "The exclusion instead happens in the read loop … identical semantics" (QueryStoreCollector.cs:831-840) — and so does DarlingSelfQueryMarkerTests' summary ("so they never top a Top Queries or Query Store list"). Both are stale for the separate-fetch path.
  2. One filter at the writer (QueryStoreTextWriter, beside the existing scrub.TryText hook) or at the text fetch would close every Darling surface at once, instead of one strpos per reader. If the narrow read-level fix is the deliberate choice for get_query_store_regressions lists the monitor's own Query Store collector statement, and databases the server no longer has #6654, the body should say the other Darling surfaces still show it rather than claim they do not.

Mechanically the predicate itself is fine (it sits in qualified, above ranked/slots/LIMIT, so total_count and the page agree), and the '' tail of the COALESCE keeps it fail-open when neither text is known.

foreach (var (q, database, text) in new[]
{
(20L, Db, "SELECT normal"),
(21L, Db, "/* " + QueryStoreCollector.SelfQueryMarker + " */ SELECT own"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixture carries inline text, which is the one shape a live Darling store does not produce — so the arm that fires in production is the one this does not test.

WriteAsync sets QueryStoreCollector.Row.QueryText, so the marker lands in query_store_stats.query_text and the new predicate finds it in the f.query_text_sample fallback. On a real Darling store that column is null for every row collected since the separate-fetch cutover (QueryStoreCollector.cs:768-770), and the marker lives only in query_store_text — the SELECT x.query_sql_text … arm (QueryStoreRegressionBaseline.cs:580-583). Nothing here puts a row in query_store_text, so a mistake in that subquery's own predicates (wrong column, database_name vs query_id swapped, a missing server_id) passes this test green: the subquery just returns NULL and the fallback answers.

This is the trap in .github/REVIEW-TRAPS.md: "A test that constructs a 'four maintenance anomalies in one run' fixture is testing a shape no live run produces (#3709…). Fixtures should carry the population the engine actually emits, or say they do not."

A second row for query 21 — INSERT INTO query_store_text (server_id, database_name, query_id, query_sql_text, …) with the marker and the inline QueryText left null — would exercise the arm that matters, and would also pin that the keyed lookup matches the projection's.

@claude

claude Bot commented Oct 24, 2026

Copy link
Copy Markdown

Non-blocking notes (none of these drive the verdict)

1. The suite filters that were run would not have run the guard that fails. The test plan lists *QueryStoreRegressions*, *DroppedDatabaseScope*, *ViewerQueriesTests*, *Census*, *Ratchet*. CurrentDatabaseScopeLoggerThreadingTests matches none of those, and it carries [Trait("Stage", "Guard")] because it walks the repo tree (Darling.Tests/CurrentDatabaseScopeLoggerThreadingTests.cs:30) — so "Total 563, Failed 0" is consistent with the new viewer call still breaking it in CI. Worth re-running with *CurrentDatabaseScope* added.

2. Stale doc comment left behind. DatabaseFilter.Excluding's summary still says "Only the top-procedures ranking applies it today (see ExcludeClause)" (Darling/PerformanceMonitor.Darling.Storage/DatabaseFilter.cs:93-94). After this PR the regressions read — MCP and viewer — applies it too. Same for DatabaseFilter.ExcludedNames' neighbours, which name CurrentDatabaseScope.NarrowAsync as "its one producer"; that is still true, but the consumer list is not.

3. The exclusion predicate is hand-written rather than aliased. QueryStoreRegressionBaseline.cs:576 spells out what DatabaseFilter.ExcludeClause already spells (DatabaseFilter.cs:130-131, pinned at DroppedDatabaseScopeLiveTests.cs:364). I'm not calling the #3684 "two copies pinned equal" trap on it: ExcludeClause takes an int index and the shared statement needs a @@X token, and the one copy is shared by both twins through Bind, so there is no per-twin duplicate to drift. But if ExcludeClause ever gains a NULL-handling subtlety, this line will not get it. A token-taking overload would remove the question.

4. The marker subquery adds a correlated lookup to the hot WHERE. strpos(COALESCE((SELECT … FROM query_store_text …), f.query_text_sample, ''), …) = 0 sits in qualified's WHERE (QueryStoreRegressionBaseline.cs:580-584), i.e. over every flagged candidate rather than over the ~30-50 output rows the projection's twin subquery runs on. Postgres sorts ANDed quals by cost and a SubPlan is expensive, so in practice it should be evaluated after the three is_* flags — but the #5506 note two hundred lines down says this stage is "3 to 4 thousand" rows on a real store, so that is a few thousand extra PK probes per call, on both the MCP read and the viewer grid. EXPLAIN (ANALYZE) on the rig would settle whether it shows up; the existing scan-census test (QueryStoreRegressionsIntervalTableLiveTests.cs:305-312) only asserts which tables are touched, not the loop count, so it will not notice either way.

5. The viewer grid omits rows with nothing on the surface saying so. The body calls this out ("no banner; the grid has no envelope"), so it stays a note — but it is worth weighing against the reason DatabasesLeftOut exists at all: "an empty page then must not read as 'the collector recorded nothing': the omitted databases may be the only place a procedure ran" (PerformanceMonitor.Common/DatabasesLeftOut.cs:16-21). The grid's empty text names a specific, different cause — "Each query is compared with the 7 days before the selected range, so a range that reaches back to the start of the stored Query Store data has nothing earlier to compare with" (ViewerServerTab.Queries.cs:QueryStoreRegressionsEmptyText) — so a store whose only regressions were in a dropped database now reads as a baseline problem. The grid already has a banner mechanism beside it (UpdateTruncationBanner, used for the #4966 window floor), and the dropped names are in hand at the call site.

6. Parity on the Lite marker filter checks out. "Considered, not added" is right, and for the reason given: QueryStoreCollector.cs:768-770 keeps the qst join for Lite, so Lite's inline query_text is populated and the client-side drop at :1792 does fire. Lite has no query_store_text table (LocalDataService.QueryStoreRegressions.cs:95) and no viewer grid on this read, so the dropped-database half plus the two payload keys is the whole of Lite's share. The two-store parity trap does not apply here.

claude[bot]
claude Bot previously requested changes Oct 24, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES REQUESTED

Findings are inline (two on the viewer call, one on the shared SQL, one on the new fixture); non-blocking notes are in a separate comment.

Claims

  • VERIFIED — the new @@X exclusion and the marker filter sit in qualified, before the rank and the cap, and Bind takes the new token — QueryStoreRegressionBaseline.cs:574-584 above ranked/slots/LIMIT at :586-676; token :239, Bind :243-249.
  • VERIFIED — $10 in the MCP read, $9 in the viewer's, bound with DatabaseFilter.ExcludeParameter — DarlingQueryStoreRegressionReader.cs:117,334; ViewerDataService.QueryStoreRegressions.cs:122,303.
  • VERIFIED — the MCP tool narrows with CurrentDatabaseScope.NarrowAsync and reports both keys on the page and on the empty answers — DarlingMcpQueryStoreRegressionTools.cs:98-100,122-123,160-161.
  • VERIFIED — a named database is read as asked — CurrentDatabaseScope.cs:220 (unchanged unless IsAll); McpQueryTools.cs:631-633. Asserted by the named/db arms of both new tests.
  • VERIFIED — the Lite twin gets the same exclusion and the same two fields — McpQueryTools.cs:631-636,679-680; LocalDataService.QueryStoreRegressions.cs:131,318,409-410 (values bound after maxRows, so the positional order holds).
  • VERIFIED — the viewer grid excludes dropped databases with no banner — ViewerDataService.QueryStoreRegressions.cs:212-214; nothing added at ViewerServerTab.Queries.cs:463-476.
  • VERIFIED — every by-hand test binder got the parameter: all four sites that bind plan.HighStart — QueryStoreRegressionsDailyBaselineLiveTests.cs:185, QueryStoreRegressionsFailureGroupsLiveTests.cs:212, QueryStoreRegressionsIntervalTableLiveTests.cs:157,182. All four statements in both twins go through Bind, so none is short a $n.
  • VERIFIED — "Considered, not added": Lite needs no marker filter — QueryStoreCollector.cs:768-770 keeps Lite's inline qst.query_sql_text, so the drop at :1792 fires; Lite has no text table (LocalDataService.QueryStoreRegressions.cs:95). RegressionCoverageSql is indeed un-narrowed (DarlingQueryStoreRegressionReader.cs:149-167).
  • UNVERIFIED — the revert-proof (Expected [20], Actual [20, 21, 30]) — needs the gated PostgreSQL rig.
  • UNVERIFIED — "Darling … Total 563, Failed 0" — needs the rig, and is consistent with the guard below still being red: none of the listed filters matches CurrentDatabaseScopeLoggerThreadingTests.
  • UNVERIFIED — "Lite … Total 41, Failed 0" — a read-only tree cannot run DuckDB tests.
  • REFUTED — "the collector drops its own marker rows at collection, so get_query_store_top … never shows them" — on Darling the drop tests row.QueryText (QueryStoreCollector.cs:1792) and the SELECT nulls that column when FetchQueryTextSeparately is on (:768-770), which it is for Darling (DarlingCollectorRunner.cs:2783). The stats rows are stored, text lands unfiltered via QueryStoreTextWriter.cs:93-110, and get_query_store_top resolves it from the same query_store_text lookup (DarlingDataReader.cs:2776-2783) behind only NOT LIKE 'WAITFOR%' (:2804).

Riskiest lines

  1. Darling/PerformanceMonitor.Darling.Viewer/ViewerDataService.QueryStoreRegressions.cs:214 — first direct CurrentDatabaseScope call from the viewer, with no logger, so the documented fail-open goes unreported — UNCOVERED as behaviour; the existing source-walking guard EveryNonTestCaller_OfACurrentDatabaseScopeLookup_PassesALogger fails on it, which is the finding, not the coverage.
  2. Darling/PerformanceMonitor.Darling.Viewer/ViewerDataService.QueryStoreRegressions.cs:212 — no asOfUtc, so a past custom range is judged against today's dropped set — UNCOVERED (ViewerQueriesTests.cs:1270 reaches the line but seeds no database_states, so the branch passes either way).
  3. Darling/PerformanceMonitor.Darling.Storage/QueryStoreRegressionBaseline.cs:580 — the correlated query_store_text lookup that decides which rows exist on both surfaces — covered by the new live test only through the query_text_sample fallback; the query_store_text arm, the one that fires in production, is UNCOVERED.

Traps near this diff

  • "Lite and Darling ship in parity … a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding" — does not apply — both SKUs got the exclusion and both envelope keys (DarlingMcpQueryStoreRegressionTools.cs:160-161, McpQueryTools.cs:679-680), and the marker filter is rightly Darling-only (QueryStoreCollector.cs:768-770).
  • "Every MCP filter is part of the query, not a post-filter on the page (#3641)" — does not apply — QueryStoreRegressionBaseline.cs:574-584, LocalDataService.QueryStoreRegressions.cs:318.
  • "A total_* assigned from a .Count is a count of the page unless the set is provably whole (#3699)" — does not apply — total_count is COUNT(*) OVER () on qualified, now below the new predicates (QueryStoreRegressionBaseline.cs:597, LocalDataService.QueryStoreRegressions.cs:331).
  • "Truncation inferred from a full page … the flag is fetched.Count > limit" — does not apply — over-fetch untouched (DarlingMcpQueryStoreRegressionTools.cs:108,125).
  • "Two copies of one definition pinned EQUAL … Alias, do not copy (#3684)" — does not apply — one copy, shared by both twins through Bind; ExcludeClause takes an int where the statement needs a token.
  • "A test that constructs a … fixture is testing a shape no live run produces (#3709). Fixtures should carry the population the engine actually emits, or say they do not." — applies — QueryStoreRegressionsFailureGroupsLiveTests.cs:251 seeds inline query_text, which Darling has not produced since the separate-fetch cutover.
  • "An empty analysis window is 'unavailable', not an all-clear (#3553)" — does not apply as a verdict item — the viewer omits rows while its empty text names a different cause, but the body documents the missing banner as deliberate.

Divergence (blind read vs PR body)

  1. The body's root cause is inverted for Darling: it treats the collection-time self-exclusion as working and this read as the only leak. The tree says the exclusion is dead on Darling, so the marker statement is a stored fact row and every reader resolving text from query_store_text can list it. One reader is patched; the body says the others need no patch.
  2. "Viewer grid read: leaves the dropped databases out" understates it. It is the first direct CurrentDatabaseScope caller in the viewer (which an existing guard forbids without a logger, and whose summary records "never CurrentDatabaseScope directly" as an invariant) and it drops the historical anchor both MCP twins pass, so the stated parity with those two does not hold.
  3. The diff makes a change the body does not mention: the shared SuffixSql means the viewer grid also stops listing the monitor's own statement. Good, but the viewer bullet and the CHANGELOG entry credit the viewer with only the dropped-database half.

Reviewer checklist

  • Build both SKUs; TryGetQueryStoreRegressionsFromTableAsync gained a parameter after its CancellationToken (ViewerDataService.QueryStoreRegressions.cs:335-336), which CA1068 flags at the viewer's latest-recommended level — a warning here, but confirm the build stays clean.
  • Re-run Darling with *CurrentDatabaseScope* added to the filters and confirm the logger guard is green.
  • On the rig, EXPLAIN (ANALYZE) both forms over a few thousand flagged candidates and compare the new SubPlan's loop count with the pre-change plan.
  • On a live Darling store, check whether get_query_store_top and the Queries grids list the monitor's own marker statement today — the decision point for a writer-side fix instead of a per-reader one.

…read (#6654)

Merge dev (one conflict, the Viewer regressions read: keeps dev's measured purge-edge margin and this PR's left-out databases).
The Viewer now passes a logger to CurrentDatabaseScope (the Stage=Guard threading test), ExcludeClause has one definition that
the shared regressions statement uses, the doc comments name the regressions consumers, and the Viewer grid and the web page say
which dropped databases were left out.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: LGTM

Claims

  • VERIFIED — "The call passes a logger... a small ViewerStoreLogger (new file) forwards warnings to ViewerLogger.Warn" — Darling/PerformanceMonitor.Darling.Viewer/ViewerStoreLogger.cs (new file) wraps ILogger, IsEnabled gates at Warning, Log calls ViewerLogger.Warn("ViewerDataService", ...); ViewerDataService.QueryStoreRegressions.cs:228 passes logger: ViewerStoreLogger.Instance to CurrentDatabaseScope.NarrowAsync.
  • VERIFIED — "New DatabaseFilter.ExcludeClause(string column, string parameter); the int overload calls it... QueryStoreRegressionBaseline uses it with ExcludeToken" — DatabaseFilter.cs:131,138 and QueryStoreRegressionBaseline.cs:576 ({{DatabaseFilter.ExcludeClause("f.database_name", ExcludeToken)}}).
  • VERIFIED — "The only change in the shared statement's text: the second @@X no longer repeats the ::text[] cast" — old hand-written clause was <> ALL(@@X::text[]); the new shared ExcludeClause emits <> ALL({parameter}) with no second cast, matching the pre-existing int overload's own shape.
  • VERIFIED — "Left-out databases are now said on both grids" (Viewer banner + web noteKey) — ViewerServerTab.Queries.cs adds DatabasesLeftOutBannerText/ShowDatabasesLeftOutBanner, wired at the new QueryStoreRegressionsLeftOutBanner XAML row (ViewerServerTab.xaml:1766, RowDefinitions grown to 4, grid moved to row 3); web side adds noteKey: "databases_left_out_note" to the Query Store Regressions table() call (server-tabs.js:2139) plus a new leftOutStrip in util.js wired into the empty-answer path in panels.js:208.
  • VERIFIED — "Tests: TheLeftOutBanner_NamesTheDatabases_...; ALeftOutNote_IsDrawnOnAnEmptyAnswer_..." — both present and exercise real behavior (ViewerQueryStoreRegressionsDataStartTests.cs, web-partial-window-harness.mjs + WebPartialWindowBehaviourTests.cs), not text/spelling checks.
  • UNVERIFIED — the cited live-run totals ("Total 563, Failed 0", "Total 4453, Failed 0", etc.) — require executing the test suites against a live PostgreSQL instance, not checkable from a read-only tree.
  • REFUTED — none found.

Riskiest lines

  1. Darling/PerformanceMonitor.Darling.Storage/QueryStoreRegressionBaseline.cs:576 — the exclude clause now shares DatabaseFilter.ExcludeClause, dropping the second ::text[] cast from the generated SQL text — covered by QueryStoreRegressionsFailureGroupsLiveTests (new #6654 review assertions against a live Postgres via GetQueryStoreRegressionsWithScopeAsync, which fail if the cast removal broke parameter typing).
  2. Darling/PerformanceMonitor.Darling.Viewer/ViewerDataService.QueryStoreRegressions.cs:228 — now passes ViewerStoreLogger.Instance into CurrentDatabaseScope.NarrowAsync — covered only by CurrentDatabaseScopeLoggerThreadingTests (a source-text regex scan, not a behaviour test) | UNCOVERED by a behaviour test.
  3. Darling/PerformanceMonitor.Darling.Viewer/ViewerServerTab.Queries.cs:472 — ShowDatabasesLeftOutBanner(QueryStoreRegressionsLeftOutBanner, read.DatabasesLeftOut) WPF wiring — the pure text builder DatabasesLeftOutBannerText is unit-tested, but the codebehind call that sets the banner's Text/Visibility on a live grid load has no WPF-level test | UNCOVERED.

Traps near this diff

  • "Lite and Darling ship in parity... a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding" — does not apply here: this follow-up's changes (Viewer logger threading, the shared ExcludeClause helper, the Viewer grid banner, the web panels.js/util.js note wiring) are all to Darling-only surfaces (the separate Viewer WPF app and web dashboard). Lite has no Query Store Regressions UI at all (grep finds only Lite/Mcp/McpQueryTools.cs and Lite/Services/LocalDataService.QueryStoreRegressions.cs — tool/service layer only, no WPF grid), and its MCP-level databases_left_out parity was already in place from round 1, untouched by this range.
  • "never answer a trap with a test that reads source text or asserts how code is spelled" — the modified EveryTabRead_OfTheGrid_HasItsProbe regex-count test (updated for the GetQueryStoreRegressionsAsync → GetQueryStoreRegressionsWithScopeAsync rename) and CurrentDatabaseScopeLoggerThreadingTests's caller scan are both source-text checks, but neither is newly introduced as the answer to a defect here — the real behaviour coverage for the banner/exclusion logic is the new live-Postgres assertions and the util.js Node-harness test; the text scans are incidental rename upkeep on pre-existing guards.

Divergence (blind read vs PR body)

None — the PR body's "Round 1 notes" (items 1-5) match the diff exactly: the ViewerStoreLogger fix, the stale doc-comment updates in DatabaseFilter.cs/CurrentDatabaseScope.cs, the ExcludeClause(string, string) consolidation (including the precise "second cast dropped" detail), and the Viewer-banner/web-noteKey pair are all present and wired as described, with the marker-subquery explicitly left untouched as stated.

Reviewer checklist

  • Run the cited live-Postgres suites (*QueryStoreRegressions*, *DroppedDatabaseScope*, the Stage=Guard trait) to confirm the claimed "Total X, Failed 0" results, and visually confirm the new QueryStoreRegressionsLeftOutBanner row renders correctly (no clipping/overlap) in the Viewer at a few window sizes.

# Conflicts:
#	Darling/Darling.Tests/InvestigationEvidenceFamiliesLiveTests.cs
@erikdarlingdata
erikdarlingdata dismissed claude[bot]’s stale review October 24, 2026 07:31

Claude posting for Erik Darling. Round 1 findings fixed; round 2 (5636828427) is LGTM.

@erikdarlingdata
erikdarlingdata enabled auto-merge (squash) October 24, 2026 07:31
@erikdarlingdata
erikdarlingdata merged commit a7fae23 into dev Oct 24, 2026
25 of 26 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/6654-regressions-marker-dropped branch October 24, 2026 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant