Skip to content

Log each tab's first load and draw Performance Trends charts as their reads land (#9232) - #9239

Open
erikdarlingdata wants to merge 2 commits into
devfrom
fix/9232-performance-trends-first-open
Open

erikdarlingdata wants to merge 2 commits into
devfrom
fix/9232-performance-trends-first-open

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Nov 21, 2026 •

Copy link
Copy Markdown
Owner

Claude posting for Erik Darling.

Refs #9232. The issue stays open until the first-open time is measured again on a field install that has this change and #9235, because the reads measured here are not slow and the field wait is not reproduced.

What this changes

  1. One INFO line per first load of a tab, and of each Queries sub-tab. First load of Queries > Performance Trends: 1234 ms, written once, when that tab's reads first complete. The timer's ticks and a return to the tab write nothing. A load a newer request superseded is not counted, so its replay gets the line. The new TabFirstLoad helper holds the decision, and RefreshVisibleTabAsync calls it around the tab's reads (the engine-edition and data-start steps before them are not in the figure).
  2. Performance Trends draws each chart as its own read lands. Before, the sub-tab awaited all five reads (four trend reads and the discontinuity markers) and then drew the four charts in a row, so the slowest read held back the chart that was ready first. Now each chart draws once its own read and the small discontinuity read are in (LandedPanels). The reads, their SQL and the chart code are unchanged.

What I measured

  • The first open already loads only the visible sub-tab. Both callers of RefreshVisibleTabAsync pass subTabOnly: true, so the full eleven-read branch of RefreshQueriesAsync (the one whose comment says it runs 8 queries) is not reached from the app. I left it alone: tests read its text, and nothing calls it. So the first visit to Performance Trends waits on its own five reads only, not on the other sub-tabs.
  • The Queries reads are not slow on a seeded store. I seeded a temporary store of the field shape: six servers, 26 hours at the collectors' one-minute cadence (query_stats 250 rows a minute, procedure_stats 40 a minute, with the collection log, Query Store 150 rows every five minutes in hourly intervals), 2.6 million query_stats rows. Server 1, run alone on this laptop:
Read Hot store, 4 h Hot store, 24 h Archived (hourly Parquet files for all but the last 90 min), 4 h cold Archived, 24 h warm
Performance Trends, all 5 reads together 32 ms 61 ms 217 ms 345 ms
Top Queries 69 ms 55 ms 539 ms 250 ms
Top Procedures 23 ms 21 ms 111 ms 87 ms
Query Store by Duration 96 ms 62 ms 462 ms 492 ms
Heatmap 39 ms 54 ms 145 ms 348 ms
Every read of the old full first open together (11 reads) 194 ms 260 ms 792 ms 2.5 s

The slowest single read anywhere was half a second, and the largest total was 2.5 s. The data-start and oldest-retained reads that run before every tab load took 59 ms cold and 0 ms after. Nothing here comes near 30 to 45 s, so there is no slow read to rewrite. I changed no SQL, so there is no old-versus-new equality test. I did not invent a rewrite to have one.

  • So the field 30 to 45 s is not the read cost on its own. It has to be time spent waiting, and the field run had the FinOps tab loading twice at window start (Lite window start: the FinOps High Impact read holds the store read lock for over 12 s, so database-state maintenance skips every server #9233 / Lite: cheaper High Impact read, FinOps loads wait for the first show, one skip line per cycle (#9233) #9235), 12 heavy reads against DuckDB's threads, plus all six servers' first collection cycles writing at once on a fresh start. I could not reproduce that mix here (no SQL Server, no app). The new log line is what will tell: it gives each first load's elapsed time in the log, to compare against the collector lines beside it. If a first open is still slow after Lite: cheaper High Impact read, FinOps loads wait for the first show, one skip line per cycle (#9233) #9235, that line is the starting point, and a lock wait on a long write (archive or compaction at start) is the next thing to look at, since the reads take the shared read lock.
  • "Connecting... waiting for the first collection". The charts do not wait for it. ConnectToServerAsync calls serverTab.RefreshData() first, so the stored data is shown at once, and then runs every collector for the server and calls FirstCollectionFinished(), which is the only thing that replaces the status line. The 8 to 48 s is that whole first collection cycle (every collector, queued behind the other servers' starting cycles), not a data read. Parked: ending the line earlier when the store already holds the server's data changes what the header says and needs a decision on the wording (for example "Showing stored data, collecting..."). I did not measure the cycle itself, since that needs a live server, so there is no figure for it from me.

Tests

Lite.Tests/TabFirstLoadTests.cs (7 tests, behaviour only, no source pins, no wall-clock bounds):

  • the first load writes one line with the elapsed milliseconds (a hand-moved clock), a repeat of the same key writes nothing, a new key writes its own line, an abandoned load is not a first open, a load that throws writes no line and the next one is still the first, a null key is not tracked.
  • a panel draws when its own read lands while a slower panel is still pending, and a two-read panel draws once with both results.

Revert proof: with the first-load guard removed (every load logs), ALaterLoadOfTheSameTab_WritesNothing_ButEachNewKeyWritesItsOwnLine fails, and with it restored all 7 pass. I did not mutate the panel helper: the old code path (await all five, then draw) is not expressible through it, so that test pins the new behaviour rather than failing on the old code. WPF cannot run in the suite, so the two call sites in ServerTab.Refresh.cs (the key and label and the four DrawAsync calls) are exercised only by the app.

Classes run (all green, 175 tests): TabFirstLoadTests, DataStartBannerQueriesTabTests, AzureSqlDatabaseCallSiteTests, AzureSqlDatabaseNotCollectedSurfaceTests, CollectorRunHistoryArchiveTests, CollectorRunHistoryReadTests, EngineGapNoteTests, OlderSkuEmptyStatesTests, RefreshCoordinatorTests, ServerTabChartCardsTests. Source pins in AzureSqlDatabaseCallSiteTests and RefreshCoordinatorTests read RefreshVisibleTabAsync, so I kept its name, its call count and the switch (MainTabControl.SelectedIndex) it spells.

Gate (bash .github/scripts/pre-push.sh): [guard-darling] Darling.Tests Total: 4980, Errors: 0, Failed: 0 and [guard-lite] Lite.Tests Total: 1242, Errors: 0, Failed: 0, then GATE STOP: selection is the whole suite (darling 3262, lite 1140), so the selected classes did not run there. I ran the classes ci-select.py local-classes lists for Lite instead (above). The Darling classes it lists name only _time, a common field name, not anything I changed.

Considered, not added

  • A timing test asserting Performance Trends' first load under 10 s: the seed takes 15 s and the reads take tens of milliseconds, so it would pass for any code and fail only on a noisy machine.
  • A per-panel loading state: there is no measured slow read for it to cover, and the issue's other asks are done above.
  • Removing the unreached full-refresh branch of RefreshQueriesAsync: tests read its text and it is not part of this fix.

Release note: Performance Trends now draws each chart as soon as its own data is read, and the Lite log gets one line the first time each tab, and each Queries sub-tab, finishes loading, with the time it took.

@erikdarlingdata
erikdarlingdata marked this pull request as ready for review November 21, 2026 07:58

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: LGTM

Claims

  • VERIFIED — One INFO line per first load of a tab and of each Queries sub-tab, written once — Lite/Helpers/TabFirstLoad.cs RunAsync (the _loaded set, then _write); wired in ServerTab.Refresh.cs RefreshVisibleTabAsync; TheFirstLoadOfATab_WritesOneLine_WithTheElapsedMilliseconds.
  • VERIFIED — A load a newer request superseded is not counted, so its replay gets the line — RunAsync returns before _loaded.Add when abandoned() is true; the call site passes () => ct.IsCancellationRequested; ALoadANewerRequestAbandoned_IsNotAFirstOpen_SoItsReplayGetsTheLine.
  • VERIFIED — A load that throws writes no line and the next load is still the first — the exception propagates before _loaded.Add; ALoadThatThrows_WritesNoLine_AndTheNextLoadIsStillTheFirst.
  • VERIFIED — Performance Trends draws each chart when its own read and the discontinuity read land; reads, SQL and chart code unchanged — the diff replaces the WhenAll plus four sequential Update*Chart calls with four LandedPanels.DrawAsync calls over the same tasks; APanel_IsDrawnWhenItsOwnReadLands_WithoutWaitingForASlowerPanel.
  • VERIFIED — No SQL changed — the diff touches no SQL.
  • VERIFIED — The switch text and the name RefreshVisibleTabAsync are kept — switch (MainTabControl.SelectedIndex) is still present inside the lambda.
  • UNVERIFIED — The timings table (all reads under 2.5 s on a seeded store) — needs a live run on a seeded store.
  • UNVERIFIED — The tests pass and the revert proof holds — needs a build and a test run.
  • UNVERIFIED — The two call sites in ServerTab.Refresh.cs work in the app — WPF cannot run in this read-only tree. The body says the same.

Riskiest lines

  1. Lite/Controls/ServerTab.Refresh.cs:427-436 — the key and label are built from the selected index before the load, so a tab switch mid-load could mislabel the line — UNCOVERED (WPF; the body says so). The ct abandonment check is the guard.
  2. Lite/Controls/ServerTab.Refresh.cs:560-568 — the four draws now run independently, so if one read throws, the other charts draw before WhenAll surfaces the exception, where before none drew — UNCOVERED (the helper tests cover only the helper).
  3. Lite/Helpers/TabFirstLoad.cs:44-56 — the _loaded set is unsynchronized; this is safe only because the call is on the UI thread — covered by ALaterLoadOfTheSameTab_WritesNothing_ButEachNewKeyWritesItsOwnLine for the logic, UNCOVERED for threading.

Traps near this diff

  • "A test that reads the clock twice measures the CI scheduler… One anchor owns both ends of an interval." — does not apply — the tests use a hand-moved HandClock (TabFirstLoadTests.cs:20) and no wall-clock bounds. Eventually polls with a 30 s deadline for a condition, not a measurement.
  • "Ruling (#5833): tests are load-bearing… never add a source pin" — does not apply — all 7 new tests are behaviour tests.
  • "Lite and Darling ship in parity" — does not apply — the change is to Lite's ServerTab and Lite helpers, and the PR's scope is Lite's tab loading. I did not search Darling for a counterpart.

Divergence (blind read vs PR body)

None in substance. One behaviour the body does not mention: Overview returns its failed lanes as a value (failedReads) rather than throwing, so a first load with failed reads still writes "First load of …" and is then marked loaded. This is non-blocking. The line says the load completed, not that it succeeded.

Reviewer checklist

  • Open Performance Trends in the Lite app and confirm that the charts draw one at a time and that the log has one First load of Queries > Performance Trends: N ms line.
  • Switch tabs mid-load and confirm the line carries the right label and that the replay is logged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant