Skip to content

test(connscale): emit five band-less diagnostic fields so a moved ratio can be attributed (BACKLOG #1366) - #624

Merged
wshallwshall merged 1 commit into
mainfrom
claude/builder-2-connscale-diagnostics
Aug 28, 2026
Merged

test(connscale): emit five band-less diagnostic fields so a moved ratio can be attributed (BACKLOG #1366)#624
wshallwshall merged 1 commit into
mainfrom
claude/builder-2-connscale-diagnostics

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Built by Builder 2. Landed by the Lander. One commit, four files, +390/-9. Zero files under messagefoundry/ or .github/.

What it does

The connscale smoke emits five band-less diagnostic fields to the step summary on every run. The existing ratio says that something moved; these say which. Drain-tail and reload-probe separate only on drain_seconds against reload_seconds; contention separates from probe-cost only on the FD probe's tick counts. All five already existed on the record and died with the test job, which uploads no artifacts.

This is directly relevant to BACKLOG #1357, whose connscale monotonicity assertion has been failing intermittently across unrelated PRs all evening with no way to attribute the movement.

The design call worth a reviewer's eye

This is a second renderer, not a parameter on the existing one.

Only two connscale metrics carry an SLO band; these five have none. Emitting prior, band-floor and margin for a band-less field would print a threshold computed from whichever reading happened to precede it -- manufactured precision that reads as measured in a job summary. So the new table carries no band, no threshold and no verdict, and says so in its own preamble.

Two defects the author's own verification caught

A duplicate test name silently deleted #236's cap test. A new test was given the name of an existing one. Python replaced the earlier definition with no error and no warning: the file held 29 def test_ and ran 28. It was caught only by counting definitions against collected. There is now a guard that fails the file if any two tests share a name.

Two of the author's own tests were hollow, and mutation proved it. One iterated the constant it was testing, so renaming a field out of the shipped set survived. The other asserted a substring that also appears in the heading, so deleting the sentence it meant to pin survived. Both are now pinned against literals.

Six mutants, all killed, each by a distinct red set.

Lander verification

commits vs origin/main                 1
behind origin/main                     0
git merge-tree --write-tree vs main    exit 0 (clean)
git diff --shortstat origin/main...    4 files changed, 390 insertions(+), 9 deletions(-)
files under messagefoundry/ or .github/    0
'+## 1366' in the BACKLOG diff             1

Exact match to the handover. I also re-ran the author's own duplicate-name control rather than taking it:

tests/test_connscale_smoke.py               defs 9    unique 9    OK
tests/test_connscale_empty_claims_per_msg.py  defs 32   unique 32   OK

The +## 1366 check is there deliberately: PR 623 was refused by the required a PR that implements BACKLOG #N must update BACKLOG.md context for exactly the missing row, and the author folded that lesson into this handover rather than needing to be told twice.

Not armed

Required approvals are 0, so arming means landing unread. This is diagnostics rather than a fix for a live defect, so nothing on main is costing anything while it waits for a reader.

Generated with Claude Code

@wshallwshall
wshallwshall added this pull request to the merge queue Aug 27, 2026
@wshallwshall
wshallwshall removed this pull request from the merge queue due to a manual request Aug 27, 2026
…ACKLOG #1366)

The ratio says THAT something moved; these say WHICH. #1211 limb one made
`empty_claims_per_msg` survive a passing run via `$GITHUB_STEP_SUMMARY`. The readings
that explain a movement were not in that scope, and they die with the job:
`tests/test_connscale_smoke.py` runs inside the `test` job, and THAT JOB UPLOADS NO
ARTIFACTS -- ci.yml's three `upload-artifact` steps are in `load-test` (:2162),
`load-test-sqlserver` (:2331) and `windows-service-smoke` (:2521).

Measured absent from the emission on origin/main, all five zero: drain_seconds,
reload_seconds, fd_probe_ticks, fd_probe_degraded_ticks, cpu_util_cores_mean.

WHY THESE FIVE AND NOT "EVERYTHING ON THE RECORD": they are the readings that separate the
competing explanations. Drain-tail and reload-probe separate ONLY on drain_seconds against
reload_seconds. Contention and probe-cost separate ONLY on the FD probe's tick counts --
`fd_probe_degraded_ticks` non-zero means the walk could not measure, zero means it measured
cleanly and any wrong reading is a wrong SUBJECT rather than a failed sample. That exact
distinction decided a live investigation into the FD gauge today, and it was unavailable
from CI.

A SECOND RENDERER RATHER THAN A PARAMETER ON THE FIRST, and the constraint decides it.
ONLY TWO metrics have a monotonic SLO -- empty_claims_monotonic and fd_count_monotonic.
Every field here has NO BAND. `render_readings_markdown` emits prior / band floor / margin;
for a band-less field those would be a threshold computed from whichever reading happened
to precede it -- false precision manufactured by the renderer and, in a job summary,
indistinguishable from a measured one. So this table carries no band, no threshold and no
verdict column, and says so in its own preamble.

`None` RENDERS AS A DASH AND NEVER AS `0`. "the probe did not measure" and "the probe
measured zero" are different verdicts, and telling them apart is the entire purpose of
fd_probe_degraded_ticks.

Emitted from the FIXTURE, before any assertion, so a passing run is recorded as fully as a
failing one -- and a test pins that the call is UNCONDITIONAL, because the renderer's own
tests cannot see a fixture that gates it on failure.

NOT IN SCOPE: artifact upload (the step-summary channel already reaches every run and needs
no ci.yml change), and the empty_claims band -- that is #1211 limb two and it stays blocked
until samples exist.

TWO OF MY OWN TESTS WERE HOLLOW AND THE MUTATION RUN PROVED IT. Both looked fine:

  - the field-set test ITERATED `DIAGNOSTIC_FIELDS` and checked each label against the
    rendered text. That reads the same constant it is testing, so renaming a field out of
    the set SURVIVED -- the test looked for the new name and found it. Now pinned against a
    literal, so an add, a removal or a rename all fail deliberately.
  - the no-band test asserted the substring "no band", which the table's HEADING also
    satisfies, so deleting the explanatory sentence SURVIVED. A substring occurring twice
    cannot witness the presence of either occurrence. Now pinned to the sentence.

A DUPLICATE TEST NAME SILENTLY DELETED #236's CAP TEST, and the count caught it. This file
had 29 `def test_` and ran 28: my new cap test was given the name of the existing one, and
Python replaced the earlier definition with no error, no warning and nothing collected.
Renamed, and a guard now fails the file if any two tests share a name -- pytest does not
warn about this and the only visible trace is a collected count one lower than the
definitions, which nobody reads.

Verified: ruff check + ruff format --check clean; mypy strict clean (9 files); 32 tests
passing, all new ones confirmed present BY NAME rather than inferred from the total;
definitions equal collected (32/32). Six mutants, ALL KILLED, EACH BY A DISTINCT RED SET:
None as zero; the no-band declaration dropped; a field dropped from the set; the
discriminates text dropped; a silent cap; the fixture gated on failure. Every mutant
asserted a unique anchor, a changed hash, an unchanged NUL count AND no introduced control
bytes, and every restore was byte-identical.

Rebased onto f8597f4. THE REBASE NEEDED A SEMANTIC RESOLUTION GIT COULD NOT SEE.
This branch replaced the local `context = {...}` dict in _record_ratio_readings with a
shared _run_context() helper, precisely so more than one emitter could read ONE definition.
PR #599 (BACKLOG #1292) meanwhile added a second caller on main, `readings_payload(...,
context=context)`, against the local dict this branch had already removed. Git merged both
sides without a conflict and produced a NameError: the definition was gone and a use
remained. Resolved the way the helper's own docstring dictates -- the new caller reads
_run_context() too. Caught by running the connscale tests, not by the merge: 1 failed and 9
errors before, 82 passed after.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall
wshallwshall force-pushed the claude/builder-2-connscale-diagnostics branch from a22337c to 8f7b4c7 Compare August 27, 2026 23:34
@wshallwshall
wshallwshall added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit 302e2ea Aug 28, 2026
40 of 41 checks passed
@wshallwshall
wshallwshall deleted the claude/builder-2-connscale-diagnostics branch August 28, 2026 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant