Skip to content

test(client): the envelope-caller census names the string-literal trap in its own failure text - #15397

Merged
os-litant merged 1 commit into
mainfrom
claude/issue-13874-census-literal-span-diagnostic
Sep 4, 2026
Merged

test(client): the envelope-caller census names the string-literal trap in its own failure text#15397
os-litant merged 1 commit into
mainfrom
claude/issue-13874-census-literal-span-diagnostic

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Fixes #13874

Triage ruled option 2 on this card (comment 5486747463, 2026-09-01): when a counted
site falls inside a string-literal span, the census's failure text says so. Options 1 and 4
were rejected, option 3 is suspended. This PR is that and only that.

The defect, restated from the measurement rather than the card

packages/client/src/envelope-caller-census.test.ts walks the workspace for four SDK
spellings and blanks comments only. That is deliberate, and the reason is recorded in
two headers: a scanner that also blanked literals opens a phantom comment on any
block-comment opener inside a string and shrinks the census in silence.

The price is that example code quoted inside a literal — a refusal message showing an
author what to write, a usage banner, a fixture embedded in a gate script — counts as a
real call site. It happened, and the merge queue ejected the PR.

What made it expensive was never the count. It was the failure text. It read
expected 21 to be 19 and said nothing about strings, masking or gate scripts, in a
package the author had not edited, naming a ledger the author had never read, at the most
expensive point in the pipeline, invisible to every local gate a scripts/** edit derives.

So the count is untouched and the message explains itself.

The one design decision worth reviewing

The obvious implementation is a small literal-span scanner written here. It is the wrong
one
, and the file's own header says why: this tree has ONE code/prose separator, and a
private one is the exact defect that separator exists to prevent.

It also turned out to be unnecessary. scripts/js-comment-mask.mjs already exports
scanSource — the single pass maskComments is itself built on — and it already reports a
per-character literal flag beside the comment flag the mask is made of. It is already
declared in the hand-written js-comment-mask.d.mts, so it resolves with types.

So the diagnostic borrows the house scanner:

  • scripts/js-comment-mask.mjs is not edited, and maskComments' behaviour is
    unchanged — the census still masks comments only;
  • the note cannot disagree with the census about what a literal is, because it is the same
    pass;
  • it inherits a scanner measured against an independent parser over the whole tree
    (check-comment-mask-corpus.mjs, re-run here: 5907 files, 0 disagree), rather than one
    written this afternoon.

Which of the two maskComments consumers the note belongs to — measured, not assumed

The file has two. The dispatch asked which one takes the diagnostic, and the answer is
the first only:

  • scanCallSites produces CENSUS.sites, i.e. the counted sites. The ruling is about a
    counted site, so every assertion that compares a census-derived number carries the note.
  • methodSource masks the SDK's index.ts to slice one method body for section 4's
    toMatch assertions. It produces no count, its assertions already carry per-method
    messages, and extending there would widen the card past its ruling.

What the note says

Five assertions carry it — section 2's producer-receiver count (where a quoted example
lands first, because it rarely spells client. in front of the method), section 3's
enumeration-versus-ledger equality, the bare ledgerTotal mismatch that is the ejecting
shape, the production-site assertion, and the SDK site count. The three verdict sums are
ledger arithmetic and cannot move on a census reading, so they deliberately do not.

When it applies, the text names each offending site with file, line, method, receiver, the
opening delimiter and the line that delimiter opened on; then where the repair belongs; then
the two prohibitions, so the reader does not reach for the ledger or the census to go green.

And it says the opposite when it does not apply. A note that only ever speaks up says
"it is the literal trap" about every failure. The negative reading is pinned as its own test.

Proof the count did not move

Re-derived with an independent scanner — the census's own METHODS and regex through
the repo's own maskComments, run outside the test — before and after the change:

filesScanned = 5882      sites total = 29      sdk = 28      service = 1      inLiteral = 0

Byte-identical in both runs (diff of the two outputs is empty). The new section 6
fixtures add zero sites, which is not luck: this file sits inside the tree the census
walks, so every fixture is assembled from parts — the spelling is split in the SOURCE and
restored in the VALUE handed to the matcher. The last test pins the whole file at zero
counted sites.

scanCallSites' per-file matching is lifted out unchanged as sitesInSource so section 6
drives the matcher that counts over its fixtures instead of a copy of it. A pin written
against a hand-rolled second matcher measures the copy and stays green through any drift in
the original.

Ablation — direction predicted first, both legs, mutation proved on disk

Prediction, recorded before running: breaking the span check leaves every count assertion
in sections 1-5 green
and reds only section 6.

leg mutation predicted measured
A span check always false positive pins red, counts green 2 failed / 18 passed — the two positive pins; sections 1-5 all green
B span check always true negative half red, counts green 4 failed / 16 passed — positive pin, negative half, never-throws, end-to-end; sections 1-5 all green

⚠️ One prediction was wrong and is reported rather than smoothed: leg B was predicted to
red the template-banner test too. It stayed green, correctly — that site genuinely is inside
a literal, so disabling the guard does not change its walk-back. Direction and the
load-bearing claim held; the count of red tests did not.

Both legs proved the mutation reached the disk before reading anything (injected marker
present = 1, original line gone = 0 — an editor that matches nothing exits 0 too). Restore
is by git checkout HEAD -- ABSOLUTE_PATH from a committed implementation, proved by blob
hash equality (03d7789e4b769267e2cc34cdda4fa44e5725e9a0 == HEAD: the file) plus an empty
git diff HEAD, with an absolute-path trap on EXIT INT TERM.

⇒ The diagnostic can be broken in either direction without moving a single count. That
is the strict-additivity proof the ruling asked for.

The extra reading triage asked for: the population under scripts/**

Derived with the census's own METHODS through the repo's own scanner, classifying each
mention by scanSource's comment and literal flags rather than by a hand grep.

Under scripts/**: 0 literals embed any of the four spellings. There are 6 mentions,
all in comments (correctly masked, contributing nothing), all in one file —
scripts/check-adr-0087-registration.mjs, which is the file whose fixture caused the
ejection and which now discusses the trap in prose instead of embedding it.

The raw grep's other 3 hits, in scripts/analytics-reconcile/**, are all
analytics.queryDataset — the protected counter-example, a different symbol that merely has
analytics.query as a prefix. The census's pattern requires the call shape, so it does not
match them.

For context, repo-wide: 70 literal-embedded mentions across 8 files, and 0 of them
are counted — none is written as a call shape. They are it(...) titles, route-ledger
rows, permission arrays and migration prose. That is the real reservoir, and every one of
them is one added argument list away from becoming a phantom site.

⇒ On the scripts/** number triage asked for, option 3's priority is not raised. The
repo-wide 70 is offered as the fuller picture for whoever files it; ⛔ nothing here widens
this card.

Changeset

None, and skip-changeset applied. packages/client publishes dist, README.md and
CHANGELOG.md only (check:published-files green, and it independently refuses any package
that admits a test file), so a change confined to src/*.test.ts releases nothing.

Verification — union re-run on the final commit afdc12c7bb, clean tree

  • pnpm --filter '@objectstack/client^...' build — VERDICT command-exit 0
  • pnpm --filter @objectstack/client exec vitest run --maxWorkers=2 src/envelope-caller-census.test.ts20 passed (20), VERDICT command-exit 0
  • pnpm --filter @objectstack/client typecheck — VERDICT command-exit 0.
    ⚠️ tsc --noEmit here excludes **/*.test.ts, so on its own it says nothing about
    this file. Proved with --listFilesOnly: the file appears 1 time in the TEST program
    (tsconfig.test.json) and 0 times in the build program. check:test-typecheck is the
    half that measured it, and it is green.
  • dispatch-gates --commands re-derived on the clean tree at final HEAD: identical to
    the pre-commit derivation, 1 path in the change set. 35 of 38 derived commands green,
    including check:comment-mask-adoption (+ self-test), check:comment-mask-corpus,
    check:cross-package-test-inputs, check:test-source-alias, check:nul-bytes,
    check:published-files, check:type-check-coverage, plus the four artifact rosters the
    tool flags as "silence is not evidence" (authz-resolver, error-code-casing,
    filter-alias-parity, swallow-census-controls).
  • 3 NOT MEASURED, none a finding about this diff — each refuses to run against an
    unbuilt world and says so: check:dual-build-cjs-loads (exit 3, PREREQUISITE NOT MET),
    check:type-check-debt (exit 3, PREREQUISITE NOT MET), spec check:skill-examples
    (refuses because packages/spec/dist holds no declarations). CI builds first and runs
    them properly.
  • Lint: a proved narrowing, not a skipped run. (1) The population is read from
    eslint.config.mjs itself, and this file is in it — it was linted, not ignored.
    (2) --format json reports 1 file, 0 errors, 0 warnings, exit 0. (3) The config
    enables no type-aware linting for any file — every parserOptions in it is
    { ecmaVersion, sourceType }, with no project and no projectService, a property its
    own header states with a measured positive control. So no untouched file's verdict can
    move because of this diff.
  • Every exit code captured before any pipe; each verdict quoted from the gate's own
    printed line, never from a bare $?.

Generated by Claude Code


Generated by Claude Code

…p in its own failure text

The census walks the workspace for four SDK spellings and blanks COMMENTS
only, deliberately: a scanner that also blanked literals would open a
phantom comment on any block-comment opener inside a string and shrink the
census in silence. The price is that example code quoted inside a literal --
a refusal message showing an author what to write, a usage banner, a fixture
embedded in a gate script -- counts as a real call site. That happened, and
the merge queue ejected the PR.

What made it expensive was never the count; it was the failure text. It read
`expected 21 to be 19` and said nothing about strings, masking or gate
scripts, in a package the author had not edited, naming a ledger the author
had never read, at the most expensive point in the pipeline.

So the count is untouched and the message explains itself. Every assertion
that compares a census-derived number now carries a note that re-reads the
counted sites and says which of them sit inside a literal -- with file, line,
receiver, the opening delimiter and the line that delimiter opened on -- plus
where the repair belongs and what must not be done to make it green.

Strictly additive:

- the literal question is answered by `scanSource`, the pass `maskComments`
  is itself built on, which already reports a per-character `literal` flag
  beside the `comment` flag. No second private scanner, and `maskComments`
  and `js-comment-mask.mjs` are untouched;
- the note is built only while a message is being built, over the counted
  sites alone, and decides nothing;
- it cannot throw: an unreadable file, a stale offset and a garbage site all
  degrade to "not in a literal", pinned in section 6;
- the negative half is pinned too -- a real call site outside any literal
  produces the note that says the trap is NOT the cause, so the positive
  reading stays worth acting on.

`scanCallSites`' per-file matching is lifted out unchanged as
`sitesInSource` so section 6 drives THE MATCHER THAT COUNTS over its
fixtures rather than a copy of it. Every fixture is assembled from parts,
because this file is inside the tree the census walks and a fixture spelled
as one literal would arm the very trap it documents; the last test pins this
file at zero counted sites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
@github-actions github-actions Bot added the size/m label Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 5ea2e38820ef96cceb40bc6796af3cfce0b2b311packageMentionDocs.

@github-actions github-actions Bot added the tests label Sep 4, 2026
@os-litant os-litant added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 4, 2026 — with Claude
@os-litant
os-litant marked this pull request as ready for review September 4, 2026 14:44
@os-litant
os-litant enabled auto-merge September 4, 2026 14:44
@os-litant
os-litant added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit 9b9f1aa Sep 4, 2026
39 checks passed
@os-litant
os-litant deleted the claude/issue-13874-census-literal-span-diagnostic branch September 4, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants