Skip to content

fix(session): reconcile registered global-* sources against render coverage - #51

Merged
andrei-hasna merged 2 commits into
mainfrom
fix/102d6d0a-render-gap-reconcile
Aug 2, 2026
Merged

fix(session): reconcile registered global-* sources against render coverage#51
andrei-hasna merged 2 commits into
mainfrom
fix/102d6d0a-render-gap-reconcile

Conversation

@andrei-hasna

@andrei-hasna andrei-hasna commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • instructions session plan|apply had no way to detect when a registered global-* instruction source was omitted from a render's --config list. The list this fleet actually uses (render-spec-station01-sh, config 54b5ea5a) is a hand-maintained shell array, and planSessionRender only warned when the resolved source list was empty -- a single omitted source silently disappeared from every rendered agent home at rc=0.
  • Measured on station01 (todos 102d6d0a, 5dcd60ec): 29 registered global-* sources, 16 in the array, 13 never rendered anywhere. Breakdown: 1 deliberately withdrawn (deployment-terms doctrine consolidation), 2 stale duplicate registrations from an earlier slug collision, 10 live gaps (6 registered the same day this was found).
  • Adds --check-global-coverage to session plan and session apply. It reads the expected set fresh from the config registry via store.listConfigs -- independent of the plan/array being audited -- and diffs it against the plan's own resolved source ids. Non-fatal (warnings + a globalSourceCoverage block in --json), so it doesn't break any existing caller.
  • A source only leaves the expected set via an explicit retired-global-source tag, so a deliberate withdrawal doesn't produce a permanent false positive.

Why this shape

A checker whose expected and actual sides both come from the array being audited always reports full coverage by construction -- it cannot observe a shortfall. src/lib/global-source-coverage.test.ts includes a constructed-shortfall test: build a full render, remove one --config entry, and assert missingSlugs moves from 0 to 1. That test is run and green, not merely argued.

Out of scope (filed separately)

  • Deleting the two stale duplicate registry rows (global-agent-rules-standard-1 etc.) -- destructive DB op unrelated to this defect, tracked as todos 991a8720.
  • Retagging global-hasna-deployment-terms with retired-global-source in the live registry, and adding the 10 live-gap sources to the actual render-spec-station01-sh array -- those are config-content changes to a DB-stored artifact, not this repo, and belong to whoever re-renders next.
  • A full registry-driven render (deriving the array itself from the registry, not just auditing it) -- flagged as the more durable fix in review discussion; this PR ships the audit because it's a complete, testable, non-breaking unit on its own.

Test plan

  • bun run typecheck -- clean
  • bun test -- 554 pass, 0 fail (45 files)
  • New regression suite src/lib/global-source-coverage.test.ts (7 tests) including the constructed-shortfall proof
  • --check-global-coverage appears in session plan --help / session apply --help output from a real build
  • Independent adversarial review (required before merge per fleet policy)

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…verage

The session render spec (config render-spec-station01-sh, GLOBAL_CONFIGS array)
is the only thing deciding which registered global-* instruction sources reach
any agent home, and planSessionRender only warns when the resolved source list
is empty. A source registered but never added to that array vanished from every
render silently, at rc=0, with no warning (todos 102d6d0a, 5dcd60ec).

Measured 2026-08-02 on station01: 29 sources registered with a global- slug
prefix, 16 in the array, 13 never rendered anywhere (1 deliberately withdrawn,
2 stale duplicate registrations, 10 live gaps -- 6 registered the same day).

Adds `instructions session plan|apply --check-global-coverage`, which reads the
expected set fresh from the config registry (independent of the plan being
audited) and compares it against the render's own resolved source ids, so a
shortfall is observable rather than definitionally impossible -- a checker that
derived both sides from the same array would report full coverage by
construction. Non-fatal: emits warnings and a globalSourceCoverage block in
--json output; does not fail the render.

A source is excluded from "expected" only via an explicit retired-global-source
tag, so an intentional withdrawal (e.g. the deployment-terms doctrine
consolidation) does not perpetually false-positive.
@andrei-hasna
andrei-hasna force-pushed the fix/102d6d0a-render-gap-reconcile branch from 11f7fd5 to 85a0d20 Compare August 2, 2026 08:08
@andrei-hasna

Copy link
Copy Markdown
Contributor Author

[REVIEW] NO-GO — #51 @ 85a0d20 — lens: coverage-fraction-and-designed-exclusions, reviewer pr51-reviewer (1 of 1)

Disclosure: I share a dispatcher (fabricius/agent-chief-staff) with this PR's author. Not arm's-length.

Sha/base confirmed current: head 85a0d20 matches PR#51 exactly (verified via gh pr view, state OPEN). Base moved to d9eaa6a (#50) after a mid-flight rebase; merge-tree of base+head equals head tree, so what would land is byte-identical to what I reviewed. Diff scope confirmed: 3 files, 258 insertions/4 deletions, exactly as briefed.

P1 BLOCKING — reproduced live against the real registry, twice

1. The fraction genuinely moves — this part is solid. Verified three ways: (a) unit test explicitly asserts 0→1 missing on removing one config; (b) I injected the exact vacuous-check anti-pattern (deriving "actual" from the same array as "expected") into computeGlobalSourceCoverage and re-ran the suite — 4 of 7 tests failed, including the shortfall test, proving the suite genuinely detects that defect class; (c) I ran the real CLI (session plan --check-global-coverage) against the live production registry with 1 vs 2 --config flags and watched missingSlugs go 29→28. Expected/actual are genuinely independent (registry query vs. plan.manifest.sources). This is not vacuous.

2. The designed-exclusion mechanism is unshipped, and it produces false positives TODAY on production data — confirmed, not hypothesized. The PR adds a retired-global-source tag and filtering logic, but tags NO real registry row with it. I queried the live registry: global-hasna-deployment-terms (withdrawn by owner ruling 2026-07-29) has tags: []. global-agent-rules-standard (07a324d0, the designed non-render fossil this PR's own doc comment names as the example) has tags: [global, mandatory, agent-policy, supply-chain, infinity-machine, coordinator-autonomy] — no retired tag. Neither is in the real render-spec-station01-sh GLOBAL_CONFIGS array (I read it from the live registry — 16 global-* entries, neither of these two among them). So turning this flag on against the actual production render spec reports BOTH as permanently missing, forever, with no shipped mechanism to silence them. Worse: two more fossils exist in the live registry right now, global-agent-rules-standard-1 and -2 (both carry the semantic-policy sentinel), also untagged, also absent from the array.

3. The coordinator's flagged concern is real and I reproduced it end-to-end. PR#50 (already merged into this PR's base) added skip-reporting for sources discarded by internal semantic-policy dedup. I ran session plan --config global:global-agent-rules-standard-1 --config global:global-agent-rules-standard-2 --check-global-coverage against the live registry. Single command, single JSON response, two self-contradicting lines:

An operator who explicitly configured the source is told in the same breath that it was intentionally discarded AND that it is a coverage gap they need to fix. There is no fix available — re-adding it triggers the same collapse. This is a permanent, unfixable false positive for any source in a semantic-policy family, and it directly contradicts #50's own accurate reporting one line above it.

Why this blocks rather than being a follow-up: the PR's stated purpose (todos 102d6d0a/5dcd60ec) is exactly "distinguish a real gap from a designed exclusion." Item 2 and item 3 above are that exact failure mode, confirmed with live production data, not edge cases. Zero integration tests exercise checkGlobalSourceCoverage against a real planSessionRender output — the only tests are pure-function unit tests against a hand-built fixture that never contains a designed-exclusion source alongside a real collapse. The interaction I reproduced above is entirely untested.

Non-blocking, for the record

  • --check-global-coverage is opt-in and non-fatal (warn only, no exit code change) — not wired into CI or the default apply path in this diff, so nothing breaks silently in the mean time. This is the only reason I'm not calling this P0.
  • resolveConfigStore() refactor (called once, reused) in index.tsx is behaviorally inert — the function is a stateless factory, confirmed by reading its implementation.
  • store.listConfigs({}) call is a legit, if heavy (~1.4MB / 151 rows on this box), read — acceptable given the flag is opt-in.

What I checked and verified with my own eyes (not taken on the code's structure)

  • Fraction moving: observed directly, both via a broken-implementation regression run and via two live CLI invocations against production data.
  • The retired-tag gap: observed directly by querying the live registry for the exact two configs named in the PR's own comment.
  • The skip-vs-missing contradiction: reproduced directly, live, in a single CLI invocation.
  • Full test suite: 561/0 pass on this branch.
  • No blast radius: registry row count before/after my session plan (read-only, dry-run) invocations was unchanged (151→151). A row did appear later (global-agent-rules-standard-3, ts 08:16:08Z) during my full bun test run, but every test file touching this feature uses an isolated new LocalConfigStore(db) with a temp DB — I found no code path in this PR's tests that writes to the shared/default store. I cannot rule out concurrent fleet activity as the cause and did not chase it further since it's outside this PR's diff; flagging for visibility, not attributing it to this PR.

What I did NOT check

  • --show-secrets / secret redaction paths — out of scope for this diff, not touched by it.
  • Whether global-agent-rules-standard-3's appearance during my test run is caused by something else entirely — I ruled out this PR's own tests as the source but did not trace the actual writer.
  • Performance of store.listConfigs({}) under load / concurrent callers.
  • Any CI-specific gate config (I ran the test suite manually; did not invoke CI's own runner).

Remedy, scoped for one remediation cycle

  1. Tag global-hasna-deployment-terms, global-agent-rules-standard, -1, and -2 with retired-global-source in the live registry (or ship a migration that does it), OR change checkGlobalSourceCoverage's "configured" side to also count sources present in plan.warnings/skip-reporting as "accounted for" rather than only plan.manifest.sources — either closes the false positive.
  2. Add one integration test that calls checkGlobalSourceCoverage-equivalent logic against a real planSessionRender output where two sources collapse via fix(session-render): report every instruction source the render discards (todos 0c7ffd33) #50's dedup, asserting the discarded one is NOT reported missing.

@andrei-hasna

Copy link
Copy Markdown
Contributor Author

[REVIEW] NO_GO — #51 @ 85a0d20 — lens: correctness+security+gates, reviewer unresolved-account001 (1 of 1)

Scope read:

  • git log --oneline origin/main..HEAD at fetched base d9eaa6abae3127fb6bf3fc4088da16e2938e53d0 and exact head 85a0d20b615937106657ededf6a790b8d49c6fec.
  • Full diff of all three changed files: src/cli/index.tsx, src/lib/global-source-coverage.ts, and src/lib/global-source-coverage.test.ts.
  • Surrounding source for CLI option parsing/output, source normalization and semantic-policy collapse in session-render.ts, manifest construction, sourceFromConfig, the local/cloud ConfigStore, server list routing, and PostgreSQL registry listing.

Commands actually run, unpiped:

  • bun install — exit 0; setup only; 158 packages installed.
  • bun run typecheck — exit 0; pass (the declared tsc --noEmit gate emits no pass/fail count).
  • bun run test — exit 0; 561 pass, 0 fail, 2347 assertions across 46 files.
  • Live positive-path reproduction: bun run src/cli/index.tsx session plan --tool codewith --profile pr51-review --config global:global-agent-rules-standard-1 --config global:global-agent-rules-standard-2 --check-global-coverage --json — exit 0.

Blocking P0/P1 findings:

  1. P1 correctness — a deliberately configured and intentionally collapsed source is reported as missing. checkGlobalSourceCoverage derives configured IDs only from plan.manifest.sources, after planSessionRender has removed semantic-policy duplicates. On the live registry command above, global-agent-rules-standard-2 appears in manifest.skippedSources with the explicit reason that it was superseded by global-agent-rules-standard-1, but the new globalSourceCoverage.missingSlugs simultaneously contains global-agent-rules-standard-2. The operator therefore cannot distinguish the exact designed exclusion from a forgotten --config entry, which is the feature's stated acceptance purpose. This is current, reachable behavior, not a speculative input.

Required remediation: count global-prefixed IDs in plan.manifest.skippedSources as accounted-for coverage, while keeping them out of the rendered-source list, and add an integration regression test using a real planSessionRender semantic-policy collapse. Re-run the declared gates and the live reproduction.

Security review:

  • No reachable secret exposure, unsafe mutation, authorization bypass, path escape, or rollback/data-integrity defect was introduced by the three-file diff.

Non-blocking follow-ups:

  • The flag is opt-in and warning-only, so existing callers are not silently broken.
  • Registry fossils that are intentionally outside every render still need the declared retired-global-source tag before the flag can produce a clean live denominator; that external registry grooming is not a repository-code blocker once the skip-accounting defect is fixed.

@andrei-hasna

Copy link
Copy Markdown
Contributor Author

[REVIEW] GO — #51 @ 4042e20 — lens: correctness+security+gates, reviewer unresolved-account001 (1 of 1)

Focused re-review only of the named P1 defect, its fix, and direct regressions:

  • 4042e20 now derives coverage-accounted IDs from both manifest.sources and manifest.skippedSources. Intentionally skipped sources remain absent from rendered files, but are no longer mislabeled as missing from the caller's configured set.
  • Added a real planSessionRender regression with two semantic-policy sources: one renders, one is explicitly skipped, both remain coverage-accounted.
  • Live reproduction against the registry now returns configuredSlugs containing both global-agent-rules-standard-1 and global-agent-rules-standard-2; the intentionally skipped -2 is absent from missingSlugs.

Verification after the fix, unpiped:

  • bun run typecheck — exit 0; pass (no count emitted by tsc --noEmit).
  • bun run test — exit 0; 562 pass, 0 fail, 2352 assertions across 46 files.
  • Live session plan ... --check-global-coverage --json reproduction — exit 0; named contradiction resolved.
  • shield review on the staged three-file fix — exit 0; no security issues found.
  • Repository pre-push hook scanned the one outgoing commit and allowed the push.

Blocking P0/P1 findings: none remain in the named remediation scope.

Non-blocking follow-up: intentionally non-rendered registry fossils still require the existing retired-global-source tag before an all-source live render reports a clean denominator; that external registry grooming remains outside this repository patch.

@andrei-hasna
andrei-hasna merged commit 04a6f46 into main Aug 2, 2026
3 checks passed
@andrei-hasna
andrei-hasna deleted the fix/102d6d0a-render-gap-reconcile branch August 2, 2026 08:34
andrei-hasna added a commit that referenced this pull request Aug 2, 2026
 follow-up) (#52)

fix(session): ship the retired-global-source tag mechanism (#52)

Closes P1 #1 from the adversarial review of PR #51: that PR introduced a
`retired-global-source` tag and the logic to filter on it, and shipped no way
to apply it. A tag nothing can set is a mechanism that reports every
designed exclusion as a permanent gap.

Adds `instructions tag <id> --add/--remove <tag>`, corrects the doc comment,
and adds tests. Exactly one production row is tagged: global-hasna-deployment-terms,
the owner-ruled withdrawal of 2026-07-29.

WHAT IS DELIBERATELY NOT TAGGED, and it is the load-bearing decision here.
global-agent-rules-standard-1/-2/-3 are byte-identical rows minted by an active
defect (43d0c1c0) which fired twice tonight, eighteen minutes apart. Tagging
them would mark a live bug's output as intentional in the one surface built to
reveal it, and the family is not finite. A new test introduces a synthetic -4
and shows it surfacing as a gap with zero code changes, so nothing is
suppressed by slug name.

A DOC COMMENT CORRECTED BECAUSE THE CLAIM IN IT WAS MINE AND WAS FALSE.
I told the author that global-agent-rules-standard feeds an embedded-baseline
fallback consumed by the version-comparison logic. It does not:
ensureGlobalAgentRulesStandardConfig only repairs the stored row and never
injects it into a render that omitted it. The author checked rather than
inherited it, and the reviewer read the function independently and agreed.

Review: GO from pr52-reviewer at 8f47c82, lens tag-mechanism-and-hardcode-resistance.
Verified independently rather than restated: the -4 test has no slug special-casing
(11/0 standalone), exactly one production row carries the tag with the duplicate
family untagged, the corrected comment matches the source, 565/0 suite and clean
typecheck in a disposable worktree, and merge-tree confirms no unreviewed content.

DISCLOSURE: author and reviewer share a dispatcher (fabricius / agent-chief-staff).
Not arm's-length, and the reviewer said so in its own verdict.

NOT VERIFIED, carried forward rather than closed: the live coverage fraction
moving 4->5->4 was not reproduced, because the installed pre-publish CLI has no
coverage command to exercise; the static inputs check out. And whether main's
already-merged accountedGlobalSourceSlugs actually resolves the original P1 #2
remains unaudited by anyone — it arrived via a concurrent commit that merged
under an unresolved NO-GO at 08:34:10Z.

Re-derived at the merge moment: head equals the reviewed sha, behind=0,
merge-tree byte-identical to the head tree, mergeStateStatus CLEAN.

Agent: fabricius
andrei-hasna added a commit that referenced this pull request Aug 2, 2026
…ommits (#53)

chore(release): instructions 0.4.18 — publish the source-visibility commits (#53)

Bumps package.json 0.4.17 -> 0.4.18 and adds the changelog entry for what the
release carries. No source file is touched and no behaviour changes.

Three commits had landed on main after the 0.4.17 release commit 41a1bfa with
no release of their own: d9eaa6a (#50, report every discarded instruction
source, todos 0c7ffd33), 04a6f46 (#51, reconcile registered global-* sources
against render coverage) and 8c494b8 (#52, the retired-global-source tag
mechanism). main still declared 0.4.17 while npm latest was 0.4.17 published
2026-08-02T02:04:23.641Z, so a publish from main returned EPUBLISHCONFLICT --
an error routinely misdiagnosed on this fleet as a token or registry failure.
Raised as P1 by the PR #50 review.

Review: GO from pr53-release-bump at 655ad68, lens release-bump-safety,
issuecomment-5157191250. The reviewer independently confirmed 0.4.18 is
unpublished and 0.4.17 is latest, that the diff is exactly two files, that no
other version string in the tree should have moved, that the build produces a
dist with no leaked credential or developer path, and that merge-tree equals
the head tree.

Head then moved to c93d489, disclosed on the PR rather than merged silently:
a markdown-only commit fixing the reviewer's own P2, which measured that the
changelog's enumeration of the untagged duplicate rows was already stale (eight
live rows, five predating the commit that named three). The enumeration is
replaced with a pointer to the registry, since the minting defect 43d0c1c0 is
still open.

Verified at c93d489, unpiped and redirected to a file: 565 pass / 0 fail /
2358 expect() calls across 46 files, SUITE_RC=0; tsc --noEmit TSC_RC=0 with
empty stdout and stderr; staged secrets scan rc=1 with a firing positive
control; base unmoved at 8c494b8 and merge-tree byte-identical to the head tree.

Agent: publius-instructions-0418
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant