harness: close the cache-blind check class (H-CHECK-CACHE-BLIND)

[dayfine]

Summary

Closes the systemic version of PR #2143's qc-behavioral finding
(H-CHECK-CACHE-BLIND): a dune (deps ...) list cannot express a path
above the dune workspace root, so any trading/devtools/checks/*.sh
script that reads such a path via repo_root() had a dune rule that
could silently serve a stale cached PASS on a warm _build after the
real file it reads regressed. #2143 fixed exactly one rule this way
(run_in_env_root_check.sh); this PR closes the class.

What changed

• Audited all 25 scripts under trading/devtools/checks/ whose
  body contains an actual repo_root()/_repo_root() call site
  (tightened detection — see "false positive" note below). 2 were
  already covered (_check_lib.sh defines the helper and is never a
  dune run-target; run_in_env_root_check.sh already had (universe)
  from harness: fix run-in-env.sh false-green on worktree builds (H-RUNENV-WORKTREE-BLIND) #2143).
• 20 of the remaining 23 genuinely needed the fix — added
  (universe) to their owning dune rule's deps: agent_compliance_check.sh,
  agent_compliance_test.sh, jj_workspace_smoke.sh,
  status_file_integrity.sh, index_size_linter.sh,
  orchestrator_plan_check.sh, all 6 deep_scan_*_check.sh structural
  smoke tests, consolidate_day_check.sh, rule_promotion_check.sh,
  budget_rollup_check.sh, posix_sh_check.sh, no_python_check.sh,
  extract_metrics_gate_smoke.sh, sweep_worktrees_smoke.sh,
  perf_catalog_check.sh.
• 3 of the 23 are provably exempt — recorded with evidence in
  trading/devtools/checks/universe_deps_exceptions.conf + inline dune
  comments. Evidence per script (see "Exemption evidence" below).
• New mechanical guard check_universe_deps.sh (+ self-test
  check_universe_deps_test.sh), wired into dune runtest, closes the
  class going forward: FAILs on any script with a repo_root() call
  site whose owning rule lacks (universe) and isn't in the exceptions
  list. The guard's own dune rule declares (universe) for itself — a
  cache-blindness linter that is itself cache-blind was exactly what
  harness: fix run-in-env.sh false-green on worktree builds (H-RUNENV-WORKTREE-BLIND) #2143's reviewer caught.

Exemption evidence (the part QC will probe hardest)

record_qc_audit.sh + write_audit.sh — both ARE invoked in
production outside of dune (the orchestrator's Step 5 calls them every
run against the real dev/reviews//dev/audit/). That production
invocation is not gated by any dune rule — it happens outside
dune runtest entirely, so it is out of scope for a dune-cache-staleness
guard by construction. The only dune rule that references either script
is the smoke test governed by record_qc_audit_test.sh. I verified
mechanically (not just by inspection) that every one of the 24
invocations of record_qc_audit.sh/write_audit.sh inside
record_qc_audit_test.sh is preceded by an explicit
REPO_ROOT="${TMP_REPO}" override pointing at a freshly-mktemp -d
fixture repo (grep + line-proximity check, not one exception). So the
dune rule that exists never reads the real dev/reviews//dev/audit/
content — there is nothing for a warm _build to serve stale. If a
future edit to record_qc_audit_test.sh ever invokes either script
without the override, this exemption goes stale silently (dune has no
way to detect that on its own) — flagged as a residual follow-up in
dev/status/harness.md.

deep_scan_followup_count_check.sh — reads
deep_scan/check_05_followup_items.sh and
deep_scan/check_08_trends.sh via repo_root() (an absolute,
sandbox-escaping path), but both files are already declared as
explicit deps on the same dune rule (deep_scan/_lib.sh deep_scan/check_05_followup_items.sh deep_scan/check_08_trends.sh) —
dune tracks their content hash regardless of how the script re-derives
their path at runtime. Internally, the rule always invokes both check
scripts with REPO_ROOT overridden to a mktemp -d synthetic fixture
tree, never real repo content.

False-positive guard (caught during development)

A naive grep -l repo_root *.sh candidate scan flags any script that
merely mentions another script's filename in a comment or echo string
(e.g. a hypothetical helper_with_repo_root.sh reference). Tightened
detection to require an actual call-site pattern (repo_root) /
_repo_root) — i.e. the name immediately followed by a closing paren,
matching both $(repo_root) command-substitution call sites and
repo_root()-style definitions/calls, but not a bare filename
substring). Pinned by check_universe_deps_test.sh assertion 5.

Warm-build re-run evidence (the property #2143 actually caught)

RED/GREEN on the guard alone only proves detection. The separate,
load-bearing property is that a rule with (universe) re-executes
against a warm _build when the real outside-workspace file changes —
rather than serving a cached PASS. Demonstrated by:

1. dune runtest warm (full build green, exit 0).
2. Added a throwaway dev/<tmp>.py file (outside the dune workspace,
   inside repo_root()) with NO dune-tracked dep changed.
3. Re-ran dune runtest — no_python_check.sh's rule (has (universe))
   re-executed and FAILed, naming the new file.
4. Removed the file, re-ran — back to green.
5. Negative control: temporarily stripped (universe) from
   no_python_check.sh's rule, re-added the throwaway .py file,
   warm-rebuilt — the rule was skipped, dune runtest returned
   exit 0 with the stale PASS, reproducing the exact bug class
   harness: fix run-in-env.sh false-green on worktree builds (H-RUNENV-WORKTREE-BLIND) #2143 fixed for one rule. Restored (universe), confirmed FAIL
   returns, then cleaned up.

Verbatim exit codes/output for both directions are in the session
transcript; summarized in the PR discussion if requested.

Test plan

• sh trading/devtools/checks/check_universe_deps_test.sh — 5/5
  assertions pass.
• dune build && dune runtest green in-container (full suite, not just
  scoped).
• dune build @fmt green.
• Guard demonstrated RED (missing (universe), named the offending
  script) and GREEN (fixed) both via direct script invocation and via
  dune runtest itself.
• Warm-build re-run property demonstrated with a positive case
  ((universe) present → re-runs, catches regression) and a negative
  control ((universe) absent → silently stale-passes), see above.

🤖 Dispatched by GHA orchestrator run 30393663268

[dayfine]

Reviewed SHA: 1b3f809

Structural QC — H-CHECK-CACHE-BLIND guard

Checklist Status

#	Check	Status	Notes
H1	dune build @fmt (format check)	PASS	Code is properly formatted; no violations in diff or guard script execution
H2	dune build	INCOMPLETE	CI still in_progress at review time; cannot gate on expectation
H3	dune runtest	INCOMPLETE	CI still in_progress. Guard script runs locally: 23 OK, 3 SKIP (exceptions), exit 0
P1	Functions ≤ 50 lines (linter)	NA	Harness infrastructure PR; no new functions added
P2	No magic numbers (linter)	NA	No hardcoded values added
P3	Config completeness	NA	No new config parameters
P4	Public-symbol export hygiene (linter)	NA	No new exported symbols
P5	Internal helpers prefixed per convention	PASS	Helper functions (_repo_root, strip_comments) properly prefixed
P6	Tests conform to project test-patterns rules	PASS	check_universe_deps_test.sh: 5 assertions covering all edge cases (missing universe, universe added, exceptions-list skip, dep-only resolution, false-positive guard)
A1	Core module modifications	NA	No changes to Portfolio/Orders/Position/Strategy/Engine
A2	Dependency-direction rules respected	PASS	Harness PR; no analysis/ imports into trading/trading/
A3	No unnecessary existing module modifications	PASS	Only 5 files touched (dune, two new check scripts, exceptions conf, status file); no drift

Focus Area Findings

1. Guard self-coverage (critical) — PASS

   • Line 524: check_universe_deps.sh rule declares (universe) ✓
   • Line 541: check_universe_deps_test.sh rule declares (universe) ✓
   • The guard's own dune wiring prevents cache-blindness in itself — exact pattern #2143's reviewer required.

2. Exceptions file review_at question

   • Three exemptions recorded (deep_scan_followup_count_check.sh, record_qc_audit.sh, write_audit.sh)
   • My call: PROVEN EXEMPTIONS (do not require review_at triggers)
   • Evidence: dune file lines 287-294 and 454-462 document why each script's repo_root() reads are already covered or always fixture-scoped. These are structural/permanent facts (test infrastructure designed to never touch real outside-workspace files), not deferrals ("we'll fix this later"). Unlike a deferral, a proven exemption is load-bearing architecture that doesn't age.
   • Minor improvement: dune comments could cite the exact exemption evidence (e.g., "REPO_ROOT always overridden to temp fixture on all call sites"); current prose is clear but a future auditor would benefit from line-number pointers. Not a finding; suggestion only.

3. Parser robustness — PASS

   • Test assertions 1–5 cover: missing universe (FAIL), universe added (PASS), exceptions-list skip, dep-only resolution, false-positive guard
   • Fixes for two real bugs confirmed in code:
     • Lines 91–110: RS order (read exceptions/candidates first under default RS="\n", then switch to paragraph mode) — prevents getline corruption. Test line 88 documents this.
     • Line 57: grep -lE 'repo_root\)' pattern — matches "repo_root" followed by closing paren only, not bare substring. Test assertion 5 (lines 184–191) verifies false-positive guard against comment mentions.
   • Regex at line 117 extracts run-target from %{dep:...}, handles nested parens and extraction edge cases correctly.

4. set -e hygiene — PASS

   • Lines 72–77: set +e / awk / CODE=$? / set -e bracket is correct; explicitly cites H-CHECK-SETE-DIAGNOSTICS item.
   • Test file (lines 119–121, 135–138, etc.): same correct pattern.
   • No diagnostic-swallowing risk.

5. A3 scope verification — PASS

   • PR file list (gh pr view API): dev/status/harness.md, trading/devtools/checks/{check_universe_deps.sh, check_universe_deps_test.sh, dune, universe_deps_exceptions.conf}
   • No dev/status/_index.md touched (correct; that's orchestrator-owned)
   • Exactly 5 files as stated

Structural Verdict

All structural gates pass. No FAIL items. No A1/A2/A3 violations. No core-module drift.

Code quality: Guard script uses solid awk + shell patterns, test covers all edge cases, comments are clear and link to authority (dune rules, universe_deps_exceptions.conf).

Cannot issue APPROVED: build-and-test CI check still in_progress at review time. Per .claude/rules/pr-merge-gates.md, "if CI has not completed when you must decide, mark the gate incomplete and say so — do NOT issue an APPROVED that rests on expectation." Perf-tier1-smoke is completed/success.

Quality Score

4 — Implementation is solid, all structural requirements met, edge cases handled; only note is that exceptions file could reference dune line numbers for future audit clarity (cosmetic improvement, not a finding).

[dayfine]

Reviewed SHA: 1b3f809

Behavioral QC — harness/check-universe-deps (H-CHECK-CACHE-BLIND)

Reviewed in a detached worktree at the tip. CI gate complete on the exact tip:
build-and-test completed/success, perf-tier1-smoke completed/success. No local
dune runtest was run as a gate; the dune runs below are mutation instruments only.

Contract Pinning Checklist

#	Check	Status	Notes
CP1	Each non-trivial claim in new .mli docstrings has an identified test that pins it	NA	No .mli in this PR — the diff is two POSIX shell scripts, a dune file, a .conf, and a status-file entry.
CP2	Each claim in the PR body "Test plan" has a corresponding committed test	PASS	Every claim reproduced independently — see the ledger below. check_universe_deps_test.sh → 5/5, exit 0. Guard against the real dune file → exit 0, 23 OK + 3 SKIP = 26 candidates (consistent with the body's "25 audited, _check_lib.sh excluded, +2 new scripts"). "20 fixed" verified arithmetically: 23 OK − run_in_env_root_check.sh (pre-existing from #2143) − the 2 new guard scripts = 20. Guard demonstrated RED via direct invocation (M1, M3); its wiring into dune runtest observed executing inside @devtools/checks/runtest.
CP3	Pass-through / identity / invariant tests pin identity, not just size	NA	No pass-through or identity semantics in this feature.
CP4	Each guard called out explicitly in code docstrings has a test exercising the guarded-against scenario	PASS	(a) "does NOT match on the bare substring repo_root" → assertion 5, plus fixture helper_with_repo_root.sh. (b) dep-only (non-run-target) resolution → assertion 4. (c) exceptions-list skip → assertion 3. (d) "this rule's OWN dune wiring declares (universe)" → pinned mechanically, not by the self-test: M3 below shows the guard flags its own rule when (universe) is stripped, and the guard runs against the real dune under runtest. (e) the set +e/set -e bracketing (H-CHECK-SETE-DIAGNOSTICS shape) → exercised by assertions 1 and 4, which both take the non-zero-awk-exit path and still print diagnostics.

Behavioral Checklist (Weinstein domain)

Entire A1 / S* / L* / C* / T* block — NA. Pure harness / build-tooling PR;
touches no strategy, screener, stops, portfolio, or simulation logic. Per
.claude/rules/qc-behavioral-authority.md §"When to skip this file entirely", CP1–CP4
is the full review. qc-structural did not flag A1.

Warm-build skip/re-execute pair — independently reproduced ✅

I did not take the author's experiment on trust. Reproduced both directions with a
throwaway dev/__qcb2148_probe.py (outside the dune workspace root, inside repo_root()):

Positive — (universe) present, @devtools/checks/runtest warmed to green, then the
.py created with no dune-tracked dep touched:

(cd _build/.sandbox/4db11cf2b02801bcfd651861321f6dd7/default/devtools/checks && /usr/bin/sh ./no_python_check.sh)
FAIL: no-python check -- unexpected *.py files found:
  /__w/trading/wt-qcb2148/dev/__qcb2148_probe.py
DUNE_EXIT=1

The rule re-executed and caught it.

Negative control — (universe) stripped from that rule (and the script exempted so the
guard itself doesn't confound the exit code), warm build with no .py first to cache a
clean PASS, then the .py re-created with nothing dune-tracked changed:

count of "no_python_check.sh" mentions in dune output: 1   (the guard's own SKIP line only —
                                                            NO `(cd _build/.sandbox/... && sh ./no_python_check.sh)` invocation)
STEP_D_DUNE_EXIT=0

The rule was never invoked and dune returned exit 0 with a real violation on disk — the
#2143 bug class reproduced live, exactly as the PR body describes. The fix is real, not decorative.

Probe file deleted; find . -name '*.py' → 0; git status --porcelain → clean.
(.claude/rules/no-python.md respected — nothing committed.)

Mutation ledger

#	Mutation	Verbatim exit	Result
M1	Strip (universe) from status_file_integrity.sh's rule	GUARD_EXIT=1	KILLED — FAIL: status_file_integrity.sh calls repo_root() ... does not declare (universe), names the script + the remediation.
M2	Strip (universe) from no_python_check.sh's rule and add one bare line to universe_deps_exceptions.conf	GUARD_EXIT=0	SURVIVED (by design) → FLAG-1 below.
M3	Strip (universe) from the guard's own rule	GUARD_EXIT=1 (guard); SELFTEST_EXIT=0	KILLED by the guard itself. Self-coverage is genuinely pinned, not merely present-in-the-file-today: the guard scans the real dune and flags its own rule. The self-test correctly stays fixture-only and does not (and should not) assert on real dune content. Editing the rule also invalidates dune's own cache key for it, so the removal can't hide behind a warm build.
M4	Add a deep_scan/zz_probe_check.sh with a genuine $(repo_root) call site, wired as a run-target with no (universe)	GUARD_EXIT=0	SURVIVED → FLAG-2 below.

Findings — all FLAG, none blocking

FLAG-1 — universe_deps_exceptions.conf is entirely unconstrained (M2).
One bare filename silently disables the guard for any script, forever. The file's header says
"Do NOT add an entry just to make the guard pass — see .claude/rules/code-health-discipline.md",
but nothing enforces it, and that rule requires path exceptions to carry "a clear review_at
trigger (date or PR number)" — the conf's documented format has no review_at field at all.
The repo already has the machinery for this on the analogous file: linter_exceptions.conf is
expiry-checked by deep_scan_linter_expiry_check.sh / deep_scan/check_11_linter_expiry.sh.
The PR is partially honest here: H-CHECK-EXEMPTION-DRIFT (filed, dev/status/harness.md)
covers the three existing entries going stale, but not unbounded future additions. Suggest
folding a review_at column + expiry check into that item.

FLAG-2 — the candidate scan is non-recursive; deep_scan/*.sh is uncovered (M4, undisclosed).
grep -lE 'repo_root\)' ./*.sh misses subdirectories. Two live subdirectory callers exist:
deep_scan/_lib.sh (REPO_ROOT="$(repo_root)", line 28) and deep_scan/main.sh (line 23).
Neither is mis-covered today — _lib.sh is an explicit dep of the exempted followup-count
rule, and main.sh is not a dune run-target at all (orchestrator-invoked, outside dune runtest,
same out-of-scope-by-construction reasoning the PR applies to record_qc_audit.sh). So this
does not reopen the class for anything currently wired; it is a prospective hole confined to one
subdirectory. Not covered by the filed residual. A one-line fix (grep -rlE ... --include='*.sh',
or globbing ./*.sh ./*/*.sh) would close it.

FLAG-3 — doc nit. check_universe_deps.sh lines 87-90 credit "assertion 5" with catching the
awk RS-ordering bug. Under that mutation the candidates file collapses to one record, so no
candidate resolves and the guard exits 0 silently — the assertion that actually fails is
assertion 1 (which expects a non-zero exit naming sample_check.sh). Comment-only.

Why this clears the bar

The PR's central claim is unusually load-bearing — its subject matter is "a check that silently
doesn't run", the exact defect #2143's reviewer caught in a 174-line check that returned exit 0
with zero output. I reproduced that failure mode live and confirmed (universe) is what fixes it.
The guard is self-covering (M3), its parser is pinned against fixtures rather than against the
file it polices, the false-positive risk is real and is tested (assertion 5), and the three
exemptions carry evidence in two places (inline dune comments + the conf) plus a filed
residual. Status quo is 23 exposed rules; this leaves 0 exposed among currently-wired top-level
rules. Both surviving mutations are prospective future-proofing gaps with zero live instances —
FLAG, not blocker, per the calibration for "leaves the class still open / introduces a new
false-green".

Quality Score

4 — Good: a genuinely well-evidenced infra PR whose load-bearing warm-build claim reproduced exactly in both directions, with a self-covering guard and fixture-isolated self-test; two prospective coverage gaps (unconstrained exceptions file, non-recursive scan) keep it off 5.

Verdict

APPROVED