harness: close the three #2148 FLAG residuals (recursive scan + review_at on exceptions)

[dayfine]

What it does

Closes the two FLAG residuals qc-behavioral raised on PR #2148
(check_universe_deps.sh, the H-CHECK-CACHE-BLIND mechanical guard),
plus the doc nit:

FLAG-2 (M4) — candidate scan was non-recursive. check_universe_deps.sh
scanned only trading/devtools/checks/*.sh, missing deep_scan/_lib.sh and
deep_scan/main.sh (both call repo_root()). Switched the scan to a
recursive find -exec grep -lE 'repo_root\)' {} +. Candidate names below
the top level now come back path-qualified (e.g. deep_scan/_lib.sh); the
existing awk dep-matching and exceptions-list lookup already accept / in
script names, so no further parser change was needed there. Resolved the
two newly-visible candidates:

• deep_scan/_lib.sh — added to universe_deps_exceptions.conf. Its only
  dune reference is as an explicit dep of deep_scan_followup_count_check.sh's
  rule, which (per the existing exemption comment on that rule) always
  invokes its scripts with REPO_ROOT overridden to a synthetic fixture —
  same evidence already recorded for that rule's other two dep-only
  candidates.
• deep_scan/main.sh — added to universe_deps_exceptions.conf with the
  honest evidence: it's not a dune run-target or dep at all, by
  construction (it's the deep-scan orchestrator entrypoint, invoked directly
  or via the deep_scan.sh shim, outside dune runtest entirely — same
  "weekly tool, not a per-PR gate" reasoning already applied to the other
  deep_scan_*_check.sh structural smoke tests in the dune file).

Also improved the guard's "not referenced by any runtest rule" FAIL message
to point at the exceptions-file escape hatch for this exact case (a script
that's intentionally never wired into dune).

FLAG-1 (M2) — universe_deps_exceptions.conf was entirely unconstrained.
A bare filename with no expiry field could silently disable the guard for
any script forever, and nothing enforced the file's own header instruction
("don't add an entry just to make the guard pass" per
.claude/rules/code-health-discipline.md). Fixed by extending the file's
format to require a trailing # review_at: <never|M1-M7|YYYY-MM-DD>
annotation, same convention as the sibling linter_exceptions.conf:

• check_universe_deps.sh itself now hard-FAILs if any exceptions entry
  has a missing or unparseable review_at — per-PR enforcement, so a bad
  entry can never land in the first place.
• deep_scan/check_11_linter_expiry.sh (the weekly deep scan's existing
  T1-K expiry checker for linter_exceptions.conf) now also scans
  universe_deps_exceptions.conf with the same milestone/date logic,
  emitting a new ## Universe-Deps Exception Expiry report section. The
  shared per-entry scanning logic was extracted into a
  _scan_exceptions_conf() function so both conf files reuse the same
  milestone-lookup / date-comparison code.

Placement rationale (presence vs. expiry): presence/parseability is
enforced in the per-PR guard (cheap, blocks the "unconstrained forever"
hole at the exact point a bad entry would be added — this is what FLAG-1
was actually about). Expiry (has the milestone/date actually passed) is
left to the weekly deep scan, matching the established linter_exceptions.conf
precedent and cadence, and because it's a "should someone look at this"
signal rather than a hard per-PR gate.

FLAG-3 (doc nit). A comment in check_universe_deps.sh credited
"assertion 5" with catching the awk RS-ordering bug; reproducing the
mutation shows assertion 1 is the one that actually fails (assertion 5 was
never touched by that mutation). Fixed the attribution.

All three existing pre-review_at entries (deep_scan_followup_count_check.sh,
record_qc_audit.sh, write_audit.sh) were given review_at: never with
a one-line reason (their exemption is an architectural invariant of the
governing rule's fixture-override wiring, not time-bound).

Updated dev/status/harness.md's H-CHECK-EXEMPTION-DRIFT item to record
that the "unconstrained future additions" half of that residual is now
closed, while the "evidence-correctness of existing entries" half remains
open (still needs a stronger mechanical check or periodic manual re-audit
if drift is ever observed in practice — unchanged from before this PR).

Test plan

• check_universe_deps_test.sh extended from 5 to 7 assertions:
  • feat: add CSV storage module with price data types and parser #6 (FLAG-2): a subdirectory script (subdir/nested_check.sh, a dep —
    not run-target — of another rule) that calls repo_root() and whose
    owning rule lacks (universe) is caught by the recursive scan, named
    path-qualified in the FAIL output.
  • Set up time period conversion module #7 (FLAG-1): a universe_deps_exceptions.conf entry with no review_at
    annotation is rejected with a FAIL naming the entry.
  • All 7 pass: sh trading/devtools/checks/check_universe_deps_test.sh.
• Ran the guard directly against the real tree: sh trading/devtools/checks/check_universe_deps.sh exits 0, and now shows SKIP (exceptions list): deep_scan/_lib.sh and SKIP (exceptions list): deep_scan/main.sh in its output (previously invisible to the pre-fix non-recursive scan).
• D1 live before/after: added a throwaway deep_scan/zz_probe_check.sh
  (real $(repo_root) call site, no (universe) anywhere). With the OLD
  non-recursive ./*.sh-only scan temporarily restored, the guard silently
  exits 0 (misses it). With the fix restored, the guard FAILs, naming
  deep_scan/zz_probe_check.sh. Probe file removed after confirming both
  directions.
• D2 live before/after: appended a bare zz_probe_no_review_at.sh line
  (no review_at) to the real universe_deps_exceptions.conf — guard FAILs
  naming it with a review_at-specific message. Reverted the conf to the
  committed version (diffed identical) — guard exits 0 again.
• sh trading/devtools/checks/deep_scan/check_11_linter_expiry.sh <tmp> <tmp>
  run standalone: emits both ## Linter Exception Expiry (existing 2
  findings, unchanged) and the new ## Universe-Deps Exception Expiry
  section (0 findings, since all current entries are review_at: never).
• sh trading/devtools/checks/deep_scan_linter_expiry_check.sh (the
  structural smoke test pinning Check 11's markers) still passes.
• dune build -j 2 && dune runtest clean (see CI / run log for this PR).

🤖 Dispatched by GHA orchestrator run 30405186411

[dayfine]

Reviewed SHA: d74a495

Structural QC — harness/check-universe-deps-residuals

Scope confirmed via GET /pulls/2155/files (canonical, not git-ancestry walk): 5 files, matches git diff main...d74a4952 --stat exactly — dev/status/harness.md, trading/devtools/checks/check_universe_deps.sh, trading/devtools/checks/check_universe_deps_test.sh, trading/devtools/checks/deep_scan/check_11_linter_expiry.sh, trading/devtools/checks/universe_deps_exceptions.conf. All shell + one conf + one status-doc line; no OCaml.

CI gate (H1): polled GET /commits/{sha}/check-runs to completion on this exact tip SHA.

• build-and-test: completed / success (completed_at 2026-07-28T23:17:12Z)
• perf-tier1-smoke: completed / success (completed_at 2026-07-28T23:08:02Z)

Both required checks green on the reviewed SHA — no new commits since.

Structural Checklist

#	Check	Status	Notes
H1	CI gate (build-and-test + perf-tier1-smoke)	PASS	Verified via GitHub check-runs API on tip SHA, both completed/success (see above).
H2	dune build (in worktree, via dev/lib/run-in-env.sh)	PASS	Exit 0. dune build @fmt also exit 0 (unrelated cosmetic ocamldoc warnings on pre-existing OCaml files not touched by this diff).
H3	dune runtest	PASS	Full-suite run completed exit code 0 (background job, confirmed via harness notification). Includes fmt_check.sh ("OK: all .ml/.mli files are correctly formatted") and linter_magic_numbers.sh ("OK: no magic numbers found") among the last steps.
P1	Functions ≤ 50 lines (linter)	NA	Shell scripts, not OCaml; fn_length_linter doesn't apply. New shell functions (_scan_exceptions_conf, awk strip_comments) are well under any reasonable line budget by inspection.
P2	No magic numbers (linter)	NA	Shell/conf diff; magic-numbers linter is OCaml-scoped. No hardcoded domain thresholds introduced — the M[1-7] milestone range and date-regex are structural parsing constants matching the existing linter_exceptions.conf convention, not tunable domain parameters.
P3	Config completeness	NA	Not applicable — this is dev tooling (a lint guard), not strategy/analysis logic subject to the config-surface principle.
P4	Public-symbol export hygiene (.mli coverage)	NA	No OCaml modules touched.
P5	Internal helpers prefixed per convention	PASS	New shell helper _scan_exceptions_conf() and its output globals _SCAN_COUNT/_SCAN_MISSING_COUNT/_SCAN_DETAILS/_SCAN_MISSING follow the underscore-prefix convention used elsewhere in this file (_milestone_num). Verified by direct read of check_11_linter_expiry.sh.
P6	Test-pattern conformance (OUnit/Matchers)	NA	No OCaml test files in this diff. Shell self-test analogue (check_universe_deps_test.sh) reviewed instead — see below.
A1	Core module modifications (Portfolio/Orders/Position/Strategy/Engine)	PASS	None touched — diff is entirely trading/devtools/checks/ + a status-doc line. No FLAG.
A2	No new analysis/ → trading/trading/ imports outside backtest exception	NA	No dune library deps changed; pure shell/conf.
A3	No unnecessary modifications to existing (non-feature) modules	PASS	File list from $PR_FILES (API) confirms the diff touches only the four check_universe_deps*/check_11_linter_expiry.sh/.conf files this feature owns, plus the one status-file line documenting it. No cross-feature drift.

Additional verification performed (beyond the generic checklist)

1. Shell self-test is fixture-isolated (P6 analogue). Read check_universe_deps_test.sh top-to-bottom: it builds a synthetic fixture tree under mktemp -d (FAKE_ROOT/FIXTURE_CHECKS) and drives check_universe_deps.sh against that fixture via REPO_ROOT="$FAKE_ROOT" override — it never asserts against the real trading/devtools/checks/dune or the real universe_deps_exceptions.conf. New assertions #6 (recursive subdir-catch, subdir/nested_check.sh) and #7 (missing-review_at rejection) both use the same fixture mechanism as assertions 1–5. This preserves the property #2148's reviewer praised in the prior version — confirmed, no regression.

2. Direct script invocation (not only via dune runtest, since these are exactly the cache-blindness-prone scripts under review):

   sh trading/devtools/checks/check_universe_deps.sh            -> GUARD_EXIT=0    (all 27 candidates resolve OK/SKIP; 5 exceptions-list SKIPs incl. new deep_scan/_lib.sh, deep_scan/main.sh)
   sh trading/devtools/checks/check_universe_deps_test.sh        -> SELFTEST_EXIT=0 (7/7 assertions pass)
   sh trading/devtools/checks/deep_scan/check_11_linter_expiry.sh /tmp/expiry_report.md -> EXPIRY_EXIT=0
   

   Exit codes read directly (no pipe-then-$?).

3. check_11_linter_expiry.sh refactor regression check (highest-risk change in the diff — it touches an existing, already-wired weekly checker). Ran it directly against the real linter_exceptions.conf + universe_deps_exceptions.conf and inspected the emitted report:

   • The pre-existing [EXPIRED] finding for analysis/technical/trend/lib/segmentation.ml (review_at: after segmentation is validated in simulation (M5), current milestone M5) still fires: [EXPIRED] magic_numbers analysis/...segmentation.ml ... — M5 <= current milestone M5. Refactor (extracting the shared _scan_exceptions_conf() function) preserved original behavior byte-for-byte in substance.
   • The pre-existing [UNRECOGNISED format] finding for price_path.ml also still fires, unchanged.
   • New ## Universe-Deps Exception Expiry section is emitted and correctly reports "No expired or missing review_at annotations found" (all five real entries are never-pinned).
   • Structural smoke test deep_scan_linter_expiry_check.sh (dune-wired, grep-based marker check) also passes directly: OK: deep scan Linter Exception Expiry section (T1-K) structural check passed.
   • The function uses plain (non-subshell) global-style out-variables (_SCAN_*) set by direct call, not command substitution, so there's no subshell-scoping hazard under set -e/sh.

4. set -eu / recursive-find shell-correctness spot checks:

   • CANDIDATES=$(cd "$CHECKS_DIR" && find . -name '*.sh' -type f -exec grep -lE 'repo_root\)' {} + 2>/dev/null | sed ... || true) — the trailing || true correctly absorbs grep's exit 1 (no matches) under set -eu; confirmed no abort.
   • find recursion is scoped to $CHECKS_DIR (trading/devtools/checks/), which contains only itself + deep_scan/ — no _build/ or other generated directory nests under this source path, so the "could recursion pick up _build/" concern does not manifest today. Verified: find trading/devtools/checks -maxdepth 2 -type d → only checks and checks/deep_scan.
   • Path-qualified candidate names (deep_scan/_lib.sh, subdir/nested_check.sh in the test fixture) flow correctly through the awk dep-matching: the depsonly character class [^A-Za-z0-9_.\/-] explicitly retains /, and the plain-string exceptions-list lookup (allow[aline]) is exact string equality, so slashes are handled. Minor latent gap (not exercised today, not a FAIL): the run-target regex %\{dep:[A-Za-z0-9_.]+\} does not include / in its character class, so a future subdirectory script wired as an actual %{dep:subdir/foo.sh} run target (as opposed to a plain dep-list entry, which is what both real subdirectory exemptions and the test fixture use) would not be matched by that regex. No current dune rule uses a path-qualified %{dep:...} run-target (grep -n '%{dep:[^}]*/' dune → no hits), so this is inert today. Recorded as a residual, not a blocking finding.

5. Minor doc-drift note (pre-existing file, not touched by this diff): the dune file's comment above the check_universe_deps.sh rule (line ~514-515) still says "the three scripts (record_qc_audit.sh, write_audit.sh, deep_scan_followup_count_check.sh) proven exempt" — now stale, since this PR adds two more exemptions (deep_scan/_lib.sh, deep_scan/main.sh) that are fully documented in universe_deps_exceptions.conf itself but not reflected in the referring dune comment. Since the dune file is outside this PR's diff, this is not a FAIL against this PR — flagging as a small follow-up opportunity only.

Quality Score

5 — All gates green (CI + build + full runtest), the highest-risk refactor (check_11_linter_expiry.sh) was independently verified to preserve original expiry behavior byte-for-byte, the new self-test assertions remain properly fixture-isolated, and the one latent gap found (path-qualified %{dep:...} run-targets) is inert today and clearly documented rather than hidden.

Verdict

APPROVED

No structural findings. The one residual noted above (run-target regex not accepting /) is not exercised by any current dune rule and does not block approval; it's recorded for awareness, not as a NEEDS_REWORK item.

[dayfine]

Reviewed SHA: d74a495

Behavioral QC — harness/check-universe-deps-residuals (PR #2155)

Pure-infra PR (mechanical guard + weekly checker maintenance). No domain
(Weinstein) logic touched. All work verified live in a detached worktree
at the reviewed SHA — every mutation below was executed by me, not taken
from the PR body on trust.

Domain checklist (S*/L*/C*/T*) — NA

All rows NA. This PR touches only trading/devtools/checks/*.sh +
universe_deps_exceptions.conf + a status-file line — a harness/linter
change with zero stage-classifier, stop, screener, or strategy code in the
diff. Per .claude/rules/qc-behavioral-authority.md §"When to skip this
file entirely."

Contract Pinning Checklist

#	Check	Status	Notes
CP1	Non-trivial header-comment claims in check_universe_deps.sh / universe_deps_exceptions.conf are each pinned by a test	PASS	(a) "recursive scan closes FLAG-2" → assertion 6 in check_universe_deps_test.sh (subdirectory script caught, path-qualified) — reproduced live, see mutation table. (b) "missing/unparseable review_at is a hard FAIL" → assertion 7 — reproduced live. (c) "this rule's own dune wiring declares (universe)" → confirmed in dune (final rule block for check_universe_deps.sh + its test both declare (universe)). (d) FLAG-3 attribution fix (comment now correctly credits "assertion 1", not "assertion 5", for the RS-ordering bug) — confirmed by diff and by re-reading the corrected comment in context.
CP2	PR body "Test plan" claims vs. committed tests	PASS	"extended from 5 to 7 assertions" → check_universe_deps_test.sh has assertions 1-7, ran 7/7 PASS. "D1 live before/after" (recursive scan catches subdir probe; old scan misses it) → reproduced independently below, both directions confirmed. "D2 live before/after" (bad exceptions entry FAILs, revert restores exit 0) → reproduced independently below. "deep_scan/check_11_linter_expiry.sh emits both sections, Linter Exception Expiry unchanged" → reproduced: byte-identical ## Linter Exception Expiry output pre/post refactor, new ## Universe-Deps Exception Expiry section present with 0 findings (all entries review_at: never). "deep_scan_linter_expiry_check.sh structural smoke test still passes" → ran it directly, PASS. No PR-body claim found unsupported by an actual test/reproduction.
CP3	Pass-through / identity / invariant tests pin identity, not just size	NA	No pass-through semantics in this PR (a linter's PASS/FAIL is not an identity-preserving transform).
CP4	Guards named in docstrings each have a test exercising the guarded-against scenario	PASS	Guard 1 ("recursive scan must catch subdirectory scripts, not just top-level") → assertion 6. Guard 2 ("exceptions entry with no review_at is a hard FAIL") → assertion 7. Guard 3 ("false-positive: bare substring match on repo_root in a comment/string must not be treated as a call site") → pre-existing assertion 5, still present and passing, unaffected by this diff's regex change (repo_root) require-paren pattern, same as base).

Mutation table (all executed live by me, detached worktree at d74a495)

Mutation	Expected	Observed	Result
Baseline: sh check_universe_deps.sh on real tree	exit 0, deep_scan/_lib.sh + deep_scan/main.sh shown as SKIP (exceptions list)	exit 0; both entries appear as SKIP exactly as claimed	KILLED (i.e. baseline holds, no bug)
Add deep_scan/zz_probe_check.sh (real $(repo_root) call, no coverage) with current (recursive) scan	FAIL naming the probe	FAIL: deep_scan/zz_probe_check.sh calls repo_root() but is not referenced...; exit 1	KILLED (guard works)
Same probe present, run the pre-#2155 non-recursive ./*.sh-only scan (negative control, copied verbatim from main)	Probe silently missed (never appears in output)	Probe never appears in any OK/FAIL/SKIP line — confirmed the FLAG-2 gap was real and is now closed	KILLED (regression proven + proven fixed)
Append bare some_new_script.sh (no review_at) to universe_deps_exceptions.conf	FAIL naming the entry, review_at-specific message	FAIL: ...entry "some_new_script.sh" has no parseable "# review_at: <value>" annotation...; exit 1	KILLED
Revert conf to committed version	exit 0	exit 0, clean OK output	KILLED
deep_scan/_lib.sh exception evidence: "only dep is deep_scan_followup_count_check.sh's rule, always fixture-overridden"	grep of dune shows exactly one rule listing it as a dep	Confirmed: single hit, in that rule's (deps ...) block; rule's own comment documents the fixture-override invariant	KILLED (evidence holds)
deep_scan/main.sh exception evidence: "never a dune run-target or dep at all"	zero hits for main.sh as a real dep/run-target anywhere in dune	Confirmed: only comment-text mentions of "main.sh" in dune/deep_scan/*.sh; zero %{dep:...} or bare-dep references	KILLED (evidence holds)
check_11_linter_expiry.sh refactor: does the known-EXPIRED segmentation.ml entry (review_at: ... (M5), current milestone M5) still fire post-extraction to _scan_exceptions_conf()?	[EXPIRED] ... M5 <= current milestone M5 still present	Present, byte-identical ## Linter Exception Expiry section vs. the pre-refactor script run side-by-side on the same tree	KILLED (no silent regression in the highest-risk change)
Self-test fixture isolation: does check_universe_deps_test.sh use a synthetic mktemp -d tree via REPO_ROOT override, never the real conf/dune?	Yes, by construction	Confirmed by reading; also ran the real self-test (7/7 PASS) independent of any conf mutation state	KILLED
My own mutation: fake _build/default/fake/generated_check.sh nested inside trading/devtools/checks/ — would the recursive find false-positive on real dune build artifacts?	Uncertain going in	Confirmed this scenario cannot occur in this repo's layout: dune-workspace lives at trading/, so dune's real _build/ is trading/_build/ — a sibling of devtools/, never nested inside trading/devtools/checks/. No real _build dir exists under the scanned directory in the actual repo. Not a live risk.	N/A — scenario unreachable in practice, not a finding
My own mutation: malformed-but-plausible review_at values (soon, M9)	Uncertain	Both correctly rejected (FAIL, unparseable) — soon matches no form, M9 fails the M[1-7] character class	KILLED
My own mutation: review_at: 2026-13-45 (digit-shaped but not a real calendar date)	Uncertain	Accepted (exit 0) — the date regex is shape-only ([0-9]{4}-[0-9]{2}-[0-9]{2}), doesn't validate month/day ranges	SURVIVED (minor, non-blocking — see note)

Note on the one survivor

review_at: 2026-13-45 (invalid month/day) is accepted as a well-formed
date by check_universe_deps.sh's presence/parseability check. This is
consistent with the sibling linter_exceptions.conf convention this file
explicitly mirrors (deep_scan/check_11_linter_expiry.sh's date regex has
the same shape-only laxity, confirmed pre-existing on main, not
introduced by this PR) — the PR's own claim is "presence of a parseable
<never|M1-M7|YYYY-MM-DD>-shaped value," not "a real calendar date," and
that narrower claim is what CP1/CP4 pin. Not a regression, not a new
false-green vector for the guarded class (a bogus date can only ever be
too permissive about which future date something reviews at — it cannot
cause a (universe)-missing rule to go undetected, which is the actual
H-CHECK-CACHE-BLIND risk this guard exists for). FLAG per the calibration
note in my dispatch brief (prospective, no live instance, does not reopen
the class for anything currently wired) — not a blocker.

Quality Score

5 — Every claim in the PR body and script docstrings was independently reproduced via live mutation (not taken on trust), including a negative control that proved the FLAG-2 gap was real before this fix. The highest-risk change (refactoring an already-wired weekly checker) was verified byte-identical on existing behavior before adding new behavior. The two new exceptions-list entries' evidence was checked against the actual dune file rather than accepted from the comment text. This is reference-quality behavioral rigor for a mechanical-guard PR.

Verdict

APPROVED