GT-602: the compiled policy.wasm is verified over the whole registry, and the row closes - #306
Open
beyondnetPeru wants to merge 16 commits into
Open
GT-602: the compiled policy.wasm is verified over the whole registry, and the row closes#306beyondnetPeru wants to merge 16 commits into
beyondnetPeru wants to merge 16 commits into
Conversation
…try, never skipped GT-602 criteria 1 and 3. The compiled-bundle block of `abac-rego-parity.spec.ts` was governed by `EVOLITH_REQUIRE_OPA_WASM`: it ran in one CI job and `it.skip`ped everywhere else, because `policy.wasm` is a gitignored build output. That is the shape that produced GT-632 — a stale wasm at the pre-`src/` path on one machine, every MCP tool denied in production, and a green suite the whole time. The block now COMPILES the policy itself. Missing or older than any `.rego` it is built from -> run `.harness/scripts/compile-opa-wasm.mjs` (the same script `npm run build:policy` runs, writing the same paths the runtime loads); if that compilation cannot happen, the suite fails with the compiler's own diagnostic. There is no environment in which it passes without having evaluated a wasm, and staleness is treated as absence because a stale bundle is the actual defect. Denominators are asserted rather than assumed: the registry must hold at least the 50 tools measured today, every one of those names must be evaluated (`evaluated.length === registered.length`), and every evaluation must come back allowed (`allowed.length === registered.length`). The ABAC-03 negative control now runs first, since an `evaluate()` returning an unexpected shape would make every ALLOW assertion pass vacuously. `compile-opa-wasm.mjs` installs by rename instead of `copyFileSync`. The test compiling on demand makes concurrent read-during-write reachable: sibling jest workers read the same `policy.wasm`, and a partial read throws in `loadPolicy`, which the evaluator reports as OPA_ERROR — a denial. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… tools (GT-606) ADR-0093 has been Accepted since 2026-06-20 and mandated a `baseSha` parameter, HEAD verification before applying, and a CONCURRENCY_CONFLICT error contract. Grepping the MCP server for any of those returned nothing, against 20 tools declaring `mutative: true`. Two agents on one workspace produced exactly the lost update the ADR was written to prevent. The MCP server is the thing doing the mutating — the tools write to disk themselves via IFileSystem/fs-extra into a caller-supplied directory — so the check belongs here and is not ceremony. Enforced at the single dispatch, keyed on `tool.mutative`, which is the same key the HITL approval gate already turns on. The protected set is therefore the mutative set by construction: a tool registered tomorrow inherits the check with no wiring, which a per-tool implementation would have made twenty chances to forget. `baseSha` is likewise derived onto every mutative tool's advertised schema by ToolRegistryService.describe rather than hand-copied into 20 input schemas, so the declared surface and the enforced behaviour cannot drift. Semantics are If-Match, not a lock: `baseSha` is optional, and a caller that declares the state it planned against gets protection. Requiring it outright would break every existing caller and make the tools that legitimately target a not-yet-initialised directory (evolith-scaffold, evolith-init-batch) unusable; EVOLITH_MCP_REQUIRE_BASE_SHA=1 gives a deployment the strict reading. When `baseSha` IS supplied, verification fails closed — an unresolvable HEAD is not evidence that the asserted base state holds. Tests. The load-bearing case is a genuine race, not a fabricated SHA: a real git repo, a real `git rev-parse` read by the client, and a second mutative call in flight at the same time landing a real commit in the window between that read and the dispatch's verification. It asserts both the conflict envelope and that the victim tool's side-effect file does not exist — a conflict reported after the write landed would be worse than no check. A deliberately stale SHA is kept as the cheaper second case. The denominator guard enumerates the mutative set off the live DI registry and drives every member through the real dispatch, so the count is observed rather than remembered and an unprotected tool fails the build with its own name in the diff. Verified non-vacuous: with the check disabled, the race fails and all 20 tools go unprotected. ADR-0093 stays Accepted and now records what backs it, including that §2 (pessimistic locking) remains deliberately unimplemented — no current tool holds an exclusive non-git resource long enough for a two-minute lease to be the right mechanism. Recording that is the point: the ADR should not keep claiming a mechanism nothing implements. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ody asked for GT-614's contract half already existed: a PipelineExecutionPlan, a reference `createKindSelectivePipeline`, and a suite proving selectivity. None of it was adopted, so the gap's own sentence was still true of the product — asking for `compliance` alone still loaded the f1..f5 gate corpus and ran every Rego rule in it, because `SatelliteEvaluationPipeline` took no plan at all. That pipeline is what `POST /api/v1/evaluate` (both branches), `evolith evaluate` and the MCP `evaluate` tool actually execute. It is now expressed as two kind-tagged stages — the SDLC phase gates (`gate`/`artifact`) and the canonical ruleset corpus (`rule`/`compliance`) — and a stage whose kinds were not requested is never entered: not its loader, not its evaluator. The plan is forwarded from the orchestrator through ValidateSatelliteUseCase on all four surfaces, so they cannot disagree about what a single-kind request costs. An unrun stage is OUT OF SCOPE, not `skipped`. `skipped` means the engine tried and the outcome is UNKNOWN: it belongs in the GT-569 denominator, becomes a coverage risk, and since GT-595 a blocking rule reported skipped FAILS the run. None of that is true of a question nobody asked. The stage therefore contributes nothing to rulesChecked/rulesSkipped/rulesErrored/rulesTotal and no gate result; it is recorded only in `coverage.outOfScopeGateIds`, so a reader can still see what the request did not buy. The pipeline now publishes that coverage quadruple (it published none before), and it keeps `checked + skipped + errored === total`. The load-bearing test is negative and about outcomes: the out-of-scope stage is one that WOULD FAIL the run. A call-count assertion would survive a refactor that re-ran the gate and discarded it; `expect(verdict.passed).toBe(true)` cannot. 8 of the 9 new tests fail against the previous code — the ninth is the baseline that proves the gate really does fail when it is in scope. Verified: npm run build (tsc -b, whole monorepo) clean; core-domain 1442, core-api 179, mcp-server 433, cli 1436 tests green; eslint boundaries clean (the new cross-layer imports are type-only). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the exit taxonomy (GT-580)
Closes the two open criteria of GT-580.
CRITERION 2 — `--format json` writes DATA ONLY to stdout.
Output formatting and process exits were ALREADY centralized: `installMachineChannelGuard`
(main.ts, armed before the command graph boots) reroutes every non-JSON stdout write to
stderr while a machine format is active, and `exit-codes.ts` + `BaseEvolithCommand.handleError`
map a thrown error onto the taxonomy in one place. What was missing was proof that the
centralization actually holds across the CLI: the existing subprocess spec pipes two
hand-picked commands. `machine-channel.registry-sweep.spec.ts` asks commander — the CLI's
own registry, built from the real AppModule — which commands declare `--format`, and sweeps
all 32 in real subprocesses with stdout and stderr on separate pipes. Each must hand back
exactly one parseable JSON document and exit from the taxonomy. The enumeration has a floor,
so a sweep over nothing fails instead of passing vacuously.
Observed red: commenting out `installMachineChannelGuard()` turns `agents` (a clack banner)
and `init-wizard` (a progress line) DIRTY while the other thirty stay clean.
CRITERION 3 — a ruleset with Rego parity fails a command that exits outside the taxonomy.
- `src/rulesets/cli/exit-code-taxonomy.rules.json` — CLI-EXIT-01 (no exit outside
{0,1,2,3}), CLI-EXIT-02 (the scan must be non-vacuous), CLI-EXIT-03 (the taxonomy may
not be widened to absorb an offender — otherwise CLI-EXIT-01 has a one-line escape).
- `src/rulesets/opa/cli-exit-code-taxonomy.rego` + tests — the same three ids decided over
the fact document the CLI's own producer emits. Aggregated into `main.rego`, and scoped
to `input.core.cli` so a satellite evaluation stays silent.
- `CliExitTaxonomyRuleHandler` — the native half, so `evolith validate` evaluates the rules
instead of reporting three blocking rules that never ran.
Observed red, end to end: planting `process.exit(7)` in a real CLI command file makes
`evolith validate --core .` emit a BLOCKING CLI-EXIT-01 naming the file, and the CLI exits 2
(BLOCKED). The same file drives the committed producer and the committed .rego to the same
verdict in `exit-code-taxonomy-ruleset.spec.ts`.
Derived artifacts moved with the corpus: native-handler 151 -> 154, ISO/IEC 5055 mapping and
the inventory summary regenerated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
release: align main — the CLI and MCP resolve the SDK that carries the security wave, 1.2.2 (GT-634)
A board row carries a STATUS but not who is working it or where, so two sessions can both read `PENDING` and both start, correctly. On 2026-07-30 that happened three times in a day. `50-validate-gap-claim` derives the claim set from OPEN PULL REQUESTS — every `GT-*` in a PR's title, body or branch name — and fails when one id is claimed by two of them, naming both with their branches and titles. DERIVED, NEVER HAND-WRITTEN, and the row asked for that on purpose: a hand-written claim goes stale in the direction that matters, because someone forgets to remove it and the next session works around a claim nobody holds. THREE CRITERIA OF FOUR, and the fourth is left open with its reason rather than stretched to fit. A claim list committed to the board would be derived from live GitHub state, so it would be stale the moment anyone opened a pull request — and GT-630's chain exists precisely to insist derived artifacts reach a fixed point. A perpetually-stale artifact inside that chain would be worse than none. Making the claim discoverable from the board needs something the board can render without committing, and that is not built. The row stays IN-PROGRESS. What it cannot see is in its own failure text, not implied: a branch with no open pull request claims nothing. Opening the PR early — draft is enough — is what makes the claim visible, which turns a convention into something with a check behind it. Anti-vacuous: a pull-request query that cannot be answered is a hard failure, not a quiet pass. A guard built because two sessions could not see each other must not stay silent when it cannot look. Zero open PRs is reported in words, because it looks identical to a broken query otherwise. Verified: 10/10 fixtures; 42-validate-guard-denominators 62 guards classified; 43-validate-guard-negative-fixtures 39/39 observed red, up from 38. Run against the real repository it reports 0 claims across 5 open PRs — checked rather than assumed: all five are Dependabot bumps naming no gap, so the zero is real and not an empty query. 08-validate-tracking (640 gaps), 01, 04, 41 within budget, 46 chain at a fixed point. Counters: 600/640, in-progress 14 -> 15, pending 22 -> 21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
feat(ci): one gap, one claim — the claim guard, advancing GT-639
`.git/EVOLITH_*` is a DIRECTORY only in the primary checkout; inside a `git worktree` it is a FILE holding a `gitdir:` pointer, so every write/read to `.git/EVOLITH_*` was ENOTDIR. In pre-commit that made `set -e` abort the commit outright; in pre-push the `rm -f` failed and TARGET stayed unread — together the reason `--no-verify` had become habitual from a worktree. `git rev-parse --git-dir` resolves both layouts; both hooks now go through it via a shared `write_push_intent`/read path, per-worktree, so the push intent belongs to the checkout that was committed in rather than the repo as a whole. Also in pre-push: `assert_checkoutable` refuses up front, before repo optimization or board sync run, when the merge target branch (main/develop) is already checked out in another worktree — `git checkout "$to"` would otherwise fail mid-dispatch with the push half-done. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fix(ci): pre-commit/pre-push hooks assumed .git is always a directory
Four conflicts, resolved by intent rather than by picking a side: - machine-channel.registry-sweep.spec.ts — another session had already landed this sweep on main AND improved it (e21e162 corrected the sweep's own invariant and found a fifth taxonomy mismatch). main's version is a strict superset, so it wins outright. Duplicated work, caught at the merge. - rule-corpus-triage.spec.ts — main refactored the counts to a SINGLE PINNED_CLASS_COUNTS declaration, deleting a second inline copy; GT-580's side edited the copy that no longer exists. Kept main's structure. Both sides agree on native-handler=154, but that is verified below, not assumed. - iso-5055-mapping.{csv,json} — generated artifacts. Regenerated rather than hand-merged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…negative half The code half landed in 086111d; the board never caught up, so the row still read IN-PROGRESS with criteria 1 and 3 unticked while the enforcement was already in `Test mcp-server`, a required check on `main`. Credited only after re-running the negative half rather than the colour: both `policy.wasm` artifacts (`src/rulesets/opa/` and `src/sdk/cli/rulesets/opa/`) were moved off disk and `abac-rego-parity.spec.ts` re-run — 11/11 green and both artifacts recompiled with fresh timestamps. That is what separates "the block no longer self-skips" from "it passed because the file happened to be there", which is the exact distinction GT-632 was registered for. Full mcp-server suite 452/452. Guards 08 (640 gaps, 583 closure records), 09 (reconciliation regenerated to match), 04 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
…ting status in prose A row carried its status in up to three places at once: the `Estado`/`Status` column, a bolded restatement inside the narrative, and — for 45 of 616 entries — a `**Status:**` field in the catalog. Reading a row meant reconciling them. Two mechanical changes, no content lost: `Qué significa` / `What it means` and `Ejemplo` / `Example` were empty in 599 of 614 rows (97.6%). They widened every row and pushed the status column out of view for the 2.4% that used them. Dropped; the 15 rows that did use them keep their text folded into the Gap cell behind bold labels. The narrative's status announcements are re-labelled by what they actually are — a dated record, not a second status field: `**COMPLETADO (x):**` -> `**Cierre (x):**`, `**EN-PROGRESO (x):**` -> `**Avance (x):**`, and the English twins to `**Closure**` / `**Progress**`. 71 replacements across 61 ES rows, 74 across 64 EN rows. Prose that discusses the vocabulary is untouched on purpose — "a row can sit IN-PROGRESS forever with no acceptance criteria" is an argument about the board, not a second status, and rewriting it would have destroyed the point. The one genuine restatement outside a bold label (GT-518 "Sigue EN-PROGRESO (...)") became "Sigue abierto (...)". Guard 08 finds its column by header name rather than by position, so removing columns does not touch it: 640 gaps, 616/616 catalog sections, 583 closure records, green. Bilingual parity green (1643 files), doc validation green (1491). Checked and NOT changed: the 45 catalog `**Status:**` fields contradict the board in zero cases, so they are redundant rather than wrong — left for a decision rather than deleted in a formatting commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
5 tasks
`Validate documentation` was red on every open PR in this stack with a single cause: guard 46 found the executive governance summary stale at link 5 of 5. The board edit (GT-602 closure + the column/status cleanup) is an input to that chain, and the derived artifact was never replayed — exactly the drift the GT-630 chain exists to catch, caught by it. Regenerated: 3 P0, 39 open, top risk Evolith Core. Guard 46 now reports the chain current AND at a fixed point (replaying all 5 links changes nothing), guard 08 green, bilingual parity green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
GT-580, GT-606 and GT-614 are owned by #305, now merged — this branch carried the same four integration merges because it was cut from a local This PR retains GT-602 only. Its closure is credited here on the negative half (both Merges last in the stack, deliberately: its residue after #305 is the board reformat — every one of the 645 rows, in both languages. Landing that into an empty queue costs nothing; landing it early forces #311 (which adds new rows) to re-express them in the 7-column shape. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes GT-602 (P0,
Evolith MCP) and carries the pending integration of GT-580, GT-606 and GT-614 that was sitting onintegration/gt-580-602-606-614with no PR.What GT-602 was
Fifteen of fifty MCP tools were FORBIDDEN in production, because
mcp-tool-dispatch.tsrequires native and OPA to allow, and the compiledpolicy.wasm— the artifact dispatch actually loads — denied them withABAC-03.Why it closes now
All three acceptance criteria hold:
abac-rego-parity.spec.tsevaluates the compiled bundle over every registered tool name and requires ALLOW for anarchitectinproduction.AbacEvaluator.toolProjection()(classified by callingclassifyTool, not by re-reading its maps) — 64 tools, zero losses.ABAC-03and turnsTest mcp-serverred — a required check onmain.The block used to be governed by
EVOLITH_REQUIRE_OPA_WASM: it ran in one CI job andit.skipped everywhere else. It now compiles the policy itself when the bundle is missing or older than any.regoit is built from, and fails with the compiler's own diagnostic if it cannot. Staleness is treated as absence, because a stale bundle is the defect GT-632 was.Verification
Credited on the negative half, not the colour: both
policy.wasmartifacts were moved off disk and the spec re-run — 11/11 green, both artifacts recompiled with fresh timestamps.npm --workspace @beyondnet/evolith-mcp test→ 452/45208-validate-tracking→ 640 gaps, 583 closure records, green09-reconcile-maturity --check→ matches (reconciliation regenerated)04-check-bilingual-parity→ greenBoard:
601 / 640 done · 14 in progress · 21 pending · 4 deferred.🤖 Generated with Claude Code