feat(010): Barrier B — oracle freeze, independent audit, and hard-gate checkpoint (T013–T024) - #88
Merged
Merged
Conversation
Barrier B, tasks T013 and T014 of feature 010 (specs/010-catalog-backstage). Barrier side: IS THE BARRIER. T013 creates the git-tracked evidence tree. Tracking is not incidental: research.md R5 mechanism 2 requires CI to re-derive the freeze hashes, which it can only do for files it can see, and ADR-0015 Condition of Acceptance 1 identifies spike 009's untracked scratch bundle as the hazard being corrected. T014 records the accept-corpus selection basis and size BEFORE the rule is applied, per ADR-0020 clause 5's requirement that both be "fixed and recorded in that same cycle, not chosen afterwards". This commit deliberately contains the rule and NOTHING that results from it. selection-basis.md here names no entity, no canonical id and no sourcePath. The enumeration lands in a separate follow-up commit that does not modify sections 1-6, so the ordering clause 5 requires is checkable in git history rather than merely asserted in prose. See selection-basis.md section 7. The rule fixed here: - corpus: github.com/backstage/community-plugins @ 92e9e4e0 - predicate P: exact catalog-info.yaml basename; parses with uniqueKeys; entity-shaped; admissible under ADR-0015's four pinned validators; canonical id unique corpus-wide, with every member of a colliding group excluded (never first-wins/last-wins, per entity-identity.md section 3) - selection S: one entity per workspace, compareCodeUnits order, first 24 eligible workspaces - size: 24 -- a maintainer judgement, explicitly NOT a minimum and NOT a production limit (FR-055, ADR-0012). Adequacy is T019's finding to make. - overlay: maintainer-authored, assigned by position from an 11-entry hand-authored pattern catalogue, exercising all three ownership states and eleven independent no-exclusive-winner pairs R14's known-failing populations are handled explicitly, and each figure was re-derived from the pin rather than copied: 156 files / 167 entity documents / 23 annotated / 0 carrying adrkit.io/owned-paths / 5 skeletons / 7 invalid metadata.name (5 character class, 2 length alone). All reproduced exactly. The 5 placeholder descriptors are excluded on INADMISSIBILITY, not on collision -- they never acquire a canonical id at all (admissibility.md section 4.1), and FR-020 fixes inadmissibility as the earlier and more specific defect. The Nexus pair is excluded as the residual valid duplicate, both documents and not one. No generator was run. No generator-derived output exists. No package exists. ADR-0014 rung 1 only; no release is scheduled, prepared or implied. Refs: ADR-0020 clauses 5 and 6, ADR-0012 gate 3, ADR-0015, ADR-0016 Discharges: FR-055 (T014); T013 enables FR-053, FR-054, FR-055 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Barrier B, tasks T015 through T018 of feature 010, applying the selection rule fixed at 654031b without modifying it. Barrier side: IS THE BARRIER. This is the fresh T014 step of ADR-0020 clause 6's T014 -> T014a cycle. The T014a step is T019 and is DELIBERATELY NOT PART OF THIS COMMIT: clause 5(a) requires a reviewer with no authoring involvement in T014-T018, and an audit by the author of the freeze would defeat the control rather than implement it. The correction the cycle exists to make: spike 009 froze derivedPathPatterns in INPUT order. This oracle records it in compareCodeUnits-sorted order, per data-model.md section 16 and owned-paths-annotation.md section 3's requirement that an explicit-paths derivation be sorted and deduplicated. The two orders disagree at the very first element here, so the distinction is visible without reading to the tail. What landed: - T015 overlay.json -- 23 maintainer-authored adrkit.io/owned-paths values over real upstream descriptors left otherwise unmodified. Zero descriptors in the pin carry this annotation; R14 records that and this freeze re-derived it. Assignment is positional (C[i mod 11]), never per-entity, so no entity was chosen for the annotation it would receive -- and so the overlay cannot be mistaken for the descriptor-parent inference FR-061 forbids the adapter. - T016 expected-paths.json -- 24 hand-derived entries. Sort-and-dedupe applied by hand from the frozen contracts, then checked against compareCodeUnits; that arithmetic check is disclosed in the artifact rather than done silently. All eleven arrays and the 25-pattern union matched on the first check. - T017 frozen-expectation-set.json -- 25 derivedPathPatterns, sorted; 24 expectedByEntity records carrying ownershipState, because owned-paths-annotation.md section 3's non-conflation rule requires the discriminator to be explicit: explicit-empty and annotation-absent both yield an empty array and must stay distinguishable. - T018 accept-corpus-freeze.json -- corpusRef, selectionBasis, size, overlay, expectedPaths, contentHash, assembled in the same cycle. - selection-basis.md gains section 8 only. 124 insertions, 0 deletions: the rule in sections 1-7 is provably untouched, which is the ordering evidence clause 5's "not chosen afterwards" actually requires. Corpus: github.com/backstage/community-plugins @ 92e9e4e0. 167 entity documents; 7 fail admissibility; 2 fail canonical-id uniqueness; 158 satisfy P; 79 eligible workspaces; 24 selected, one per workspace, 24 distinct ids. Recorded content hashes: frozen-expectation-set.json e641ae5e4201a099e92e98fbaa7683bc0eb0290adb01f93abbd474c695c2430c accept-corpus-freeze.json f98e6d464b53ba334298c1e5b76bbd0222ff2a460f7637439f0e8d54c9294aca Both re-derived with an independently written serializer before commit. T019 must recompute them from the artifacts, never copy them -- data-model.md section 16: "An audit that transcribes the author's declared hash has verified nothing." Barrier state at this commit: no generator was created, built or run; no SnapshotEnvelope exists; no derived-ownership result exists, persisted, in memory or asserted in a test (research.md R4); no input manifest exists anywhere in the tree and no evidence artifact carries InputManifest keys; no comparison harness exists anywhere. packages/adapters holds only spec-kit. Nothing derived ownership at any point. The corpus work evaluated ADR-0015 admissibility and entity-identity canonicalization only -- both placed before Barrier B by plan.md, and both required BY clause 5 itself, which demands a corpus that is admissible and free of duplicate canonical ids. Verified: bun test 857 pass / 0 fail; bun run typecheck clean; bun run adr lint 20 records 0 errors 0 warnings; bun run check:deps ok. Plus 17 cross-artifact consistency checks (orderings, dedupe, oracle/freeze agreement, glob-dialect validity of all 25 patterns, no manifest shape). NOT claimed and not done: T019 audit, so SC-010 is NOT satisfied and no part of it is reported as satisfied. T020/T021/T023 observed-failing runs, so under ADR-0016 nothing here counts as coverage yet. Barrier B has NOT cleared; T024 is unchecked. ADR-0014 rung 1 only -- not reference-verified, not externally validated, no release scheduled, prepared or implied. Only the corpus data is third-party; the overlay, the expectations and the selection are the maintainer's own. This freeze fixes what output is expected to be; it is not correctness evidence, which is FR-056 / SC-011's separate job in Phase F. Refs: ADR-0020 clauses 5 and 6, ADR-0012 gate 3, ADR-0015, ADR-0016 Discharges: FR-054 (overlay half, expected-paths half, same-cycle freeze), FR-053 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Independent auditor (no authoring involvement in T014-T018) records the T014a step of the ADR-0020 clause 6 oracle cycle. Verdict: PASS. - Recomputed both content hashes from the artifact bytes with a from-scratch README §3 serializer (Node + Bun cross-checked); both match: frozen-expectation-set e641ae5e…c695c2430c, accept-corpus-freeze f98e6d46…c9294aca. Freeze is byte-stable. - Confirmed derivedPathPatterns is in compareCodeUnits order, not input order (spike-009 defect absent); localeCompare gives a different, inadmissible order; re-derived the sequence as the dedup union of all 24 entities' expectedPaths. - Explicit ADEQUACY finding: size 24 is ADEQUATE for the technical- compatibility claim it gates (all 3 ownership states; every glob/dedup trap with >=2 independent instances; 11 overlap instances; deterministic pre-committed selection rule; hand-verifiable). No minimum count invented. - Verified pre-commitment (654031b rule / 4fff13d +124/-0 §8), funnel (167-7-2=158, 24<=79, name-split preserved), exclusion reasoning (placeholders inadmissible, Nexus pair residual valid duplicate), and the no-generator-run barrier by inspecting the tree. Bounded limitation: pinned Backstage corpus not checked out, so funnel figures verified for consistency, not reproduced against ground truth. Writes only the two audit sibling files (hashes unchanged) and the T019 checkbox. No audited artifact modified. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Encode the T019 oracle-freeze audit as an executable procedure and attack it with two ADR-0016 observed-failing negative cases, build R5 mechanism 2 (the CI freeze-hash drift check) with its own observed-failing case, and record the Barrier B hard-gate checkpoint confirming all three R5 mechanisms by inspection. - T020: retained input-order oracle variant (hash recomputed so ordering is the sole defect); real audit observed FAIL [ordering], restore observed PASS. - T021: retained integrity-only audit case; real audit observed FAIL [adequacy] against SC-010 rather than a silent accept, restore observed PASS. - T022: scripts/check-freeze-hashes.ts re-derives each frozen artifact's canonical hash and fails on drift; wired into ci.yml + check:freeze-hashes script. - T023: single-byte mutation observed FAIL (freeze drift) on an isolated copy, restore observed PASS; retained at evidence/negative-cases/freeze-drift/. - T024: BARRIER_B_CLEARED = true. (1) input absence — no input-manifest instance in feature-010's tree; adapter/consumer on origin/main are Phase A placeholders with no discovery, proven by observed-firing no-dynamic-loader guards. (2) hash match — drift check green over both frozen trees. (3) ordering — no comparison harness in this worktree, on origin/main, on any branch, or in scratch. Cross- tree inspection of origin/main (99ba8d2) disclosed in the record. Gates: bun test 866 pass / 0 fail, typecheck clean, adr lint 0/0. Frozen artifacts unmodified; deliberate variants retained as permanent negatives. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
One conflict, in tasks.md T013, resolved by taking main's reconciled task text together with this branch's completed checkbox. Both sides were right about different things: main had gained the reconciliation's correction (T013 must also create negative-cases/, and that tree is shared across phases A-G with per-task subdirectories, which is what makes it safe for concurrent worktrees), while this branch had genuinely completed the task. Verified after resolution: 100 contiguous tasks, 66 checked, T013-T024 all present, and the only remaining unchecked are T058/T062 (the five-steps three-reasons defect, now corrected in spec but not yet re-run) and T069-T100 (phases E, F, G). Gates on the merged tree: bun test 1504 pass / 0 fail, typecheck clean, check:deps ok, check:freeze-hashes ok over both frozen trees, adr lint 20 records 0/0. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Implements Barrier B by freezing and auditing the feature-010 oracle before generator output can exist.
Changes:
- Adds frozen corpus expectations, independent audit evidence, and negative cases.
- Adds executable audit and hash-drift checks with tests and CI wiring.
- Records T013–T024 as complete and marks Barrier B cleared.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
specs/010-catalog-backstage/tasks.md |
Marks Barrier B tasks complete. |
specs/010-catalog-backstage/evidence/README.md |
Documents evidence structure and hashing. |
specs/010-catalog-backstage/evidence/barrier-b-checkpoint.json |
Records the hard-gate checkpoint. |
specs/010-catalog-backstage/evidence/accept-corpus-freeze/README.md |
Documents the corpus freeze. |
specs/010-catalog-backstage/evidence/accept-corpus-freeze/selection-basis.md |
Defines and applies corpus selection. |
specs/010-catalog-backstage/evidence/accept-corpus-freeze/overlay.json |
Adds the ownership overlay. |
specs/010-catalog-backstage/evidence/accept-corpus-freeze/expected-paths.json |
Records expected paths. |
specs/010-catalog-backstage/evidence/accept-corpus-freeze/accept-corpus-freeze.json |
Assembles the frozen corpus artifact. |
specs/010-catalog-backstage/evidence/accept-corpus-freeze/adequacy-audit.json |
Records the adequacy audit. |
specs/010-catalog-backstage/evidence/frozen-expectations/README.md |
Documents the re-frozen oracle. |
specs/010-catalog-backstage/evidence/frozen-expectations/frozen-expectation-set.json |
Adds corrected expectations. |
specs/010-catalog-backstage/evidence/frozen-expectations/audit-record.json |
Records the independent oracle audit. |
specs/010-catalog-backstage/evidence/negative-cases/oracle-input-order/README.md |
Documents the ordering failure case. |
specs/010-catalog-backstage/evidence/negative-cases/oracle-input-order/frozen-expectations/frozen-expectation-set.json |
Adds an input-order oracle variant. |
specs/010-catalog-backstage/evidence/negative-cases/oracle-input-order/accept-corpus-freeze/adequacy-audit.json |
Supplies copied adequacy evidence. |
specs/010-catalog-backstage/evidence/negative-cases/oracle-input-order/accept-corpus-freeze/accept-corpus-freeze.json |
Supplies the fixture corpus freeze. |
specs/010-catalog-backstage/evidence/negative-cases/audit-integrity-only/README.md |
Documents the missing-adequacy case. |
specs/010-catalog-backstage/evidence/negative-cases/audit-integrity-only/frozen-expectations/frozen-expectation-set.json |
Supplies intact oracle evidence. |
specs/010-catalog-backstage/evidence/negative-cases/audit-integrity-only/accept-corpus-freeze/adequacy-audit.json |
Omits adequacy intentionally. |
specs/010-catalog-backstage/evidence/negative-cases/audit-integrity-only/accept-corpus-freeze/accept-corpus-freeze.json |
Supplies intact corpus evidence. |
specs/010-catalog-backstage/evidence/negative-cases/freeze-drift/README.md |
Documents the drift observation. |
specs/010-catalog-backstage/evidence/negative-cases/freeze-drift/observed-fail.json |
Records the observed hash failure. |
scripts/audit-oracle-freeze.ts |
Implements the executable audit. |
scripts/audit-oracle-freeze.test.ts |
Tests passing and failing audits. |
scripts/check-freeze-hashes.ts |
Implements freeze-drift checking. |
scripts/check-freeze-hashes.test.ts |
Tests drift detection and restoration. |
package.json |
Adds the hash-check command. |
.github/workflows/ci.yml |
Runs the hash check in CI. |
Suppressed comments (1)
scripts/check-freeze-hashes.ts:75
- This skip means T022 does not hash “everything under” the two directories as required by
tasks.md:361-363. The audit records, selection basis, overlay, expected paths, and READMEs can all drift while this check remains green; in particular, the independent PASS/adequacy evidence is unprotected. Use an external hash manifest (to avoid self-reference) that enumerates every tracked file in both trees and fail on additions, removals, or byte changes.
// Only artifacts that carry a recorded contentHash are in scope. Sibling audit
// records (no contentHash) are intentionally skipped, not failed.
if (typeof obj.contentHash !== 'string') continue;
Comment on lines
+100
to
+103
| // 3. adequacy — an explicit finding is mandatory (ADR-0020 clause 5(a) / SC-010). | ||
| if (input.adequacy?.finding !== 'adequate' && input.adequacy?.finding !== 'inadequate') { | ||
| findings.push({ check: 'adequacy', reason: REASON_NO_ADEQUACY }); | ||
| } |
| }, | ||
| "reasoningJudged": { | ||
| "upperBound": "Reviewability. Expected paths are hand-authored so they can be checked line-by-line by a reviewer; a materially larger corpus would force code-derived expectations, which clause 6 constrains. This auditor accepts the upper bound as sound for a hand-verifiable oracle.", | ||
| "lowerBound": "Coverage-with-repetition: every ownership state and every catalogued glob/dedup feature exercised with >=2 independent instances so that a single transcription slip cannot silently pass. This auditor independently confirmed the >=2 repetition holds for all 11 catalogue entries and for each ordering trap. The lower bound is met by the data, not merely asserted.", |
Comment on lines
+40
to
+45
| let entries: import('node:fs').Dirent[]; | ||
| try { | ||
| entries = await readdir(dir, { withFileTypes: true }); | ||
| } catch { | ||
| return []; | ||
| } |
Comment on lines
+10
to
+12
| "requirement3_adequacyFinding": { | ||
| "mandate": "ADR-0020 clause 5(a) requires an explicit adequacy judgement on the accept corpus; an integrity/hash confirmation alone does not satisfy it. The authoring session fixed size = 24 (one entity per eligible workspace, first 24 of 79 eligible workspaces in compareCodeUnits order) and stated its reasoning. This auditor judges that reasoning.", | ||
| "claimTheCorpusGates": "Technical compatibility only (ADR-0020 clause 5 step (a)): that the ownership-derivation contract can be exercised end-to-end against real Backstage descriptor structure and that the frozen oracle's ordering and dedup behaviour are the contract-admissible ones. The freeze explicitly does NOT claim scale ratification (ADR-0012 / FR-055), correctness of the generator, SC-010 satisfaction, or adoption. The adequacy judgement is scoped to that narrow claim.", |
| "limitationsDisclosed": [ | ||
| "The adapter and consumer packages (packages/adapters/catalog-backstage, packages/catalog-envelope) do not exist in THIS worktree (branched before Phase A merged); they were inspected on origin/main via git show/git ls-tree at 99ba8d2. This cross-tree inspection is stated explicitly rather than implied.", | ||
| "input-manifest.md §5 (cited by T024 mechanism 1) is a spike-009 contract document, not a feature-010 artifact; the mechanism-1 conclusion rests on the absence of any input-manifest INSTANCE in feature-010's tree plus the adapter placeholder's absence of discovery, which is the substance the clause is guarding.", | ||
| "Mechanism (2)'s 'green in CI' is confirmed by the wired step running green locally; the CI run itself will execute on push. The check logic and its wiring are both present and were observed both passing and failing." |
Comment on lines
+10
to
+12
| "requirement3_adequacyFinding": { | ||
| "mandate": "ADR-0020 clause 5(a) requires an explicit adequacy judgement on the accept corpus; an integrity/hash confirmation alone does not satisfy it. The authoring session fixed size = 24 (one entity per eligible workspace, first 24 of 79 eligible workspaces in compareCodeUnits order) and stated its reasoning. This auditor judges that reasoning.", | ||
| "claimTheCorpusGates": "Technical compatibility only (ADR-0020 clause 5 step (a)): that the ownership-derivation contract can be exercised end-to-end against real Backstage descriptor structure and that the frozen oracle's ordering and dedup behaviour are the contract-admissible ones. The freeze explicitly does NOT claim scale ratification (ADR-0012 / FR-055), correctness of the generator, SC-010 satisfaction, or adoption. The adequacy judgement is scoped to that narrow claim.", |
| }, | ||
| "reasoningJudged": { | ||
| "upperBound": "Reviewability. Expected paths are hand-authored so they can be checked line-by-line by a reviewer; a materially larger corpus would force code-derived expectations, which clause 6 constrains. This auditor accepts the upper bound as sound for a hand-verifiable oracle.", | ||
| "lowerBound": "Coverage-with-repetition: every ownership state and every catalogued glob/dedup feature exercised with >=2 independent instances so that a single transcription slip cannot silently pass. This auditor independently confirmed the >=2 repetition holds for all 11 catalogue entries and for each ordering trap. The lower bound is met by the data, not merely asserted.", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Barrier B stack: the fresh T014 → T014a oracle cycle, its independent audit, the ADR-0016 observations against the audit machinery, R5 mechanism 2, and the hard-gate checkpoint. Four commits, T013–T024.
ADR-0020 clause 6 and clause 5(a) require this to complete before any generator-derived output exists. It is the anti-backfilling control, and Phases E, F and G are graph-blocked on T024 — all 32 of their tasks name it in
Depends.Why this stack exists
Spike 009's oracle carried a known-wrong
derivedPathPatternsordering (input order rather thancompareCodeUnits-sorted), recorded as a carry-forward blocker in its own evidence index. ADR-0015's Condition of Acceptance 1 requires a fresh cycle before any new generator output. This is that cycle.The freeze (T013–T018)
Accept corpus:
github.com/backstage/community-plugins @ 92e9e4e0. Funnel: 167 entity documents → 7 inadmissible → 2 colliding → 158 satisfy P → 79 eligible workspaces → 24 selected, 24 distinct canonical ids.The 5 placeholder descriptors are excluded on inadmissibility, not collision — they never acquire a canonical id (
admissibility.md§4.1). The Nexus pair is excluded as the residual valid duplicate, both documents, leaving that workspace ineligible. Two mechanisms, correctly distinguished.The pre-commitment is structural, not asserted. Commit 1 contains only the selection rule and names no selected entity; commit 2 adds §8 with 124 insertions, 0 deletions — so §1–§7 are provably untouched after the entities were known. Verified by
git diff --numstat.The correction:
derivedPathPatternsis 25 patterns incompareCodeUnitsorder. It disagrees with input order at the first element (**vsworkspaces/alpha-tools/**), so a wrong variant is visibly wrong rather than subtly so. The overlay deliberately encodes the traps —-<.</,*before letters, uppercase before lowercase,v10beforev2— every one of which a locale-aware sort gets wrong.The independent audit (T019)
Performed by a session with no authoring involvement in T014–T018. Spike 009's equivalent audit used a non-compliant reviewer, was caught in PR review, and had to be redone; this one is scoped to prevent a recurrence.
compareCodeUnits' actual semantics, withlocaleCompareconfirmed to give a different answer, so the comparator choice is load-bearing.The observations (T020–T023)
The audit was encoded as an executable (
scripts/audit-oracle-freeze.ts) so its FAIL behaviour is genuinely observed rather than asserted.FAIL [ordering], restore → PASS.FAIL [adequacy]against SC-010 rather than silently accepted. This is the exact failure mode the adequacy finding exists to prevent, demonstrated rather than described.scripts/check-freeze-hashes.ts, R5 mechanism 2, wired intoci.yml. (This is the commit that requiredworkflowscope to push.)Barrier B checkpoint (T024) —
BARRIER_B_CLEARED = trueAll three R5 mechanisms confirmed simultaneously and by inspection, including cross-tree inspection of
origin/main(Phase A merged after this branch was cut):input-manifest.md§5 already forbids discovery, so the only route to a corpus is a manifest naming files explicitly: absent manifest, absent corpus, absent output.origin/main, on any branch, or in scratch.Independent corroboration by the coordinating session
Not merely reviewed — re-derived:
docs/**→docz/**,recomputed=8dd1f152…), then passing on restore — so R5 mechanism 2 is observed working by a second party on a mutation its author did not select.origin/mainand every other worktree found no comparison harness, noSnapshotEnvelope, and no input-manifest instance.Checks
bun test866 pass / 0 fail ·typecheckclean ·check:deps→core-has-no-adapter-deps: ok·adr lint20 records, 0 errors.Standing
ADR-0014 rung 1 only. This PASS does not clear ADR-0012 gate 3 as an external claim, does not assert rung 2 or rung 3, does not assert SC-010 beyond the technical-compatibility step this freeze gates, and does not ratify 24 as a production sample size. Gate 4 remains unmet and not yet testable. No release scheduled, implied, or prepared.