GT-597: a posture score that can go red, an SLSA target with an honest gap, and an SSDF map of controls that exist - #313
Merged
Conversation
release: align main — the CLI and MCP resolve the SDK that carries the security wave, 1.2.2 (GT-634)
A board row carries a STATUS but not who is working it or where, so two sessions can both read `PENDING` and both start, correctly. On 2026-07-30 that happened three times in a day. `50-validate-gap-claim` derives the claim set from OPEN PULL REQUESTS — every `GT-*` in a PR's title, body or branch name — and fails when one id is claimed by two of them, naming both with their branches and titles. DERIVED, NEVER HAND-WRITTEN, and the row asked for that on purpose: a hand-written claim goes stale in the direction that matters, because someone forgets to remove it and the next session works around a claim nobody holds. THREE CRITERIA OF FOUR, and the fourth is left open with its reason rather than stretched to fit. A claim list committed to the board would be derived from live GitHub state, so it would be stale the moment anyone opened a pull request — and GT-630's chain exists precisely to insist derived artifacts reach a fixed point. A perpetually-stale artifact inside that chain would be worse than none. Making the claim discoverable from the board needs something the board can render without committing, and that is not built. The row stays IN-PROGRESS. What it cannot see is in its own failure text, not implied: a branch with no open pull request claims nothing. Opening the PR early — draft is enough — is what makes the claim visible, which turns a convention into something with a check behind it. Anti-vacuous: a pull-request query that cannot be answered is a hard failure, not a quiet pass. A guard built because two sessions could not see each other must not stay silent when it cannot look. Zero open PRs is reported in words, because it looks identical to a broken query otherwise. Verified: 10/10 fixtures; 42-validate-guard-denominators 62 guards classified; 43-validate-guard-negative-fixtures 39/39 observed red, up from 38. Run against the real repository it reports 0 claims across 5 open PRs — checked rather than assumed: all five are Dependabot bumps naming no gap, so the zero is real and not an empty query. 08-validate-tracking (640 gaps), 01, 04, 41 within budget, 46 chain at a fixed point. Counters: 600/640, in-progress 14 -> 15, pending 22 -> 21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
feat(ci): one gap, one claim — the claim guard, advancing GT-639
`.git/EVOLITH_*` is a DIRECTORY only in the primary checkout; inside a `git worktree` it is a FILE holding a `gitdir:` pointer, so every write/read to `.git/EVOLITH_*` was ENOTDIR. In pre-commit that made `set -e` abort the commit outright; in pre-push the `rm -f` failed and TARGET stayed unread — together the reason `--no-verify` had become habitual from a worktree. `git rev-parse --git-dir` resolves both layouts; both hooks now go through it via a shared `write_push_intent`/read path, per-worktree, so the push intent belongs to the checkout that was committed in rather than the repo as a whole. Also in pre-push: `assert_checkoutable` refuses up front, before repo optimization or board sync run, when the merge target branch (main/develop) is already checked out in another worktree — `git checkout "$to"` would otherwise fail mid-dispatch with the push half-done. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fix(ci): pre-commit/pre-push hooks assumed .git is always a directory
…report
The Scorecard workflow landed in a previous wave and publishes a number.
Nothing in that path can fail: a score falling from 7.1 to 4.2 produces an
identical green run, which is the audit failure mode this row exists to remove,
reproduced weekly with better ergonomics. This adds the half that goes red.
- 51-validate-scorecard-regression.mjs compares each run against floors
committed in .harness/security/scorecard-baseline.json and exits non-zero on
a dropped aggregate, a dropped check, a check that stopped running, a check
scored -1, a check with no floor, or a payload it cannot read. An improvement
never fails; it prints a ratchet suggestion, because a baseline that updates
itself cannot detect anything. Observed red on all six failure shapes and
green on an unchanged run; 42 and 43 both accept it, and 43 has now seen it
refuse the empty fixture.
- openssf-scorecard.yml gains a JSON pass (the aggregate score is not in the
SARIF) and runs the gate last, so evidence still uploads on the run that most
needs it. A red scheduled job on the default branch mails the owner — that is
the visibility mechanism, not a dashboard someone has to remember.
- The baseline ships UNSEEDED on purpose. No Scorecard run has ever executed
here and the public OpenSSF API 404s for this repository, so invented floors
would look exactly like a measurement without being one. The gate fails closed
and prints the exact document to commit; seeding is owner action.
- supply-chain-posture.{md,es.md} declares the SLSA target (Build L3, SLSA v1.2
Build track) and records the gap honestly: 7 of 8 packages now carry npm
provenance — the audit's "0 of 8" is stale — evolith-contracts does not and
only the owner can fix that, pre-npm-release.yml versions never will, and
Build L3 needs the reusable-workflow isolation that belongs to GT-570.
No level is claimed as verified.
- The SSDF v1.1 mapping is 20 rows covering 23 practice IDs read from
SP 800-218's own table, each row carrying the command that verified the
control. It includes NOT IMPLEMENTED rows on purpose: the SBOM is generated
and discarded, and dependabot_security_updates is disabled. A mapping where
everything is green is a mapping nobody checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nch collision `feat/gt-587-semconv-genai` (PR #308) already claims `51-` for `51-validate-semconv-pin.mjs`. Two guards sharing a prefix is the GT-638 id-collision pattern one level down: neither branch conflicts in git, because the filenames differ, so the clash would have landed silently and only shown up as a broken numbering convention afterwards. The older claim keeps the number. Re-verified after the rename: self-test 10/10, guard 42 "every one of the 63 CI guard(s) is classified", guard 43 "all 40 exercised guard(s) refused the empty fixture", bilingual parity green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
To create skeletons: node .harness/scripts/generate-es-skeleton.mjs <file.md>Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes GT-597 (P1, complexity S). All three criteria met, with one step that is the owner's by construction.
Criterion 1 — the Scorecard workflow existed but could not fail
A previous wave (
6bb43cfa, already onmain) landed.github/workflows/openssf-scorecard.yml. It publishes but cannot go red — exactly the failure mode the row warns about, a scheduled job nobody looks at. This adds the half that fails:52-validate-scorecard-regression.mjscompares each run against floors in.harness/security/scorecard-baseline.json.The workflow now runs Scorecard a second time with
results_format: json, because the aggregate score is not in the SARIF — deriving it from SARIF would have meant reading a field that does not exist and reporting green. The gate runs last, so SARIF upload and public publication still happen on the run that most needs them.Observed failing:
Also red on: no
--results, an unseeded baseline, and a SARIF file handed to it by mistake ("Refusing to report a pass over a payload this guard cannot read"). Green on an unchanged run. Meta-guards agree:42— every one of the 63 CI guards classified;43— all 40 exercised guards refused the empty fixture; self-test 10/10. Wired intodocs.yml;ci-cd.yml/docker-images.ymluntouched.How regression is actually visible: the scheduled job turns red on the default branch, which fires GitHub's own scheduled-workflow failure email. That is the load-bearing mechanism — not a dashboard nobody opens.
Deliberately not done: the baseline ships unseeded (
aggregate: null), so the gate stays red until a real run seeds it. No Scorecard run has ever executed here (gh run listempty; the public API returns 404). Inventing floors would produce a file indistinguishable from a measurement. Dispatching the workflow publishes a score to a public API — a publication decision, and the owner's. The gate prints the exact JSON to commit.Criterion 2 — target declared, gap recorded, nothing implied
Target: Build L3, SLSA v1.2 Build track (checked against slsa.dev: v1.2 is current and supersedes v1.1; levels are Build L0–L3). Current position is stated as a claim pending verification, not an achievement: Sigstore-signed provenance on a hosted runner is the Build L2 shape, npm asserts no level, and no independent verification was performed.
The audit's headline number is now stale, and this was checked live rather than repeated: "0 of 8 packages carry
dist.attestations" → 7 of 8 now carryhttps://slsa.dev/provenance/v1onlatest. Recorded gap:@beyondnet/evolith-contracts@1.1.0has none (owner only — needsNPM_TOKEN, neither used nor requested); every pre-npm-release.ymlversion can never be retro-attested (permanent); Build L3 needs reusable-workflow isolation, left to GT-570 on purpose; plus no automated attestation check and no per-version inventory. Also recorded and owner-gated:dependabot_security_updates: disabled,required_signatures: false,require_code_owner_reviews: false.Criterion 3 — 20 rows, 23 task IDs, every control verified to exist
Taken from SP 800-218 v1.1's own practice table, extracted from the NIST PDF rather than quoted from memory — note it retired
PW.3.2/PW.4.5/PW.5.2, so nothing maps to them. Every row carries the command that verified its control (gh apiagainst live branch protection / repo settings / rulesets, or agrepat a named file:line).Two findings that changed the mapping:
enforce_adminsis now true,developis protected, andCodeQL SASTis required — all four audit findings self-healed since 2026-07-26, with nothing measuring that they stay healed. That is this row's thesis, confirmed by accident.required_pull_request_reviewsis absent from the protection summary payload, but the dedicated endpoint returnsrequired_approving_review_count: 1. Trusting the first call would have recorded "review not required" — a wrong finding.NOT IMPLEMENTED rows are included on purpose: the CycloneDX SBOM is generated at
sdk-cli-release.yml:129and never uploaded (grep -rn "sbom" .github/workflows/returns exactly that one line), and Dependabot security updates are disabled.Renumbered 51 → 52 during integration
feat/gt-587-semconv-genai(PR #308) already claims51-for51-validate-semconv-pin.mjs. Two guards sharing a prefix is the GT-638 id-collision pattern one level down: the filenames differ, so neither branch conflicts in git and the clash would have landed silently. The older claim keeps the number. Re-verified after the rename — self-test 10/10, guards 42 and 43 green, bilingual parity green (1493 files).Deliberately left out
npm-release.yml/sdk-cli-release.ymluntouched — the reusable-workflow refactor for Build L3 and an attestation-presence check both belong to GT-570, and editing that pipeline from here is how this repo has previously duplicated work. No workflow was dispatched, no repository setting changed.SECURITY.md's Supported Versions table still says1.1.xwhilelatestis1.2.2— a stale row left alone, because changing a support policy is an owner decision.One assumption not verifiable by observation: the gate parses Scorecard's documented JSON v2 shape read from
pkg/scorecard/json.goupstream, not from a real run of this repo. It fails closed on any unrecognised payload, and both the script header and the doc say so plainly.🤖 Generated with Claude Code