From 9a34f8dadee9c26021ba608b8aa78857c2e29b64 Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Tue, 21 Jul 2026 18:09:05 -0700 Subject: [PATCH] =?UTF-8?q?Retire=20self-generated=20id=E2=86=94label=20va?= =?UTF-8?q?lidator=20sha256=20pin=20(Phase=202=20step=202d)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `vendored-sync` CI job (scripts/check_vendored_sync.sh, shared-reference diff against CultureBotAI/CultureMech@<.vendored_canon_ref>, merged in #176) has replaced the self-generated sha256 pin, which could only compare a copy to a hash from the same repo — so all four repos could pass verify-validator-pin while diverged. Remove the redundant second mechanism: - delete verify-/refresh-validator-pin recipes + VENDORED_IDLABEL_FILES manifest - delete scripts/.validate_id_label_correspondence.sha256 - fix a stale "validator-pin job below" CI comment + reword the vendored-sync guard comment (no longer a self-pinned sha256) - refresh NEXT_TASKS + the next-tasks skill note (keep the out-flow caveat: TraitMech has originated cross-repo validator fixes before) schema-pin is a separate vendored set with no drift-check replacement — kept. Co-Authored-By: Claude Opus 4.8 --- .claude/skills/next-tasks/SKILL.md | 17 +++++---- .github/workflows/label-correspondence.yaml | 7 ++-- NEXT_TASKS.md | 23 +++++++----- justfile | 36 +++++-------------- .../.validate_id_label_correspondence.sha256 | 5 --- 5 files changed, 37 insertions(+), 51 deletions(-) delete mode 100644 scripts/.validate_id_label_correspondence.sha256 diff --git a/.claude/skills/next-tasks/SKILL.md b/.claude/skills/next-tasks/SKILL.md index f3c748f2..b3437333 100644 --- a/.claude/skills/next-tasks/SKILL.md +++ b/.claude/skills/next-tasks/SKILL.md @@ -105,13 +105,16 @@ citations validate, verify it locally. ## Notes & limitations -- `scripts/validate_id_label_correspondence.py` and `scripts/chem_formula.py` are - vendored **byte-identical** across the four Mech repos behind a sha256 pin. - `just verify-validator-pin` only checks *this* repo's copy against its own pin - — it passes even when the four copies have diverged. Compare md5s across repos - before assuming they're in sync. TraitMech has originated cross-repo fixes - before (the `id(adapter)` cache-reuse fix), so check which direction a sync - should flow rather than assuming this copy is behind. +- `scripts/validate_id_label_correspondence.py`, `scripts/chem_formula.py`, and the + three `tests/test_id_label_*.py` files are vendored **byte-identical** across the + four Mech repos. The old self-generated sha256 pin was retired (it compared a copy + to a hash from the *same* repo, so all four could pass while diverged). Drift is now + caught by the `vendored-sync` CI job (`scripts/check_vendored_sync.sh`), which diffs + these files against `CultureBotAI/CultureMech@`. To + propagate a change: PR into that hub → merge → bump `.vendored_canon_ref` here. + TraitMech has originated cross-repo fixes before (the `id(adapter)` cache-reuse + fix), so a change may need to flow *out* to the hub, not just in — land it in + CultureMech first, then re-pin the ref everywhere. - `.claude/` is gitignored here — a new skill or command needs `git add -f`. - Without `gh` or a network, reconcile from `git log` alone and say so. diff --git a/.github/workflows/label-correspondence.yaml b/.github/workflows/label-correspondence.yaml index 933b1e9f..b514508c 100644 --- a/.github/workflows/label-correspondence.yaml +++ b/.github/workflows/label-correspondence.yaml @@ -9,8 +9,9 @@ name: label-correspondence # `exceptions:` allow-list in conf/id_label_targets.yaml. The drift report is # still generated and uploaded as a triage artifact even when the gate passes. # -# The validator-pin job below is also blocking: it is fast and dependency-free, -# and guards the cross-repo byte-identical vendoring of the validator + tests. +# The vendored-sync job below is also blocking: it is fast and dependency-free, +# and guards the cross-repo byte-identical vendoring of the validator + tests via +# a shared-reference diff against the canonical hub (not a self-generated pin). on: pull_request: @@ -29,7 +30,7 @@ permissions: jobs: # Cross-repo durability guard: the validator is vendored byte-identical across - # the Mech repos, so fail fast if this copy drifts from its pinned sha256. + # the Mech repos, so fail fast if this copy drifts from the canonical hub's copy. # Fast and dependency-free (no uv / OAK), so it blocks even though the drift # report below is non-blocking. vendored-sync: diff --git a/NEXT_TASKS.md b/NEXT_TASKS.md index 0afdce87..93cace1e 100644 --- a/NEXT_TASKS.md +++ b/NEXT_TASKS.md @@ -25,17 +25,24 @@ further action unless a newer deepwalk lands. ## 2. id↔label validator — ADOPTED + ENFORCING (Phase 2) — DONE -TraitMech joined the Mech group — the byte-identical pin is now a **4-repo -invariant** (CultureMech / MIM / CommunityMech / TraitMech). Vendored -byte-identical (sha256 matches all three siblings): -`scripts/validate_id_label_correspondence.py` + the two shared tests + -`scripts/.validate_id_label_correspondence.sha256` pin. +TraitMech joined the Mech group — the byte-identical vendoring is now a **4-repo +invariant** (CultureMech / MIM / CommunityMech / TraitMech): the validator + the +three shared tests + `chem_formula.py`. `conf/id_label_targets.yaml` targets the two ontology grounding tables (`mappings/node_grounding.tsv`, `mappings/predicate_grounding.tsv`) with CHEBI/GO/ENVO/PATO/RO adapters; METPO/traitmech/biolink/rdfs/UniProtKB are -ignored prefixes. Recipes: `validate-products`, `report-label-drift`, -`verify-validator-pin`, `refresh-validator-pin`. CI workflow -`label-correspondence.yaml`: pin guard + `validate-products` both **blocking**. +ignored prefixes. Recipes: `validate-products`, `report-label-drift`. CI workflow +`label-correspondence.yaml`: `vendored-sync` drift check + `validate-products` +both **blocking**. + +Update (2026-07-21): the self-generated sha256 pin was **retired** (Phase 2 step +2d). It only compared a copy to a hash from the *same* repo, so all four could +pass while diverged. The `vendored-sync` job now runs `scripts/check_vendored_sync.sh`, +which diffs the vendored files against `CultureBotAI/CultureMech@` +— the reference lives in another repo, so a one-copy edit fails CI. Deleted: +`verify-/refresh-validator-pin`, the `VENDORED_IDLABEL_FILES` manifest, and +`scripts/.validate_id_label_correspondence.sha256`. Propagation: PR into the hub → +merge → bump `.vendored_canon_ref` here. `schema-pin` is a separate set, unaffected. The 15 pre-existing MISMATCHES found at adoption were all wrong CURIEs in `mappings/node_grounding.tsv` (e.g. `PATO:0000383` is "female", not "decreased diff --git a/justfile b/justfile index 5180547d..6111b10e 100644 --- a/justfile +++ b/justfile @@ -279,34 +279,14 @@ validate-products: report-label-drift: uv run python scripts/validate_id_label_correspondence.py -c conf/id_label_targets.yaml --report reports/label_drift.tsv -# Vendored id-label files that must stay byte-identical across the CultureMech / -# MIM / CommunityMech / TraitMech Mech repos and must not silently diverge: the -# validator + its two shared tests. conf/id_label_targets.yaml is deliberately -# per-repo (different adapters/targets) so it is NOT here. -VENDORED_IDLABEL_FILES := "scripts/validate_id_label_correspondence.py scripts/chem_formula.py tests/test_id_label_empty_adapter.py tests/test_id_label_unknown_prefix.py tests/test_id_label_plausibility.py" - -# Durability guard: fail if any vendored id-label file drifts from its pinned -# sha256. CI runs this so an accidental edit to one copy can't silently diverge. -verify-validator-pin: - #!/usr/bin/env bash - set -euo pipefail - if command -v sha256sum >/dev/null 2>&1; then - sha256sum -c scripts/.validate_id_label_correspondence.sha256 - else - shasum -a 256 -c scripts/.validate_id_label_correspondence.sha256 - fi - -# Intentional sync only: re-pin the sha256 manifest to the CURRENT contents of -# the vendored files after a deliberate, all-repos byte-identical update. -refresh-validator-pin: - #!/usr/bin/env bash - set -euo pipefail - : > scripts/.validate_id_label_correspondence.sha256 - for f in {{VENDORED_IDLABEL_FILES}}; do - if command -v sha256sum >/dev/null 2>&1; then h=$(sha256sum "$f" | cut -d' ' -f1); else h=$(shasum -a 256 "$f" | cut -d' ' -f1); fi - printf '%s %s\n' "$h" "$f" >> scripts/.validate_id_label_correspondence.sha256 - echo "re-pinned $f to $h" - done +# NOTE: the id↔label validator + its shared tests are vendored byte-identical +# across the Mech repos. The old self-generated sha256 pin (verify-/refresh- +# validator-pin) was retired — it only compared a copy to a hash from the SAME +# repo, so all four could pass while diverged. Drift is now caught by the +# shared-reference check: the `vendored-sync` CI job runs +# scripts/check_vendored_sync.sh, which diffs these files against +# CultureBotAI/CultureMech@. To propagate a change: +# PR into that hub → merge → bump .vendored_canon_ref here. # Discussions / knowledge-gap browser (shared kg_microbe_discussions in claw). # Writes app/discussions/{index.html,data.js} from every record's discussions. diff --git a/scripts/.validate_id_label_correspondence.sha256 b/scripts/.validate_id_label_correspondence.sha256 deleted file mode 100644 index 9eee7342..00000000 --- a/scripts/.validate_id_label_correspondence.sha256 +++ /dev/null @@ -1,5 +0,0 @@ -79b9de5e6edb04c8b4bf110bc6b80e1312f9dae78912098d67764caa616a12ba scripts/validate_id_label_correspondence.py -9852db629e76aee781a934be1fac58a07f8f88e28b792d9cfb098097b157c8e6 scripts/chem_formula.py -55a432e04c402939c3fdc985524e749b7d7d09b58ccbd036874865bc4250b115 tests/test_id_label_empty_adapter.py -f01d2264a4728285c521b0c3edc084f97504e8ff3bc1eaf90531d7479c04786d tests/test_id_label_unknown_prefix.py -fcca339acab6ec7bbe54ec5921eefe422be620b0f93cba13a58d44bd257e1ac7 tests/test_id_label_plausibility.py