From a5772f5e109c653642197fd2339188314ee112d1 Mon Sep 17 00:00:00 2001 From: Arena Agent Date: Tue, 22 Sep 2026 09:46:19 +0000 Subject: [PATCH] =?UTF-8?q?fix(ci):=20adopt=20upstream=20CI-honesty=20gate?= =?UTF-8?q?s=20downstream=20=E2=80=94=20dogfood-gate=20+=20hypatia=20(issu?= =?UTF-8?q?e=20#49)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 2026-07-17 failure-path audit found checks that CANNOT fail when something is wrong. The template origin (rsr-template-repo / standards#hypatia-scan-reusable) has since been repaired; this change adopts those fixes here, per the issue's fix-at-origin-then-fan-out rule: * empty-lint: 🔴 GATE — runs the shared byte-safe scanner (scripts/check-invisible-characters.sh, verbatim from rsr-template-repo) and EXITS 1 on blocking (C0/NUL) findings; a scanner failure exits 2 instead of silently passing. (Pre-fix: 'set +e' + ::warning annotations only — could never fail.) * groove-check: 🟡 CHECK — gate logic extracted to scripts/check-groove-manifest.sh; an INVALID manifest now exits 1 (pre-fix: ::error annotation on a still-green job). Missing-endpoint advisories stay advisories and say so. * dogfood-summary: ℹī¸ ADVISORY — labelled non-gating in the job name and the scorecard, so green is never mistaken for enforcement. * hypatia-scan: adds the estate-reusable's block-on-high gate as an opt-in (repository variable HYPATIA_BLOCK_ON_HIGH=true), logic in scripts/check-hypatia-findings.sh; advisory step relabelled '(ADVISORY — does not gate)' and suppressed when the gate is on. * gate-self-test job: scripts/test-ci-honesty.sh runs every gate's REAL logic against fixtures that MUST pass and MUST FAIL, plus structural drift checks that the workflows still wire the gates in. A future edit that re-fakes a gate turns CI red. * just test-ci-honesty + .github/workflows/README.md gate-honesty docs. Part of issue #49 (codeql rust coverage landed in #61; README scorecard reference previously removed; contractile drift resolved by the 2026-08-01 decision record). --- .github/workflows/README.md | 36 ++++- .github/workflows/dogfood-gate.yml | 173 +++++++++++++----------- .github/workflows/hypatia-scan.yml | 47 ++++--- Justfile | 6 + scripts/check-groove-manifest.sh | 90 +++++++++++++ scripts/check-hypatia-findings.sh | 73 +++++++++++ scripts/check-invisible-characters.sh | 72 ++++++++++ scripts/test-ci-honesty.sh | 182 ++++++++++++++++++++++++++ 8 files changed, 590 insertions(+), 89 deletions(-) create mode 100755 scripts/check-groove-manifest.sh create mode 100755 scripts/check-hypatia-findings.sh create mode 100755 scripts/check-invisible-characters.sh create mode 100755 scripts/test-ci-honesty.sh diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 2271cf7..5d04866 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -26,7 +26,41 @@ private or sibling `hyperpolymath` repos (`standards` reusable workflows, `hypatia-scan.yml`, `instant-sync.yml`, `mirror.yml`, `push-email-notify.yml`, `secret-scanner.yml` -`codeql.yml` is standard GitHub scanning and runs anywhere. +`codeql.yml` is standard GitHub scanning and runs anywhere — its matrix +covers **both** `javascript-typescript` and `rust` (the Rust core has been +scanned since #61; buildless `build-mode: none` is correct for CodeQL Rust). Branch protection should require `rust-ci` and `ui-ci`; estate workflows should stay non-required. + +## Gate honesty (issue #49) + +The verdict test for CI here: *can this check fail when something is actually +wrong?* Every check we own carries a tier label, and every gate is proven by +fail fixtures: + +| Tier | Meaning | Examples | +|---|---|---| +| 🔴 GATE | Fails the run on findings | `empty-lint` (C0/NUL bytes), `hypatia-scan` **only when opted in**, `gate-self-test` | +| 🟡 CHECK | Fails on real errors; advisory otherwise | `groove-check` (invalid manifest JSON fails; missing endpoint only warns) | +| ℹī¸ ADVISORY | Never fails; labelled so | `dogfood-summary`, Hypatia "Check for critical issues" | + +The gate logic lives in runnable scripts (not inline workflow shell), so the +exact code that runs in CI can be exercised locally: + +* `scripts/check-invisible-characters.sh` — byte-safe invisible-character scanner +* `scripts/check-groove-manifest.sh` — Groove manifest gate (exit 1 on invalid JSON) +* `scripts/check-hypatia-findings.sh` — Hypatia severity counts + blocking decision + +**Prove it:** `scripts/test-ci-honesty.sh` (also `just test-ci-honesty`) runs +each gate against fixtures that MUST pass and fixtures that MUST FAIL, and +structurally checks the workflows still wire the gates in. The `gate-self-test` +job in `dogfood-gate.yml` runs the same proofs on every PR — if a future edit +turns a gate back into an annotation-only fake, CI goes red. + +**Opting into the Hypatia gate:** set the repository variable +`HYPATIA_BLOCK_ON_HIGH=true` (Settings → Secrets and variables → Actions → +Variables). Then critical/high findings fail the scan — mirroring the +`block-on-high` input of the estate reusable in `hyperpolymath/standards`. +Default (and estate doctrine) is advisory: findings land on the Security → +Code scanning surface (SARIF category `hypatia`) instead. diff --git a/.github/workflows/dogfood-gate.yml b/.github/workflows/dogfood-gate.yml index ab34f93..9d10fc4 100644 --- a/.github/workflows/dogfood-gate.yml +++ b/.github/workflows/dogfood-gate.yml @@ -1,12 +1,25 @@ # This workflow is managed by gh actions-lock. # SPDX-License-Identifier: MPL-2.0 -# This workflow is managed by gh actions-lock. -# This workflow is managed by gh actions-lock. # Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) # # dogfood-gate.yml — Hyperpolymath Dogfooding Quality Gate # Validates that the repo uses hyperpolymath's own formats and tools. # Companion to static-analysis-gate.yml (security) — this is for format compliance. +# +# Provenance / CI-honesty status (issue #49, adopted 2026-09-22): +# Adopts the fixed template from hyperpolymath/rsr-template-repo, where the +# "checks that cannot fail" were repaired at their origin. Gate tiers follow +# the upstream taxonomy — 🔴 GATE (blocks), 🟡 CHECK (fails on real errors), +# ℹī¸ ADVISORY (never gates, labelled): +# * empty-lint: 🔴 GATE — runs scripts/check-invisible-characters.sh and +# EXITS 1 on blocking (C0/NUL) findings; a scanner failure exits 2 +# rather than silently passing. (Pre-fix: `set +e` + warnings only.) +# * groove-check: 🟡 CHECK — runs scripts/check-groove-manifest.sh; an +# invalid manifest EXITS 1. Advisories (missing manifest with server +# code) stay advisories and are labelled as such. +# * dogfood-summary: ℹī¸ ADVISORY — never gates; labelled so. +# The gate-self-test job proves, on every run, that each gate can still fail: +# scripts/test-ci-honesty.sh runs the real gate logic against fail fixtures. name: Dogfood Gate on: @@ -15,13 +28,33 @@ on: push: branches: [main, master] +# Estate guardrail: cancel superseded runs; safe — read-only checks only. +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + permissions: actions: read contents: read jobs: # --------------------------------------------------------------------------- - # Job 1: A2ML manifest validation + # Job 0: Gate honesty self-test — proves the gates can fail (issue #49) + # --------------------------------------------------------------------------- + gate-self-test: + name: "🔴 GATE: CI honesty self-test (fail fixtures)" + runs-on: ubuntu-latest + timeout-minutes: 10 + + steps: + - name: Checkout repository + uses: actions/checkout@v7.0.1 + + - name: Run the fail-fixture proofs + run: scripts/test-ci-honesty.sh + + # --------------------------------------------------------------------------- + # Job 1: DEED manifest validation (REAL — kept as-is per issue #49 audit) # --------------------------------------------------------------------------- a2ml-validate: name: Validate DEED manifests @@ -65,7 +98,7 @@ jobs: fi # --------------------------------------------------------------------------- - # Job 2: K9 contract validation + # Job 2: K9 contract validation (REAL — kept as-is; skips when k9_count==0) # --------------------------------------------------------------------------- k9-validate: name: Validate K9 contracts @@ -115,10 +148,13 @@ jobs: # --------------------------------------------------------------------------- # Job 3: Empty-linter — invisible character detection + # 🔴 GATE (adopted from rsr-template-repo): blocking findings fail the check; + # a scanner failure refuses rather than passing silently. # --------------------------------------------------------------------------- empty-lint: - name: Empty-linter (invisible characters) + name: "🔴 GATE: Empty-linter (invisible characters)" runs-on: ubuntu-latest + timeout-minutes: 15 steps: - name: Checkout repository @@ -127,36 +163,46 @@ jobs: - name: Scan for invisible characters id: lint run: | - # Inline invisible character detection (from empty-linter's core patterns). - # Checks for: zero-width spaces, zero-width joiners, BOM, soft hyphens, - # non-breaking spaces, null bytes, and other invisible Unicode in source files. - set +e - PATTERNS='(*UTF)[\x00-\x08\x0B\x0C\x0E-\x1F\x{a0}\x{ad}\x{200b}-\x{200f}\x{202a}-\x{202f}\x{2060}\x{2066}-\x{2069}\x{feff}]' - find "$GITHUB_WORKSPACE" \ - -not -path '*/.git/*' -not -path '*/node_modules/*' \ - -not -path '*/.deno/*' -not -path '*/target/*' \ - -not -path '*/_build/*' -not -path '*/deps/*' \ - -not -path '*/external_corpora/*' -not -path '*/.lake/*' \ - -type f \( -name '*.rs' -o -name '*.ex' -o -name '*.exs' -o -name '*.res' \ - -o -name '*.js' -o -name '*.ts' -o -name '*.json' -o -name '*.toml' \ - -o -name '*.yml' -o -name '*.yaml' -o -name '*.md' -o -name '*.adoc' \ - -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ - -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ - -exec grep -aPl "$PATTERNS" {} + > /tmp/empty-lint-results.txt 2>/dev/null - EL_EXIT=$? - set -e - - FINDINGS=$(wc -l < /tmp/empty-lint-results.txt 2>/dev/null || echo 0) + RESULTS_FILE="$RUNNER_TEMP/empty-lint-results.bin" + BLOCKING_FILE="$RUNNER_TEMP/empty-lint-blocking-results.bin" + if ! scripts/check-invisible-characters.sh \ + "$GITHUB_WORKSPACE" "$RESULTS_FILE" "$BLOCKING_FILE"; then + echo "::error::Invisible-character scanner failed; refusing a partial pass" + exit 2 + fi + + FINDINGS=0 + while IFS= read -r -d '' filepath; do + FINDINGS=$((FINDINGS + 1)) + REL_PATH="${filepath#"$GITHUB_WORKSPACE"/}" + SAFE_PATH="${REL_PATH//'%'/'%25'}" + SAFE_PATH="${SAFE_PATH//$'\r'/'%0D'}" + SAFE_PATH="${SAFE_PATH//$'\n'/'%0A'}" + SAFE_PATH="${SAFE_PATH//':'/'%3A'}" + SAFE_PATH="${SAFE_PATH//','/'%2C'}" + echo "::warning file=${SAFE_PATH}::Invisible Unicode or C0 characters detected" + done < "$RESULTS_FILE" + + BLOCKING=0 + while IFS= read -r -d '' filepath; do + BLOCKING=$((BLOCKING + 1)) + REL_PATH="${filepath#"$GITHUB_WORKSPACE"/}" + SAFE_PATH="${REL_PATH//'%'/'%25'}" + SAFE_PATH="${SAFE_PATH//$'\r'/'%0D'}" + SAFE_PATH="${SAFE_PATH//$'\n'/'%0A'}" + SAFE_PATH="${SAFE_PATH//':'/'%3A'}" + SAFE_PATH="${SAFE_PATH//','/'%2C'}" + echo "::error file=${SAFE_PATH}::C0 control character or NUL byte detected" + done < "$BLOCKING_FILE" + echo "findings=$FINDINGS" >> "$GITHUB_OUTPUT" - echo "exit_code=$EL_EXIT" >> "$GITHUB_OUTPUT" + echo "blocking=$BLOCKING" >> "$GITHUB_OUTPUT" echo "ready=true" >> "$GITHUB_OUTPUT" - # Emit annotations for each file with invisible chars - while IFS= read -r filepath; do - [ -z "$filepath" ] && continue - REL_PATH="${filepath#$GITHUB_WORKSPACE/}" - echo "::warning file=${REL_PATH}::Invisible Unicode characters detected (zero-width space, BOM, NBSP, etc.)" - done < /tmp/empty-lint-results.txt + if [ "$BLOCKING" -gt 0 ]; then + echo "## Empty-linter: BLOCKED — $BLOCKING file(s) contain C0/NUL corruption" >> "$GITHUB_STEP_SUMMARY" + exit 1 + fi - name: Write summary run: | @@ -178,11 +224,14 @@ jobs: fi # --------------------------------------------------------------------------- - # Job 4: Groove manifest check (for repos that should expose services) + # Job 4: Groove manifest check + # 🟡 CHECK: an invalid manifest FAILS the job (exit 1 in the shared script); + # "server code but no endpoint" remains a labelled ADVISORY warning. # --------------------------------------------------------------------------- groove-check: - name: Groove manifest check + name: "🟡 CHECK: Groove manifest check" runs-on: ubuntu-latest + timeout-minutes: 15 steps: - name: Checkout repository @@ -191,57 +240,32 @@ jobs: - name: Check for Groove manifest id: groove run: | - # Check for static or dynamic Groove endpoints - HAS_MANIFEST="false" - HAS_GROOVE_CODE="false" - - if [ -f ".well-known/groove/manifest.json" ]; then - HAS_MANIFEST="true" - # Validate the manifest JSON - if ! jq empty .well-known/groove/manifest.json 2>/dev/null; then - echo "::error file=.well-known/groove/manifest.json::Invalid JSON in Groove manifest" - else - SVC_ID=$(jq -r '.service_id // "unknown"' .well-known/groove/manifest.json) - echo "service_id=$SVC_ID" >> "$GITHUB_OUTPUT" - fi - fi - - # Check for Groove endpoint code (Rust, Elixir, Zig, V) - if grep -rl 'well-known/groove' --include='*.rs' --include='*.ex' --include='*.zig' --include='*.v' --include='*.res' . 2>/dev/null | head -1 | grep -q .; then - HAS_GROOVE_CODE="true" - fi - - # Check if this repo likely serves HTTP (has server/listener code) - HAS_SERVER="false" - if grep -rl 'TcpListener\|Bandit\|Plug.Cowboy\|httpz\|vweb\|axum::serve\|actix_web' --include='*.rs' --include='*.ex' --include='*.zig' --include='*.v' . 2>/dev/null | head -1 | grep -q .; then - HAS_SERVER="true" - fi - - echo "has_manifest=$HAS_MANIFEST" >> "$GITHUB_OUTPUT" - echo "has_groove_code=$HAS_GROOVE_CODE" >> "$GITHUB_OUTPUT" - echo "has_server=$HAS_SERVER" >> "$GITHUB_OUTPUT" - - if [ "$HAS_SERVER" = "true" ] && [ "$HAS_MANIFEST" = "false" ] && [ "$HAS_GROOVE_CODE" = "false" ]; then - echo "::warning::This repo has server code but no Groove endpoint. Add .well-known/groove/manifest.json for service discovery." - fi + # Gate logic lives in scripts/check-groove-manifest.sh so the exact + # code that runs here is exercised by scripts/test-ci-honesty.sh + # against pass AND fail fixtures (issue #49). + scripts/check-groove-manifest.sh "$GITHUB_WORKSPACE" - name: Write summary + if: always() run: | echo "## Groove Protocol Check" >> "$GITHUB_STEP_SUMMARY" echo "" >> "$GITHUB_STEP_SUMMARY" echo "| Check | Status |" >> "$GITHUB_STEP_SUMMARY" echo "|-------|--------|" >> "$GITHUB_STEP_SUMMARY" - echo "| Static manifest (.well-known/groove/manifest.json) | ${{ steps.groove.outputs.has_manifest }} |" >> "$GITHUB_STEP_SUMMARY" + echo "| Static manifest (www/.well-known/groove/manifest.json) | ${{ steps.groove.outputs.has_manifest }} |" >> "$GITHUB_STEP_SUMMARY" echo "| Groove endpoint in code | ${{ steps.groove.outputs.has_groove_code }} |" >> "$GITHUB_STEP_SUMMARY" echo "| Has HTTP server code | ${{ steps.groove.outputs.has_server }} |" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "_🟡 CHECK tier: an invalid manifest fails this job; advisories (e.g. missing endpoint with server code) do not gate._" >> "$GITHUB_STEP_SUMMARY" # --------------------------------------------------------------------------- - # Job 5: Dogfooding summary + # Job 5: Dogfooding summary — ℹī¸ ADVISORY. Never gates, by design, and says + # so in its name so a green run is never mistaken for enforced compliance. # --------------------------------------------------------------------------- dogfood-summary: - name: Dogfooding compliance summary + name: "ℹī¸ ADVISORY: Dogfooding compliance summary (non-gating)" runs-on: ubuntu-latest - needs: [a2ml-validate, k9-validate, empty-lint, groove-check] + needs: [gate-self-test, a2ml-validate, k9-validate, empty-lint, groove-check] if: always() steps: @@ -278,7 +302,7 @@ jobs: fi # Groove manifest or code? - if [ -f ".well-known/groove/manifest.json" ] || grep -rl 'well-known/groove' --include='*.rs' --include='*.ex' --include='*.zig' . 2>/dev/null | head -1 | grep -q .; then + if [ -f "www/.well-known/groove/manifest.json" ] || [ -f ".well-known/groove/manifest.json" ] || grep -rl 'well-known/groove' --include='*.rs' --include='*.ex' --include='*.zig' . 2>/dev/null | head -1 | grep -q .; then SCORE=$((SCORE + 1)) GROOVE_STATUS=":white_check_mark:" else @@ -296,11 +320,14 @@ jobs: cat <> "$GITHUB_STEP_SUMMARY" ## Dogfooding Scorecard + > ℹī¸ **ADVISORY — this job never gates.** It only reports; enforcement + > tier of each check is stated in its own job (🔴/🟡/ℹī¸). + **Score: ${SCORE}/${MAX}** | Tool/Format | Status | Notes | |-------------|--------|-------| - | DEED repo deed (`_chora.deed`) | ${A2ML_STATUS} | Required for all RSR repos | + | DEED repo deed (\`_chora.deed\`) | ${A2ML_STATUS} | Required for all RSR repos | | K9 contracts | ${K9_STATUS} | Required for repos with config files | | .editorconfig | ${EC_STATUS} | Required for all repos | | Groove endpoint | ${GROOVE_STATUS} | Required for service repos | diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index caf5c2a..7bc31f4 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -252,6 +252,20 @@ jobs: # (codeql.yml) instead of overwriting them on the same surface. category: hypatia + - name: Block critical/high findings when HYPATIA_BLOCK_ON_HIGH is enabled + # 🔴 OPT-IN GATE (issue #49): by default this scan is ADVISORY by estate + # doctrine. Setting the repository variable HYPATIA_BLOCK_ON_HIGH=true + # (Settings → Secrets and variables → Actions → Variables) turns the + # check into a real gate WITHOUT editing this canonical workflow — + # mirroring the `block-on-high` input of the estate reusable + # (hyperpolymath/standards .github/workflows/hypatia-scan-reusable.yml). + # The decision logic lives in scripts/check-hypatia-findings.sh so the + # fail path is proven by fail fixtures (scripts/test-ci-honesty.sh, + # run by the gate-self-test job in dogfood-gate.yml). + if: vars.HYPATIA_BLOCK_ON_HIGH == 'true' + run: | + scripts/check-hypatia-findings.sh hypatia-findings.json --block --high-too + - name: Submit findings to gitbot-fleet (Phase 2) if: steps.scan.outputs.findings_count > 0 # Phase 2 is the collaborative LEARNING side-channel ("bots share @@ -321,19 +335,20 @@ jobs: # Cleanup rm -rf "$FLEET_DIR" - - name: Check for critical issues - if: steps.scan.outputs.critical > 0 + - name: Check for critical issues (ADVISORY — does not gate) + # Only surfaces an advisory when the opt-in gate above is OFF; + # when HYPATIA_BLOCK_ON_HIGH=true the blocking step owns the verdict. + if: vars.HYPATIA_BLOCK_ON_HIGH != 'true' && steps.scan.outputs.critical > 0 # GATING POLICY (explicit, by design — not an oversight): - # Hypatia is ADVISORY here. Critical findings are surfaced - # (step annotation + SARIF alert on the code-scanning page + - # PR comment) but do NOT fail this check. Enforcement is - # delegated to the code-scanning surface: tighten by adding a - # branch-protection "required" status on the `hypatia` SARIF - # category, not by reintroducing an `exit 1` here. This keeps - # the gate decision in one auditable place (hypatia#213 gate - # decoupling) and lets a repo opt into fail-on-critical without - # editing this canonical workflow. To change the policy, change - # branch protection — deliberately no commented-out `exit 1`. + # Hypatia is ADVISORY here unless the repository opts into the gate + # (repository variable HYPATIA_BLOCK_ON_HIGH=true — see the 🔴 step + # above). Findings are surfaced (step annotation + SARIF alert on the + # code-scanning page + PR comment) but do NOT fail this check in + # advisory mode. Enforcement is delegated to one of two auditable + # switches: (a) the HYPATIA_BLOCK_ON_HIGH variable, or (b) a + # branch-protection "required" status on the `hypatia` SARIF category. + # This keeps the gate decision in one auditable place (hypatia#213 + # gate decoupling) — deliberately no commented-out `exit 1`. run: | echo "::warning::Hypatia found critical security issue(s) — advisory." echo "See the Security → Code scanning page (category: hypatia)" @@ -364,9 +379,11 @@ jobs: CodeQL alerts. 2. The full finding set is also attached as the \`hypatia-findings.json\` build artifact for offline review. - 3. Findings are **advisory** today (surfaced, not gated); the - gating policy is documented in the workflow's "Check for - critical issues" step. + 3. Findings are **advisory** unless the repository variable + \`HYPATIA_BLOCK_ON_HIGH=true\` is set (surfaced, not gated in + advisory mode); the gating policy is documented in the + workflow's "Check for critical issues (ADVISORY — does not + gate)" step. ## Learning diff --git a/Justfile b/Justfile index a90eb3c..c96e822 100644 --- a/Justfile +++ b/Justfile @@ -32,6 +32,12 @@ test: build test-rust: bun run test:rust +# Prove the CI gates can fail: run every gate's real logic against fixtures +# that MUST pass and MUST FAIL, plus drift checks that the workflows still wire +# the gates in (issue #49). Also runs as the gate-self-test job in CI. +test-ci-honesty: + scripts/test-ci-honesty.sh + # Run the development server (http://localhost:5173) run: bun run dev diff --git a/scripts/check-groove-manifest.sh b/scripts/check-groove-manifest.sh new file mode 100755 index 0000000..1bbf94e --- /dev/null +++ b/scripts/check-groove-manifest.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) +# +# check-groove-manifest.sh — Groove protocol manifest gate logic, extracted +# from the dogfood-gate workflow so the *same* code that runs in CI can be +# exercised locally against fail fixtures (issue #49 — a gate that cannot be +# run is a gate that cannot fail). +# +# Exit codes: +# 0 — everything fine (advisories may have been printed; advisories do not gate) +# 1 — a manifest exists but is INVALID JSON (real error — this is the gate) +# 2 — internal/usage error (refuse a partial pass) +# +# Behaviour (faithful to rsr-template-repo/.github/workflows/groove-check.yml): +# * canonical manifest location is www/.well-known/groove/manifest.json; +# the repository-root path is accepted, with a warning, during migration. +# * a manifest that does not parse as JSON is a HARD FAILURE (this is what +# turns the check from an annotation-only fake into a real gate). +# * a repo with production HTTP-server markers but no Groove endpoint is an +# ADVISORY warning only — not every HTTP service is meant to be discovered. +set -u + +root="${1:-}" +if [ -z "$root" ] || [ ! -d "$root" ]; then + echo "usage: $0 REPO_ROOT" >&2 + exit 2 +fi +cd "$root" || exit 2 + +command -v jq >/dev/null 2>&1 || { echo "jq is required" >&2; exit 2; } + +emit_output() { + # Only write step outputs when running inside GitHub Actions. + if [ -n "${GITHUB_OUTPUT:-}" ]; then + echo "$1=$2" >> "$GITHUB_OUTPUT" + else + echo "$1=$2" + fi +} + +HAS_MANIFEST="false" +HAS_GROOVE_CODE="false" +HAS_SERVER="false" +SERVICE_ID="" + +MANIFEST="" +if [ -f "www/.well-known/groove/manifest.json" ]; then + MANIFEST="www/.well-known/groove/manifest.json" +elif [ -f ".well-known/groove/manifest.json" ]; then + MANIFEST=".well-known/groove/manifest.json" + echo "::warning::Groove manifest at legacy root .well-known/ — canonical location is www/.well-known/ (run scripts/migrate-wellknown-to-www.sh)" +fi + +if [ -n "$MANIFEST" ]; then + HAS_MANIFEST="true" + if ! jq empty "$MANIFEST" 2>/dev/null; then + echo "::error file=$MANIFEST::Invalid JSON in Groove manifest" + emit_output has_manifest "$HAS_MANIFEST" + emit_output has_groove_code "$HAS_GROOVE_CODE" + emit_output has_server "$HAS_SERVER" + exit 1 # <-- THE GATE: invalid manifests fail the check. + fi + SERVICE_ID=$(jq -r '.service_id // "unknown"' "$MANIFEST") +fi + +# Groove endpoint implemented in code? +if grep -rl 'well-known/groove' --include='*.rs' --include='*.ex' --include='*.zig' --include='*.v' --include='*.res' . 2>/dev/null | grep -v '^./scripts/' | head -1 | grep -q .; then + HAS_GROOVE_CODE="true" +fi + +# Production HTTP-server markers only. A bare `TcpListener` is deliberately NOT +# a signal (dominated by test/utility use); real servers pair the listener with +# a framework entry point. +if grep -rl 'Bandit\|Plug.Cowboy\|httpz\|vweb\|axum::serve\|actix_web\|hyper::Server\|rocket::build\|warp::serve' --include='*.rs' --include='*.ex' --include='*.zig' --include='*.v' . 2>/dev/null | head -1 | grep -q .; then + HAS_SERVER="true" +fi + +emit_output has_manifest "$HAS_MANIFEST" +emit_output has_groove_code "$HAS_GROOVE_CODE" +emit_output has_server "$HAS_SERVER" +if [ -n "$SERVICE_ID" ]; then + emit_output service_id "$SERVICE_ID" +fi + +if [ "$HAS_SERVER" = "true" ] && [ "$HAS_MANIFEST" = "false" ] && [ "$HAS_GROOVE_CODE" = "false" ]; then + echo "::warning::This repo has server code but no Groove endpoint. Add www/.well-known/groove/manifest.json for service discovery. (ADVISORY — this does not fail the check.)" +fi + +exit 0 diff --git a/scripts/check-hypatia-findings.sh b/scripts/check-hypatia-findings.sh new file mode 100755 index 0000000..0e5e410 --- /dev/null +++ b/scripts/check-hypatia-findings.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) +# +# check-hypatia-findings.sh — severity counts + optional blocking decision for +# a Hypatia findings JSON file. Extracted from the hypatia-scan workflow so the +# gate decision is runnable (and provable) outside CI (issue #49). +# +# Semantics mirror the estate canonical reusable +# (hyperpolymath/standards .github/workflows/hypatia-scan-reusable.yml): +# * advisory by default — findings are surfaced, never gate; +# * when blocking is enabled (the workflow passes --block, gated on the +# HYPATIA_BLOCK_ON_HIGH repository variable, mirroring the reusable's +# `block-on-high` input) the decision refuses *critical* findings, and +# additionally *high* findings when --high-too is given. +# +# Exit codes: +# 0 — no blocking finding under the selected policy (or advisory mode) +# 1 — blocking finding(s) present while --block is active +# 2 — the findings file is missing/malformed (fail loud, never silently pass) +set -u + +findings_file="" +block="false" +high_too="false" + +for arg in "$@"; do + case "$arg" in + --block) block="true" ;; + --high-too) high_too="true" ;; + -*) echo "unknown flag: $arg" >&2; exit 2 ;; + *) findings_file="$arg" ;; + esac +done + +if [ -z "$findings_file" ] || [ ! -f "$findings_file" ]; then + echo "usage: $0 FINDINGS_JSON [--block] [--high-too]" >&2 + exit 2 +fi +command -v jq >/dev/null 2>&1 || { echo "jq is required" >&2; exit 2; } + +# Malformed input must fail LOUDLY: a scanner whose output we cannot read is a +# scanner in an unknown state, not a clean scanner (the verdict test of #49 — +# "can this check fail when something is actually wrong?"). +if ! jq -e 'type == "array"' "$findings_file" >/dev/null 2>&1; then + echo "::error::Hypatia findings file is not a JSON array: $findings_file" + exit 2 +fi + +CRITICAL=$(jq '[.[] | select(.severity == "critical")] | length' "$findings_file") +HIGH=$(jq '[.[] | select(.severity == "high")] | length' "$findings_file") +MEDIUM=$(jq '[.[] | select(.severity == "medium")] | length' "$findings_file") +TOTAL=$(jq 'length' "$findings_file") + +echo "hypatia findings: total=$TOTAL critical=$CRITICAL high=$HIGH medium=$MEDIUM" + +blocking="$CRITICAL" +policy="critical" +if [ "$high_too" = "true" ]; then + blocking=$((CRITICAL + HIGH)) + policy="critical+high" +fi + +if [ "$block" = "true" ] && [ "$blocking" -gt 0 ]; then + echo "::error::Hypatia found $blocking blocking finding(s) (policy: $policy) — refusing, HYPATIA_BLOCK_ON_HIGH is enabled" + exit 1 +fi + +if [ "$blocking" -gt 0 ]; then + echo "::warning::Hypatia found $blocking $policy finding(s) — ADVISORY only, this does not fail the check" +fi + +exit 0 diff --git a/scripts/check-invisible-characters.sh b/scripts/check-invisible-characters.sh new file mode 100755 index 0000000..8cc49fc --- /dev/null +++ b/scripts/check-invisible-characters.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# Byte-safe scanner for invisible Unicode encodings and forbidden C0 controls. +set -u + +scan_root="${1:-}" +results_file="${2:-}" +blocking_results_file="${3:-}" +grep_bin="${INVISIBLE_GREP_BIN:-grep}" +find_bin="${INVISIBLE_FIND_BIN:-find}" + +if [[ -z "$scan_root" || ! -d "$scan_root" || -z "$results_file" ]]; then + echo "usage: $0 SCAN_ROOT RESULTS_FILE" >&2 + exit 2 +fi + +# Scan bytes under the C locale. This detects UTF-8 encodings even when another +# byte in the file is invalid UTF-8, while excluding permitted TAB/LF/CR bytes. +pattern='[\x00-\x08\x0B\x0C\x0E-\x1F]|\xC2(?:\xA0|\xAD)|\xE2\x80[\x8B-\x8F\xAA-\xAF]|\xE2\x81(?:\xA0|[\xA6-\xA9])|\xEF\xBB\xBF' +blocking_pattern='[\x00-\x08\x0B\x0C\x0E-\x1F]' +: > "$results_file" || exit 2 +if [[ -n "$blocking_results_file" ]]; then + : > "$blocking_results_file" || exit 2 +fi +scan_error=0 +enumeration_file="$(mktemp)" || exit 2 # TMPDIR-respecting; Hypatia hardcoded_tmp (alert #122) +# Invoked indirectly by the EXIT trap. +# shellcheck disable=SC2329 +cleanup() { + rm -f -- "$enumeration_file" +} +trap cleanup EXIT + +if ! "$find_bin" "$scan_root" \ + -not -path '*/.git/*' -not -path '*/node_modules/*' \ + -not -path '*/.deno/*' -not -path '*/target/*' \ + -not -path '*/_build/*' -not -path '*/deps/*' \ + -not -path '*/external_corpora/*' -not -path '*/.lake/*' \ + -type f \( -name '*.rs' -o -name '*.ex' -o -name '*.exs' -o -name '*.res' \ + -o -name '*.js' -o -name '*.ts' -o -name '*.json' -o -name '*.toml' \ + -o -name '*.yml' -o -name '*.yaml' -o -name '*.md' -o -name '*.adoc' \ + -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ + -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \ + -o -name '*.a2ml' -o -name '*.txt' -o -name '*.just' \ + -o -name 'Justfile' -o -name 'Mustfile' -o -name 'Trustfile' -o -name 'Bustfile' \) \ + -print0 > "$enumeration_file"; then + echo "file enumeration failed: $scan_root" >&2 + exit 1 +fi + +while IFS= read -r -d '' filepath; do + LC_ALL=C "$grep_bin" -aPq "$pattern" "$filepath" + status=$? + case "$status" in + 0) + printf '%s\0' "$filepath" >> "$results_file" || scan_error=1 + if [[ -n "$blocking_results_file" ]]; then + LC_ALL=C "$grep_bin" -aPq "$blocking_pattern" "$filepath" + blocking_status=$? + case "$blocking_status" in + 0) printf '%s\0' "$filepath" >> "$blocking_results_file" || scan_error=1 ;; + 1) ;; + *) echo "blocking-classifier error ($blocking_status): $filepath" >&2; scan_error=1 ;; + esac + fi + ;; + 1) ;; + *) echo "scanner error ($status): $filepath" >&2; scan_error=1 ;; + esac +done < "$enumeration_file" + +exit "$scan_error" diff --git a/scripts/test-ci-honesty.sh b/scripts/test-ci-honesty.sh new file mode 100755 index 0000000..a125f8d --- /dev/null +++ b/scripts/test-ci-honesty.sh @@ -0,0 +1,182 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) +# +# test-ci-honesty.sh — fail-fixture proofs for nexia-list's CI gates. +# +# Issue #49 verdict test: *can this check fail when something is actually +# wrong?* This script answers it empirically, for every gate we own, by +# running the REAL gate logic (the same scripts the workflows invoke) against +# fixtures that MUST pass and fixtures that MUST FAIL, plus structural checks +# that the workflows still wire the gates in (drift alarms). +# +# Usage: scripts/test-ci-honesty.sh [REPO_ROOT] +# Exit: 0 = every proof passed; 1 = at least one gate could not fail when it +# should (or passed when it must not); 2 = environment problem. +set -u + +ROOT="${1:-$(cd "$(dirname "$0")/.." && pwd)}" +SCANNER="$ROOT/scripts/check-invisible-characters.sh" +GROOVE="$ROOT/scripts/check-groove-manifest.sh" +HYPATIA="$ROOT/scripts/check-hypatia-findings.sh" +DOGFOOD="$ROOT/.github/workflows/dogfood-gate.yml" +HYPATIA_WF="$ROOT/.github/workflows/hypatia-scan.yml" + +failures=0 +pass() { echo "PASS $1"; } +fail() { echo "FAIL $1"; failures=$((failures + 1)); } + +expect_exit() { + # expect_exit DESCRIPTION EXPECTED_CODE CMD... + local desc="$1" want="$2"; shift 2 + "$@" >/dev/null 2>&1 + local got=$? + if [ "$got" -eq "$want" ]; then pass "$desc (exit $got)"; else fail "$desc (wanted exit $want, got $got)"; fi +} + +for f in "$SCANNER" "$GROOVE" "$HYPATIA" "$DOGFOOD" "$HYPATIA_WF"; do + [ -f "$f" ] || { echo "missing: $f" >&2; exit 2; } +done + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +echo "=== scanner [1/5] empty-linter (invisible characters) ===" + +# Fixture: a clean source file — must produce NO findings. +mkdir -p "$WORK/clean" +printf 'fn main() { println!("clean"); }\n' > "$WORK/clean/main.rs" +"$SCANNER" "$WORK/clean" "$WORK/clean.res" "$WORK/clean.blk" +rc=$? +if [ "$rc" -eq 0 ] && [ ! -s "$WORK/clean.res" ] && [ ! -s "$WORK/clean.blk" ]; then + pass "clean fixture yields no findings" +else + fail "clean fixture should yield no findings (rc=$rc)" +fi + +# Fixture: zero-width space (E2 80 8B) — warning tier, NOT blocking. +mkdir -p "$WORK/zwsp" +printf 'fn main() { let s = "a\xe2\x80\x8bb"; }\n' > "$WORK/zwsp/main.rs" +"$SCANNER" "$WORK/zwsp" "$WORK/zwsp.res" "$WORK/zwsp.blk" +rc=$? +if [ "$rc" -eq 0 ] && [ -s "$WORK/zwsp.res" ] && [ ! -s "$WORK/zwsp.blk" ]; then + pass "ZWSP fixture is advisory (findings, no blocking)" +else + fail "ZWSP fixture should be advisory-only (rc=$rc)" +fi + +# Fixture: NUL byte — BLOCKING tier. This is the fail fixture: the gate must bite. +mkdir -p "$WORK/nul" +printf 'fn main() { let b = 0;\x00 }\n' > "$WORK/nul/main.rs" +"$SCANNER" "$WORK/nul" "$WORK/nul.res" "$WORK/nul.blk" +rc=$? +if [ -s "$WORK/nul.blk" ]; then + pass "NUL fixture lands in blocking results — gate has input to fail on" +else + fail "NUL fixture MUST land in blocking results (rc=$rc)" +fi + +# Decision-rule honesty: the exact rule the workflow applies after the scan +# (`if [ "$BLOCKING" -gt 0 ]; then exit 1`). Blocking results exist => the +# rule would fire. Verified structurally below and empirically by the runner +# exercising the decision here. +BLOCKING=$(tr -cd '\0' < "$WORK/nul.blk" | wc -c) +expect_exit "NUL fixture trips the gate decision rule (blocking=$BLOCKING > 0 => workflow exits 1)" 0 \ + test "$BLOCKING" -gt 0 + +echo "=== drift [2/5] workflow wiring (structural honesty) ===" + +if grep -q 'scripts/check-invisible-characters.sh' "$DOGFOOD"; then + pass "dogfood-gate runs the shared scanner (not a silent inline copy)" +else + fail "dogfood-gate.yml MUST call scripts/check-invisible-characters.sh" +fi +if grep -qE 'if \[ "\$BLOCKING" -gt 0 \]' "$DOGFOOD" && grep -q 'exit 1' "$DOGFOOD"; then + pass "dogfood-gate empty-lint contains a real exit-1 gate" +else + fail "dogfood-gate.yml empty-lint lost its exit-1 gate" +fi +if grep -q 'ADVISORY' "$DOGFOOD"; then + pass "non-gating summary job is labelled ADVISORY" +else + fail "dogfood-gate summary must be labelled ADVISORY (non-gating)" +fi +if grep -q 'scripts/check-groove-manifest.sh' "$DOGFOOD"; then + pass "dogfood-gate groove-check runs the shared manifest check" +else + fail "dogfood-gate.yml groove-check MUST call scripts/check-groove-manifest.sh" +fi + +echo "=== groove [3/5] manifest check ===" + +# Fixture: valid canonical manifest — must pass. +mkdir -p "$WORK/g-ok/www/.well-known/groove" +printf '{"service_id":"demo","endpoints":[]}\n' > "$WORK/g-ok/www/.well-known/groove/manifest.json" +expect_exit "valid groove manifest passes" 0 "$GROOVE" "$WORK/g-ok" + +# Fixture: INVALID JSON manifest — must FAIL. (The pre-fix workflow annotated +# ::error and stayed green; this is the fixture that proves the fix.) +mkdir -p "$WORK/g-bad/www/.well-known/groove" +printf '{"service_id": broken\n' > "$WORK/g-bad/www/.well-known/groove/manifest.json" +expect_exit "INVALID groove manifest FAILS the check" 1 "$GROOVE" "$WORK/g-bad" + +# Fixture: server code without a manifest — advisory only, must still pass. +mkdir -p "$WORK/g-srv/src" +printf 'use axum; fn main() { axum::serve(listener, app); }\n' > "$WORK/g-srv/src/main.rs" +"$GROOVE" "$WORK/g-srv" > "$WORK/g-srv.out" 2>&1 +rc=$? +if [ "$rc" -eq 0 ] && grep -q '::warning::' "$WORK/g-srv.out"; then + pass "missing manifest with server code is advisory (exit 0 + warning)" +else + fail "missing manifest with server code must be advisory, not failing (rc=$rc)" +fi + +echo "=== hypatia [4/5] findings gate ===" + +# Fixture: one critical finding. +cat > "$WORK/findings-critical.json" <<'JSON' +[{"rule_module":"demo","severity":"critical","type":"vuln","file":"src/x.rs","reason":"fixture"}] +JSON +expect_exit "advisory mode does not gate on critical" 0 "$HYPATIA" "$WORK/findings-critical.json" +expect_exit "--block refuses critical findings" 1 "$HYPATIA" "$WORK/findings-critical.json" --block + +# Fixture: one HIGH finding only. +cat > "$WORK/findings-high.json" <<'JSON' +[{"rule_module":"demo","severity":"high","type":"vuln","file":"src/x.rs","reason":"fixture"}] +JSON +expect_exit "--block alone tolerates high-only findings" 0 "$HYPATIA" "$WORK/findings-high.json" --block +expect_exit "--block --high-too refuses high findings" 1 "$HYPATIA" "$WORK/findings-high.json" --block --high-too + +# Fixture: clean scan blocks nothing. +printf '[]\n' > "$WORK/findings-clean.json" +expect_exit "clean findings never block" 0 "$HYPATIA" "$WORK/findings-clean.json" --block --high-too + +# Fixture: malformed findings must fail LOUDLY (exit 2), never silently pass. +printf 'this is not json\n' > "$WORK/findings-broken.json" +expect_exit "malformed findings fail loudly" 2 "$HYPATIA" "$WORK/findings-broken.json" --block + +echo "=== hypatia [5/5] workflow wiring (structural honesty) ===" + +if grep -q 'HYPATIA_BLOCK_ON_HIGH' "$HYPATIA_WF"; then + pass "hypatia-scan wires the HYPATIA_BLOCK_ON_HIGH opt-in gate" +else + fail "hypatia-scan.yml MUST wire the HYPATIA_BLOCK_ON_HIGH gate" +fi +if grep -q 'scripts/check-hypatia-findings.sh' "$HYPATIA_WF"; then + pass "hypatia-scan runs the shared findings gate script" +else + fail "hypatia-scan.yml MUST call scripts/check-hypatia-findings.sh" +fi +if grep -q 'ADVISORY — does not gate' "$HYPATIA_WF"; then + pass "hypatia advisory step is labelled ADVISORY — does not gate" +else + fail "hypatia-scan.yml advisory step lost its honesty label" +fi + +echo +if [ "$failures" -gt 0 ]; then + echo "CI HONESTY: $failures proof(s) FAILED — a gate cannot fail when it should (or fails when it must not)." + exit 1 +fi +echo "CI HONESTY: all proofs passed — every gate can fail and every advisory is labelled." +exit 0