Block only the vulnerabilities a merge adds

.claude/skills/codetrial-triage/pulls.md

@@ -335,15 +335,20 @@ says why it reads the whole log rather than `--log-failed`, and which markers
 count. A check outside Actions, cubic among them, comes back with its link
 only; read it from the PR's checks tab.
 
-A `cargo-audit` advisory that `main` also has is the maintainer's, and the PR
-must not carry a lockfile bump to clear it: #304 was told it "needs nothing from
-you", and #305 to drop its unrelated `Cargo.lock` change. Once `main` lands the
+For historical runs with `cargo-audit` in the portable lane, an advisory that
+`main` also has is the maintainer's, and the PR must not carry a lockfile bump
+to clear it: #304 was told it "needs nothing from you", and #305 to drop its
+unrelated `Cargo.lock` change. Once `main` lands the
 fix (#331 moved to zip 9), a PR still failing on that advisory has a stale base,
 and the sweep puts it in `rebase` under the same limits as a conflicting one,
 so a request in any wording on its head, such as #326's own, keeps it out. A
 PR that touches a Cargo file is never a stale base: read whether its lockfile
 change brought the advisory before asking for anything.
 
+The independent `audit` job compares the actual merge against its base tip;
+inherited findings do not fail it. Read its summary for newly introduced
+vulnerabilities or an incomplete scan instead of classifying it as stale.
+
 A clean merge does not establish readiness. For a dependent draft, check whether
 its prerequisite merged and whether the author still needs to rebase, remove
 prerequisite commits, or mark it ready. Compare overlapping PRs by behavior as

.claude/skills/codetrial-verify/SKILL.md

@@ -15,12 +15,16 @@ make check                # every gate, plus a live Gemini credential check
 
 CI runs one `gates (<lane>)` job per lane in parallel: `gates (rust)`,
 `gates (portable)` and `gates (browser)`. The `check` job only collects their
-results and the mutation jobs', so its log does not say what failed: open the
-failing lane, or, when every lane is green, the failing `plan-mutants` job or
+results, the independent `audit` job and the mutation jobs', so its log does
+not say what failed: open the failing lane, `audit`, `plan-mutants` job or
 `mutants` shard. On a pull request the portable lane checks the commit
 messages first, and a bad message skips the rest of that lane only. A gate
 names its lane where it is declared, one naming none is portable, and a test
-proves the three lanes together cover the full run.
+proves the three lanes plus the independent audit cover the full run.
+`cargo-audit` runs only in the default local gate; CI uses `scripts/audit.py`
+for merge-relative PR checks and full main checks. See
+[dependency security checks](../../../docs/development.md#dependency-security-checks).
+Its regression suite is `tests/test_audit.py`.
 Reproduce a failure with the same lane locally.
 
 `scripts/test.sh` is a list of named gates that all run before it reports, so
@@ -149,16 +153,22 @@ What this adds is one call that reads all of it at the PR's head:
 Read its `failing` entries in this order. `current` false is history, not the
 current failure: say so and stop there. `current` or `on_main` null means its
 lookup failed, and `gates`, `errors` and `excerpt` null mean the job log could
-not be read; check either with the raw calls below. `on_main` true means the latest
-finished run on `main` fails the same job, so the failure is the maintainer's
-and the author is not asked to fix it. Otherwise `gates` names what
-`scripts/test.sh` failed and `errors` the lines that say why; rerun that lane
-locally with the same head. A `cargo-audit` failure on a PR whose base predates
-the advisory fix is a stale base, answered by a rebase (#334), not a code
-change; `stale_base` (defined in the script's header) says exactly that, and
-the PR sweep lists such a PR in `rebase` on its own. The aggregate `check` job
-only collects results, so its entry points at the failing job it collected,
-as in the lanes section above. A check outside Actions has only its link, and
+not be read; check either with the raw calls below. `on_main` true means the
+latest finished run on `main` fails the same job. Read whether the cause is
+shared before asking the author for a fix: the PR `audit` compares added
+findings, while main runs a full scan, so both can fail for different reasons.
+Otherwise `gates` names what `scripts/test.sh` failed and `errors` the lines
+that say why; rerun that lane locally with the same head. For historical runs
+that still put `cargo-audit` in the portable lane, a failure on a PR whose base
+predates the advisory fix is a stale base, answered by a rebase (#334), not a
+code change; `stale_base` (defined in the script's header) says exactly that,
+and the PR sweep lists such a historical PR in `rebase` on its own.
+The independent `audit` job already compares the proposed merge with the base
+tip: inherited findings are informational, while added findings and incomplete
+dependency-change scans require investigation rather than an automatic rebase.
+The aggregate `check` job only collects results, so its entry points at the
+failing job it collected, as in the lanes section above. A check outside
+Actions has only its link, and
 `held` checks have no log until a maintainer approves the run.
 
 The raw calls behind it, when one answer needs checking:

.claude/skills/codetrial-verify/ci-failure.sh

@@ -20,7 +20,9 @@
 #               Actions, its link only. current and on_main are null when the
 #               lookup behind them failed, and gates, errors and excerpt when
 #               the job log could not be read.
-#   stale_base  true when every failing lane failed only gates that read
+#   stale_base  legacy portable-audit runs only; the independent audit job
+#               reports inherited findings without failing. True when every
+#               failing lane failed only gates that read
 #               outside data (the table in the last jq below: cargo-audit
 #               reads the advisory database), on the current head, main's
 #               latest run started after those runs and succeeded in each of

.github/actions/install-audit/action.yml

@@ -0,0 +1,14 @@
+name: Install cargo-audit
+description: Install the pinned, digest-checked RustSec release binary.
+runs:
+  using: composite
+  steps:
+    - shell: sh
+      env:
+        GH_TOKEN: ${{ github.token }}
+        CARGO_AUDIT_VERSION: 0.22.2
+      run: |
+        name="cargo-audit-x86_64-unknown-linux-musl-v$CARGO_AUDIT_VERSION"
+        ./scripts/install-release-binary.sh rustsec/rustsec \
+          "cargo-audit/v$CARGO_AUDIT_VERSION" "$name.tgz" "$name/cargo-audit"
+        cargo audit --version | grep -qxF "cargo-audit-audit $CARGO_AUDIT_VERSION"

.github/workflows/audit.yml

@@ -0,0 +1,76 @@
+name: security audit
+
+on:
+  schedule:
+    - cron: '43 4 * * *'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: security-audit
+
+jobs:
+  audit:
+    # Only trusted main code may update the repository's tracking issue.
+    if: github.ref == 'refs/heads/main'
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+    permissions:
+      contents: read
+      issues: write
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Install cargo-audit
+        uses: ./.github/actions/install-audit
+      - name: Audit main
+        id: audit
+        run: python3 scripts/audit.py --report "$RUNNER_TEMP/audit.md"
+      - name: Track remediation
+        if: ${{ !cancelled() && steps.audit.outputs.findings != '' }}
+        env:
+          GH_TOKEN: ${{ github.token }}
+          GH_REPO: ${{ github.repository }}
+          FINDINGS: ${{ steps.audit.outputs.findings }}
+        run: |
+          title='Dependency security audit'
+          fingerprint() { sed -n '/^Findings fingerprint:/p'; }
+          # gh refuses --slurp with --jq, so filter the slurped pages with jq.
+          # Closed issues count too: reopening keeps existing subscribers.
+          found=$(gh api --paginate --slurp --method GET "repos/$GH_REPO/issues" \
+            -f state=all -f 'creator=github-actions[bot]' -f per_page=100 \
+            | jq -r --arg title "$title" '[.[][] | select(.pull_request == null and .user.login == "github-actions[bot]" and .title == $title)][0] // empty | "\(.number) \(.state) \(.closed_at // "")"')
+          read -r issue state closed <<< "$found"
+          if [ "$FINDINGS" != true ]; then
+            # The clean body replaces the old fingerprint, so findings that
+            # return later count as a change and notify subscribers again.
+            if [ "$state" = open ]; then
+              gh issue edit "$issue" --body-file "$RUNNER_TEMP/audit.md"
+              gh issue close "$issue" --reason completed
+            fi
+            exit 0
+          fi
+          if [ -z "$issue" ]; then
+            gh issue create --title "$title" --body-file "$RUNNER_TEMP/audit.md"
+            exit 0
+          fi
+          current=$(fingerprint < "$RUNNER_TEMP/audit.md")
+          previous=$(gh issue view "$issue" --json body --jq .body | fingerprint)
+          # Keep the old fingerprint until notification succeeds so failures
+          # retry, but skip the comment if a failed edit left it posted. Only
+          # comments since a close count, so recurring findings notify again.
+          if [ "$previous" != "$current" ]; then
+            notified=$(gh api --paginate --slurp "repos/$GH_REPO/issues/$issue/comments" \
+              | jq -r --arg since "$closed" '[.[][] | select(.user.login == "github-actions[bot]" and .created_at > $since)] | last | .body // ""' \
+              | fingerprint)
+            if [ "$notified" != "$current" ]; then
+              gh issue comment "$issue" --body-file "$RUNNER_TEMP/audit.md"
+            fi
+          fi
+          gh issue edit "$issue" --body-file "$RUNNER_TEMP/audit.md"
+          if [ "$state" = closed ]; then
+            gh issue reopen "$issue"
+          fi

.github/workflows/check.yml

@@ -54,23 +54,51 @@ env:
   # install, and a guard skips the install whenever a cached binary exists:
   # bumping only the install line would leave the old key matching, the old
   # binary restored, and CI running a version nobody asked for.
-  CARGO_AUDIT_VERSION: 0.22.2
   CARGO_MUTANTS_VERSION: 27.1.0
 
-  # Pinned for the same reason the two above are: a linter that moves under you
+  # Pinned for the same reason as the one above: a linter that moves under you
   # turns an unrelated push red, and the fix is then a version bump nobody chose
   # to make. Kept in step with whatever contributors run locally, which is
   # whatever `ruff` is on their PATH; the gate skips rather than fails when they
   # have none.
   RUFF_VERSION: 0.15.2
 
-  # The workflow's own linter, pinned like the three above. Installed rather
+  # The workflow's own linter, pinned like the tools above. Installed rather
   # than assumed: `scripts/test.sh` skips this gate when the binary is absent,
   # and absent is what it was here, so the file that describes every other
   # check was the one file nothing checked.
   ACTIONLINT_VERSION: 1.7.12
 
 jobs:
+  audit:
+    if: github.event_name != 'schedule' && github.event_name != 'workflow_dispatch'
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 2
+          persist-credentials: false
+      - name: Install cargo-audit
+        uses: ./.github/actions/install-audit
+      - name: Audit the tested merge or main
+        env:
+          EVENT: ${{ github.event_name }}
+          GROUP_BASE: ${{ github.event.merge_group.base_sha }}
+        run: |
+          case "$EVENT" in
+            pull_request)
+              base=$(git rev-parse HEAD^1)
+              python3 scripts/audit.py --base "$base"
+              ;;
+            merge_group)
+              git cat-file -e "$GROUP_BASE^{commit}" || git fetch --no-tags --depth=1 origin "$GROUP_BASE"
+              python3 scripts/audit.py --base "$GROUP_BASE"
+              ;;
+            push) python3 scripts/audit.py ;;
+            *) exit 2 ;;
+          esac
+
   gates:
     name: gates (${{ matrix.lane }})
     if: github.event_name != 'schedule' && github.event_name != 'workflow_dispatch'
@@ -225,25 +253,6 @@ jobs:
         id: chromium
         uses: ./.github/actions/chromium
 
-      # The gate skips this audit when the binary is missing, which is right for
-      # a contributor's laptop and wrong for CI: a silent skip here means the
-      # dependency surface stops being checked and nothing says so. Installed
-      # ahead of the gate so the skip branch is never the one CI takes.
-      #
-      # The upstream release binary, checked against the digest its release
-      # records, rather than `cargo install`: that compiled the crate graph for
-      # minutes on every cache miss. musl, so the binary does not depend on the
-      # runner's glibc. The version check proves the pin is what installed.
-      - name: Install cargo-audit
-        if: matrix.lane == 'portable'
-        env:
-          GH_TOKEN: ${{ github.token }}
-        run: |
-          name="cargo-audit-x86_64-unknown-linux-musl-v$CARGO_AUDIT_VERSION"
-          ./scripts/install-release-binary.sh rustsec/rustsec \
-            "cargo-audit/v$CARGO_AUDIT_VERSION" "$name.tgz" "$name/cargo-audit"
-          cargo audit --version | grep -qxF "cargo-audit-audit $CARGO_AUDIT_VERSION"
-
       # The Python here generates the problem bank and drives two integration
       # harnesses, and `ruff check` is its only linter. The gate skips when the
       # binary is missing, which is right on a laptop and wrong here. pipx is on
@@ -366,7 +375,7 @@ jobs:
             target
           key: ${{ steps.cargo-cache.outputs.key }}
 
-  # The one status: every gate lane passed, and on a pull request the mutation
+  # The one status: every gate lane and audit passed, and on a PR the mutation
   # plan and every shard did too. Planning only runs on pull requests, and the
   # shards only after a plan that found mutants, so a skipped shard job after a
   # successful plan means the plan counted none. A push or a merge group plans
@@ -377,21 +386,26 @@ jobs:
   # reaches this job and fails it. Not on a schedule or a dispatch, which run
   # no gates.
   check:
-    needs: [gates, plan-mutants, mutants]
+    needs: [gates, audit, plan-mutants, mutants]
     if: ${{ !cancelled() && github.event_name != 'schedule' && github.event_name != 'workflow_dispatch' }}
     runs-on: ubuntu-24.04
     timeout-minutes: 5
     steps:
       - name: Require every gate lane and mutation shard
         env:
           GATES: ${{ needs.gates.result }}
+          AUDIT: ${{ needs.audit.result }}
           PLAN: ${{ needs.plan-mutants.result }}
           MUTANTS: ${{ needs.mutants.result }}
         run: |
           if [ "$GATES" != success ]; then
             echo "::error::gate lanes: $GATES. This job only collects them; the failed gates lane has the cause."
             exit 1
           fi
+          if [ "$AUDIT" != success ]; then
+            echo "::error::security audit: $AUDIT. See the audit job for new vulnerabilities or an incomplete scan."
+            exit 1
+          fi
           case "$MUTANTS/$PLAN" in
             success/success | skipped/success | skipped/skipped) ;;
             *) echo "::error::mutants: $MUTANTS (plan: $PLAN)"; exit 1 ;;
@@ -653,11 +667,11 @@ jobs:
   # compiles `web/` (including its WASM) into it. Native runners avoid a
   # cross-platform linker or packaging toolchain.
   #
-  # After `gates` rather than `check`: on a push `check` adds nothing, and the
-  # mutation jobs it waits on are skipped, which the default `success()` reads
+  # After `gates` and `audit` rather than `check`: on a push `check` adds
+  # nothing, and its mutation jobs are skipped, which the default `success()` reads
   # as a reason to skip everything downstream.
   build:
-    needs: gates
+    needs: [gates, audit]
     if: github.event_name == 'push' && github.ref == 'refs/heads/main'
     name: ${{ matrix.name }}
     timeout-minutes: 45
@@ -970,7 +984,7 @@ jobs:
   # One rolling release for the current main build. GitHub releases require a
   # tag; `latest` avoids exposing an implementation SHA as a release name.
   release:
-    needs: [gates, build]
+    needs: [gates, audit, build]
     if: github.event_name == 'push' && github.ref == 'refs/heads/main'
     runs-on: ubuntu-24.04
     timeout-minutes: 15

README.md

@@ -57,7 +57,7 @@ rather than guessing.
 | Phase | Required | Notes |
 |---|---|---|
 | Build time | Rust stable, `curl` or `wget`, and `sha256sum` or `shasum`; Linux also needs Clang 21+ and GLib development headers | Cargo resolves application crates from `Cargo.lock`. `make build` fetches the checksum-pinned browser assets on the first build. |
-| Test time | Build-time tools, Node.js 18+, and Python 3.9+ | Node runs browser and fixture checks; Python verifies generated problem-bank files. `npm ci` adds pinned Prettier, ESLint and Playwright for the full local browser gate. `ruff`, `shellcheck`, `shfmt`, `commentflow`, `actionlint`, and `cargo-audit` are optional: each gate reports that it skipped rather than failing without them. CI installs all of them, so those lanes are enforced on a pull request whatever a checkout can run; locally the `actionlint` lane also takes its container image. |
+| Test time | Build-time tools, Node.js 18+, and Python 3.9+ | Node runs browser and fixture checks; Python verifies generated problem-bank files. `npm ci` adds pinned Prettier, ESLint and Playwright for the full local browser gate. `ruff`, `shellcheck`, `shfmt`, `commentflow`, `actionlint`, and `cargo-audit` are optional: each gate reports that it skipped rather than failing without them; locally the `actionlint` lane also takes its container image. CI installs these tools and enforces its test lanes whatever a checkout can run. The independent [dependency security check](docs/development.md#dependency-security-checks) blocks newly introduced vulnerabilities on PRs and runs full audits on main. |
 | Runtime | The compiled `codetrial` binary and a config file | No Node.js, Python, or `node_modules` is required. Rust dependencies are compiled into the binary; browser dependencies are vendored, checksum-pinned, and embedded, so a `web/` directory is optional and only overrides what is already inside. |
 
 On Linux x86_64, `make` downloads the checksum-pinned WebRTC compiler into