Skip to content

Validation now runs the pull request's own tools/ - #1994

Merged
andrewboudreau merged 1 commit into
mainfrom
validator-runs-pr-tools-docs
Aug 30, 2026
Merged

Validation now runs the pull request's own tools/#1994
andrewboudreau merged 1 commit into
mainfrom
validator-runs-pr-tools-docs

Conversation

@andrewboudreau

Copy link
Copy Markdown
Collaborator

The validator used to check out all of tools/ from the base commit after making the test merge, so no pull request could exercise its own tool change. Worse, tools/ was not in the scope that decides whether a PR changed anything, so a tools-only PR got noverify: no source/build-data changes in this PR and a green check that had measured nothing — #1986 is exactly that.

Both halves are fixed in sm64ds-validator. The committed test merge runs whole, tools/ included, and a tools-only PR is now a full base-vs-merge ROM comparison. This PR is only the two places in this repo that documented the old behaviour.

.github/workflows/pr-validate.yml — the pull_request_target safety argument is still correct about this workflow (it never checks out or runs PR code), but "sandboxed there" undersold what the worker does. It runs the branch's tools/ on purpose. That happens in the worker's container, never on a GitHub runner and never with these secrets — but a tools/ diff on a fork PR is worth reading before approving.

notes/plan-cpp-conversion-queue.md §7 — said not to let a batch's correctness depend on a tool the validator will delete. It no longer will: a batch ships tu_create.py / tu_preflight.py and is validated by them.

Three things a merge still may not decide, all unchanged or newly enforced:

tools/mwccarm/, extracted/ refused outright — the compiler and the bytes every verdict is compared against are the operator's, gitignored here, never committed
config/rombuild-versions.txt validated against the installed compilers, never substituted
config/arm9/config.yaml, config/rombuild-exclude.txt still restored from base, so a PR cannot narrow what is built and compared

No code changes — comments and notes only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S7zcPKb3LNpLAJq4dH8NRf

The worker used to check out all of tools/ from the base commit after making
the test merge, so no pull request could exercise its own tool change -- and
a tools-only PR was not even counted as a change, so it got "noverify" and a
green check that had measured nothing. Both halves are fixed in
sm64ds-validator; the merge tree runs whole, tools/ included.

Two notes here were describing the old world:

  - pr-validate.yml's pull_request_target safety argument is still correct
    about THIS workflow (it never checks out or runs PR code), but "sandboxed
    there" undersold what the worker does. It runs the branch's tools on
    purpose. That is in its container, never on a runner and never with these
    secrets -- but a tools/ diff on a fork PR is worth reading before approving.
  - the C++ conversion plan's Tooling caveat said not to let a batch's
    correctness depend on a tool the validator will delete. It no longer will.
    A batch ships tu_create.py / tu_preflight.py and is validated by them, and
    a byte-neutral tool change now gets a real base-vs-merge ROM comparison
    instead of a no-op pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S7zcPKb3LNpLAJq4dH8NRf
@tangos-validator

tangos-validator Bot commented Aug 30, 2026

Copy link
Copy Markdown

✅ PR validation — Passed

noverify: no source/build-data changes in this PR

Each changed src/*.c|*.cpp is compiled and its relocated bytes compared to the binary data on a private build box. Passing requires every changed file to reproduce the ROM byte-for-byte with correct relocation targets — this catches WRONG-DEST relocations and non-reproducing near-misses that ledger-scoped linkcheck skips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant