Run benchmark workflows on pull_request instead of pull_request_target#4664
Open
tautschnig wants to merge 2 commits into
Open
Run benchmark workflows on pull_request instead of pull_request_target#4664tautschnig wants to merge 2 commits into
tautschnig wants to merge 2 commits into
Conversation
actions/checkout v6.1.0 (published 2026-07-20, with backports v5.1.0 and v4.4.0) refuses to check out fork pull-request code from workflows running in the trusted pull_request_target context: Refusing to check out fork pull request code from a 'pull_request_target' workflow. [...] Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. Since the floating v6 tag moved to v6.1.0, the label-gated benchmark jobs (invoked from extra_jobs.yml, which runs on pull_request_target because the auto-labeler needs write permissions) fail at their checkout step for every pull request from a fork: they check out and then build and execute the proposed code, which is exactly the pattern the new default blocks. Pushes to main are unaffected since the same workflows then compare trusted before/after refs. Rather than opting out via allow-unsafe-pr-checkout, move the benchmark invocations to a new workflow triggered by plain pull_request events, where fork code runs in the untrusted, fork-scoped context as GitHub recommends. Gating still uses the Z-EndToEndBenchCI and Z-CompilerBenchCI labels, now read from the event payload instead of the labeler job's outputs. One behavioral difference: labels added by the auto-labeler do not re-trigger workflows (events created with the default GITHUB_TOKEN never do), so on freshly opened pull requests the benchmarks only start with the next push to the pull request, or when a label is added manually. Subsequent pushes to labeled pull requests (the common case) behave as before. Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR moves the label-gated benchmark runs off the trusted pull_request_target path (used for auto-labeling) and onto a dedicated pull_request-triggered workflow, aligning with the new actions/checkout safety behavior and avoiding running fork PR code in a trusted context.
Changes:
- Remove benchmark reusable-workflow invocations from
extra_jobs.yml, leaving only the minimal auto-labeler onpull_request_target. - Add
bench-pr.ymlto run the benchmark reusable workflows onpull_requestevents when the correspondingZ-*BenchCIlabel is present. - Simplify PR-ref selection logic in the benchmark reusable workflows (but see below: the current condition breaks
workflow_calluse).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| .github/workflows/extra_jobs.yml | Stops invoking benchmark workflows from the trusted pull_request_target labeler workflow. |
| .github/workflows/bench-pr.yml | New pull_request workflow that label-gates and calls the reusable benchmark workflows with concurrency cancellation. |
| .github/workflows/bench-e2e.yml | Updates PR ref-selection condition for setting OLD_REF/NEW_REF (currently problematic under workflow_call). |
| .github/workflows/bench-compiler.yml | Same ref-selection condition update in both compiler benchmark jobs (currently problematic under workflow_call). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
In reusable workflows, github.event_name and github.event are those of
the caller workflow ('workflow_call' is never a runtime event name), so
the pull_request condition holds when invoked from bench-pr.yml. Add a
comment to preempt confusion.
Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
Member
Author
|
We need to get the changes in #4666 merged first to unblock the CI jobs. |
Contributor
|
@tautschnig I just approved #4666 and got it into the merge queue. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since 2026-07-20, the label-gated benchmark jobs (
End-to-End Benchmarks / perf-benchcomp,Compiler Benchmarks / compile-timer-{short,long}) fail within seconds on every pull request from a fork, while pushes tomainstay green.Root cause: actions/checkout v6.1.0 was published on 2026-07-20 (with coordinated backports v5.1.0 and v4.4.0) and made the floating
v6tag refuse to check out fork PR code from workflows running in the trustedpull_request_targetcontext (changelog):Our benchmark workflows are invoked via
workflow_callfromextra_jobs.yml, which runs onpull_request_target(the auto-labeler needspull-requests: write). They check outgithub.event.pull_request.head.sha— fork code — and then build and execute it, which is precisely the pattern the new default blocks. Pushes tomainrun the same workflows via thepushtrigger comparing trustedbefore/afterrefs, hence unaffected.Fix: rather than opting out via
allow-unsafe-pr-checkout: true, move the benchmark invocations to a newbench-pr.ymlworkflow triggered by plainpull_requestevents, so fork code builds and runs in the untrusted, fork-scoped context as GitHub recommends. Label gating (Z-EndToEndBenchCI/Z-CompilerBenchCI) is preserved, now reading the labels from the event payload instead of the labeler job's outputs. The event-name conditions inside the reusable workflows are simplified accordingly, andextra_jobs.ymlkeeps only the auto-labeler.Behavioral difference: labels added by the auto-labeler do not re-trigger workflows (events created with the default
GITHUB_TOKENnever do), so on freshly opened PRs the benchmarks only start with the next push, or when someone adds the label manually (human-added labels do trigger via thelabeledevent type). Subsequent pushes to labeled PRs — the common case — behave exactly as before. Aconcurrencygroup cancels superseded benchmark runs.Validated with actionlint 1.7.7 (clean) on all four touched workflow files.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.