ci: parallelize game builds with a per-game matrix#95
Merged
Conversation
The publish and validate workflows built every changed game serially in a single job's `while read` loop, so a mass update (a 14-game art release, or a kit autobump touching every game) paid the sum of all per-game build+conformance times. On a recent 14-game validate run that was ~15m43s of wall-clock, ~14m20s of it in the serial loop — and one heavy game (floorfall, a ~5.5min TinyGo build) alone blocked the other 13 behind it. Split each workflow into three jobs: - changed: computes the changed game dirs BEFORE any game code runs (git diff + the maintainer-permission gate + the static dir validator) and emits a compact JSON array for the matrix. This preserves the property that the work list — and publish's release DIRS — is decided by trusted code only. - build: one matrix leg PER GAME (fail-fast: false), each compiling untrusted community code on its OWN read-only, secret-less runner. Wall-clock becomes O(slowest single game) instead of O(sum). A malicious leg can no longer even observe a sibling game's build. - release / smoke-merge: unchanged trust boundary. publish's release job still holds the secrets and runs zero game code, now reading dirs from the changed job and downloading every leg's artifact (release-wasm-*, merge-multiple). validate's new smoke-merge job (no secrets, no game code) stitches the per-leg preview parts back into the single smoke-previews artifact + merged manifest.json, so smoke-preview.yml is untouched. Cache safety: the build legs set setup-go cache:false and add no actions/cache. They compile untrusted code, and the Go build cache also feeds the TRUSTED conformance CLI — a poisoned cache entry could subvert the checker gating the wasm and hand a malicious game a false pass. The matrix's parallelism delivers the win without caching poisonable build output; each leg pays a cold ~30-40s kit-CLI compile, which is dwarfed by running all games concurrently. fail-fast: false means one bad game no longer blocks the others' publishes: the release job skips any game whose build leg failed (that leg is the red signal) and ships the rest. Also silenced a pre-existing SC2016 false positive (literal backticks in a printf format) so actionlint is fully clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mass-update pushes (kit autobumps, art waves like #94) currently build serially — today's 14-game validate ran 15m43s, 91% of it the per-game build+conformance loop, with a single slow game (floorfall, 5.5min TinyGo build) blocking the other 13.
Change: both
validate.ymlandpublish.ymlfan out to a per-game matrix build job (fail-fast: false— one bad game can't cancel the rest; publish skips failed legs so good games still ship). Wall-clock becomes O(slowest game), not O(sum): ~2× today's outlier-bound case, ~3.4× a typical mass update (~2.7min vs ~9min), degrading gracefully as the catalog grows.Security posture preserved and documented inline:
changedjob still computes the work list, maintainer gate, and static validation before any game code runs; it emits the matrix.persist-credentials: false, read-only, matrix dir passed viaenv(shell data, never spliced).cache: falseeverywhere. Toolchains stay pinned + checksum-verified per leg.smoke-mergejob stitching per-leg previews sosmoke-preview.ymlis untouched.Trade-off: ~14× the setup runner-minutes (parallel and cheap). Add
max-parallel: Nunderstrategyif billing needs a bound.Worth watching the first real matrix run after merge (per-leg artifact naming + merge-multiple is the newest moving part).
🤖 Generated with Claude Code