Repository navigation
[ci] DRY proto pipeline: shared scripts, composite action, pinned toolchain - #486
Merged
Merged
Conversation
… toolchain Introduces a toolchain-pinned, script-backed proto generation pipeline so macOS build, tests, and CodeQL all invoke the same generation logic: scripts/install-proto-tooling.sh - Installs pinned protoc + Go plugins (go install @Pinned version) + swift- protobuf (brew) + protoc-gen-grpc-swift-2 (built from grpc-swift-protobuf tag 2.0.0) on macOS; Go plugins + protoc on Linux. - --check mode verifies all tools are present without installing. scripts/generate-protos.sh go|swift|all - Single canonical protoc invocation for each language, repo-root-relative. - Fails fast with clear error + install hint if a tool is missing. Makefile - proto-gen-go / proto-gen-swift now delegate to generate-protos.sh. - build-mac / test-mac call proto-gen-swift before xcodebuild so fresh checkouts always have generated stubs. .github/actions/setup-proto/action.yml - Composite action: caches protoc-gen-grpc-swift-2, runs install script, optionally generates (generate: swift|go|all|none). .github/workflows/codeql.yml - Swift job uses setup-proto before xcodebuild — fixes Velocity_Visualiser_V1_* symbol resolution failures on clean runners. .github/workflows/mac-ci.yml - setup job uses composite action instead of inline brew/clone steps. - Trigger paths include install/generate scripts and composite action. Docs (CLAUDE.md, COMMANDS.md, DEBUGGING.md) - Bootstrap command documented alongside make proto-gen. - Troubleshooting entry for missing Velocity_Visualiser_V1_* symbols. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR DRYs and stabilises the protobuf generation/tooling pipeline across local dev and CI, especially for the macOS Swift visualiser, by centralising tool installation and generation and reusing it via a composite GitHub Action.
Changes:
- Added
scripts/install-proto-tooling.sh(pinned proto/gRPC toolchain installer) andscripts/generate-protos.sh(canonical generation entrypoint). - Updated Makefile proto targets to delegate to the script and ensured
build-mac/test-macauto-generate Swift stubs. - Introduced
.github/actions/setup-protoand updated macOS CI + CodeQL Swift job to run shared proto setup + generation.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/install-proto-tooling.sh | Adds a single installer for protoc + Go plugins + Swift tooling (including building grpc-swift plugin). |
| scripts/generate-protos.sh | Adds canonical protoc invocation for Go/Swift/all generation. |
| Makefile | Replaces inline protoc commands with script delegation; ensures mac build/test generate Swift stubs. |
| .github/actions/setup-proto/action.yml | Adds composite action to install toolchain, cache grpc-swift binary, and run generation. |
| .github/workflows/mac-ci.yml | Switches mac CI setup to the composite action and updates trigger paths. |
| .github/workflows/codeql.yml | Ensures Swift CodeQL job installs toolchain and generates stubs before xcodebuild. |
| DEBUGGING.md | Documents new bootstrap + regen workflow and missing-symbol troubleshooting. |
| COMMANDS.md | Documents one-time toolchain bootstrap and clarifies proto-gen targets. |
| CLAUDE.md | Updates developer bootstrap instructions to include pinned proto tooling install. |
| pnpm-lock.yaml | Aligns lockfile specifier with root package.json pin for Prettier. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
…ereq checks, --force - Remove unused GRPC_SWIFT_BIN variable (install always targets /usr/local/bin directly) - Clarify header: Go plugins are pinned by module version; protoc and swift-protobuf are Homebrew latest (no formula-version pinning available) - Add explicit `need git` / `need swift` checks with actionable error messages before git clone / swift build - Add --force flag to force reinstall of protoc-gen-grpc-swift-2 even when a binary is already on PATH; default behaviour is unchanged Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces hardcoded '2.0.0' in the grpc-swift cache key with
hashFiles('scripts/install-proto-tooling.sh') so the cache busts
automatically whenever the install script changes (including version
bumps), with no manual key update required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Swift stubs (tools/visualiser-macos/.../Generated/) are gitignored; Go stubs (internal/lidar/l9endpoints/pb/) are committed. The previous wording "Generated files are gitignored" implied both, which was wrong. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SHA-pin actions/setup-go@v5 → @d35c59ab... # v5.5.0 in codeql.yml to fix the action-pin CI check (same SHA used by go-ci.yml and nightly-full-ci.yml). Add 'Setup Go' step (go-version-file: go.mod) to mac-ci.yml setup job before the setup-proto composite action: the install script calls `go install` for the protoc-gen-go plugins and was failing with 'go not found' on the macOS runner which has no Go by default. All Go CI now uses go-version-file: go.mod — no hardcoded version strings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…licon On GitHub-hosted ARM macOS runners the CodeQL action launches job steps under Rosetta (x86_64 emulation). Homebrew's native ARM prefix (/opt/homebrew) then refuses to install with: "Cannot install under Rosetta 2 in ARM default prefix" Detect arm64 at runtime and prefix the brew invocation with `arch -arm64` so the formula install runs natively regardless of the parent shell's architecture mode. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/install-proto-tooling.sh— single place for all tool version pins; installs protoc, Go plugins, swift-protobuf, and protoc-gen-grpc-swift-2 (built fromgrpc-swift-protobuftag2.0.0)scripts/generate-protos.sh go|swift|all— canonical protoc invocation, extracted from the Makefileproto-gen-go/proto-gen-swiftnow delegate to the script (removes duplicated inlineprotoccalls)make build-macandmake test-maccallmake proto-gen-swiftbefore xcodebuild — fresh checkouts work without a manual generation step.github/actions/setup-proto/action.yml— composite action that caches the grpc-swift binary, runs the install script, and optionally runs generationcodeql.ymlSwift job uses the composite action before xcodebuild — fixesVelocity_Visualiser_V1_*symbol resolution failures on clean runnersmac-ci.ymlsetup job uses the composite action instead of inline brew/clone steps; trigger paths include the new scripts and actionAcceptance criteria satisfied
make proto-gen,make proto-gen-go,make proto-gen-swiftall route throughscripts/generate-protos.shmake build-macandmake test-maccallmake proto-gen-swiftautomaticallyscripts/install-proto-tooling.sh)./scripts/install-proto-tooling.sh && make build-macprotoccommand lines remain in multiple placesTest plan
codeql.ymlSwift job completes without missing-symbol build errors (requires [ci] swift CodeQL workflow config #484 merged and default CodeQL setup disabled in repo settings)mac-ci.ymlsmoke build passes (setup job uses composite action, proto artifact still uploaded)make proto-genlocally produces identical output to beforescripts/install-proto-tooling.sh --checkexits 0 on a machine with tools installed🤖 Generated with Claude Code