Skip to content

Harden CI against transient Bazelisk setup failures#264

Merged
tameware merged 7 commits into
developfrom
copilot/fix-failing-build-and-test
Jul 24, 2026
Merged

Harden CI against transient Bazelisk setup failures#264
tameware merged 7 commits into
developfrom
copilot/fix-failing-build-and-test

Conversation

Copilot AI commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

build_and_test failed before any Bazel build/test work due to a timeout in bazel-contrib/setup-bazel (api.github.com connect timeout). This change makes Bazelisk setup resilient to transient network failure in the macOS CI path.

  • Workflow resilience update (.github/workflows/ci_macos.yml)

    • Added a two-attempt Bazelisk setup sequence in build_and_test.
    • First setup attempt is allowed to fail; second attempt runs only when the first did not succeed.
    • Second attempt is strict (fails job if unsuccessful).
  • Fail-fast verification

    • Added an explicit post-setup check to ensure bazelisk is actually available before fetch/build/test steps.
    • Emits a clear failure message when setup still does not produce a usable Bazelisk binary.
- name: Setup Bazelisk (attempt 1)
  id: setup_bazelisk_1
  continue-on-error: true
  uses: bazel-contrib/setup-bazel@0.19.0
  with:
    bazelisk-version: "1.29.0"

- name: Setup Bazelisk (attempt 2)
  if: steps.setup_bazelisk_1.outcome != 'success'
  continue-on-error: false
  uses: bazel-contrib/setup-bazel@0.19.0
  with:
    bazelisk-version: "1.29.0"

- name: Verify Bazelisk installation
  run: bazelisk version || (echo "Failed to install Bazelisk after two setup attempts, or Bazelisk is not available on PATH." && exit 1)

Copilot AI changed the title [WIP] Fix failing GitHub Actions job build_and_test Harden macOS build_and_test against transient Bazelisk setup failures Jul 23, 2026
Copilot AI requested a review from tameware July 23, 2026 17:19

@tameware tameware left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hope it will help!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves GitHub Actions CI resilience by retrying Bazelisk setup and adding an explicit “tool is installed” verification step before running Bazel fetch/build/test work.

Changes:

  • Add a two-attempt bazel-contrib/setup-bazel (Bazelisk) installation sequence with a conditional second attempt.
  • Add a post-setup verification step intended to fail fast when Bazelisk is not usable.
  • Apply the same pattern across macOS and Linux CI workflows/jobs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
.github/workflows/ci_macos.yml Adds two-attempt Bazelisk setup plus a verification step in macOS build_and_test and sanitizers jobs.
.github/workflows/ci_linux.yml Adds two-attempt Bazelisk setup plus a verification step across Linux build_and_test, asan, and tsan jobs.
Comments suppressed due to low confidence (2)

.github/workflows/ci_linux.yml:118

  • The workflow verifies bazelisk, but subsequent steps in this job use the bazel command (bazel fetch/test). To fail fast on a missing toolchain, verify bazel is available (or verify both bazel and bazelisk).
      - name: Verify Bazelisk installation
        run: bazelisk version || (echo "Failed to install Bazelisk after two setup attempts, or Bazelisk is not available on PATH." && exit 1)

.github/workflows/ci_linux.yml:179

  • The workflow verifies bazelisk, but subsequent steps in this job use the bazel command (bazel fetch/test). To fail fast on a missing toolchain, verify bazel is available (or verify both bazel and bazelisk).
      - name: Verify Bazelisk installation
        run: bazelisk version || (echo "Failed to install Bazelisk after two setup attempts, or Bazelisk is not available on PATH." && exit 1)

Comment thread .github/workflows/ci_linux.yml Outdated
Comment thread .github/workflows/ci_linux.yml Outdated
@tameware tameware changed the title Harden macOS build_and_test against transient Bazelisk setup failures Harden CI against transient Bazelisk setup failures Jul 23, 2026
@tameware
tameware marked this pull request as ready for review July 23, 2026 18:23
@tameware
tameware requested a review from zzcgumn July 23, 2026 18:23

@zzcgumn zzcgumn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an improvement which I am happy to merge. Is there a way to move the duplicated code to a separate script that is invoked from two places?

Deduplicate the retry-and-verify Bazelisk install across CI workflows so version and retry policy live in one place.

Co-authored-by: Cursor <cursoragent@cursor.com>
@tameware

Copy link
Copy Markdown
Collaborator

Is there a way to move the duplicated code to a separate script that is invoked from two places?

Possible and desirable. Done. This also highlighted that the change was missing from Windows and WASM builds. It's now everywhere. All four CI workflows call it with:

- name: Setup Bazelisk
  uses: ./.github/actions/setup-bazelisk

That’s 7 call sites (Linux ×3, macOS ×2, Windows, WASM).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

.github/workflows/ci_linux.yml:32

  • The PR description still frames the resilience change as macOS-specific, but this PR updates all CI workflows (Linux/Windows/WASM/macOS) to use the new ./.github/actions/setup-bazelisk composite action. Please update the PR description to reflect the broader scope (or, if unintended, revert the non-macOS workflow changes).
      # 3️⃣ Install Bazelisk (recommended Bazel launcher)
      - name: Setup Bazelisk
        uses: ./.github/actions/setup-bazelisk

Comment thread .github/actions/setup-bazelisk/action.yml
tameware and others added 2 commits July 24, 2026 14:18
Avoid a Bash-only dependency on Windows runners while keeping one composite action for all CI platforms.

Co-authored-by: Cursor <cursoragent@cursor.com>
pwsh is not always available on Linux runners after free-disk-space, so pick the verify shell per OS in the shared action.

Co-authored-by: Cursor <cursoragent@cursor.com>
@tameware
tameware merged commit 8a182c1 into develop Jul 24, 2026
7 checks passed
@tameware
tameware deleted the copilot/fix-failing-build-and-test branch July 24, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants