Skip to content

Required status checks name each matrix leg, so adding a Julia version silently leaves it ungated #6

Description

@sotashimozono

Branch protection on main requires six contexts, and four of them name a single matrix leg:

version / check-version
format / format-check
julia 1.11 — ubuntu-latest
julia 1.12 — ubuntu-latest
julia 1.12 — windows-latest
Docs build

The trap

Adding a Julia version to the matrix in .github/workflows/CI.yml does not add it to the
gate. The new leg runs, can fail, and the PR stays mergeable — silently, because nothing
connects the workflow to the protection settings. Renaming a leg is worse: the required context
then never reports, and main becomes permanently unmergeable until someone edits the
protection by hand.

TestShards.jl avoids this by requiring one aggregate context, test / All shards passed, and
letting the matrix change underneath it.

Options

  1. Aggregate gate job. Add a job that needs the legs that must pass and require only that
    context. The matrix then changes freely. Costs one extra job per run, and the job has to
    needs a subset of the matrix — see below — so the matrix likely has to split in two.
  2. Leave it, and write the coupling down. A comment in CI.yml saying that changing a job
    name or adding a version means editing branch protection. Cheaper, and honest, but it is a
    comment guarding a setting in a different system.

Deliberately excluded, and why

Not currently required, and this should stay true under either option:

  • julia nightly — ubuntu-latestcontinue-on-error, allowed to fail.
  • julia 1.12 — macos-latest — sat in the free-tier queue for roughly 50 minutes on
    2026-09-01. Requiring it would block every merge for that long. It still runs, and it has
    passed on every run so far.

Both exclusions mean a regression on nightly or macOS can merge. That is the accepted cost; it
is recorded here so it is a decision rather than an oversight.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions