Skip to content

Add optional multi-version Python test matrix#26

Open
leynos wants to merge 1 commit into
mainfrom
add-python-matrix-testing
Open

Add optional multi-version Python test matrix#26
leynos wants to merge 1 commit into
mainfrom
add-python-matrix-testing

Conversation

@leynos

@leynos leynos commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

Adds an optional multi-version Python test matrix to the template, selectable at render time, modelled on the estate's existing practice:

  • femtologgingbuild-test matrix over 3.12/3.13/3.14 with a 3.15 experimental lane excluded from required checks;
  • falcon-correlate — lint job plus a test matrix (3.12, 3.13, 3.14);
  • cuprumtypecheck-test legs plus a 3.15a experimental leg with continue-on-error: true.

The template's CI is a single lint-test job, so the change follows cuprum's shape: lint-test is untouched and a new typecheck-test matrix job is added when the toggle is on.

Toggle semantics

New copier.yml boolean multi_python_tests (default false). Help text notes it suits libraries for external consumption; the estate baselines libraries at 3.12 and tests 3.12–3.14. No hidden computed variable was needed — the version list is derived inline in ci.yml.jinja.

Version derivation

The matrix is derived in Jinja from the existing python_version answer: one stable leg per minor version from range(python_version minor, 15) (so a 3.10 baseline yields 3.10–3.14; a 3.14 baseline yields just 3.14), plus a final experimental leg python-version: '3.15', labelled 3.15a per cuprum, with allow-prereleases: true.

Experimental lane is non-gating

The job carries continue-on-error: ${{ matrix.experimental }} and strategy.fail-fast: false; legs are named Typecheck and test (Python <label>) so required-check names stay predictable and the 3.15a leg can be left out of branch protection.

Design notes

  • Lint, make audit, Makefile validation, and coverage (plus the CodeScene upload) stay single-legged in lint-test, matching the models. Each matrix leg runs make build, make typecheck, and make test only. Note the 3.13-ish leg re-runs tests that lint-test also runs via the coverage action — cuprum accepts the same duplication.
  • With use_rust, every leg sets up the Rust toolchain (shared setup-rust at the template's existing pin) and Swatinem/rust-cache (which keys on matrix values automatically), because make build compiles the PyO3 extension per interpreter — mirroring cuprum and femtologging's per-leg toolchain setup.
  • The experimental 3.15 leg may fail to build the PyO3 extension against a prerelease interpreter; that is precisely what the non-gating lane is for.
  • Action pins, ${{ "{{" }} … {{ "}}" }} escaping, and indented-tag trailing-whitespace conventions match the existing ci.yml.jinja. The outer toggle uses Jinja whitespace control so the off-state output carries no residue.

Validation

  • Rendered all four toggle combinations (matrix × Rust) plus a python_version=3.14 reduced-matrix case with uvx copier copy.
  • Matrix-off renders are byte-identical to renders from main for both Rust states (verified with diff).
  • Every rendered ci.yml YAML-parses and passes actionlint cleanly.
  • Full template contract suite: 38 passed, 2 skipped (act-gated), 3 snapshots passed — including five new parametrized contract cases covering both toggle states, both Rust states, and the reduced 3.14-baseline matrix.
  • The developers' guide documents the new behaviour.

🤖 Generated with Claude Code

Generated repositories can now opt into CI testing across multiple
Python versions via a new multi_python_tests boolean answer (default
false). When enabled, the generated CI workflow gains a typecheck-test
matrix job modelled on the estate's library repositories
(femtologging, falcon-correlate, and cuprum): one leg per Python minor
version from the project's python_version baseline through 3.14, plus
an experimental 3.15 lane that installs prerelease interpreters and
sets continue-on-error so it never gates merges. Lint, audit, and
coverage stay single-legged in lint-test, and Rust-enabled variants
set up the toolchain in every leg because make build compiles the
extension.

When the toggle is off, the rendered CI workflow is byte-identical to
the previous output. Contract tests cover both toggle states and a
reduced matrix for a 3.14 baseline, and the developers' guide
documents the behaviour.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 6 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3557d6fe-a9a1-4147-a868-0150c8419876

📥 Commits

Reviewing files that changed from the base of the PR and between 09a4d88 and beb5022.

📒 Files selected for processing (6)
  • copier.yml
  • docs/developers-guide.md
  • template/.github/workflows/ci.yml.jinja
  • tests/helpers/ci_contracts.py
  • tests/helpers/tooling_contracts.py
  • tests/test_template.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch add-python-matrix-testing

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant