ci: weekly grouped pip auto-bump-and-merge (docker OS/base image excluded)

[amd-vivekag]

Summary

Set up a weekly, grouped, ==-pinned pip auto-bump-and-merge so Python
dependencies are raised to their latest releases every week and land without
human review once required checks are green — while the docker OS/base image is
never auto-bumped.

• Pin requirements*.txt to == at the latest release that still supports
  the project's minimum Python (3.10) so Dependabot's weekly pip run produces a
  real diff (>= floors never move). pyproject.toml keeps its >= library
  floors; the TraceLens git+ VCS dep is left untouched.
• .github/dependabot.yml: pip → interval: weekly,
  open-pull-requests-limit: 1, grouped python-deps
  (applies-to: version-updates, patterns: ["*"]); remove the docker
  package-ecosystem block so the ROCm/Ubuntu OS/base image is never
  auto-bumped; github-actions stays weekly + human-merged.
• .github/workflows/dependabot-auto-merge.yml: for dependabot[bot] pip
  PRs, a validate-requirements matrix job installs requirements.txt +
  requirements-dev.txt across Python 3.10/3.11/3.12, then the merge step
  (needs: that job) enables GitHub auto-merge (squash). fetch-metadata
  is pinned to a commit SHA (this workflow has write perms + merges).
• docs/ci-nightly-eval.md / docker/Dockerfile.ci-gpu: document the
  weekly grouped pip auto-merge, the one-open-PR rule, the security-update
  exception, the files a bump PR touches / never touches, and the manual docker
  OS/base-image bump flow.

Behavior

• One grouped pip PR weekly; auto-merges (squash) after the requirements
  install job passes on 3.10/3.11/3.12 and the repo's other required checks
  (the aggregate CPU tests check and pre-commit) are green. No human review.
• Why the extra install job: no existing gate exercises requirements*.txt
  (the CPU gate installs pyproject.toml extras, pre-commit installs only
  pre-commit), so an incompatible exact pin could otherwise merge green.
• Security updates are the exception: Dependabot opens security-update PRs
  outside open-pull-requests-limit and the version-update group, so more than
  one pip PR can be open; those also auto-merge on green. Only version updates
  obey the one-PR/grouped contract.
• Only one version-update bump PR open at a time; Dependabot rebases it upward
  each week.
• A bump PR edits exactly requirements.txt (pyyaml, matplotlib,
  numpy, pandas, openpyxl, seaborn, beautifulsoup4, click) and
  requirements-dev.txt (pytest, pytest-cov, pre-commit,
  pytest-timeout, pytest-xdist, pytest-forked). It never edits
  pyproject.toml floors, docker/**, or config/ci/ci-constraints.txt.
• docker OS/base image: bump manually via
  docker buildx imagetools inspect rocm/pytorch:<tag> → update the FROM ...@sha256:... in docker/Dockerfile.ci-gpu → run Refresh baselines.

Relationship to #345

Not fully independent: bump-validate.yml (deleted by #345) triggers the
self-hosted GPU eval on any requirements*.txt change for same-repo PRs —
including Dependabot's — and can't run on a Dependabot PR (no ROCM_SHARED_KEY),
which would block auto-merge. Merge #345 first (or together). gpu-tests.yml
itself skips to Success since requirements*.txt isn't a GPU-relevant path.

Required repo settings (prerequisites)

• Allow auto-merge (Settings → General → Pull Requests).
• Actions: Read and write permissions + Allow GitHub Actions to create
  and approve pull requests (Settings → Actions → General).
• Branch protection on main: required status checks configured (CPU tests, pre-commit) with no blocking human review for pip bump PRs (or a
  Ruleset bypass for Dependabot). If approvals are still required with no bypass,
  auto-merge waits.

Review updates (f092769, addressing Copilot)

• Added the validate-requirements matrix job gating auto-merge.
• Fixed pins that broke Python 3.10: matplotlib 3.11.1→3.10.9,
  numpy 2.5.1→2.2.6, pandas 3.0.5→2.3.3 (3.11/2.3+/3.0 require ≥3.11).
• SHA-pinned dependabot/fetch-metadata; explicit applies-to: version-updates
  • documented the security-update exception; refreshed the Dockerfile.ci-gpu
    comment; corrected the required-check names and the ci: validate stack bumps nightly instead of on every PR raise #345 relationship.

Closes #346
Refs #303

[Copilot]

Pull request overview

Adds nightly grouped pip dependency updates with automatic squash merging while excluding Docker base-image updates.

Changes:

• Exact-pins Python dependencies.
• Configures daily grouped Dependabot updates and pip-only auto-merge.
• Documents automated and manual dependency-update procedures.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
requirements.txt	Pins runtime and analysis dependencies.
requirements-dev.txt	Pins development dependencies.
.github/dependabot.yml	Configures grouped daily pip updates.
.github/workflows/dependabot-auto-merge.yml	Enables pip Dependabot auto-merge.
docs/ci-nightly-eval.md	Documents dependency-update operations.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/workflows/dependabot-auto-merge.yml:19

• This enables auto-merge without any required check validating the files being bumped. The CPU matrix explicitly installs .[tests,hw-queue] from pyproject.toml rather than requirements*.txt (cpu-tests.yml:227-255), pre-commit installs only pre-commit, and the GPU evaluation installs wheel extras. Thus an incompatible exact pin can pass every stated gate and merge automatically. Add a required supported-Python matrix that installs these requirement files (and ideally exercises their scripts) before enabling auto-merge.

      - if: steps.meta.outputs.package-ecosystem == 'pip'
        env:
          PR_URL: ${{ github.event.pull_request.html_url }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: gh pr merge --auto --squash "$PR_URL"

docs/ci-nightly-eval.md:141

• This is not true while .github/workflows/bump-validate.yml:11-32 remains in the base: that workflow explicitly triggers on both requirements files and runs the reusable GPU evaluation for same-repository PRs, which includes Dependabot branches. Either account for that GPU run here or make this PR depend on/remediate the separate deletion; as written, the claimed behavior is not independent of #345.

  required checks are green (`pytest (CPU, py3.10/3.11/3.12)` and `pre-commit`;
  GPU checks auto-skip to Success for a requirements-only change). Only

[Copilot]

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

.github/workflows/dependabot-auto-merge.yml:56

• Waiting for this job only protects the first call that enables auto-merge; it does not make validation a merge requirement. Auto-merge can remain enabled while an open Dependabot PR is rebased (an explicitly expected weekly path), so a later revision can merge as soon as the configured required checks pass even if its new validate-requirements run is still pending or fails. Expose an always-present validation aggregate and require it in branch protection, or use a merge mechanism that reevaluates this validation for the current head SHA before merging.

    needs: [meta, validate-requirements]
    if: needs.meta.outputs.ecosystem == 'pip'

[amd-vivekag]

Follow-up on the latest review's note that validate-requirements only gates enabling auto-merge (via needs:) and isn't a merge requirement across a rebase -- so a rebased Dependabot PR could merge once the other required checks pass while its re-validation is still pending: fixed in 737252f.

Added an always-present Requirements validation aggregate job (same pattern as cpu-tests.yml's required / "CPU tests" check, per actions/runner#952 -- the per-version matrix legs can't be required because a skipped matrix never emits their contexts). Branch protection on main should require this one stable context; since gh pr merge --auto waits on required checks on the current head SHA, a rebased bump is re-validated on its merge commit rather than only when auto-merge was first enabled. The aggregate reports a pass on non-pip / non-Dependabot PRs, so it's safe to require globally. Documented the branch-protection requirement in docs/ci-nightly-eval.md.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (2)

docs/ci-nightly-eval.md:148

• This check is mandatory for the rebase safety described here, but both the PR's “Required repo settings” section and linked #346 list only CPU tests and pre-commit. Following those prerequisites would leave Requirements validation optional and permit the exact unvalidated-head merge race documented below. Add this check to the rollout prerequisites in the PR and reference issue.

  check and `pre-commit`) are green. **`Requirements validation` must be a
  required status check on `main`** (branch protection): it is the stable
  aggregate over the `validate-requirements` matrix (the per-version legs can't
  be required — a skipped matrix never emits their contexts, actions/runner#952),
  and requiring it is what enforces the install gate on the *current head SHA*.

.github/workflows/dependabot-auto-merge.yml:37

• This guard still lets a GitHub Actions Dependabot PR execute the newly proposed fetch-metadata SHA from its own branch. Because that action's output is the sole condition for the write-capable job, a compromised update could report pip and auto-merge itself, bypassing the stated human-review policy for action bumps. Gate metadata on the expected pip branch namespace before executing the action.

    if: github.event.pull_request.user.login == 'dependabot[bot]'