ci: validate stack bumps nightly instead of on every PR raise

[amd-vivekag]

Summary

Moves Phase-4 bump validation (from #303) to the nightly run instead of running the full GPU evaluation matrix on every PR raise.

bump-validate.yml triggered eval-reusable.yml (timeout-minutes: 150) on the single self-hosted MI350 runner for every PR touching docker/**, requirements*.txt, or config/ci/ci-constraints.txt. Dependabot (.github/dependabot.yml) opens weekly pip / docker (ROCm digest) / github-actions bump PRs that hit exactly those paths, so every automated bump kicked off a heavy per-PR GPU run and serialized the shared runner behind bump/CI PRs.

nightly-eval.yml already installs the wheel, builds the pinned ROCm container from the repo, and runs the identical matrix + baseline comparison against main — so a merged stack bump is exercised by the very next nightly. This PR retires the per-PR trigger and relies on the nightly run, keeping workflow_dispatch for on-demand confirmation of a specific bump.

Changes

• Delete .github/workflows/bump-validate.yml (the per-PR GPU eval trigger).
• Update eval-reusable.yml / nightly-eval.yml comments to reflect that the reusable eval is nightly + on-demand dispatch, not per-PR.
• Document in docs/ci-nightly-eval.md (and the nightly_eval_matrix.yaml comment) that bumps are validated nightly rather than on each PR raise.

Why this is safe

• Fail-closed regression coverage for bumps is preserved — it just runs post-merge in the nightly rather than pre-merge per PR.
• The single self-hosted GPU runner is no longer tied up by a full matrix run on every bump/stack PR.
• Maintainers can still validate a specific bump pre-merge via Actions → Nightly Evaluation → Run workflow.

Test plan

• Confirm a PR touching docker/** / requirements*.txt / config/ci/ci-constraints.txt no longer triggers a GPU eval on raise.
• Confirm the nightly Nightly Evaluation still runs the matrix + baseline comparison and files/closes the nightly-regression issue.
• Confirm workflow_dispatch on Nightly Evaluation still works for on-demand bump validation.

Closes #344. Refs #303.

[Copilot]

Pull request overview

Moves stack-bump GPU validation from per-PR execution to nightly and on-demand evaluation, reducing contention on the shared MI350 runner.

Changes:

• Removes the per-PR bump validation workflow.
• Updates workflow comments to describe nightly/on-demand validation.
• Documents the revised bump-validation process.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
.github/workflows/bump-validate.yml	Removes per-PR GPU validation.
.github/workflows/eval-reusable.yml	Updates reusable workflow documentation.
.github/workflows/nightly-eval.yml	Documents nightly and dispatched validation.
config/ci/nightly_eval_matrix.yaml	Clarifies an excluded workload’s nightly impact.
docs/ci-nightly-eval.md	Documents nightly and on-demand bump validation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

docs/ci-nightly-eval.md:143

• This dispatch path is not isolated bump validation. nightly-eval.yml always calls the reusable workflow with do_alert: true, and its publish job runs for every non-skipped dispatch. Running a PR branch can therefore file or close the canonical nightly-regression issue and overwrite ci-results/results/<today>.json with unmerged results. Please gate alerting/publishing to canonical nightly runs (or add a non-publishing bump-validation mode) before recommending this path; the instructions should also say to select the bump branch.

  matrix + baselines; use **Actions -> Nightly Evaluation -> Run workflow**
  (`workflow_dispatch`) if you want to confirm a specific bump on demand.