Skip to content

[ci] Refresh stale .buildkite/README and remove dead ci_tags_from_change.sh#64375

Open
dstrodtman wants to merge 3 commits into
ray-project:masterfrom
dstrodtman:cleanup-pre-rayci-buildkite-docs
Open

[ci] Refresh stale .buildkite/README and remove dead ci_tags_from_change.sh#64375
dstrodtman wants to merge 3 commits into
ray-project:masterfrom
dstrodtman:cleanup-pre-rayci-buildkite-docs

Conversation

@dstrodtman

Copy link
Copy Markdown
Contributor

What

Two cleanups to the Buildkite CI docs/scripts that predate the rayci migration:

  1. Rewrite .buildkite/README.md. It described the pre-rayci model: the conditions / NO_WHEELS_REQUIRED test-selection field and the pipeline.{build,test,ml,gpu,gpu.large}.yml files. None of those exist anymore. Pipelines are now the *.rayci.yml files run by rayci, and conditional test selection is the .buildkite/*.rules.txt tag rules. The new README describes that and points at the rules files for the format.

  2. Remove ci/ci_tags_from_change.sh. It's orphaned: nothing in this repo references it (git grep finds zero callers) and rayci doesn't either. It also execs python ci/pipeline/determine_tests_to_run.py ci/pipeline/test_rules.txt, and ci/pipeline/test_rules.txt no longer exists, so it would fail if it ran.

Why

The stale README and the orphaned script both point at ci/pipeline/determine_tests_to_run.py as the live test-selection mechanism. It isn't: the live tag computation is rayci's test-rules step, which has its own parser and reads the .buildkite/*.rules.txt files. The mismatch is an easy trap for anyone trying to understand or modify CI test selection.

Left for CI owners to decide

ci/pipeline/determine_tests_to_run.py is now referenced only by the (also-removed) wrapper and a py_library target with no dependents. It's an in-repo reference implementation of the same rule-file format rayci's parser uses, not the live evaluator. I've kept it (with a clarifying note in the README) since it can serve as a readable reference and is handy for a local rules dry-run. If you'd rather retire it (and its ci/pipeline/BUILD.bazel target), say the word and I'll add that here.

Opening as a draft since .buildkite/ and ci/ are CI-owned — flagging for review before this goes anywhere.

…nge.sh

.buildkite/README.md described the pre-rayci pipeline model: the `conditions` /
`NO_WHEELS_REQUIRED` test-selection field and the `pipeline.*.yml` files, none
of which exist anymore. Rewrite it to describe the current state: pipelines are
the `*.rayci.yml` files run by rayci, and conditional test selection is driven
by the `.buildkite/*.rules.txt` tag rules that rayci's `test-rules` evaluates.

ci/ci_tags_from_change.sh is orphaned: nothing in this repo or in rayci
references it, and it execs `ci/pipeline/determine_tests_to_run.py
ci/pipeline/test_rules.txt`, a rules file that no longer exists. Remove it.

ci/pipeline/determine_tests_to_run.py is left in place: it is an in-repo
reference implementation of the rule-file format, not the live evaluator
(rayci's own parser is). Flagging for CI owners whether it should also be
retired or kept as a reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the .buildkite/README.md documentation to reflect the transition to rayci for pipeline orchestration and conditional test selection via rule files, while removing an obsolete shell script. The review feedback suggests improving the documentation's navigability by turning file paths and rule filenames into clickable relative links.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread .buildkite/README.md
Comment on lines +13 to +14
- `test.rules.txt` is the conditional rule set. Rules are evaluated in order and the first match wins. A rule with no `@` tags is a skip rule (it matches but emits nothing), and a trailing `*` catch-all fans unmatched changes out to the full suite.
- `always.rules.txt` is an always-on overlay applied to every change.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To improve navigability and developer experience, we can make the rule filenames clickable links pointing directly to the files in the repository.

Suggested change
- `test.rules.txt` is the conditional rule set. Rules are evaluated in order and the first match wins. A rule with no `@` tags is a skip rule (it matches but emits nothing), and a trailing `*` catch-all fans unmatched changes out to the full suite.
- `always.rules.txt` is an always-on overlay applied to every change.
- [`test.rules.txt`](test.rules.txt) is the conditional rule set. Rules are evaluated in order and the first match wins. A rule with no `@` tags is a skip rule (it matches but emits nothing), and a trailing `*` catch-all fans unmatched changes out to the full suite.\n- [`always.rules.txt`](always.rules.txt) is an always-on overlay applied to every change.

Comment thread .buildkite/README.md
are run on the `ml.Dockerfile` image.
- `pipeline.gpu.yml` contains jobs that require one GPU. The tests are run on the `gpu.Dockerfile` image.
- `pipeline.gpu.large.yml` contains jobs that require multi-GPUs (currently 4). The tests are run on the `gpu.Dockerfile` image.
`ci/pipeline/determine_tests_to_run.py` is an in-repo reference implementation of the same rule-file format. It is not the live evaluator; rayci is.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

We can make the path to the reference script a relative link so that developers reading the README on GitHub can easily jump to the implementation.

Suggested change
`ci/pipeline/determine_tests_to_run.py` is an in-repo reference implementation of the same rule-file format. It is not the live evaluator; rayci is.
[`ci/pipeline/determine_tests_to_run.py`](../ci/pipeline/determine_tests_to_run.py) is an in-repo reference implementation of the same rule-file format. It is not the live evaluator; rayci is.

@dstrodtman dstrodtman marked this pull request as ready for review June 26, 2026 18:06
@dstrodtman dstrodtman added the go add ONLY when ready to merge, run all tests label Jun 26, 2026
@ray-gardener ray-gardener Bot added docs An issue or change related to documentation core Issues that should be addressed in Ray Core devprod labels Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Issues that should be addressed in Ray Core devprod docs An issue or change related to documentation go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants