Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/actions/check-quality/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ runs:
shell: bash
run: python scripts/check_markdown_links.py

- name: Check test layout
shell: bash
run: python scripts/check_test_layout.py

- name: Test Specula CI wiring
shell: bash
working-directory: .github/specula
Expand Down
4 changes: 2 additions & 2 deletions .github/actions/validate-nvx/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ runs:
set -euo pipefail
python3 -m pip install --requirement scripts/setup/azure/requirements.txt
python3 -m compileall -q scripts
python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_rs_mutants.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py -v
python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_rs_mutants.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py scripts/test_test_layout.py scripts/test_test_changes.py -v
python3 scripts/test_adversarial.py -v
python3 .github/skills/nvx-host-connect/scripts/test_hosts.py -v
python3 scripts/nvx.py --help
Expand All @@ -32,7 +32,7 @@ runs:
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
python -m compileall -q scripts
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_rs_mutants.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py -v
python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_rs_mutants.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py scripts/test_test_layout.py scripts/test_test_changes.py -v
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
python scripts\test_adversarial.py -v
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
Expand Down
13 changes: 13 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@
nested repository. Do not claim unavailable hardware or cross-platform gates
passed; report the missing prerequisite and rely on CI when appropriate.

## Tests

- Put a Rust module's unit tests in its `tests` module, in files of their own
(`src/a/b.rs` declares `#[cfg(test)] mod tests;` and keeps them in
`src/a/b/tests.rs`), never inline or as test-only items in production files.
Python unit tests are `test_*.py` modules in `scripts/`. Run
`python3 scripts/check_test_layout.py`; a test hook that cannot move needs an
entry in `scripts/test-layout-allowlist.json`, and moving test code out needs
`--tighten`. See [Test layout](../doc/contribute.md#test-layout).
- A change to an existing test, its data, or its CI wiring gets the
`tests-modified` label; explain each such change in the pull request. Run
`python3 scripts/check_test_changes.py --base origin/dev` to list them first.

## Documentation

- In [`nvx-rs/README.md`](../nvx-rs/README.md#layout), each row of the module
Expand Down
80 changes: 80 additions & 0 deletions .github/workflows/test-changes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: Test changes

# Flags a pull request that modifies or removes existing tests, test data,
# end-to-end scenarios, test doubles, test policy, or CI test wiring: it
# labels the pull request tests-modified and keeps one comment that lists the
# changes (doc/ci.md#test-changes). pull_request_target runs the workflow as
# the default branch holds it, with a token that may label and comment, for
# pull requests from forks too. So the job runs the default branch's script,
# which reads the pull request's commits as data and never runs their code.
on:
pull_request_target:
types:
- opened
- synchronize
- reopened
- ready_for_review
- edited

permissions: {}

jobs:
flag:
name: Flag changes to existing tests
# An edit changes what the job reads only when it changes the base branch.
if: github.event.action != 'edited' || github.event.changes.base != null
runs-on: ubuntu-latest
timeout-minutes: 10
# In the job, so that a run whose job the condition skips, as an edit of
# the title does, cancels no comparison.
concurrency:
group: test-changes-${{ github.event.pull_request.number }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
steps:
# The default branch's script runs, whatever the pull request's base:
# pull_request_target runs the default branch's workflow, and the
# checkout names that branch, so that no other branch's script runs
# with the token. It fetches every branch, the base among them.
- name: Checkout
uses: actions/checkout@v5
with:
ref: ${{ github.event.repository.default_branch }}
fetch-depth: 0
persist-credentials: false
Comment thread
ppenna marked this conversation as resolved.
Resolved

# The pull request's commits are data: nothing of them runs.
- name: Fetch the pull request's commits
id: fetch
env:
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
NUMBER: ${{ github.event.pull_request.number }}
run: |
set -euo pipefail
git fetch --no-tags --quiet origin \
"+refs/pull/${NUMBER}/head:refs/remotes/pull/head"
# A push after the event that started this run starts another, which
# flags the newer head.
if [[ "$(git rev-parse refs/remotes/pull/head)" == "${HEAD_SHA}" ]]; then
echo "current=true" >> "${GITHUB_OUTPUT}"
else
echo "current=false" >> "${GITHUB_OUTPUT}"
echo "The pull request's head moved on; a newer run flags it."
fi

- name: Flag changes to existing tests
if: steps.fetch.outputs.current == 'true'
env:
GH_TOKEN: ${{ github.token }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
NUMBER: ${{ github.event.pull_request.number }}
run: >-
python3 scripts/check_test_changes.py
--base "origin/${BASE_REF}"
--head "${HEAD_SHA}"
--repository "${GITHUB_REPOSITORY}"
--summary "${GITHUB_STEP_SUMMARY}"
--publish "${NUMBER}"
42 changes: 41 additions & 1 deletion doc/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -1811,7 +1811,9 @@ start the workflow, and a manually dispatched run notifies the person who
started it.
The job checks out the default branch, never the run's code. Its token can
read the repository, the run's jobs and logs, and their check annotations,
and write issues, and no other workflow can write issues. The job runs one
and write issues, and no other workflow can write issues; the
[Test changes](#test-changes) workflow labels and comments on pull requests
with a token that writes pull requests instead. The job runs one
report at a time; a newer reporter can replace one that waits for its turn,
but it reports the latest nightly result anyway. Its job summary repeats
the report.
Expand Down Expand Up @@ -1866,6 +1868,44 @@ owner triages a new report within a working day:
To hand triage over or to rotate it, change the workflow's `--assignee`
options, and reassign the open report.

## Test changes

The [`test-changes.yml`](../.github/workflows/test-changes.yml) workflow flags
a pull request that modifies or removes an existing test asset, as
[Changes to existing tests](contribute.md#changes-to-existing-tests)
describes: it labels the pull request `tests-modified`, which the workflow
creates when the repository lacks it, and keeps one comment that lists the
changes, which a hidden marker at its start identifies. It runs when a pull
request to any branch, a stacked one included, is opened, reopened, pushed
to, marked ready for review, or retargeted, drafts included, and replaces a
run of the same pull request in progress. A run that finds no change to an
existing test removes the label and rewrites the comment, if there is one,
to say so; the label then blocks nothing, and `Required status check` does
not wait for the workflow. Its job summary lists every change, where the
comment, which GitHub limits to 65,536 characters, may cut a section short.

`pull_request_target` gives the job a token that can label and comment, for
pull requests from forks too, so the job never runs a pull request's code. It
runs the workflow and script of the default branch: `pull_request_target`
always runs that branch's workflow, and its checkout names that branch, with
every other fetched and without the token's credentials kept. It fetches the
pull request's commits as data, and
[`check_test_changes.py`](../scripts/check_test_changes.py) reads their files
from Git's objects: it parses Rust and Python without running them and
compares the merge base of the pull request's base branch and head with the
head. A head that moved on since the event exits, since the push starts
another run. The token can read the repository's contents and write pull
requests, with which GitHub labels and comments on them, but not issues, which
only the [nightly failure report](#nightly-failure-reports) writes. A change
to the workflow or to the script takes effect once it merges into `dev`, and
the script itself counts as test policy, so a pull request that changes it is
flagged once it does.

The `quality` job checks the [test layout](contribute.md#test-layout) with
[`check_test_layout.py`](../scripts/check_test_layout.py), whose rules say
where tests live, which test hooks a production file may hold, and which the
flagging script shares to read Rust.

## crates.io releases

A scheduled or manually dispatched full run of `dev` publishes the
Expand Down
Loading
Loading