Skip to content

Test layout umbrella: move tests out of production code, and flag changes to existing tests #1207

Closed
5 of 5 issues completed

Description

Objective. Keep every test of NVX in files of its own, apart from the code that it tests; check that layout deterministically, so that new tests follow it; and flag each pull request that changes an existing test for careful review.

Why. 121 production files held their tests inline, about 24,000 lines, besides about 100 test-only helpers, accessors, and re-exports. A change to a test hid in a change to the code, a change to a guest module's tests rebuilt every guest image, and nothing pointed a reviewer at a pull request that weakened an existing test.

Children. One for each pull request of the stack, in order.

Step Issue Pull request Scope
1 #1202 #1188 check_test_layout.py and its allowlist; check_test_changes.py and the Test changes workflow, which label a pull request tests-modified; their documentation.
2 #1203 #1195 The tests of the core modules.
3 #1204 #1196 The tests of the guest modules, and the code that only they compile.
4 #1205 #1197 The tests of the host modules, and the code that only they compile.
5 #1206 #1198 The tests of the runtime adapter, and the end of the allowlist's baseline.

Order. Each pull request is stacked on the previous one's branch, so each child is blocked by the one before it. When a pull request merges, GitHub retargets the next one to dev, which starts no ci.yml run: close and reopen it to start its CI.

Scope. In: the Rust tests of nvx-rs and tests/runtime-adapter, unit and integration; the Python unit tests; and the end-to-end scenario drivers and guest workloads, which already lived in files of their own and which the flagger watches. Out: OpenVMM's tests in the openvmm/ submodule; doctests, which are documentation; and the test doubles and probes that NVX ships, such as nvx::testing, nvx-fake-openvmm, and the guest probes, which the flagger also watches.

Done when: every child is closed. No production file then holds test code other than the hooks that the allowlist's seams allow, and the Test changes workflow flags each pull request that changes an existing test.

Activity

  1. ppenna commented on Nov 2, 2026

    @ppenna
    ContributorAuthor

    All five child issues (#1202–#1206) are closed as completed, and their corresponding pull requests (#1188, #1195, #1196, #1197, and #1198) are merged. I verified that all five merge commits are ancestors of the freshly fetched origin/dev (tip 716324a60). The test-layout check and 80 targeted layout/flagger tests pass, and the final PR's test-change report was generated. This completes the umbrella issue's stated criteria.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions