Repository navigation
Closed
Description
Goal
Report which lines, functions, and branches of the nvx crate's production code its own tests run, on Linux and Windows. #726 measured this only for the families of golden tests that #722 retired, and 10035af deleted its workflow once the last family was gone.
Tasks
- Add a workflow, beside
nvx-rs-mutants.yml, that runscargo llvm-covat a pinned release, with--all-featuresandNVX_BUNDLE=skip, onubuntu-latestandwindows-latest, for pushes todevor on a schedule, and on a manual dispatch. No required check depends on it. - Choose the compiler: the release that
rust-toolchain.tomlpins, with itsllvm-toolscomponent, measures lines, regions, and functions, while branches still need a nightly compiler (-Z coverage-options=branch), which nvx-rs: measure line, branch, and mutation coverage on Linux and Windows #726 pinned apart fromrust-toolchain.toml. - Count production code only. cargo-llvm-cov already leaves out
tests/and thetests.rsfiles ofsrc/, but it countstests/vectors/openvmm/platform_command_line_token_is_invariant.rs, which a unit test includes, and the fake OpenVMM ofsrc/testing/, a test double of 1,611 lines on Windows. Leave out the first and report the second apart. - Keep the profiles of the processes that tests start:
nvx-rs,nvx-fake-openvmm, and, on Linux, the guest binaries thattests/guest_tools.rsruns. Both host binaries already return their statuses frommainso that Windows writes their profiles (84edb1f). - Carry over what nvx-rs: measure line, branch, and mutation coverage on Linux and Windows #726's script handled on Linux: let the owner read and write every profile before
llvm-profdatamerges them, since the store tests run processes whose umasks clear these permissions andllvm-profdatacrashed on their profiles; and run failing tests once more, naming in the summary those that passed only then. - Upload each OS's LCOV and HTML reports as artifacts, and write the totals and a table per module to the run's summary.
- Test the workflow as
scripts/test_nvx_rs_mutants.pytests its own, its triggers, pinned tools, and artifact names, and add the new module tovalidate-nvx. - Document it in
doc/ci.md,doc/project-structure.md, and the Testing section ofnvx-rs/README.md.
Acceptance criteria
- A run on
devreports both OSes from profiles that include thenvx-rsandnvx-fake-openvmmprocesses that tests start. - Test code, test data, and the fake OpenVMM don't count as production code.
- No required check changes.
Metadata
Metadata
Assignees
Labels
No labels
Activity
setup: install cargo-llvm-cov and the llvm-tools component
nvx-rs: keep the coverage profiles of the agent helper's refusals
ci: report the coverage of the nvx crate's tests on Linux and Windows
nvx-rs: keep the coverage profiles of the agent helper's refusals
ci: report the coverage of the nvx crate's tests on Linux and Windows
ci: report the coverage of the nvx crate's tests on Linux and Windows
setup: install cargo-llvm-cov and the llvm-tools component
nvx-rs: keep the coverage profiles of the agent helper's refusals
ci: report the coverage of the nvx crate's tests on Linux and Windows
ppenna commentedon Nov 7, 2026
#1333 did the tasks of this issue, and the first runs of
nvx-rs-coverage.ymlondevmeet its acceptance criteria, so this closes it. As #1333 chose, the workflow measures with the Rust release thatrust-toolchain.tomlpins and itsllvm-tools, so branches go unmeasured until they no longer need a nightly compiler.Runs on
devpushpushBoth ran Rust 1.97.1, which
rust-toolchain.tomlpins, and cargo-llvm-cov 0.9.1, whichscripts/setup/tool-versions.confpins, and uploadednvx-rs-coverage-<os>-<run-id>withlcov.info,html/, and the summary that the run's page shows. No test ran again and no profile strayed into the crate's directory:retriesandstray_profilesare empty in eachmeasurement.json.Acceptance criteria
Checked against the artifacts of run 42387413841:
A run on
devreports both OSes from profiles that include thenvx-rsandnvx-fake-openvmmprocesses that tests start: met. Each OS's summary says that the profiles include processes ofnvx-rsandnvx-fake-openvmm, and on Linux of the ten guest binaries ofsrc/bin/too.coverage.jsonagrees: a binary'smainruns only in its own processes, and the tests reached 5 of the 6 lines ofsrc/bin/nvx-rs.rsand the 3 lines ofsrc/bin/nvx-fake-openvmm.rson both OSes.Test code, test data, and the fake OpenVMM don't count as production code: met.
lcov.infoandhtml/hold only files undersrc/, none in atestsdirectory, atests.rsfile, orsrc/testing/: 262 files on Linux and 164 on Windows, which no longer countstests/vectors/openvmm/platform_command_line_token_is_invariant.rs.coverage.jsonadds onlysrc/testing/fake_openvmm.rsandsrc/bin/nvx-fake-openvmm.rs, which the summary counts apart:No required check changes: met. The workflow runs only for pushes to
devand on dispatch,dev's only required check isRequired status check, and ci: report the coverage of the nvx crate's tests on Linux and Windows #1333 leftci.ymlunchanged.ci: measure the nvx-rs host code that the KVM microVM scenarios reach
ci: measure the nvx-rs host code that the KVM microVM scenarios reach
ci: measure the nvx-rs host code that the KVM microVM scenarios reach