Repository navigation
nvx-rs: retire the golden CLI scenarios - #830
Merged
Pedro Henrique Penna (ppenna) merged 4 commits intoOct 25, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Retiring extensive cross-platform regression coverage alongside a persisted-format change warrants human confirmation of the stacked coverage and mutation-test equivalence gates.
No open issues 🎉
What changed in this PR
This PR completes the CLI transcript retirement for #728, building on #825’s structured scenario tests. It replaces recorded-output replay with contract-focused checks and standardizes newly provisioned sandbox configurations on format 7.
Changes:
- Replace the golden CLI replay with scenario tests covering errors, launches, lifecycle behavior, and configuration compatibility.
- Share a dedicated run-help fixture between Rust and Python tests, and remove obsolete replay coverage tracking.
- Write format 7 for all sandboxes while retaining format 1 reads; update documentation accordingly.
| File | Description |
|---|---|
scripts/test_nvx_tools.py |
Check Python run help against the shared fixture. |
scripts/test_guest_binaries.py |
Replace the retired fixture reference in source-identity testing. |
scripts/nvx_rs_coverage.py |
Remove the retired replay coverage family. |
nvx-rs/tests/fixtures/state/format-7.json |
Add a format 7 configuration with shares. |
nvx-rs/tests/fixtures/state/format-7-without-shares.json |
Add format 7 with a null export. |
nvx-rs/tests/fixtures/state/format-6.json |
Add an unsupported format 6 fixture. |
nvx-rs/tests/fixtures/state/format-5.json |
Add an unsupported format 5 fixture. |
nvx-rs/tests/fixtures/state/format-4.json |
Add an unsupported format 4 fixture. |
nvx-rs/tests/fixtures/state/format-3.json |
Add an unsupported format 3 fixture. |
nvx-rs/tests/fixtures/state/format-2.json |
Add an unsupported format 2 fixture. |
nvx-rs/tests/fixtures/state/format-1.json |
Add a supported legacy configuration. |
nvx-rs/tests/fixtures/state/format-1-nvx-py.json |
Add a legacy Python configuration for rejection testing. |
nvx-rs/tests/fixtures/state/format-1-network.json |
Add legacy format coverage with networking. |
nvx-rs/tests/fixtures/run-help.txt |
Store shared reference help text. |
nvx-rs/tests/cli/usage.rs |
Check parser errors and help output. |
nvx-rs/tests/cli/openvmm.rs |
Provide command, image, and outcome assertions. |
nvx-rs/tests/cli/one_shot.rs |
Check launch arguments, output, and exit statuses. |
nvx-rs/tests/cli/harness.rs |
Provide isolated roots and structured CLI assertions. |
nvx-rs/tests/cli/formats.rs |
Check supported formats and legacy refusals. |
nvx-rs/tests/cli/egress.rs |
Check policy compilation output and errors. |
nvx-rs/tests/cli/dry_run.rs |
Check dry-run commands and argument round trips. |
nvx-rs/tests/cli/artifacts.rs |
Check artifact listings and guest order. |
nvx-rs/tests/cli.rs |
Replace replay helpers with scenario modules and shared setup. |
nvx-rs/src/host/cli/sandbox/state.rs |
Write format 7 universally and preserve format 1 reads. |
nvx-rs/src/host/cli/sandbox/lifecycle.rs |
Update comments describing persisted configurations. |
nvx-rs/src/host/artifacts.rs |
Update the memory-default test reference in documentation. |
nvx-rs/README.md |
Document scenario tests and configuration compatibility. |
nvx-rs/Cargo.toml |
Enable Windows command-line splitting for tests. |
doc/project-structure.md |
Describe scenario tests and configuration fixtures. |
doc/ci.md |
Document retirement of the replay coverage family. |
.gitattributes |
Remove the retired fixture’s generated-file designation. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-728-retire-cli-transcripts
branch
2 times, most recently
from
October 24, 2026 00:23
d334b16 to
b77ccbc
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-728-retire-cli-transcripts
branch
from
October 24, 2026 00:53
b77ccbc to
e6d11c3
Compare
nvx.py run parses the options of nvx-rs run before it runs it, so it prints a help of its own, which must be that of nvx-rs run, and which gives the default memory of each guest that guests.py also gives test-microvm. A test of nvx.py compared that help, and the dry runs that nvx.py run recorded for the default of each guest, with the transcripts of tests/fixtures/cli.json, which #728 retires. tests/fixtures/run-help.txt now holds the help of nvx-rs run, as the transcripts recorded it, byte for byte. run_prints_the_help_of_nvx_py_run checks that nvx-rs run prints it, and the test of nvx.py, now test_run_help_is_the_help_of_nvx_rs_run, that nvx.py run prints it too, and that it gives the default memory of each guest of guests.py. The dry runs of the scenario tests check that nvx-rs run boots each guest with that memory. The Testing section of the README and the documentation of Guest::default_memory_mib name the fixture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The coverage measurement of #722 compares what each family of golden tests reaches with what the other tests reach, so that the tests that replace a family can show that they reach it too. The family of the CLI's recorded scenarios, nvx_rs_reproduces_the_recorded_scenarios, retires with tests/fixtures/cli.json (#728): the scenario tests of tests/cli/ reach every line and branch of production code that it reached beyond the tests outside the families, on Linux and Windows, and what it alone reached among all the tests was code that only some runs reach. nvx_rs_coverage.py no longer measures the family, whose filter would select no test once the replay is gone, and doc/ci.md notes its retirement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tests/fixtures/cli.json held the 315 scenarios that scripts recorded from the Python implementations of nvx.py sandbox and nvx.py run, and nvx_rs_reproduces_the_recorded_scenarios replayed them byte for byte once placeholders replaced the root and volatile values. The scenario tests of tests/cli/ check what callers rely on in their stead (#728): every scenario, and every regression that #722 traces, maps to them; on Linux and Windows, the replay reaches no line or branch of production code beyond the tests outside its family; and cargo-mutants finds no mutant of src/host/cli that the replay catches and the other tests of the CLI miss. The fixture goes, with the replay, its normalization and population helpers, and the mapping of tests/cli/scenarios.rs, whose tables of cases are private again, as is Root::empty(), which only Root::new() calls now. .gitattributes no longer marks the fixture as generated, and the test of the guest binaries' source identity names another file of tests/fixtures, root-tokens.json, which holds the cases of a root's boot tokens rather than behavior recorded from Python, so that no retirement of #722 removes it. The README describes the scenario tests in a section of their own, which replaces Golden scenarios and records how they replaced the transcripts, and its Python parity and Testing sections point to it. #828, which merged after #825 added the scenario tests, changed two messages that they check, and recorded the change in the transcripts alone: an unknown field of an egress policy is quoted as Rust quotes it, "other" where Python wrote 'other', and a policy file that cannot be read is named as it was given, ./ included, where Python normalized its path. The tests of egress compile and of the refusals of run expect these messages, which the transcripts no longer check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
provision wrote the configuration of a sandbox without shares in format 1, with a null mount, as nvx.py did, and that of one with shares in format 7, with the export plan of its shares. No nvx.py reads either any more: nvx.py runs nvx-rs, and start refuses a configuration without the bindings that only nvx-rs records. #728 asks for the newest format alone. provision now writes format 7 for every sandbox, with export null for one without shares, and start reads format 7, with export a plan or null, and still reads format 1, which earlier builds wrote for sandboxes without shares. A configuration whose format is not the one that its export needs, format 7 with export and format 1 without it, fails as before. The scenario tests expect format 7 of every provision, start a hand-written configuration of format 7 without shares as they start that of format 1, and check that a start refuses a configuration of format 7 without export and one of format 1 with it. The README and the documentation of the state directory describe the formats. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-728-retire-cli-transcripts
branch
from
October 25, 2026 03:16
e6d11c3 to
7b0feef
Compare
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-728-retire-cli-transcripts
branch
October 25, 2026 11:02
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Build on the behavior-preserving specifications of #823 for the retirement half of #730. Remove the six Python-oracle tests of src/host/sandbox/golden.rs and their fixtures, tests/fixtures/sandbox.json and tests/fixtures/sandbox_options.json. Preserve all four accepted launch shapes, eight POSIX and Windows records with identical contracts, as explicit argument, token, and exact-budget regressions in the specification module rather than as fixtures. Remove the launch-profile family from the coverage inventory of scripts/nvx_rs_coverage.py, and update its inventory test to recognize the specification module. #828 and #830 retired the egress-policy and CLI-scenario families on dev, so the inventory and its table in doc/ci.md keep only the Python-semantics family, and doc/ci.md notes this retirement beside that of the CLI family. Update the coverage and mutation documentation, remove the temporary section mapping that #823 kept in the README, and drop the sandbox fixtures from the README's list of Python records. The native input and API changes follow in separate commits. Validation on Windows, rebased on dev: 701 library tests and 51 CLI tests pass with all features, and the coverage-script tests, Ruff, and the Ruff format check pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Retire the Python-backed scalar compatibility layer of #730 in favor of native sandbox values while preserving the existing wire contracts. - Remove the public `Int` and `IntError` exports; use `Option<u64>` for the memory limit and `Option<u32>` for the process limit, including the bound that leaves room for the guest's init process. - Share ASCII-digit-guarded `FromStr` parsing for UID and GID, the CLI resource limits, and the limits that sandbox configurations persist, which they keep writing as JSON integers. - Use the canonical native `Uuid` parser while preserving tokens and all valid plan and configuration wire encodings. - Reject malformed limits at the CLI usage stage with clear native errors: `--memory-max` and `--pids-max` fail with `argument-invalid` and exit status 2, naming the accepted ASCII decimal range, instead of Python's `invalid int value` and the later `--pids-max must be positive` refusal with exit status 1. - Quote layer and persisted-plan values with Rust debug formatting. - Remove the now-unused Python integer arithmetic, keep only the test-only historical Python UUID oracle, and narrow the visibility of the remaining private integer helpers. - Update the scenario tests that replaced the CLI transcripts (#825, #830) to match: the usage table expects the double-quoted layer UUID and roles and the native `--memory-max` message, and the `--pids-max 0` case moves from the validation table, whose cases exit with status 1, to the usage table. - Document the input contract and the API change in the README, whose Python parity section no longer lists the sandbox, nor UUIDs among the values that it checks as Python does; #828 already removed the egress policy compiler from that section. Validation on Windows, rebased on dev: 702 library tests and 51 CLI tests pass with all features, and `cargo fmt --all --check` passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Retire the Python JSON and surrogate behavior from the sandbox metadata checks for #730. Parse bounded environment files directly as Vec<String> with serde_json and validate ordinary UTF-8 KEY=VALUE strings. Read distro manifests as strict JSON, require the integer format 1, and refuse non-finite numbers and unpaired surrogate escapes, even in unused fields. Preserve the file, entry, and byte limits, the binding of the artifact name and digest, the file seals, and the independent refusal of systemd. Quote sandbox values and paths with Rust debug formatting, and keep the native JSON and I/O causes in diagnostics. Move the now-unused Python JSON lookup and UTF-8 encode-oracle helpers out of production code; the unrelated historical Python family keeps its test-only oracles. Update the focused checks of tests/cli.rs that name these values and paths; the transcripts of tests/fixtures/cli.json, which #830 retired, no longer record them. Document the accepted-input and diagnostic changes. Update the scenario tests that replaced the transcripts (#825, #830) to match. The validation table expects the double-quoted --mount modes and targets, and the serde_json causes of environment files that hold an object or a non-string entry. Its environment-file cases name the file as nvx-rs now quotes it: Root::expand, in the shared harness, gains the {root:?} and {root/RELATIVE:?} placeholders, which quote a path as Rust's Debug quotes it, so that the fragments match the escaped backslashes of Windows paths as well as POSIX paths. Validation on Windows, rebased on dev: 703 library tests and 51 CLI tests pass with all features, `cargo fmt --all --check` passes, and Clippy passes with warnings denied for all targets with all features and for the library without default features. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Complete the kernel-token part of #730 with the existing protocol::cmdline Linux lexer rather than Python's whitespace splitting. Refuse control characters, Unicode whitespace, quotes, and the bytes that Linux treats as whitespace, including the 0xa0 inside a UTF-8 sequence, in entrypoints, workload arguments, and guest targets, including the target of the single directory share of run. Check reserved names after Linux's quote splitting and its aliasing of hyphens and underscores, and also in the view of the guest's shell bootstrap, which splits the expanded command line on IFS whitespace without interpreting quotes, so that a quoted value such as extra="ignored nvx_memory_max=max ignored" cannot override a configured entrypoint, memory limit, process limit, or share; ordinary quoted caller values remain accepted. Trim the caller's command line once, before both reservation checks and rendering, so that the checks see exactly the text that is rendered and leading Unicode whitespace, such as U+2003 EM SPACE, cannot hide a reserved name. Confirm that Linux can still see every generated root parameter, so that -- and unmatched quotes cannot hide the appended boot settings. Keep the guest root parser and the exact 1024-byte budgets for both share-delivery modes and adapter plans. Broaden the properties to quoted and dashed reserved inputs, leading Unicode whitespace, and the known separator bytes, add explicit hidden-root and quoted-override regressions for both share-delivery modes, and make the host-path unit tests native-only. Update the documentation, and the validation table of the scenario tests that replaced the CLI transcripts (#825, #830): a sandbox argument with a space is now refused for its whitespace rather than its spaces. Build the Python compatibility code that only the CLI and the historical tests use for them alone, so that the default-feature library has no dead code, and add `cargo clippy --lib -- -D warnings` to check-nvx-rs so that it stays so. Gate the Python split and strip oracles to tests, and the integer, codec, and Unicode-digit helpers to the cli feature and tests. Since #828 moved the egress policy compiler to serde_json, and the previous commit the sandbox metadata readers, only the CLI's state files use the Python JSON reader and Python strings, so gate the json and text modules whole, as their dumps, shell, and value siblings are. The README's Python parity section now credits the nvx-rs CLI, their only remaining user, with the whitespace, int, and json.loads semantics, and the module documentation of the JSON reader no longer cites the retired egress and distro-manifest readers. Validation on Windows, rebased on dev: 706 library tests and 51 CLI tests pass with all features, and 630 library tests with default features; `cargo fmt --all --check`, Clippy with warnings denied for all targets with all features, for the default-feature library, and for the library without default features, rustdoc with warnings denied, and the Markdown link check pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Build on the behavior-preserving specifications of #823 for the retirement half of #730. Remove the six Python-oracle tests of src/host/sandbox/golden.rs and their fixtures, tests/fixtures/sandbox.json and tests/fixtures/sandbox_options.json. Preserve all four accepted launch shapes, eight POSIX and Windows records with identical contracts, as explicit argument, token, and exact-budget regressions in the specification module rather than as fixtures. Remove the launch-profile family from the coverage inventory of scripts/nvx_rs_coverage.py, and update its inventory test to recognize the specification module. #828 and #830 retired the egress-policy and CLI-scenario families on dev, so the inventory and its table in doc/ci.md keep only the Python-semantics family, and doc/ci.md notes this retirement beside that of the CLI family. Update the coverage and mutation documentation, remove the temporary section mapping that #823 kept in the README, and drop the sandbox fixtures from the README's list of Python records. The native input and API changes follow in separate commits. Validation on Windows, rebased on dev: 701 library tests and 51 CLI tests pass with all features, and the coverage-script tests, Ruff, and the Ruff format check pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Retire the Python-backed scalar compatibility layer of #730 in favor of native sandbox values while preserving the existing wire contracts. - Remove the public `Int` and `IntError` exports; use `Option<u64>` for the memory limit and `Option<u32>` for the process limit, including the bound that leaves room for the guest's init process. - Share ASCII-digit-guarded `FromStr` parsing for UID and GID, the CLI resource limits, and the limits that sandbox configurations persist, which they keep writing as JSON integers. - Use the canonical native `Uuid` parser while preserving tokens and all valid plan and configuration wire encodings. - Reject malformed limits at the CLI usage stage with clear native errors: `--memory-max` and `--pids-max` fail with `argument-invalid` and exit status 2, naming the accepted ASCII decimal range, instead of Python's `invalid int value` and the later `--pids-max must be positive` refusal with exit status 1. - Quote layer and persisted-plan values with Rust debug formatting. - Remove the now-unused Python integer arithmetic, keep only the test-only historical Python UUID oracle, and narrow the visibility of the remaining private integer helpers. - Update the scenario tests that replaced the CLI transcripts (#825, #830) to match: the usage table expects the double-quoted layer UUID and roles and the native `--memory-max` message, and the `--pids-max 0` case moves from the validation table, whose cases exit with status 1, to the usage table. - Document the input contract and the API change in the README, whose Python parity section no longer lists the sandbox, nor UUIDs among the values that it checks as Python does; #828 already removed the egress policy compiler from that section. Validation on Windows, rebased on dev: 702 library tests and 51 CLI tests pass with all features, and `cargo fmt --all --check` passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Retire the Python JSON and surrogate behavior from the sandbox metadata checks for #730. Parse bounded environment files directly as Vec<String> with serde_json and validate ordinary UTF-8 KEY=VALUE strings. Read distro manifests as strict JSON, require the integer format 1, and refuse non-finite numbers and unpaired surrogate escapes, even in unused fields. Preserve the file, entry, and byte limits, the binding of the artifact name and digest, the file seals, and the independent refusal of systemd. Read distro manifests with the strict JSON reader of egress policy files that #828 added, which host::network::policy::json now shares as read_document, with the name that its errors give the root of the document. serde_json's own Value depends on the features of serde_json, which another crate of a build may enable: with arbitrary_precision, it reads an object that holds a number as that number, so that a format of {"$serde_json::private::Number": "1"} would pass the integer check, and it accepts numbers beyond the range of a double in unused fields. The shared reader reads both as it does without the feature. It also refuses an object that repeats a field, as `manifest has duplicate field "format"` or `packages[0] has duplicate field "name"`, and arrays and objects nested more than MAX_JSON_DEPTH, 32, deep, and a manifest that is not UTF-8 now fails as such. Quote sandbox values and paths with Rust debug formatting, and keep the native JSON and I/O causes in diagnostics. Move the now-unused Python JSON lookup and UTF-8 encode-oracle helpers out of production code; the unrelated historical Python family keeps its test-only oracles. Update the focused checks of tests/cli.rs that name these values and paths; the transcripts of tests/fixtures/cli.json, which #830 retired, no longer record them. Document the accepted-input and diagnostic changes. Update the scenario tests that replaced the transcripts (#825, #830) to match. The validation table expects the double-quoted --mount modes and targets, and the serde_json causes of environment files that hold an object or a non-string entry. Its environment-file cases name the file as nvx-rs now quotes it: Root::expand, in the shared harness, gains the {root:?} and {root/RELATIVE:?} placeholders, which quote a path as Rust's Debug quotes it, so that the fragments match the escaped backslashes of Windows paths as well as POSIX paths. The manifest tests refuse that object-valued format as a mismatch, and refuse numbers beyond a double, repeated fields at the root and in a package, nesting beyond the limit, and a manifest that is not UTF-8. A test of the shared reader simulates arbitrary_precision, as its other tests do, and checks that the object stays an object there, and another that read_document names the root of a document in its errors. Validation on Windows, rebased on dev: 705 library tests and 51 CLI tests pass with all features, `cargo fmt --all --check` passes, Clippy passes with warnings denied for all targets with all features, and the Markdown link check passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
that referenced
this pull request
Oct 25, 2026
Complete the kernel-token part of #730 with the existing protocol::cmdline Linux lexer rather than Python's whitespace splitting. Refuse control characters, Unicode whitespace, quotes, and the bytes that Linux treats as whitespace, including the 0xa0 inside a UTF-8 sequence, in entrypoints, workload arguments, and guest targets, including the target of the single directory share of run. Check reserved names after Linux's quote splitting and its aliasing of hyphens and underscores, and also in the view of the guest's shell bootstrap, which splits the expanded command line on IFS whitespace without interpreting quotes, so that a quoted value such as extra="ignored nvx_memory_max=max ignored" cannot override a configured entrypoint, memory limit, process limit, or share; ordinary quoted caller values remain accepted. Trim the caller's command line once, before both reservation checks and rendering, so that the checks see exactly the text that is rendered and leading Unicode whitespace, such as U+2003 EM SPACE, cannot hide a reserved name. Confirm that Linux can still see every generated root parameter, so that -- and unmatched quotes cannot hide the appended boot settings. Keep the guest root parser and the exact 1024-byte budgets for both share-delivery modes and adapter plans. Broaden the properties to quoted and dashed reserved inputs, leading Unicode whitespace, and the known separator bytes, add explicit hidden-root and quoted-override regressions for both share-delivery modes, and make the host-path unit tests native-only. Update the documentation, and the validation table of the scenario tests that replaced the CLI transcripts (#825, #830): a sandbox argument with a space is now refused for its whitespace rather than its spaces. Build the Python compatibility code that only the CLI and the historical tests use for them alone, so that the default-feature library has no dead code, and add `cargo clippy --lib -- -D warnings` to check-nvx-rs so that it stays so. Gate the Python split and strip oracles to tests, and the integer, codec, and Unicode-digit helpers to the cli feature and tests. Since #828 moved the egress policy compiler to serde_json, and the previous commit the sandbox metadata readers, only the CLI's state files use the Python JSON reader and Python strings, so gate the json and text modules whole, as their dumps, shell, and value siblings are. The README's Python parity section now credits the nvx-rs CLI, their only remaining user, with the whitespace, int, and json.loads semantics, and the module documentation of the JSON reader no longer cites the retired egress and distro-manifest readers. Validation on Windows, rebased on dev: 708 library tests and 51 CLI tests pass with all features, and 632 library tests with default features; `cargo fmt --all --check`, Clippy with warnings denied for all targets with all features, for the default-feature library, and for the library without default features, rustdoc with warnings denied, and the Markdown link check pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #728: the second of its two pull requests. It builds on #825, which added the scenario tests, and is rebased on
devsince #825 and #828 merged, so its four commits apply todevdirectly.Changes
run(76b47ad):nvx.py runparses the options ofnvx-rs runbefore it runs it, so it prints a help of its own, which a test ofnvx.pycompared with the help thatcli.jsonrecorded.tests/fixtures/run-help.txtnow holds that help, byte for byte:run_prints_the_help_of_nvx_py_runchecks thatnvx-rs runprints it, andtest_run_help_is_the_help_of_nvx_rs_runthatnvx.py runprints it too, with the default memory of each guest ofguests.py.scripts/nvx_rs_coverage.pyno longer measures the family of the replay, whose filter would select no test, anddoc/ci.mdnotes its retirement. With the egress family that nvx-rs: retire the egress policy fixture and read policies as strict JSON #828 retired, the Python semantics and launch profile families remain.tests/fixtures/cli.jsongoes, withnvx_rs_reproduces_the_recorded_scenarios, its normalization and population helpers, and the mapping oftests/cli/scenarios.rs; the tables of cases are private again. The README's Golden scenarios section gives way to Scenario tests, which records how they replaced the transcripts, and.gitattributesand a test of the guest binaries' source identity no longer name the fixture: that test namesroot-tokens.json, which holds no behavior recorded from Python, so that no retirement of nvx-rs: retire the Python-parity golden tests and make Rust the reference #722 removes it. The scenario tests also expect the two messages that nvx-rs: retire the egress policy fixture and read policies as strict JSON #828 changed while it updated only the transcripts: an unknown field of an egress policy is quoted as Rust quotes it,"other", and a policy file that cannot be read is named as it was given,./included.provisionwrites format 7 for every sandbox, withexportnullwithout shares, where it wrote format 1 for those, andstartstill reads format 1. The scenario tests start a hand-written configuration of format 7 without shares, and check that a start refuses one of format 7 withoutexportand one of format 1 with it.Differences from the issue
startreads format 1 and refuses those as before.nvx.pycould operate a sandbox thatnvx-rsprovisioned:startalready refuses a configuration without the bindings that onlynvx-rsrecords.Gates
#825 holds the evidence, from runs on its branch, whose CLI sources this pull request leaves unchanged but for the format of
config.json:--test clicatch or time out on every mutant that the replay catches. On Linux, that covers 791 mutants ofsrc/host/cli/**/*.rs, a superset ofsandbox/{mod,lifecycle,state}.rs. On Windows, it covers 505 mutants of the files with platform-specific code.Rebase on
devThe rebase onto
devat ae0ceb4 left the patches of the first and last commits unchanged. #828 conflicted with the other two:egress-policy.jsonand its coverage family. The coverage script,doc/ci.md,.gitattributes, and the README's Python parity section now drop both families and fixtures, the retirement deletes thecli.jsonthat nvx-rs: retire the egress policy fixture and read policies as strict JSON #828 updated, and the test of the guest binaries' source identity, which namedegress-policy.jsonbefore the rebase, namesroot-tokens.json.devhas failed two of them since it merged, on Linux and Windows (run 40153682491):egress_compile_prints_the_library_loweringexpected'other', andvalidation_errors_exit_1_before_any_side_effecta normalized path. The retirement commit updates both expectations, so the first two commits fail those two tests, asdevdoes, and the last two pass them.Validation
After the rebase, on Windows, Rust 1.95.0:
-D warningsfor all targets and features and for the library without default features; the tests with all features and with the default ones; the library without default features; the artifact bundling test; rustdoc with-D warnings;cargo +1.89 check --all-targets --all-features; and the runtime adapter's protocol-only and host-only configurations. Every Windows step of check-aci-edge-sandboxes, its agent bundling and Rust 1.89 check included.clitests, Ruff, the Markdown link check, andtest_nvx_tools.py,test_guest_binaries.py,test_nvx_rs_coverage.py, andtest_markdown_links.py. Theclitests of the first two commits fail only the two tests thatdevfails.test_adversarial.py, andtest_hosts.py, with Git'sbashfirst onPATH, as on the hosted runners.Before the rebase, on Linux (WSL), at each commit: rustfmt, both Clippy configurations, and the
clitests; at the head: the full test suite with all features, and the Rust 1.89 check.Left to CI: Linux after the rebase, the Specula tests, which need
fcntl, the macOS check, the guest binaries, and the hypervisor, OpenVMM, and benchmark lanes.