diff --git a/doc/build.md b/doc/build.md index 2c95156c7..c6c91d0c3 100644 --- a/doc/build.md +++ b/doc/build.md @@ -149,6 +149,16 @@ The normal `build-guest --architecture aarch64 --guest alpine|ubuntu` command selects these same producers and namespaces, adding `--native` on Linux to avoid Docker. ARM rejects the x86 debug-kernel and all-guests distro-layer modes instead of mixing their outputs. +The separate maintained Ubuntu EROFS producer accepts +`build-distro-layer --guest ubuntu --architecture aarch64`. It uses the same +root builder, ARM package lock and input identity as the Ubuntu initramfs, +and writes `build/aarch64/ubuntu-distro.erofs` and its manifest. The existing +ISA-neutral scratch producer can target +`--output build/aarch64/ubuntu-smoke-scratch.ext4`. BuildKit's +`arm64-ubuntu-distro-layer-artifacts` target exports both through the existing +local-output mechanism; select `--platform linux/arm64` and +`--build-arg NVX_ARCHITECTURE=aarch64`. These are maintained producer inputs, +not a qualification-only filesystem fixture. The existing Alpine one-shot container entry selects its control guest's fixed x86-64 or AArch64 musl loader; it keeps the same namespace and privilege-drop policy and refuses other control ISAs. @@ -166,6 +176,10 @@ before any process is launched. Existing x86 paths and default selection remain unchanged. This native runtime default does not change the build commands' x86-64 default or the existing artifact API's explicit architecture selection. +The public `sandbox` CLI likewise selects native local artifacts and validates +the ARM Image, initramfs manifest and Windows OpenVMM PE identity. Its options, +saved formats and one-VP lifecycle contract are unchanged. ARM release +publication and snapshot modes remain outside this enablement. **Deferred ARM cacheinfo metadata.** Phase 1 defers complete Linux sysfs cache geometry and sharing metadata, an existing upstream Standard ARM diff --git a/doc/run.md b/doc/run.md index 8c6cc3f51..26e6d0438 100644 --- a/doc/run.md +++ b/doc/run.md @@ -1652,7 +1652,10 @@ template, so it has the same artifact requirements. It checks that: file survives `stop` and a new `start`, because the overlay's upper directory is on scratch; - `/sbin/nvx-sandbox-smoke` passes its security profile checks in a sandbox - provisioned with `--hostname`, after earlier requests; + provisioned with `--hostname`, after earlier requests. The x86 port check + still requires the fault and permission refusal. ARM instead requires the + x86-only diagnostic to remain unsupported and raw platform devices to be + absent; the capability, namespace and exact private-device checks stay common; - a request that writes more than the 1 MiB output bound exits with status 125 and an `output-limit` outcome report after the host has received more than 1 MiB less one 32 KiB read chunk and at most 1 MiB, and a request whose diff --git a/doc/usage.md b/doc/usage.md index aa837d9ed..3a340937e 100644 --- a/doc/usage.md +++ b/doc/usage.md @@ -970,13 +970,17 @@ and runtime qualification gates are not waived. ```text python3 scripts/nvx.py build-distro-layer --guest ubuntu + [--architecture {x86_64,aarch64}] [--output PATH] [--replace] ``` Builds the immutable Ubuntu EROFS `distro` layer directly on Linux. The default -output is `build/ubuntu-distro.erofs`. Existing output or manifest files are -rejected unless `--replace` is present. +output is `build/ubuntu-distro.erofs`. `--architecture` defaults to x86-64; +`aarch64` selects the pinned ARM64 Ubuntu Base and supplemental lock and defaults +to `build/aarch64/ubuntu-distro.erofs`, with its matching manifest beside it. +An explicit `--output` is honored for either ISA. Existing output or manifest +files are rejected unless `--replace` is present. ### `build-scratch-template` @@ -1000,7 +1004,11 @@ fixes the clock of `mkfs.ext4` with `E2FSPROGS_FAKE_TIME` and its directory hash seed, and ignores the caller's other `E2FSPROGS_*` and `MKE2FS_*` variables, so e2fsprogs 1.46.5 and 1.47.0 of Debian 12, Ubuntu 22.04 and 24.04, and Azure Linux 3 format the same bytes, whose SHA-256 -`UbuntuBuildConstants.SCRATCH_TEMPLATE_SHA256` pins. It warns when +`UbuntuBuildConstants.SCRATCH_TEMPLATE_SHA256` pins. When libext2fs's compiler +defaults to unsigned `char`, the producer uses the same package's `debugfs` +to select the existing signed-directory-hash format and update its checksums. +Already canonical x86 output is untouched; the UUID, layout and pinned hash +are unchanged. It warns when `mkfs.ext4` formats other bytes. The guest Docker build formats the template with the Ubuntu distro layer, in the pinned Debian image, for [`build-guest`](#build-guest) and for CI, which caches, records, and hands it @@ -1133,6 +1141,7 @@ OpenVMM's tests, and CI checks that it passed the pinned revision with ```text python3 scripts/nvx.py test-microvm --backend {kvm,mshv,whp} + [--architecture {x86_64,aarch64}] [--guest {alpine,ubuntu,azurelinux}] [--scenario SCENARIO]... [--debug-kernel] @@ -1167,6 +1176,11 @@ the CI debug kernel, `build/vmlinux-debug`, whose soft-lockup and hung-task detectors the guest's time ABI watcher reports; without `--scenario`, it runs only the same-host restore scenarios `smp`, `smp-snapshot`, `restore-processors`, `restore-downtime`, and `snapshot-tiers`. +The architecture defaults to x86-64. ARM requires explicit `--scenario` +selection and uses `build/aarch64/Image`, matching initramfs and managed-layer +inputs; it does not qualify every scenario listed here for ARM. Public sandbox +scenarios retain the existing one-VP CLI contract; `--processors` does not add +a sandbox processor option or change saved configurations. `--shard INDEX/COUNT`, such as `1/2`, runs shard INDEX of the COUNT shards of the default Alpine suite on the default kernel, the scenarios of a run without `--scenario` but the public sandbox scenarios, which CI runs in its guest and diff --git a/docker/Dockerfile b/docker/Dockerfile index 54cff2ebf..c33d59168 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -94,7 +94,7 @@ RUN python3 scripts/nvx.py build-initramfs --guest ubuntu --architecture "$NVX_A FROM ubuntu-initramfs AS ubuntu-distro-layer RUN python3 scripts/nvx.py build-distro-layer \ --guest ubuntu \ - --output build/ubuntu-distro.erofs + --architecture "$NVX_ARCHITECTURE" # The sandbox scratch template needs only the base image's mkfs.ext4, so it # builds no guest binary. @@ -296,6 +296,11 @@ COPY --from=kernel /repo/build/aarch64/Image.provenance.json /Image.provenance.j COPY --from=ubuntu-initramfs /repo/build/aarch64/initramfs-ubuntu.cpio.gz /initramfs-ubuntu.cpio.gz COPY --from=ubuntu-initramfs /repo/build/aarch64/initramfs-ubuntu.cpio.gz.packages.json /initramfs-ubuntu.cpio.gz.packages.json +FROM scratch AS arm64-ubuntu-distro-layer-artifacts +COPY --from=ubuntu-distro-layer /repo/build/aarch64/ubuntu-distro.erofs /ubuntu-distro.erofs +COPY --from=ubuntu-distro-layer /repo/build/aarch64/ubuntu-distro.erofs.manifest.json /ubuntu-distro.erofs.manifest.json +COPY --from=ubuntu-scratch-template /repo/build/ubuntu-smoke-scratch.ext4 /ubuntu-smoke-scratch.ext4 + FROM scratch AS kernel-debug-artifacts COPY --from=kernel-debug /repo/build/vmlinux-debug /vmlinux-debug COPY --from=kernel-debug /repo/build/vmlinux-debug.config /vmlinux-debug.config diff --git a/guest/common/nvx-sandbox-smoke b/guest/common/nvx-sandbox-smoke index 0a12524fc..0c26f8fb4 100644 --- a/guest/common/nvx-sandbox-smoke +++ b/guest/common/nvx-sandbox-smoke @@ -242,6 +242,25 @@ check_machine_id() { # The workload can neither use the processor's I/O ports, whose read ends # the reader with SIGSEGV without the permission, nor obtain the permission. check_ports() { + case "${1:-$(uname -m)}" in + x86_64) ;; + aarch64) + for raw_device in /dev/mem /dev/port /dev/nvx-platform; do + [ ! -e "$raw_device" ] && [ ! -L "$raw_device" ] || + fail "the workload can reach raw device $raw_device" + done + for probe in snapshot "snapshot grant"; do + port_status=0 + # The ARM helper must refuse the x86-only diagnostic itself. + # shellcheck disable=SC2086 + /sbin/nvx-time test port-probe $probe 2>/dev/null || port_status=$? + [ "$port_status" -eq 2 ] || + fail "an ARM x86-port diagnostic ended with status $port_status" + done + return + ;; + *) fail "unsupported workload architecture ${1:-$(uname -m)}" ;; + esac port_status=0 # The shell reports the signal on standard error, so silence it. { /sbin/nvx-time test port-probe snapshot; } 2>/dev/null || port_status=$? diff --git a/nvx-rs/public-api/linux.txt b/nvx-rs/public-api/linux.txt index 9c61fd1d4..5d17e61da 100644 --- a/nvx-rs/public-api/linux.txt +++ b/nvx-rs/public-api/linux.txt @@ -8,6 +8,7 @@ const nvx::Artifacts::ENV_OPENVMM method nvx::Artifacts::artifact_dir method nvx::Artifacts::bundled method nvx::Artifacts::discover +method nvx::Artifacts::from_artifact_dir method nvx::Artifacts::from_dir method nvx::Artifacts::from_release_dir method nvx::Artifacts::from_repo_layout @@ -1690,6 +1691,7 @@ const nvx::host::artifacts::Artifacts::ENV_OPENVMM method nvx::host::artifacts::Artifacts::artifact_dir method nvx::host::artifacts::Artifacts::bundled method nvx::host::artifacts::Artifacts::discover +method nvx::host::artifacts::Artifacts::from_artifact_dir method nvx::host::artifacts::Artifacts::from_dir method nvx::host::artifacts::Artifacts::from_release_dir method nvx::host::artifacts::Artifacts::from_repo_layout diff --git a/nvx-rs/public-api/windows.txt b/nvx-rs/public-api/windows.txt index baa71e6c3..d9ebf0c26 100644 --- a/nvx-rs/public-api/windows.txt +++ b/nvx-rs/public-api/windows.txt @@ -8,6 +8,7 @@ const nvx::Artifacts::ENV_OPENVMM method nvx::Artifacts::artifact_dir method nvx::Artifacts::bundled method nvx::Artifacts::discover +method nvx::Artifacts::from_artifact_dir method nvx::Artifacts::from_dir method nvx::Artifacts::from_release_dir method nvx::Artifacts::from_repo_layout @@ -634,6 +635,7 @@ const nvx::host::artifacts::Artifacts::ENV_OPENVMM method nvx::host::artifacts::Artifacts::artifact_dir method nvx::host::artifacts::Artifacts::bundled method nvx::host::artifacts::Artifacts::discover +method nvx::host::artifacts::Artifacts::from_artifact_dir method nvx::host::artifacts::Artifacts::from_dir method nvx::host::artifacts::Artifacts::from_release_dir method nvx::host::artifacts::Artifacts::from_repo_layout diff --git a/nvx-rs/src/host/artifacts/architecture/tests.rs b/nvx-rs/src/host/artifacts/architecture/tests.rs index c8227d23f..d028adbe6 100644 --- a/nvx-rs/src/host/artifacts/architecture/tests.rs +++ b/nvx-rs/src/host/artifacts/architecture/tests.rs @@ -53,7 +53,82 @@ fn explicit_arm_paths_never_change_legacy_default_or_openvmm_location() { assert_eq!(arm.openvmm(), legacy.openvmm()); assert_eq!(arm.openvmm().file_name().unwrap(), openvmm_executable()); assert_eq!(arm.paths().guests.len(), 2); - assert!(arm.distro_layer(Guest::Ubuntu).is_none()); + assert_eq!( + arm.distro_layer(Guest::Ubuntu).unwrap(), + root.path().join("build/aarch64/ubuntu-distro.erofs") + ); + assert!(arm.sandbox_artifacts().is_err()); +} + +#[test] +fn arm_managed_artifacts_require_matching_native_kernel_and_manifest() { + let root = tempfile::tempdir().unwrap(); + let arm = ArtifactDir::repository(root.path()) + .unwrap() + .with_architecture(Architecture::Aarch64) + .unwrap(); + fs::create_dir_all(arm.openvmm().parent().unwrap()).unwrap(); + let mut executable = [0; 70]; + executable[..2].copy_from_slice(b"MZ"); + executable[60..64].copy_from_slice(&64_u32.to_le_bytes()); + executable[64..68].copy_from_slice(b"PE\0\0"); + executable[68..70].copy_from_slice(&0xaa64_u16.to_le_bytes()); + fs::write(arm.openvmm(), executable).unwrap(); + fs::create_dir_all(arm.kernel().parent().unwrap()).unwrap(); + fs::write(arm.kernel(), image()).unwrap(); + fs::write( + arm.initramfs(Guest::Alpine), + b"unit fixture, not a bootable CPIO", + ) + .unwrap(); + assert!(arm.sandbox_artifacts().is_err()); + fs::write( + arm.package_manifest(Guest::Alpine), + r#"{"architecture":"x86_64"}"#, + ) + .unwrap(); + assert!(arm.sandbox_artifacts().is_err()); + fs::write( + arm.package_manifest(Guest::Alpine), + r#"{"architecture":"aarch64"}"#, + ) + .unwrap(); + let artifacts = arm.sandbox_artifacts().unwrap(); + assert_eq!(artifacts.kernel, arm.kernel()); + assert_eq!(artifacts.initrd, arm.initramfs(Guest::Alpine)); + #[cfg(any(target_os = "linux", windows))] + { + artifacts.probe().unwrap(); + assert!(artifacts.runtime_identity().is_err()); + let replacement = root.path().join("foreign-artifact"); + fs::write(&replacement, b"foreign artifact").unwrap(); + for description in ["OpenVMM executable", "guest kernel", "guest initramfs"] { + let mut replaced = artifacts.clone(); + let path = match description { + "OpenVMM executable" => &mut replaced.openvmm, + "guest kernel" => &mut replaced.kernel, + _ => &mut replaced.initrd, + }; + *path = replacement.clone(); + let error = replaced.probe().unwrap_err(); + assert_eq!(error.code(), crate::ErrorCode::Unavailable); + assert_eq!( + error.message(), + format!( + "the {description} {} is not the one of {}, whose install metadata \ + identifies the artifacts", + replacement.display(), + arm.root().display() + ) + ); + } + fs::remove_file(arm.kernel()).unwrap(); + assert_eq!( + artifacts.probe().unwrap_err().code(), + crate::ErrorCode::ArtifactMissing + ); + } + fs::write(arm.kernel(), b"foreign kernel").unwrap(); assert!(arm.sandbox_artifacts().is_err()); } diff --git a/nvx-rs/src/host/artifacts/dir.rs b/nvx-rs/src/host/artifacts/dir.rs index ddef271fe..0d2939c26 100644 --- a/nvx-rs/src/host/artifacts/dir.rs +++ b/nvx-rs/src/host/artifacts/dir.rs @@ -157,17 +157,11 @@ impl ArtifactDir { /// Returns the path of a guest's distro layer, if the guest provides one. pub fn distro_layer(&self, guest: Guest) -> Option { - if self.architecture == Architecture::Aarch64 { - return None; - } guest.distro_layer_name().map(|name| self.artifact(name)) } /// Returns the path of the manifest of a guest's distro layer, if the guest provides one. pub fn distro_manifest(&self, guest: Guest) -> Option { - if self.architecture == Architecture::Aarch64 { - return None; - } guest .distro_manifest_name() .map(|name| self.artifact(&name)) @@ -180,7 +174,12 @@ impl ArtifactDir { /// /// [`ErrorCode::ArtifactMissing`]: crate::ErrorCode::ArtifactMissing pub fn require_openvmm(&self) -> Result { - required(self.openvmm(), "OpenVMM release binary") + let path = required(self.openvmm(), "OpenVMM release binary")?; + #[cfg(windows)] + if self.architecture == Architecture::Aarch64 { + super::check_arm64_pe(&path)?; + } + Ok(path) } /// Checks that the guest kernel is a regular file and returns its path. @@ -257,14 +256,13 @@ impl ArtifactDir { /// /// [`ErrorCode::ArtifactMissing`]: crate::ErrorCode::ArtifactMissing pub fn sandbox_artifacts(&self) -> Result { - if self.architecture == Architecture::Aarch64 { - return Err(Error::unsupported( - "ARM64 managed sandboxes are not qualified by the Phase 1 cold-boot contract", - )); - } self.require_openvmm()?; self.require_kernel()?; - required(self.initramfs(Guest::Alpine), "initramfs")?; + if self.architecture == Architecture::Aarch64 { + self.require_initramfs(Guest::Alpine)?; + } else { + required(self.initramfs(Guest::Alpine), "initramfs")?; + } Ok(Artifacts::located(self)) } diff --git a/nvx-rs/src/host/artifacts/identity.rs b/nvx-rs/src/host/artifacts/identity.rs index 43a1d1e9a..351982653 100644 --- a/nvx-rs/src/host/artifacts/identity.rs +++ b/nvx-rs/src/host/artifacts/identity.rs @@ -117,7 +117,7 @@ pub(super) fn provenance_identity( } /// Rejects a Windows OpenVMM executable from another ISA before any process or WHP call. -#[cfg(all(windows, feature = "cli"))] +#[cfg(windows)] pub(crate) fn check_arm64_pe(path: &Path) -> Result<()> { use std::io::{Seek, SeekFrom}; diff --git a/nvx-rs/src/host/artifacts/mod.rs b/nvx-rs/src/host/artifacts/mod.rs index bd8f85286..ab79d7f27 100644 --- a/nvx-rs/src/host/artifacts/mod.rs +++ b/nvx-rs/src/host/artifacts/mod.rs @@ -46,7 +46,7 @@ use crate::host::store::RuntimeIdentity; pub use self::architecture::Architecture; pub use self::dir::{ArtifactDir, ArtifactPaths, GuestPaths, Layout}; pub use self::guest::Guest; -#[cfg(all(windows, feature = "cli"))] +#[cfg(windows)] pub(crate) use self::identity::check_arm64_pe; #[cfg(feature = "cli")] pub(crate) use self::require::require_artifact; @@ -128,6 +128,12 @@ impl Artifacts { them in an NVX release or repository checkout, or supply their runtime identity", )); }; + self.check_directory_paths(directory)?; + directory.runtime_identity() + } + + /// Checks that the public paths still name the artifacts of `directory`. + fn check_directory_paths(&self, directory: &ArtifactDir) -> Result<()> { let located = Self::located(directory); for (path, expected, description) in [ (&self.openvmm, &located.openvmm, "OpenVMM executable"), @@ -143,11 +149,14 @@ impl Artifacts { ))); } } - directory.runtime_identity() + Ok(()) } /// Checks that this host can run NVX VMs and that the three files exist. /// + /// Located ARM artifacts must still name their directory's files, whose native format and + /// initramfs architecture are checked without reading runtime identity metadata. + /// /// Fails with [`ErrorKind::Unavailable`](crate::ErrorKind::Unavailable) otherwise, with the /// code [`ErrorCode::ArtifactMissing`] for a missing file. It does not check the hypervisor; /// [`probe_hypervisor`](super::platform::probe_hypervisor) does. @@ -169,6 +178,14 @@ impl Artifacts { )); } } + if let Some(directory) = self.artifact_dir() + && directory.architecture() == Architecture::Aarch64 + { + self.check_directory_paths(directory)?; + directory.require_openvmm()?; + directory.require_kernel()?; + directory.require_initramfs(Guest::Alpine)?; + } Ok(()) } @@ -202,7 +219,7 @@ impl Artifacts { /// /// Fails with [`ErrorKind::Unavailable`](crate::ErrorKind::Unavailable) as /// [`check_manifest`] does. - fn from_artifact_dir(directory: &ArtifactDir) -> Result { + pub fn from_artifact_dir(directory: &ArtifactDir) -> Result { check_manifest(&directory.root().join(MANIFEST_NAME))?; Ok(Self::located(directory)) } diff --git a/nvx-rs/src/host/artifacts/tests.rs b/nvx-rs/src/host/artifacts/tests.rs index 3e4de6351..067a61fa0 100644 --- a/nvx-rs/src/host/artifacts/tests.rs +++ b/nvx-rs/src/host/artifacts/tests.rs @@ -754,6 +754,27 @@ fn located_artifacts_have_the_identity_of_their_directory() { ); } +#[test] +#[cfg(any(target_os = "linux", windows))] +fn probing_explicit_and_x86_artifacts_keeps_path_override_support() { + let directory = tempfile::tempdir().unwrap(); + release(directory.path()); + let replacement = directory.path().join("replacement"); + fs::write(&replacement, b"explicit artifact").unwrap(); + let mut located = Artifacts::from_dir(directory.path()).unwrap(); + located.openvmm = replacement.clone(); + located.kernel = replacement.clone(); + located.initrd = replacement.clone(); + located.probe().unwrap(); + let named = Artifacts::new(&replacement, &replacement, &replacement); + named.probe().unwrap(); + fs::remove_file(&replacement).unwrap(); + assert_eq!( + named.probe().unwrap_err().code(), + ErrorCode::ArtifactMissing + ); +} + #[cfg(feature = "bundled")] #[test] fn the_bundle_is_a_compatible_release() { @@ -764,6 +785,7 @@ fn the_bundle_is_a_compatible_release() { if requested && (cfg!(target_os = "linux") || cfg!(windows)) { assert!(bundled.is_some(), "NVX_BUNDLE_DIR was not staged"); } + if let Some(artifacts) = bundled { for path in [&artifacts.openvmm, &artifacts.kernel, &artifacts.initrd] { assert!(path.is_file(), "{} is missing", path.display()); diff --git a/nvx-rs/src/host/cli/sandbox/mod.rs b/nvx-rs/src/host/cli/sandbox/mod.rs index 8ab296e7f..d8485dee7 100644 --- a/nvx-rs/src/host/cli/sandbox/mod.rs +++ b/nvx-rs/src/host/cli/sandbox/mod.rs @@ -24,12 +24,12 @@ mod state; use std::time::Instant; -use super::artifacts::{Located, OPENVMM_VARIABLE, locate}; +use super::artifacts::{Located, OPENVMM_VARIABLE}; use super::download::{self, Hold}; use super::shell::format_command; use super::{Exit, Result, output}; use crate::ErrorCode; -use crate::host::artifacts::{Guest, KERNEL_NAME, require_artifact}; +use crate::host::artifacts::{Architecture, Guest, require_artifact}; use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding}; use crate::host::sandbox::mount::{plan_mounts, validate_mounts}; use crate::host::sandbox::options::{ @@ -51,6 +51,13 @@ pub(super) use self::options::{ pub(crate) use self::options::{Options, path_string}; use self::state::RecordedDistro; +/// Selects native managed artifacts without changing the legacy artifact API's x86 default. +fn locate(root: Option<&str>) -> Result { + let mut located = super::artifacts::locate(root)?; + located.dir = located.dir.with_architecture(Architecture::native())?; + Ok(located) +} + /// The artifacts of a sandbox, found when an operation first needs them. pub(crate) struct Artifacts<'a> { /// `--root`, if given. @@ -84,10 +91,17 @@ impl Artifacts<'_> { pub(crate) fn require(&self) -> Result<[String; 3]> { let Located { dir, openvmm } = locate(self.root)?; require_artifact(&openvmm, "OpenVMM release binary")?; - let kernel = dir.artifact(KERNEL_NAME); - require_artifact(&kernel, "Linux direct kernel")?; + #[cfg(windows)] + if dir.architecture() == Architecture::Aarch64 { + crate::host::artifacts::check_arm64_pe(&openvmm)?; + } + let kernel = dir.require_kernel()?; let initrd = dir.initramfs(Guest::Alpine); - require_artifact(&initrd, "initramfs")?; + if dir.architecture() == Architecture::Aarch64 { + dir.require_initramfs(Guest::Alpine)?; + } else { + require_artifact(&initrd, "initramfs")?; + } Ok([ path_string(&openvmm)?, path_string(&kernel)?, diff --git a/nvx-rs/tests/cli.rs b/nvx-rs/tests/cli.rs index e6d67002d..1d67a0038 100644 --- a/nvx-rs/tests/cli.rs +++ b/nvx-rs/tests/cli.rs @@ -54,7 +54,7 @@ use std::time::{Duration, Instant}; use nvx::host::process::{Process, ProcessRef}; use serde_json::{Value, json}; -use self::harness::{FAKE_OPENVMM, NVX_RS, Root, canonical}; +use self::harness::{FAKE_OPENVMM, NVX_RS, Root, canonical, sandbox_relative}; /// Runs `nvx-rs --root ROOT ARGUMENTS` in the root, without the variables that change its /// output. @@ -282,9 +282,9 @@ fn a_mapping_that_the_guest_refuses_leaves_no_usable_start() { } assert!( !root - .path - .join("build") - .join(format!("fake-openvmm-{state}-maps.json")) + .below(&sandbox_relative(&format!( + "build/fake-openvmm-{state}-maps.json" + ))) .exists() ); let mut exec = lifecycle("exec", &root, state); @@ -903,7 +903,7 @@ fn a_restarted_cli_drives_only_a_vm_of_its_runtime_identity() { // Artifacts of another kernel did not start the VM, so no transition drives it, and it runs // on until a stop terminates it. - let kernel = root.below("build/vmlinux.provenance.json"); + let kernel = root.below(&sandbox_relative("build/vmlinux.provenance.json")); let provenance = fs::read_to_string(&kernel).unwrap(); fs::write(&kernel, other_kernel_provenance()).unwrap(); let pid = record["process"]["pid"].as_u64().unwrap(); @@ -976,8 +976,10 @@ fn terminated(parts: &str, pid: u64) -> String { /// Returns how many connections to its control endpoint the fake OpenVMM of the sandbox in /// `state` below the root has accepted. fn sessions(root: &Root, state: &str) -> usize { - fs::read_to_string(root.below(&format!("build/fake-openvmm-{state}-sessions.log"))) - .map_or(0, |log| log.lines().count()) + fs::read_to_string(root.below(&sandbox_relative(&format!( + "build/fake-openvmm-{state}-sessions.log" + )))) + .map_or(0, |log| log.lines().count()) } #[test] @@ -992,7 +994,7 @@ fn a_stop_that_cannot_identify_the_runtime_keeps_the_vm_for_the_next_one() { // Without the install metadata of the initramfs, the runtime identity is indeterminate, and so // is whether the VM may be driven: the stop signals nothing, and keeps its record. - let initramfs = root.below("build/initramfs.provenance.json"); + let initramfs = root.below(&sandbox_relative("build/initramfs.provenance.json")); fs::remove_file(&initramfs).unwrap(); assert_eq!( transition(&root, "stop", "state", &[]), @@ -1032,7 +1034,7 @@ fn a_stop_that_cannot_identify_the_runtime_keeps_the_vm_for_the_next_one() { fn a_stop_under_another_identity_never_ends_a_process_that_its_record_does_not_name() { let root = Root::new(); provisioned(&root, "state"); - let kernel = root.below("build/vmlinux.provenance.json"); + let kernel = root.below(&sandbox_relative("build/vmlinux.provenance.json")); let provenance = fs::read_to_string(&kernel).unwrap(); let path = root.path.join("state").join("runtime.json"); // The record names the VMM's ID with another start time, as after the VMM exited and another @@ -1116,7 +1118,7 @@ fn a_start_that_was_interrupted_is_ended_under_another_identity_too() { // Under another runtime identity, the next transition ends the start, whose OpenVMM it // verifies through the journal. fs::write( - root.below("build/vmlinux.provenance.json"), + root.below(&sandbox_relative("build/vmlinux.provenance.json")), other_kernel_provenance(), ) .unwrap(); @@ -1995,11 +1997,8 @@ fn start_hands_openvmm_the_images_that_it_checked() { .unwrap(); // OpenVMM has started, and has not opened its images yet. - wait_for(&root.path.join("build").join("fake-openvmm-state.json")); - let blocks = root - .path - .join("build") - .join("fake-openvmm-state-blocks.json"); + wait_for(&root.below(&sandbox_relative("build/fake-openvmm-state.json"))); + let blocks = root.below(&sandbox_relative("build/fake-openvmm-state-blocks.json")); assert!(!blocks.exists()); for (path, replacement) in [ (&distro, "replaced distro\n"), @@ -2111,10 +2110,7 @@ fn a_start_that_ends_before_openvmm_opens_the_images_leaves_it_the_checked_ones( let process = Process::open(&reference) .unwrap() .expect("the OpenVMM of the start runs"); - let blocks = root - .path - .join("build") - .join("fake-openvmm-state-blocks.json"); + let blocks = root.below(&sandbox_relative("build/fake-openvmm-state-blocks.json")); assert!(!blocks.exists()); for (path, replacement) in [ (&distro, "replaced distro\n"), @@ -2309,7 +2305,7 @@ fn a_run_passes_openvmm_no_handle_of_its_caller() { let mut cli = cli.unwrap(); // Once OpenVMM has started, the CLI holds the handle, and OpenVMM does not. - wait_for(&root.path.join("build").join("fake-openvmm-.json")); + wait_for(&root.below(&sandbox_relative("build/fake-openvmm-.json"))); let value = unrelated.as_raw_handle() as usize; let openvmm = children_named(cli.id(), "openvmm.exe"); let cli_holds = holds(cli.id(), value); @@ -2336,7 +2332,15 @@ fn a_run_passes_openvmm_no_handle_of_its_caller() { fn batch_openvmm(root: &Root) -> (BTreeMap, PathBuf, String) { let marker = root.path.join("ran"); let wrapper = root.path.join("openvmm.cmd"); - fs::write(&wrapper, format!("@echo ran>\"{}\"\r\n", marker.display())).unwrap(); + fs::write( + &wrapper, + format!( + "@echo ran>\"{}\"\r\n@rem {}\r\n", + marker.display(), + "x".repeat(64) + ), + ) + .unwrap(); let environment = BTreeMap::from([( "NVX_OPENVMM".to_owned(), wrapper.to_str().unwrap().to_owned(), @@ -2355,6 +2359,14 @@ fn a_run_refuses_a_batch_file_as_openvmm() { // cmd.exe would read the `&` of the scratch image's path as a second command. let root = Root::new(); let (environment, marker, refusal) = batch_openvmm(&root); + let refusal = if cfg!(target_arch = "aarch64") { + format!( + "error[unavailable]: ARM64 OpenVMM {}: not a Windows PE executable\n", + environment["NVX_OPENVMM"] + ) + } else { + refusal + }; let scratch = root.path.join("scratch&echo.ext4"); fs::copy(root.path.join("scratch.ext4"), &scratch).unwrap(); let layer = format!( diff --git a/nvx-rs/tests/cli/download.rs b/nvx-rs/tests/cli/download.rs index ebdeab97e..aa94b8854 100644 --- a/nvx-rs/tests/cli/download.rs +++ b/nvx-rs/tests/cli/download.rs @@ -7,7 +7,7 @@ use std::path::Path; use std::process::Command; use std::time::{Duration, Instant}; -use crate::harness::{LAYER, NVX_RS, OPENVMM, Root, SCRATCH, UUID}; +use crate::harness::{LAYER, NVX_RS, OPENVMM, Root, SCRATCH, UUID, sandbox_relative}; /// The files of a test root that a release installs. const INSTALLED: [&str; 6] = [ @@ -26,11 +26,20 @@ fn checkout(removed: &[&str]) -> Root { root.write("SOURCE-MANIFEST.json", "{}\n"); fs::create_dir(root.below(".git")).unwrap(); for relative in removed { - root.remove(relative); + remove_installed(&root, relative); } root } +/// Removes both fixture ISAs when the same case invokes explicit-x86 run and native sandbox. +fn remove_installed(root: &Root, relative: &str) { + root.remove(relative); + let native = sandbox_relative(relative); + if native != relative && root.below(&native).is_file() { + root.remove(&native); + } +} + /// Asserts that no download left its staging directory in the root's `build/`. fn assert_no_staging(root: &Root) { let Ok(entries) = fs::read_dir(root.below("build")) else { @@ -53,6 +62,14 @@ fn run_downloads_the_release_of_a_checkout_without_artifacts() { vec!["sandbox", "run", "--layer", LAYER, "--scratch", SCRATCH], ] { let outcome = root.nvx_rs(&arguments); + if cfg!(target_arch = "aarch64") && arguments[0] == "sandbox" { + outcome.assert_error( + "unsupported", + &["ARM64 release downloads are not qualified"], + ); + assert_no_staging(&root); + continue; + } assert_eq!(outcome.status, Some(1), "{}", outcome.describe()); assert!(outcome.stdout.is_empty(), "{}", outcome.describe()); let stderr = outcome.stderr(); @@ -112,6 +129,16 @@ fn run_downloads_nothing_over_some_artifacts() { vec!["sandbox", "run", "--layer", LAYER, "--scratch", SCRATCH], ] { let outcome = root.nvx_rs(&arguments); + if cfg!(target_arch = "aarch64") && arguments[0] == "sandbox" { + outcome.assert_error( + "unsupported", + &["ARM64 release downloads are not qualified"], + ); + let mut after = root.snapshot(); + assert!(after.remove("build/.nvx-download.lock").is_some()); + assert_eq!(after, before, "{arguments:?}"); + continue; + } assert_eq!(outcome.status, Some(1), "{}", outcome.describe()); assert!(outcome.stdout.is_empty(), "{}", outcome.describe()); let stderr = outcome.stderr(); @@ -322,9 +349,17 @@ fn start_downloads_the_release_of_a_checkout_without_artifacts() { let outcome = root.nvx_rs(&crate::provision(&root, "state", "")); assert_eq!(outcome.status, Some(0), "{}", outcome.describe()); for relative in INSTALLED { - root.remove(relative); + remove_installed(&root, relative); } let outcome = root.nvx_rs(&crate::lifecycle("start", &root, "state")); + if cfg!(target_arch = "aarch64") { + outcome.assert_error( + "unsupported", + &["ARM64 release downloads are not qualified"], + ); + assert_no_staging(&root); + return; + } assert_eq!(outcome.status, Some(1), "{}", outcome.describe()); let stderr = outcome.stderr(); let lines: Vec<&str> = stderr.lines().collect(); @@ -401,7 +436,7 @@ fn the_variable_turns_downloads_off_with_0() { fn roots_that_are_no_git_checkout_download_nothing() { let root = Root::new(); for relative in INSTALLED { - root.remove(relative); + remove_installed(&root, relative); } root.write("SOURCE-MANIFEST.json", "{}\n"); let outcome = root.nvx_rs(&["run", "--dry-run"]); diff --git a/nvx-rs/tests/cli/dry_run.rs b/nvx-rs/tests/cli/dry_run.rs index f8d796528..5adc0d92f 100644 --- a/nvx-rs/tests/cli/dry_run.rs +++ b/nvx-rs/tests/cli/dry_run.rs @@ -5,7 +5,9 @@ use proptest::prelude::*; use proptest::test_runner::{Config, TestRunner}; -use crate::harness::{Content, Hosts, LAYER, OPENVMM, Outcome, Root, SCRATCH, in_parallel, native}; +use crate::harness::{ + Content, Hosts, LAYER, OPENVMM, Outcome, Root, SCRATCH, in_parallel, native, sandbox_relative, +}; use crate::openvmm::{ DISCONNECTED_CONSOLE, NO_CONSOLE, Options, TERMINAL_CONSOLE, WORKLOAD, child, cold_boot, one_shot, restore, share_token, snapshot, @@ -1194,11 +1196,22 @@ fn dry_runs_print_the_openvmm_command_and_launch_nothing() { .collect(); in_parallel(&cases, |case| { let root = Root::new(); + let fixture = |text: &str| { + if case + .arguments + .first() + .is_some_and(|argument| argument == "sandbox") + { + sandbox_relative(text) + } else { + text.to_owned() + } + }; for (relative, content) in &case.files { - root.write_content(relative, *content); + root.write_content(&fixture(relative), *content); } for relative in &case.removed { - root.remove(relative); + root.remove(&fixture(relative)); } let before = root.snapshot(); let arguments: Vec<&str> = case.arguments.iter().map(String::as_str).collect(); diff --git a/nvx-rs/tests/cli/formats.rs b/nvx-rs/tests/cli/formats.rs index bf0178ffd..e8a5a4977 100644 --- a/nvx-rs/tests/cli/formats.rs +++ b/nvx-rs/tests/cli/formats.rs @@ -15,7 +15,7 @@ use nvx::host::sandbox::scratch::ScratchBinding; use nvx::host::sandbox::{LayerRole, SandboxLayer}; use serde_json::{Value, json}; -use crate::harness::{HYPERVISOR, Root, UUID, entries, in_parallel, json}; +use crate::harness::{HYPERVISOR, Root, UUID, entries, in_parallel, json, sandbox_relative}; use crate::lifecycle::{PROVISIONED, provision, sandbox}; use crate::openvmm::{Options, WORKLOAD, assert_checked_blocks, assert_opened_images, managed}; @@ -207,14 +207,15 @@ fn configurations_of_every_format_start_and_exec() { .as_str() .unwrap() .to_owned(); - let mut options = - Options::parse(&root.openvmm_arguments("build/fake-openvmm-state.json")); + let mut options = Options::parse( + &root.openvmm_arguments(&sandbox_relative("build/fake-openvmm-state.json")), + ); assert_checked_blocks(&root, &options.take("--microvm-sandbox-block")); assert_opened_images(&root, "state"); let mut expected = openvmm(managed("state", &endpoint)); expected.take("--microvm-sandbox-block"); expected.assert_matches(&root, &options, name); - let mapped = root.below("build/fake-openvmm-state-maps.json"); + let mapped = root.below(&sandbox_relative("build/fake-openvmm-state-maps.json")); match maps { Some(table) => assert_eq!(json(&mapped), *table, "{name}"), None => assert!(!mapped.exists(), "{name}"), diff --git a/nvx-rs/tests/cli/harness.rs b/nvx-rs/tests/cli/harness.rs index 063b6503e..98279ef8f 100644 --- a/nvx-rs/tests/cli/harness.rs +++ b/nvx-rs/tests/cli/harness.rs @@ -44,6 +44,18 @@ pub(crate) const OPENVMM: &str = "openvmm/target/release/openvmm{exe}"; /// How the lines that `nvx-rs` prints end on this host, as Python's `print` ends them. pub(crate) const LINE_END: &str = if cfg!(windows) { "\r\n" } else { "\n" }; +/// The native sandbox fixture's counterpart of a legacy guest-build path. +pub(crate) fn sandbox_relative(relative: &str) -> String { + if cfg!(target_arch = "aarch64") { + relative + .replace("build/vmlinux", "build/aarch64/Image") + .replace("build/initramfs", "build/aarch64/initramfs") + .replace("build/fake-openvmm", "build/aarch64/fake-openvmm") + } else { + relative.to_owned() + } +} + /// The hosts on which a case applies. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum Hosts { @@ -168,6 +180,24 @@ impl Root { for (relative, text) in FILES { root.write(relative, text); } + if cfg!(target_arch = "aarch64") { + let mut image = [0_u8; 64]; + image[16..24].copy_from_slice(&4096_u64.to_le_bytes()); + image[24..32].copy_from_slice(&10_u64.to_le_bytes()); + image[56..60].copy_from_slice(b"ARM\x64"); + root.write(&sandbox_relative("build/vmlinux"), image); + root.write(&sandbox_relative("build/initramfs.cpio.gz"), "initramfs\n"); + root.write( + &sandbox_relative("build/initramfs.cpio.gz.packages.json"), + r#"{"architecture":"aarch64"}"#, + ); + for (name, member, digest) in &PROVENANCE[1..] { + root.write( + &sandbox_relative(&format!("build/{name}")), + format!("{{\n \"format\": 1,\n \"{member}\": \"{digest}\"\n}}\n"), + ); + } + } for relative in DIRECTORIES { fs::create_dir_all(root.below(relative)).unwrap(); } diff --git a/nvx-rs/tests/cli/lifecycle.rs b/nvx-rs/tests/cli/lifecycle.rs index 7078aba9e..a5e2c86af 100644 --- a/nvx-rs/tests/cli/lifecycle.rs +++ b/nvx-rs/tests/cli/lifecycle.rs @@ -21,7 +21,7 @@ use serde_json::{Value, json}; use crate::harness::mode; use crate::harness::{ Content, HYPERVISOR, Hosts, LAYER, OPENVMM, Outcome, PROVENANCE, Root, SCRATCH, UUID, entries, - in_parallel, is_hex, json, + in_parallel, is_hex, json, sandbox_relative, }; use crate::openvmm::{ NO_NETWORK, NetworkPolicy, Options, WORKLOAD, assert_checked_blocks, assert_checked_distro, @@ -582,14 +582,17 @@ fn check_lifecycle(case: &Configuration) { let (endpoint, process) = assert_runtime_record(&root, case.state, &json(&state.join("runtime.json"))); assert_capability(&state); - let mut options = - Options::parse(&root.openvmm_arguments(&format!("build/fake-openvmm-{token}.json"))); + let mut options = Options::parse(&root.openvmm_arguments(&sandbox_relative(&format!( + "build/fake-openvmm-{token}.json" + )))); assert_checked_blocks(&root, &options.take("--microvm-sandbox-block")); assert_opened_images(&root, token); let mut expected = (case.openvmm)(managed(case.state, &endpoint)); expected.take("--microvm-sandbox-block"); expected.assert_matches(&root, &options, &context); - let maps = root.below(&format!("build/fake-openvmm-{token}-maps.json")); + let maps = root.below(&sandbox_relative(&format!( + "build/fake-openvmm-{token}-maps.json" + ))); match &case.maps { Some(table) => assert_eq!(json(&maps), *table, "{context}"), None => assert!(!maps.exists(), "{context}"), @@ -676,7 +679,9 @@ fn lifecycle_of_a_sandbox_whose_upper_is_in_ram() { assert_eq!(entries(&state), running(), "{context}"); let (endpoint, process) = assert_runtime_record(&root, "state", &json(&state.join("runtime.json"))); - let mut options = Options::parse(&root.openvmm_arguments("build/fake-openvmm-state.json")); + let mut options = Options::parse( + &root.openvmm_arguments(&sandbox_relative("build/fake-openvmm-state.json")), + ); assert_checked_distro(&root, &options.take("--microvm-sandbox-block")); assert_opened_distro(&root, "state"); let mut expected = managed("state", &endpoint).in_ram(); @@ -1047,7 +1052,7 @@ fn managed_execs_exit_with_the_workload_status() { outcome.assert_error(code, &fragments); // A refusal changes nothing, but the fake OpenVMM's log of the connections to its // control endpoint, and the workload did not run. - let log = "build/fake-openvmm-state-sessions.log".to_owned(); + let log = sandbox_relative("build/fake-openvmm-state-sessions.log"); let mut after = root.snapshot(); after.insert(log.clone(), before[&log].clone()); assert_eq!(after, before, "{}", case.name); diff --git a/nvx-rs/tests/cli/one_shot.rs b/nvx-rs/tests/cli/one_shot.rs index c209e45c7..3481a9d84 100644 --- a/nvx-rs/tests/cli/one_shot.rs +++ b/nvx-rs/tests/cli/one_shot.rs @@ -3,7 +3,9 @@ //! standard streams, which carry the workload's output, or OpenVMM's, byte for byte, and exits //! with OpenVMM's status, which a one-shot sandbox's OpenVMM takes from its workload. -use crate::harness::{Content, LAYER, OPENVMM, Root, SCRATCH, in_parallel, native}; +use crate::harness::{ + Content, LAYER, OPENVMM, Root, SCRATCH, in_parallel, native, sandbox_relative, +}; use crate::openvmm::{ DISCONNECTED_CONSOLE, NO_CONSOLE, NO_NETWORK, NetworkPolicy, Options, TERMINAL_CONSOLE, WORKLOAD, assert_checked_blocks, assert_checked_distro, assert_opened_distro, @@ -667,11 +669,22 @@ fn one_shot_runs_exit_with_the_workload_or_openvmm_status() { }); in_parallel(&cases, |case| { let root = Root::new(); + let fixture = |text: &str| { + if case + .arguments + .first() + .is_some_and(|argument| argument == "sandbox") + { + sandbox_relative(text) + } else { + text.to_owned() + } + }; for (relative, content) in &case.files { - root.write_content(relative, *content); + root.write_content(&fixture(relative), *content); } for relative in &case.removed { - root.remove(relative); + root.remove(&fixture(relative)); } let arguments: Vec<&str> = case.arguments.iter().map(String::as_str).collect(); let environment: Vec<(&str, &str)> = case @@ -707,7 +720,12 @@ fn one_shot_runs_exit_with_the_workload_or_openvmm_status() { ); // The `>> ` line splits back into the arguments that OpenVMM received. if let Some(record) = case.record { - assert_eq!(root.openvmm_arguments(record), arguments, "{}", case.name); + assert_eq!( + root.openvmm_arguments(&fixture(record)), + arguments, + "{}", + case.name + ); } let mut options = Options::parse(arguments); let mut expected = case.options.clone(); diff --git a/nvx-rs/tests/cli/openvmm.rs b/nvx-rs/tests/cli/openvmm.rs index b25d02c20..f7b41ece8 100644 --- a/nvx-rs/tests/cli/openvmm.rs +++ b/nvx-rs/tests/cli/openvmm.rs @@ -10,7 +10,7 @@ use std::sync::LazyLock; use serde_json::{Value, json}; -use crate::harness::{Content, HYPERVISOR, Root, is_hex}; +use crate::harness::{Content, HYPERVISOR, Root, is_hex, sandbox_relative}; /// The options of OpenVMM that take no value. const FLAGS: [&str; 3] = [ @@ -189,8 +189,17 @@ pub(crate) fn one_shot() -> Options { .flag("--single-process") .with("--hypervisor", [HYPERVISOR]) .with("--memory", ["256M"]) - .with("--kernel", ["{root/build/vmlinux}"]) - .with("--initrd", ["{root/build/initramfs.cpio.gz}"]) + .with( + "--kernel", + [format!("{{root/{}}}", sandbox_relative("build/vmlinux"))], + ) + .with( + "--initrd", + [format!( + "{{root/{}}}", + sandbox_relative("build/initramfs.cpio.gz") + )], + ) .sandbox_kernel("", &WORKLOAD) } @@ -425,7 +434,9 @@ pub(crate) fn assert_opened_distro(root: &Root, token: &str) { /// Asserts that the fake OpenVMM of the launch whose records are named for `token` read the block /// devices of `expected`, each with its role and content, whole, in order. fn assert_opened(root: &Root, token: &str, expected: &[(&str, &str)]) { - let opened = root.json(&format!("build/fake-openvmm-{token}-blocks.json")); + let opened = root.json(&sandbox_relative(&format!( + "build/fake-openvmm-{token}-blocks.json" + ))); let opened: Vec<(&str, &str)> = opened .as_array() .unwrap() diff --git a/nvx-rs/tests/cli/validation.rs b/nvx-rs/tests/cli/validation.rs index 441e54fc9..97230ac84 100644 --- a/nvx-rs/tests/cli/validation.rs +++ b/nvx-rs/tests/cli/validation.rs @@ -2,7 +2,9 @@ //! anything changes: each refusal exits with status 1, names what it refuses, leaves the root as it //! was, with no state directory, and launches no OpenVMM. -use crate::harness::{Content, Hosts, LAYER, Outcome, Root, SCRATCH, UUID, in_parallel}; +use crate::harness::{ + Content, Hosts, LAYER, Outcome, Root, SCRATCH, UUID, in_parallel, sandbox_relative, +}; use crate::lifecycle::provision; /// An environment file of one entry. @@ -1450,11 +1452,22 @@ fn validation_errors_exit_1_before_any_side_effect() { .collect(); in_parallel(&cases, |case| { let root = Root::new(); + let fixture = |text: &str| { + if case + .arguments + .first() + .is_some_and(|argument| argument == "sandbox") + { + sandbox_relative(text) + } else { + text.to_owned() + } + }; for (relative, content) in &case.files { - root.write_content(relative, *content); + root.write_content(&fixture(relative), *content); } for relative in &case.removed { - root.remove(relative); + root.remove(&fixture(relative)); } let before = root.snapshot(); let arguments: Vec<&str> = case.arguments.iter().map(String::as_str).collect(); @@ -1462,7 +1475,7 @@ fn validation_errors_exit_1_before_any_side_effect() { let fragments: Vec = case .fragments .iter() - .map(|fragment| root.expand(fragment)) + .map(|fragment| root.expand(&fixture(fragment))) .collect(); let fragments: Vec<&str> = fragments.iter().map(String::as_str).collect(); eprintln!("case: {}", case.name); diff --git a/nvx-rs/tests/managed_lifecycle.rs b/nvx-rs/tests/managed_lifecycle.rs index bea08f7fc..a1eaab25a 100644 --- a/nvx-rs/tests/managed_lifecycle.rs +++ b/nvx-rs/tests/managed_lifecycle.rs @@ -15,12 +15,19 @@ //! - `NVX_MANAGED_LIFECYCLE_TIMEOUT`: the longest wait for each operation, in seconds. //! - `NVX_MANAGED_LIFECYCLE_OUTPUT_DIR`: receives OpenVMM's output, `managed-lifecycle.log`, and //! the guest console, `managed-lifecycle-guest.log`, which the scenario then scans. +//! - `NVX_MANAGED_LIFECYCLE_PROCESSORS`: optionally asserts the fixture's online CPU count. +//! - `NVX_MANAGED_LIFECYCLE_ROOT_MODE`: `direct` (the default) or `sandbox`, whose existing +//! working-directory refusal reports status 125 rather than a direct-agent error record. +//! - `NVX_MANAGED_LIFECYCLE_PREVIOUS_STATE`: optionally asserts `present` or `absent` for the +//! state left by an earlier cycle using the same scratch, or a fresh RAM upper. //! //! The test also records OpenVMM's process ID in `NVX_ADVERSARIAL_OPENVMM_PID_JOURNAL`, if set, //! as the microVM scenarios do for the adversarial campaigns. //! //! These checks were `run_managed_lifecycle` of `scripts/nvx_tools/microvm_tests.py`, which drove //! the control session from Python until microsoft/nvx#359 left the protocol to nvx-rs alone. +//! Stale-target checks send an old numeric request ID through the current session; they do not +//! replay an old-epoch outer record or establish old-epoch wire/output rejection. #![cfg(all(feature = "host", any(target_os = "linux", windows)))] use std::env; @@ -115,6 +122,31 @@ impl Guest { .expect("the control endpoint accepts the capability") } + /// Starts a workload that stays active after its readiness marker. + fn waiting(&self, session: &mut ControlSession, body: &str) -> u64 { + let deadline = Instant::now() + self.timeout; + let workload = Workload { + argv: vec![ + "/bin/sh".to_owned(), + "-c".to_owned(), + format!("printf ready; {body}"), + ], + timeout_ms: 0, + cwd: None, + environment: WorkloadEnvironment::Default, + }; + let id = session.start_exec(&workload, deadline).unwrap(); + let mut ready = Vec::new(); + while ready.len() < b"ready".len() { + match session.next_exec_event(id, Some(deadline)).unwrap() { + ExecEvent::Stdout(bytes) => ready.extend(bytes), + event => panic!("the waiting workload did not become ready: {event:?}"), + } + } + assert_eq!(ready, b"ready"); + id + } + /// Runs `argv` in `session` and returns how it ended. fn exec( &self, @@ -135,11 +167,22 @@ impl Guest { let request = session .start_exec(&workload, deadline) .expect("the guest accepts the request"); + self.finish_exec(session, request, deadline, &format!("{argv:?}")) + } + + /// Collects the terminal outcome and all output of an already started request. + fn finish_exec( + &self, + session: &mut ControlSession, + request: u64, + deadline: Instant, + description: &str, + ) -> Outcome { let mut outcome = Outcome::default(); loop { match session .next_exec_event(request, Some(deadline)) - .unwrap_or_else(|error| panic!("{argv:?} did not end: {error}")) + .unwrap_or_else(|error| panic!("{description} did not end: {error}")) { ExecEvent::Stdout(chunk) => outcome.stdout.extend(chunk), ExecEvent::Stderr(chunk) => outcome.stderr.extend(chunk), @@ -374,6 +417,12 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { .expect("NVX_MANAGED_LIFECYCLE_TIMEOUT is a number of seconds"), ); let output_dir = PathBuf::from(variable("NVX_MANAGED_LIFECYCLE_OUTPUT_DIR")); + let sandbox = + env::var_os("NVX_MANAGED_LIFECYCLE_ROOT_MODE").is_some_and(|mode| match mode.to_str() { + Some("sandbox") => true, + Some("direct") => false, + _ => panic!("NVX_MANAGED_LIFECYCLE_ROOT_MODE must be direct or sandbox"), + }); fs::create_dir_all(&output_dir).unwrap(); let log = File::create(output_dir.join("managed-lifecycle.log")).unwrap(); let capability = generate_capability().unwrap(); @@ -416,6 +465,48 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { session .ping(Instant::now() + timeout) .expect("the guest agent is ready"); + if let Ok(processors) = env::var("NVX_MANAGED_LIFECYCLE_PROCESSORS") { + let processors: u32 = processors + .parse() + .expect("the fixture VP count is an integer"); + assert!(processors > 0); + let expected = if processors == 1 { + "0\n".to_owned() + } else { + format!("0-{}\n", processors - 1) + }; + let online = guest.exec( + &mut session, + &["/bin/cat", "/sys/devices/system/cpu/online"], + 5_000, + None, + WorkloadEnvironment::Default, + ); + assert!( + online.printed(&expected), + "the fixture's online CPUs: {online:?}" + ); + println!("NVX-MANAGED-TOPOLOGY-OK cpus={processors}"); + } + if let Ok(expected) = env::var("NVX_MANAGED_LIFECYCLE_PREVIOUS_STATE") { + assert!(matches!(expected.as_str(), "present" | "absent")); + let state = guest.exec( + &mut session, + &[ + "/bin/sh", + "-c", + "if [ -e /tmp/nvx-managed-state ]; then printf present; else printf absent; fi", + ], + 5_000, + None, + WorkloadEnvironment::Default, + ); + assert!( + state.printed(&expected), + "the fixture's persisted state: {state:?}" + ); + println!("NVX-MANAGED-PERSISTENCE-OK expected={expected}"); + } let exact = vec!["EMPTY=".to_owned(), COMPLEX.to_owned()]; let first = guest.exec( &mut session, @@ -523,6 +614,60 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { after_timeout.printed("still-usable"), "the guest was not usable after a timeout: {after_timeout:?}" ); + let active = guest.waiting(&mut session, "sleep 5; printf survived-stale; exit 0"); + let other = Workload { + argv: vec!["/bin/true".to_owned()], + timeout_ms: 0, + cwd: None, + environment: WorkloadEnvironment::Default, + }; + for attempt in 0..2 { + let deadline = Instant::now() + timeout; + let rejected = session.start_exec(&other, deadline).unwrap(); + assert_ne!(active, rejected); + assert_eq!( + session.next_exec_event(rejected, Some(deadline)).unwrap(), + ExecEvent::Rejected { + status: 16, + category: nvx::protocol::BUSY.to_owned(), + } + ); + if attempt == 0 { + // The second busy response orders this stale cancel before natural completion. + session.cancel_exec(rejected, deadline).unwrap(); + } + } + let survived = guest.finish_exec( + &mut session, + active, + Instant::now() + timeout, + "the active workload after stale-target cancellation", + ); + assert!(survived.printed("survived-stale"), "{survived:?}"); + println!("NVX-MANAGED-BUSY-STALE-TARGET-OK natural-success"); + + let cancelled = guest.waiting(&mut session, "exec sleep 30"); + let deadline = Instant::now() + timeout; + session.cancel_exec(cancelled, deadline).unwrap(); + assert_eq!( + session.next_exec_event(cancelled, Some(deadline)).unwrap(), + ExecEvent::Exit { + category: ExitCategory::Cancelled, + status: 137, + } + ); + println!("NVX-MANAGED-TARGET-CANCEL-OK status=137"); + let nonzero = guest.exec( + &mut session, + &["/bin/sh", "-c", "printf out; printf err >&2; exit 7"], + 5_000, + None, + WorkloadEnvironment::Default, + ); + assert_eq!(nonzero.exit, Some((ExitCategory::Exit, 7)), "{nonzero:?}"); + assert_eq!(nonzero.stdout, b"out"); + assert_eq!(nonzero.stderr, b"err"); + println!("NVX-MANAGED-NONZERO-DRAIN-OK status=7"); // A workload that leaves a descendant in a session of its own, outside the workload's process // group, which keeps the workload's output open, ends at once with its own status: the guest // finds the descendant in the workload's cgroup and ends it. The next workload runs, which the @@ -551,8 +696,8 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { after_detached.printed("still-usable"), "the guest did not settle a workload's descendant: {after_detached:?}" ); - // The guest refuses a working directory that the workload cannot enter, with its error - // number and a diagnostic that names it, before anything of the workload runs. + // Both agents refuse the workload before it runs; the sandbox preserves its helper's + // diagnostic and status 125, while the direct agent sends a typed error with its errno. for (description, cwd, error) in REFUSED_CWDS { let refused = guest.exec( &mut session, @@ -562,11 +707,16 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { WorkloadEnvironment::Default, ); let diagnostic = String::from_utf8_lossy(&refused.stderr); + let (expected_exit, expected_refusal, verb) = if sandbox { + (Some((ExitCategory::Exit, 125)), None, "use") + } else { + (None, Some((error, CWD_FAILED.to_owned())), "enter") + }; assert!( - refused.exit.is_none() - && refused.refusal == Some((error, CWD_FAILED.to_owned())) + refused.exit == expected_exit + && refused.refusal == expected_refusal && refused.stdout.is_empty() - && diagnostic.contains(&format!("cannot enter working directory {cwd}: ")), + && diagnostic.contains(&format!("cannot {verb} working directory {cwd}: ")), "the {description} working directory {cwd} was not refused clearly: {refused:?} \ ({diagnostic})" ); @@ -587,6 +737,30 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { "a refused workload ran, or the guest was not usable after refusals: {after_refusals:?}" ); + let old_request = guest.waiting(&mut session, "exec sleep 30"); + drop(session); + let mut session = guest.connect(); + let new_request = guest.waiting(&mut session, "sleep 5; printf after-reset; exit 0"); + assert_ne!(old_request, new_request); + let deadline = Instant::now() + timeout; + session.cancel_exec(old_request, deadline).unwrap(); + let barrier = session.start_exec(&other, deadline).unwrap(); + assert_eq!( + session.next_exec_event(barrier, Some(deadline)).unwrap(), + ExecEvent::Rejected { + status: 16, + category: nvx::protocol::BUSY.to_owned(), + } + ); + let after_reset = guest.finish_exec( + &mut session, + new_request, + Instant::now() + timeout, + "the new session's active workload after an old target was cancelled", + ); + assert!(after_reset.printed("after-reset"), "{after_reset:?}"); + println!("NVX-MANAGED-RESET-RECONNECT-OK stale-target-survived"); + // Stop ends the VM, and OpenVMM exits successfully once it has written its report. session .stop(Instant::now() + timeout) diff --git a/scripts/nvx.py b/scripts/nvx.py index 1cd8334b2..80ba30a40 100644 --- a/scripts/nvx.py +++ b/scripts/nvx.py @@ -265,17 +265,15 @@ def command_build_initramfs(args: argparse.Namespace) -> None: def command_build_distro_layer(args: argparse.Namespace) -> None: + architecture = getattr(args, "architecture", "x86_64") + selected = BuildConfig.distro_layer_config(args.guest, architecture) build_distro_layer( DistroLayerBuildConfig( guest=args.guest, - work=( - BuildConstants.BUILD_DIR - / InitramfsBuildConstants.DISTRO_WORK_DIRECTORY_TEMPLATE.format( - guest=args.guest - ) - ), - output=args.output, + work=selected.work, + output=args.output if args.output is not None else selected.output, replace=args.replace, + architecture=architecture, ) ) @@ -627,7 +625,11 @@ def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace: distro_layer.add_argument( "--output", type=Path, - default=artifact_path(UbuntuBuildConstants.DISTRO_NAME), + ) + distro_layer.add_argument( + "--architecture", + choices=GuestRustBuildConstants.ARCHITECTURES, + default="x86_64", ) distro_layer.add_argument("--replace", action="store_true") distro_layer.set_defaults(handler=command_build_distro_layer) diff --git a/scripts/nvx_tools/benchmark.py b/scripts/nvx_tools/benchmark.py index 0e19d8cd2..79242a230 100644 --- a/scripts/nvx_tools/benchmark.py +++ b/scripts/nvx_tools/benchmark.py @@ -2835,6 +2835,7 @@ def run_guest_script( host_state_on_timeout: Callable[[subprocess.Popen[bytes]], str] | None = None, on_stderr: Callable[[bytes], None] | None = None, expected_exit_status: int = 0, + architecture: str | None = None, ) -> GuestCommandResult: """Run ``script`` once the guest boots and wait for ``completion_marker``. @@ -2858,6 +2859,7 @@ def run_guest_script( then holds alone, and the log and the error reports interleave the two streams by whole lines. ``SeparatedOutput`` keeps the last MiB of each. """ + monitor = TimeAbiMonitor(command, architecture=architecture) environment = os.environ.copy() environment["OPENVMM_LOG"] = "off" started_ns = time.perf_counter_ns() @@ -2868,7 +2870,6 @@ def run_guest_script( separate_stderr=on_stderr is not None, ) process = interaction.process - monitor = TimeAbiMonitor(command) query_status = time_abi_status and monitor.cold_boot if windows_cpus is not None: set_windows_affinity(process.pid, windows_cpus) diff --git a/scripts/nvx_tools/build.py b/scripts/nvx_tools/build.py index ecb1c054a..abba37746 100644 --- a/scripts/nvx_tools/build.py +++ b/scripts/nvx_tools/build.py @@ -1158,19 +1158,36 @@ def build_distro_layer(config: DistroLayerBuildConfig) -> None: initramfs_config = InitramfsBuildConfig( guest=descriptor.name, work=config.work, + architecture=config.architecture, ) root = _prepare_guest_root(initramfs_config, descriptor) helpers = _install_guest_files(initramfs_config, root, descriptor) ubuntu.apply_metadata_policy(root) + packages = ( + ubuntu.package_records(root) + if config.architecture == "x86_64" + else ubuntu.package_records(root, architecture=config.architecture) + ) if (root / "sbin" / "init").exists() or any( - package["name"] == "systemd" for package in ubuntu.package_records(root) + package["name"] == "systemd" for package in packages ): raise ScriptError("systemd is unsupported in the Ubuntu sandbox layer profile") _normalize_initramfs_metadata(root) - input_sha256 = ubuntu.converter_input_sha256(_guest_customization_files(descriptor)) + input_sha256 = ( + ubuntu.converter_input_sha256(_guest_customization_files(descriptor)) + if config.architecture == "x86_64" + else ubuntu.converter_input_sha256( + ubuntu.customization_files(config.architecture), + architecture=config.architecture, + ) + ) filesystem_uuid = ubuntu.erofs_uuid(input_sha256) - document = ubuntu.package_manifest(root, helpers) + document = ( + ubuntu.package_manifest(root, helpers) + if config.architecture == "x86_64" + else ubuntu.package_manifest(root, helpers, architecture=config.architecture) + ) document.update( { "artifact": output.name, @@ -1260,23 +1277,56 @@ def scratch_template_environment() -> dict[str, str]: return environment -def _mkfs_ext4() -> str: - """Return mkfs.ext4 from PATH, or else from the system directories that - distributions such as Debian leave out of an unprivileged user's PATH.""" +def _e2fsprogs_tool(name: str) -> str: + """Find an e2fsprogs tool, including Debian's system directories.""" search = os.pathsep.join( directory for directory in (os.environ.get("PATH", ""), "/usr/sbin", "/sbin") if directory ) - executable = shutil.which("mkfs.ext4", path=search) + executable = shutil.which(name, path=search) if executable is None: raise ScriptError( - "mkfs.ext4 was not found on PATH, in /usr/sbin, or in /sbin; " + f"{name} was not found on PATH, in /usr/sbin, or in /sbin; " "install e2fsprogs" ) return executable +def _mkfs_ext4() -> str: + return _e2fsprogs_tool("mkfs.ext4") + + +def _scratch_hash_flags(image: Path) -> int: + with image.open("rb") as volume: + volume.seek(1024) + superblock = volume.read(0x164) + if len(superblock) != 0x164 or superblock[0x38:0x3A] != b"\x53\xef": + raise ScriptError(f"scratch template has no valid ext4 superblock: {image}") + flags = int.from_bytes(superblock[0x160:0x164], "little") + if flags not in (1, 2): + raise ScriptError( + f"scratch template has unexpected directory-hash flags {flags}: {image}" + ) + return flags + + +def _canonicalize_scratch_hash(image: Path) -> None: + # libext2fs chooses signed/unsigned hashing from its compiler's char type. + # Keep the existing signed-hash format; let e2fsprogs update its checksums. + if _scratch_hash_flags(image) == 1: + return + run_checked( + [_e2fsprogs_tool("debugfs"), "-w", os.fspath(image)], + env=scratch_template_environment(), + input_bytes=b"set_super_value flags 1\nclose -a\n", + ) + if _scratch_hash_flags(image) != 1: + raise ScriptError( + f"debugfs did not set the scratch directory-hash format: {image}" + ) + + def build_scratch_template(config: ScratchTemplateBuildConfig) -> None: """Format the sandbox scratch template at `config.output` with mkfs.ext4, and warn when its bytes are not those that @@ -1305,6 +1355,7 @@ def build_scratch_template(config: ScratchTemplateBuildConfig) -> None: [mkfs, *scratch_template_arguments(temporary)], env=scratch_template_environment(), ) + _canonicalize_scratch_hash(temporary) digest = sha256_file(temporary) temporary.replace(output) finally: diff --git a/scripts/nvx_tools/build_config.py b/scripts/nvx_tools/build_config.py index a0ab7066d..98d23ffa1 100644 --- a/scripts/nvx_tools/build_config.py +++ b/scripts/nvx_tools/build_config.py @@ -90,6 +90,13 @@ class DistroLayerBuildConfig: ) output: Path = artifact_path(UbuntuBuildConstants.DISTRO_NAME) replace: bool = False + architecture: str = "x86_64" + + def __post_init__(self) -> None: + if self.architecture not in ("x86_64", "aarch64"): + raise ScriptError( + f"unsupported distro-layer architecture: {self.architecture}" + ) @dataclass(frozen=True, slots=True) @@ -233,18 +240,22 @@ def initramfs_config( @staticmethod def distro_layer_config( guest: str = UbuntuBuildConstants.GUEST_NAME, + architecture: str = "x86_64", ) -> DistroLayerBuildConfig: descriptor = guest_descriptor(guest) + directory = BuildConstants.BUILD_DIR + if architecture != "x86_64": + directory /= architecture return DistroLayerBuildConfig( guest=descriptor.name, work=( - BuildConstants.BUILD_DIR + directory / InitramfsBuildConstants.DISTRO_WORK_DIRECTORY_TEMPLATE.format( guest=descriptor.name ) ), - output=artifact_path( - BuildConstants.DISTRO_NAME_TEMPLATE.format(guest=descriptor.name) - ), + output=directory + / BuildConstants.DISTRO_NAME_TEMPLATE.format(guest=descriptor.name), replace=True, + architecture=architecture, ) diff --git a/scripts/nvx_tools/common.py b/scripts/nvx_tools/common.py index 60192f745..513a22af2 100644 --- a/scripts/nvx_tools/common.py +++ b/scripts/nvx_tools/common.py @@ -71,8 +71,13 @@ def bytes_to_mib(value: int | float) -> float: return value / (1024 * 1024) -def artifact_path(name: str) -> Path: - return BuildConstants.BUILD_DIR / name +def artifact_path(name: str, *, architecture: str = "x86_64") -> Path: + if architecture not in ("x86_64", "aarch64"): + raise ScriptError(f"unsupported artifact architecture: {architecture}") + directory = BuildConstants.BUILD_DIR + if architecture != "x86_64": + directory /= architecture + return directory / name def path_exists(path: Path) -> bool: diff --git a/scripts/nvx_tools/managed_exec_tests.py b/scripts/nvx_tools/managed_exec_tests.py index b45baa5c5..0e41d01e8 100644 --- a/scripts/nvx_tools/managed_exec_tests.py +++ b/scripts/nvx_tools/managed_exec_tests.py @@ -35,11 +35,24 @@ class WorkloadArtifacts: scratch_template: Path -def workload_artifact_paths() -> tuple[tuple[Path, str], ...]: +def workload_artifact_paths( + *, architecture: str = "x86_64" +) -> tuple[tuple[Path, str], ...]: """Return the paths of the Ubuntu workload layer, its manifest, and the sandbox scratch template, which the public scenarios read, each with its description.""" - distro = artifact_path(UbuntuBuildConstants.DISTRO_NAME) + distro = ( + artifact_path(UbuntuBuildConstants.DISTRO_NAME) + if architecture == "x86_64" + else artifact_path(UbuntuBuildConstants.DISTRO_NAME, architecture=architecture) + ) + scratch = ( + artifact_path(UbuntuBuildConstants.SCRATCH_TEMPLATE_NAME) + if architecture == "x86_64" + else artifact_path( + UbuntuBuildConstants.SCRATCH_TEMPLATE_NAME, architecture=architecture + ) + ) return ( (distro, "Ubuntu workload layer"), ( @@ -47,16 +60,16 @@ def workload_artifact_paths() -> tuple[tuple[Path, str], ...]: "Ubuntu layer manifest", ), ( - artifact_path(UbuntuBuildConstants.SCRATCH_TEMPLATE_NAME), + scratch, "sandbox scratch template", ), ) -def load_workload_artifacts() -> WorkloadArtifacts: +def load_workload_artifacts(*, architecture: str = "x86_64") -> WorkloadArtifacts: distro, manifest_path, scratch_template = ( require_file(path, description) - for path, description in workload_artifact_paths() + for path, description in workload_artifact_paths(architecture=architecture) ) try: manifest: object = json.loads(manifest_path.read_text(encoding="utf-8")) @@ -64,7 +77,10 @@ def load_workload_artifacts() -> WorkloadArtifacts: raise ScriptError(f"invalid Ubuntu layer manifest: {manifest_path}") from error if not isinstance(manifest, dict): raise ScriptError("Ubuntu layer manifest must contain a UUID string") - layer_uuid = cast(dict[str, object], manifest).get("uuid") + document = cast(dict[str, object], manifest) + if architecture != "x86_64" and document.get("architecture") != "arm64": + raise ScriptError("ARM64 Ubuntu layer manifest must declare architecture arm64") + layer_uuid = document.get("uuid") if not isinstance(layer_uuid, str): raise ScriptError("Ubuntu layer manifest must contain a UUID string") return WorkloadArtifacts(distro, layer_uuid, scratch_template) @@ -174,9 +190,9 @@ def format_errors(errors: list[Exception]) -> str: def run_managed_exec_configuration( - backend: str, *, timeout: float, output_dir: Path + backend: str, *, timeout: float, output_dir: Path, architecture: str = "x86_64" ) -> None: - artifacts = load_workload_artifacts() + artifacts = load_workload_artifacts(architecture=architecture) distro = artifacts.distro layer_uuid = artifacts.layer_uuid scratch_template = artifacts.scratch_template diff --git a/scripts/nvx_tools/microvm_tests.py b/scripts/nvx_tools/microvm_tests.py index 9e63a2931..6d29947be 100644 --- a/scripts/nvx_tools/microvm_tests.py +++ b/scripts/nvx_tools/microvm_tests.py @@ -53,6 +53,7 @@ from .benchmark import ( run_guest_script as _run_guest_script, ) +from .build_config import BuildConfig, KernelBuildConfig from .build_constants import ( BuildConstants, KernelBuildConstants, @@ -66,7 +67,11 @@ sha256_file, ) from .guests import GUEST_NAMES, GuestDescriptor, guest_descriptor -from .managed_exec_tests import run_managed_exec_configuration, workload_artifact_paths +from .managed_exec_tests import ( + WorkloadArtifacts, + run_managed_exec_configuration, + workload_artifact_paths, +) from .microvm_durations import REPORT_NAME, DurationReport from .microvm_shards import configure_parser as configure_shards_parser from .microvm_shards import load_partition, shard_scenarios @@ -784,6 +789,9 @@ def configure_parser(parser: argparse.ArgumentParser) -> None: required=True, ) parser.add_argument("--guest", choices=GUEST_NAMES, default="alpine") + parser.add_argument( + "--architecture", choices=("x86_64", "aarch64"), default="x86_64" + ) parser.add_argument( "--scenario", action="append", @@ -873,6 +881,7 @@ def run_guest_script( boot_marker: bytes = BOOT_MARKER, on_stderr: Callable[[bytes], None] | None = None, expected_exit_status: int = 0, + architecture: str | None = None, ) -> GuestScriptResult: """Run ``script`` in the shell of the guest that ``command`` boots, and require it to print the line ``completion_marker``. @@ -910,6 +919,7 @@ def run_guest_script( time_abi_status=True, on_stderr=on_stderr, expected_exit_status=expected_exit_status, + architecture=architecture, ) output = _guest_script_output(result["text"]) if not contains_output_line(output.encode("utf-8"), completion_marker): @@ -1258,6 +1268,7 @@ def run_guest_boot( memory_mib: int, timeout: float, log_path: Path, + architecture: str | None = None, ) -> None: command = workload_boot_command( executable, @@ -1279,6 +1290,7 @@ def run_guest_boot( GUEST_BOOT_COMPLETION_MARKER, timeout=timeout, log_path=log_path, + architecture=architecture, ) @@ -1458,6 +1470,10 @@ def run_managed_lifecycle( memory_mib: int, timeout: float, output_dir: Path, + processors: int = 1, + layered: WorkloadArtifacts | None = None, + scratch: Path | None = None, + expected_previous_state: str | None = None, ) -> None: """Run the managed lifecycle of a guest whose init hands the boot to the managed agent. @@ -1465,8 +1481,16 @@ def run_managed_lifecycle( The nvx-rs test MANAGED_LIFECYCLE_TEST starts OpenVMM with the command built here and drives the guest's control session with nvx-rs, so that no Python client of the control protocol remains. This checks what the test - leaves: OpenVMM's outcome report and the guest console. + leaves: OpenVMM's outcome report and the guest console. Optional fixture + inputs run the same Rust assertions with a layered root and a selected VP + count; they do not extend the public sandbox CLI or its saved formats. """ + if processors not in MICROVM_PROCESSOR_COUNTS: + raise ScriptError("managed test processors must be one of 1, 2, 4, or 8") + if scratch is not None and layered is None: + raise ScriptError("managed test scratch requires a layered-root fixture") + if expected_previous_state not in (None, "present", "absent"): + raise ScriptError("managed test previous state must be present or absent") cargo = shutil.which("cargo") if cargo is None: raise ScriptError( @@ -1488,15 +1512,37 @@ def run_managed_lifecycle( console_endpoint = _console_endpoint(root) guest_log_path = output_dir / "managed-lifecycle-guest.log" guest_log_path.unlink(missing_ok=True) + cmdline = "quiet loglevel=0" + if layered is not None: + cmdline += ( + f" nvx_sandbox=1 nvx_layer=distro,0xd0003000,erofs:{layered.layer_uuid}" + ) + cmdline += ( + " nvx_scratch=0xd0006000,ext4" + if scratch is not None + else " nvx_overlay_upper=ramfs" + ) command = workload_boot_command( executable.resolve(), backend, kernel.resolve(), initrd.resolve(), memory_mib, - "quiet loglevel=0", + cmdline, boot_console=False, + processors=processors, ) + if layered is not None: + command.extend( + [ + "--microvm-sandbox-block", + f"distro:file:{layered.distro.resolve()},ro", + ] + ) + if scratch is not None: + command.extend( + ["--microvm-sandbox-block", f"scratch:file:{scratch.resolve()}"] + ) command.extend( ( "--microvm-workload-identity", @@ -1513,6 +1559,11 @@ def run_managed_lifecycle( ) ) environment = os.environ.copy() + environment.pop("NVX_MANAGED_LIFECYCLE_PREVIOUS_STATE", None) + if expected_previous_state is not None: + environment["NVX_MANAGED_LIFECYCLE_PREVIOUS_STATE"] = ( + expected_previous_state + ) # The test records OpenVMM's process ID for an adversarial campaign # as the microVM scenarios do, but from the nvx-rs directory, so a # relative journal resolves here. @@ -1529,6 +1580,10 @@ def run_managed_lifecycle( "NVX_MANAGED_LIFECYCLE_CONSOLE": console_endpoint, "NVX_MANAGED_LIFECYCLE_TIMEOUT": str(timeout), "NVX_MANAGED_LIFECYCLE_OUTPUT_DIR": os.fspath(output_dir), + "NVX_MANAGED_LIFECYCLE_PROCESSORS": str(processors), + "NVX_MANAGED_LIFECYCLE_ROOT_MODE": ( + "sandbox" if layered is not None else "direct" + ), } ) test_command = managed_lifecycle_test_command(cargo) @@ -1590,6 +1645,7 @@ def run_managed_lifecycle( initrd, memory_mib, "quiet loglevel=0", + processors=processors, ) invalid.extend( ( @@ -11313,7 +11369,13 @@ def require_debug_kernel(config: Path) -> None: def run(args: argparse.Namespace) -> int: validate_openvmm_test_backend(args.backend) descriptor = guest_descriptor(args.guest) + architecture = getattr(args, "architecture", "x86_64") debug_kernel = getattr(args, "debug_kernel", False) + if architecture == "aarch64" and args.scenario is None: + raise ScriptError( + "ARM64 correctness runs require explicit --scenario selection; " + "the full x86 scenario matrix is not qualified for ARM64" + ) shard: tuple[int, int] | None = getattr(args, "shard", None) if shard is not None and ( args.scenario is not None or debug_kernel or descriptor.name != "alpine" @@ -11326,7 +11388,17 @@ def run(args: argparse.Namespace) -> int: if args.memory_mib is None: args.memory_mib = descriptor.default_memory_mib executable = require_file(openvmm_binary_path(), "OpenVMM release binary") - if debug_kernel: + if architecture != "x86_64": + selected = KernelBuildConfig.for_architecture(architecture, debug=debug_kernel) + if debug_kernel: + require_debug_kernel( + selected.output.with_name(f"{selected.output.name}.config") + ) + kernel = require_file( + selected.output, + "microVM Linux direct kernel", + ) + elif debug_kernel: require_debug_kernel(artifact_path(KernelBuildConstants.DEBUG_CONFIG_NAME)) kernel = require_file( artifact_path(KernelBuildConstants.DEBUG_BINARY_NAME), @@ -11337,8 +11409,14 @@ def run(args: argparse.Namespace) -> int: artifact_path(KernelBuildConstants.BINARY_NAME), "microVM Linux direct kernel", ) + initrd_path = ( + artifact_path(descriptor.initramfs_name) + if architecture == "x86_64" + else BuildConfig.initramfs_config(descriptor.name, architecture).output + ) + assert initrd_path is not None initrd = require_file( - artifact_path(descriptor.initramfs_name), + initrd_path, f"microVM {descriptor.distribution} initramfs", ) output_dir: Path = args.output_dir @@ -11377,16 +11455,23 @@ def run(args: argparse.Namespace) -> int: ] if public: try: - for path, description in workload_artifact_paths(): + for path, description in workload_artifact_paths(architecture=architecture): require_file(path, description) except ScriptError as error: + advice = ( + "build them with `build-distro-layer --guest ubuntu " + "--architecture aarch64` and `build-scratch-template " + "--output build/aarch64/ubuntu-smoke-scratch.ext4` on Linux" + if architecture == "aarch64" + else "install them with `nvx-rs download` or build them with " + "`build-guest --guest all`, format the template alone with " + "`build-scratch-template` on Linux" + ) raise ScriptError( f"{error}; {', '.join(public)} " f"{'reads' if len(public) == 1 else 'read'} the Ubuntu distro " - "layer and the sandbox scratch template: install them with " - "`nvx-rs download` or build them with `build-guest --guest " - "all`, format the template alone with `build-scratch-template` " - "on Linux, or name other scenarios with --scenario" + f"layer and the sandbox scratch template: {advice}, " + "or name other scenarios with --scenario" ) from error with DurationReport( @@ -11431,6 +11516,7 @@ def _run_scenarios( ) -> None: """Run the selected scenarios in their fixed order, and time each execution in `report`.""" + architecture = getattr(args, "architecture", "x86_64") if "guest-boot" in scenarios: report.start( "guest-boot", @@ -11446,6 +11532,7 @@ def _run_scenarios( memory_mib=args.memory_mib, timeout=args.timeout, log_path=output_dir / f"{descriptor.name}-guest-boot.log", + architecture=architecture, ) if "guest-identity" in scenarios: report.start( @@ -11631,28 +11718,44 @@ def _run_scenarios( f"Running public managed execution configuration on OpenVMM/{args.backend}", ) run_managed_exec_configuration( - args.backend, timeout=args.timeout, output_dir=output_dir + args.backend, + timeout=args.timeout, + output_dir=output_dir, + architecture=architecture, ) if "sandbox-lifecycle" in scenarios: report.start( "sandbox-lifecycle", f"Running public managed sandbox lifecycle on OpenVMM/{args.backend}", ) - run_sandbox_lifecycle(args.backend, timeout=args.timeout, output_dir=output_dir) + run_sandbox_lifecycle( + args.backend, + timeout=args.timeout, + output_dir=output_dir, + architecture=architecture, + ) if "sandbox-proxy-only" in scenarios: report.start( "sandbox-proxy-only", f"Running public proxy-only sandbox networking on OpenVMM/{args.backend}", ) run_sandbox_proxy_only( - args.backend, timeout=args.timeout, output_dir=output_dir + args.backend, + timeout=args.timeout, + output_dir=output_dir, + architecture=architecture, ) if "sandbox-ram-upper" in scenarios: report.start( "sandbox-ram-upper", f"Running public RAM-upper sandboxes on OpenVMM/{args.backend}", ) - run_sandbox_ram_upper(args.backend, timeout=args.timeout, output_dir=output_dir) + run_sandbox_ram_upper( + args.backend, + timeout=args.timeout, + output_dir=output_dir, + architecture=architecture, + ) if "managed-lifecycle" in scenarios: report.start( "managed-lifecycle", diff --git a/scripts/nvx_tools/sandbox_lifecycle_tests.py b/scripts/nvx_tools/sandbox_lifecycle_tests.py index c6cf3dd34..5f85d5a19 100644 --- a/scripts/nvx_tools/sandbox_lifecycle_tests.py +++ b/scripts/nvx_tools/sandbox_lifecycle_tests.py @@ -37,7 +37,7 @@ KernelBuildConstants, OpenVMMBuildConstants, ) -from .common import sha256_file +from .common import ScriptError, sha256_file from .managed_exec_tests import ( WorkloadArtifacts, bounded_text, @@ -159,11 +159,22 @@ def is_other_runtime_termination(stderr: bytes, pid: int) -> bool: ) -def read_build_provenance() -> dict[str, dict[str, Any]]: +def read_build_provenance(*, architecture: str = "x86_64") -> dict[str, dict[str, Any]]: """Returns the build provenance from which nvx-rs reads the runtime identity of the checkout, by file name.""" provenance: dict[str, dict[str, Any]] = {} - for name in PROVENANCE_NAMES: + if architecture not in ("x86_64", "aarch64"): + raise ScriptError(f"unsupported provenance architecture: {architecture}") + names = ( + PROVENANCE_NAMES + if architecture == "x86_64" + else ( + OpenVMMBuildConstants.PROVENANCE_NAME, + "aarch64/Image.provenance.json", + f"aarch64/{InitramfsBuildConstants.PROVENANCE_NAME}", + ) + ) + for name in names: path = BuildConstants.BUILD_DIR / name try: value: object = json.loads(path.read_text(encoding="utf-8")) @@ -399,12 +410,14 @@ def __init__( output_dir: Path, artifacts: WorkloadArtifacts, root: Path, + architecture: str = "x86_64", ) -> None: self.backend = backend self.timeout = timeout self.output_dir = output_dir self.artifacts = artifacts self.root = root + self.architecture = architecture self.state = root / "state" self.identity_state = root / "identity-state" self.scratch = root / "scratch.ext4" @@ -887,8 +900,13 @@ def write_other_runtime(self) -> None: there that differs from the checkout's in the initramfs.""" build = self.other_root / BuildConstants.BUILD_DIRECTORY_NAME build.mkdir(parents=True) - for name, provenance in read_build_provenance().items(): - if name == InitramfsBuildConstants.PROVENANCE_NAME: + provenance_files = ( + read_build_provenance() + if self.architecture == "x86_64" + else read_build_provenance(architecture=self.architecture) + ) + for name, provenance in provenance_files.items(): + if Path(name).name == InitramfsBuildConstants.PROVENANCE_NAME: digest = provenance.get(INITRAMFS_DIGEST) if not isinstance(digest, str): raise RuntimeError(f"build provenance {name} names no initramfs") @@ -896,7 +914,9 @@ def write_other_runtime(self) -> None: **provenance, INITRAMFS_DIGEST: hashlib.sha256(digest.encode()).hexdigest(), } - (build / name).write_text(json.dumps(provenance), encoding="utf-8") + path = build / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(provenance), encoding="utf-8") def check_other_runtime(self, runtime: dict[str, Any]) -> None: # The CLI of other artifacts, as after an upgrade, reattaches to the VM @@ -1090,8 +1110,10 @@ def run_acceptance(acceptance: LifecycleAcceptance) -> None: shutil.rmtree(acceptance.root) -def run_sandbox_lifecycle(backend: str, *, timeout: float, output_dir: Path) -> None: - artifacts = load_workload_artifacts() +def run_sandbox_lifecycle( + backend: str, *, timeout: float, output_dir: Path, architecture: str = "x86_64" +) -> None: + artifacts = load_workload_artifacts(architecture=architecture) output_dir.mkdir(parents=True, exist_ok=True) root = Path(tempfile.mkdtemp(prefix="nvx-public-lifecycle-")) run_acceptance( @@ -1101,5 +1123,6 @@ def run_sandbox_lifecycle(backend: str, *, timeout: float, output_dir: Path) -> output_dir=output_dir, artifacts=artifacts, root=root, + architecture=architecture, ) ) diff --git a/scripts/nvx_tools/sandbox_proxy_tests.py b/scripts/nvx_tools/sandbox_proxy_tests.py index 9429298cc..75fecc73d 100644 --- a/scripts/nvx_tools/sandbox_proxy_tests.py +++ b/scripts/nvx_tools/sandbox_proxy_tests.py @@ -496,8 +496,10 @@ def check_listeners( ) -def run_sandbox_proxy_only(backend: str, *, timeout: float, output_dir: Path) -> None: - artifacts = load_workload_artifacts() +def run_sandbox_proxy_only( + backend: str, *, timeout: float, output_dir: Path, architecture: str = "x86_64" +) -> None: + artifacts = load_workload_artifacts(architecture=architecture) output_dir.mkdir(parents=True, exist_ok=True) root = Path(tempfile.mkdtemp(prefix="nvx-public-proxy-")) run_acceptance( @@ -507,5 +509,6 @@ def run_sandbox_proxy_only(backend: str, *, timeout: float, output_dir: Path) -> output_dir=output_dir, artifacts=artifacts, root=root, + architecture=architecture, ) ) diff --git a/scripts/nvx_tools/sandbox_ram_upper_tests.py b/scripts/nvx_tools/sandbox_ram_upper_tests.py index 0128c142d..4680a14f3 100644 --- a/scripts/nvx_tools/sandbox_ram_upper_tests.py +++ b/scripts/nvx_tools/sandbox_ram_upper_tests.py @@ -259,8 +259,10 @@ def stop_sandbox(self, description: str) -> None: self.require_processes((), f"after the {description}") -def run_sandbox_ram_upper(backend: str, *, timeout: float, output_dir: Path) -> None: - artifacts = load_workload_artifacts() +def run_sandbox_ram_upper( + backend: str, *, timeout: float, output_dir: Path, architecture: str = "x86_64" +) -> None: + artifacts = load_workload_artifacts(architecture=architecture) output_dir.mkdir(parents=True, exist_ok=True) root = Path(tempfile.mkdtemp(prefix="nvx-public-ram-upper-")) run_acceptance( @@ -270,5 +272,6 @@ def run_sandbox_ram_upper(backend: str, *, timeout: float, output_dir: Path) -> output_dir=output_dir, artifacts=artifacts, root=root, + architecture=architecture, ) ) diff --git a/scripts/nvx_tools/time_abi.py b/scripts/nvx_tools/time_abi.py index b9a966b5e..b49e797a0 100644 --- a/scripts/nvx_tools/time_abi.py +++ b/scripts/nvx_tools/time_abi.py @@ -598,7 +598,9 @@ def _counter_architecture(command: Sequence[str]) -> str: ) kernel = _command_option(command, "--kernel") return ( - "aarch64" if kernel is not None and Path(kernel).name == "Image" else "x86_64" + "aarch64" + if kernel is not None and Path(kernel).name in ("Image", "Image-debug") + else "x86_64" ) @@ -702,11 +704,19 @@ class TimeAbiMonitor: their boot and restore lines only for ``nvx-time status``: when a query ends, a cold boot must have reported a valid boot line, and a restored guest a restore line. The query's runtime line is kept as ``runtime``. + ``architecture`` supplies the selected ISA when known; otherwise the + command's explicit options or maintained product names select it. """ - def __init__(self, command: Sequence[str] = ()) -> None: + def __init__( + self, command: Sequence[str] = (), *, architecture: str | None = None + ) -> None: + if architecture is not None and architecture not in ("x86_64", "aarch64"): + raise ValueError(f"unsupported counter architecture {architecture!r}") self.backend = _command_option(command, "--hypervisor") - self.architecture = _counter_architecture(command) + self.architecture = ( + _counter_architecture(command) if architecture is None else architecture + ) self.cold_boot = is_cold_boot_command(command) self.restored = _command_option(command, "--restore-snapshot") is not None self.boot: dict[str, str] | None = None diff --git a/scripts/test_guest_binaries.py b/scripts/test_guest_binaries.py index 714377e22..945a79242 100644 --- a/scripts/test_guest_binaries.py +++ b/scripts/test_guest_binaries.py @@ -24,6 +24,9 @@ create_linux_source_archive, elf, guest_binaries, + managed_exec_tests, + microvm_tests, + sandbox_lifecycle_tests, ubuntu, ) from nvx_tools.build_constants import ( # noqa: E402 @@ -1502,6 +1505,230 @@ def test_arm_cli_routes_only_explicit_architecture(self): args.handler(args) called.assert_called_once_with(None, architecture="aarch64") + def test_distro_cli_selects_architecture_qualified_work_and_output(self): + for architecture in ("x86_64", "aarch64"): + directory = BuildConstants.BUILD_DIR + if architecture != "x86_64": + directory /= architecture + for output in (None, Path("custom/distro.erofs")): + arguments = [ + "build-distro-layer", + "--guest", + "ubuntu", + "--architecture", + architecture, + ] + if output is not None: + arguments.extend(("--output", str(output))) + args = nvx.parse_args(arguments) + with patch.object(nvx, "build_distro_layer") as called: + args.handler(args) + selected = called.call_args.args[0] + self.assertEqual(selected.architecture, architecture) + self.assertEqual(selected.work.parent, directory) + self.assertEqual( + selected.output, output or directory / "ubuntu-distro.erofs" + ) + self.assertFalse(selected.replace) + with self.assertRaisesRegex(build.ScriptError, "unsupported distro-layer"): + build_config.DistroLayerBuildConfig(architecture="arm64") + + def test_arm_distro_producer_uses_matching_root_packages_and_manifest(self): + with tempfile.TemporaryDirectory() as temporary: + directory = Path(temporary) + root = directory / "root" + root.mkdir() + output = directory / "distro.erofs" + config = build_config.DistroLayerBuildConfig( + guest="ubuntu", + work=directory / "work", + output=output, + architecture="aarch64", + ) + customization = (root / "input",) + helpers = {"nvx-init": {"source_sha256": "b" * 64}} + + def mkfs(command: Sequence[str | os.PathLike[str]]) -> None: + Path(command[-2]).write_bytes(b"unit-test-erofs") + + with ( + patch.object(build, "_require_linux"), + patch.object( + build, "_prepare_guest_root", return_value=root + ) as prepare, + patch.object(build, "_install_guest_files", return_value=helpers), + patch.object(ubuntu, "apply_metadata_policy"), + patch.object(ubuntu, "package_records", return_value=[]) as records, + patch.object(build, "_normalize_initramfs_metadata"), + patch.object( + ubuntu, "customization_files", return_value=customization + ) as sources, + patch.object( + ubuntu, "converter_input_sha256", return_value="a" * 64 + ) as digest, + patch.object( + ubuntu, + "package_manifest", + return_value={"architecture": "arm64", "packages": []}, + ) as manifest, + patch.object(build, "require_tool", return_value="mkfs.erofs"), + patch.object(build, "run_checked", side_effect=mkfs), + contextlib.redirect_stdout(io.StringIO()), + ): + build.build_distro_layer(config) + self.assertEqual(prepare.call_args.args[0].architecture, "aarch64") + records.assert_called_once_with(root, architecture="aarch64") + sources.assert_called_once_with("aarch64") + digest.assert_called_once_with(customization, architecture="aarch64") + manifest.assert_called_once_with(root, helpers, architecture="aarch64") + document = json.loads( + output.with_name("distro.erofs.manifest.json").read_text() + ) + self.assertEqual(document["architecture"], "arm64") + self.assertEqual(document["input_sha256"], "a" * 64) + self.assertEqual(document["uuid"], ubuntu.erofs_uuid("a" * 64)) + + def test_arm_public_scenarios_require_namespaced_matching_layer_inputs(self): + with tempfile.TemporaryDirectory() as temporary: + directory = Path(temporary) + with patch.object(BuildConstants, "BUILD_DIR", directory): + paths = managed_exec_tests.workload_artifact_paths( + architecture="aarch64" + ) + for path, _ in paths: + self.assertEqual(path.parent, directory / "aarch64") + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(b"unit fixture") + manifest = paths[1][0] + manifest.write_text( + json.dumps({"architecture": "arm64", "uuid": "unit-uuid"}) + ) + selected = managed_exec_tests.load_workload_artifacts( + architecture="aarch64" + ) + self.assertEqual(selected.distro, paths[0][0]) + manifest.write_text( + json.dumps({"architecture": "amd64", "uuid": "unit-uuid"}) + ) + with self.assertRaisesRegex(build.ScriptError, "architecture arm64"): + managed_exec_tests.load_workload_artifacts(architecture="aarch64") + paths[0][0].unlink() + (directory / paths[0][0].name).write_bytes(b"foreign x86 fixture") + with self.assertRaises(build.ScriptError): + managed_exec_tests.load_workload_artifacts(architecture="aarch64") + + def test_arm_lifecycle_identity_fixture_uses_matching_provenance(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + initramfs = sandbox_lifecycle_tests.InitramfsBuildConstants.PROVENANCE_NAME + names = ( + sandbox_lifecycle_tests.OpenVMMBuildConstants.PROVENANCE_NAME, + "aarch64/Image.provenance.json", + f"aarch64/{initramfs}", + ) + expected = { + name: {sandbox_lifecycle_tests.INITRAMFS_DIGEST: "a" * 64} + for name in names + } + for name, value in expected.items(): + path = root / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(value), encoding="utf-8") + artifacts = microvm_tests.WorkloadArtifacts( + root / "distro.erofs", "fixture-uuid", root / "scratch.ext4" + ) + case = sandbox_lifecycle_tests.LifecycleAcceptance( + "whp", + timeout=5, + output_dir=root, + artifacts=artifacts, + root=root / "case", + architecture="aarch64", + ) + with patch.object(BuildConstants, "BUILD_DIR", root): + self.assertEqual( + sandbox_lifecycle_tests.read_build_provenance( + architecture="aarch64" + ), + expected, + ) + case.write_other_runtime() + (root / "aarch64" / initramfs).unlink() + with self.assertRaisesRegex(RuntimeError, "unreadable"): + sandbox_lifecycle_tests.read_build_provenance( + architecture="aarch64" + ) + copied = json.loads( + (case.other_root / "build" / "aarch64" / initramfs).read_text() + ) + self.assertNotEqual(copied, expected[f"aarch64/{initramfs}"]) + self.assertFalse((case.other_root / "build" / initramfs).exists()) + kernel = case.other_root / "build" / "aarch64" / "Image.provenance.json" + self.assertEqual(json.loads(kernel.read_text()), expected[names[1]]) + + def test_arm_microvm_runner_selects_image_and_requires_explicit_scenarios(self): + args = nvx.parse_args( + [ + "test-microvm", + "--backend", + "whp", + "--architecture", + "aarch64", + "--scenario", + "managed-lifecycle", + ] + ) + with tempfile.TemporaryDirectory() as temporary: + args.output_dir = Path(temporary) + + def require(path: Path, _description: str) -> Path: + return path + + with ( + patch.object(microvm_tests, "validate_openvmm_test_backend"), + patch.object( + microvm_tests, "require_file", side_effect=require + ) as required, + patch.object(microvm_tests, "_run_scenarios") as run, + patch.object(microvm_tests, "report_time_abi_evidence"), + patch.object(microvm_tests, "DurationReport"), + contextlib.redirect_stdout(io.StringIO()), + ): + self.assertEqual(microvm_tests.run(args), 0) + kernel = run.call_args.kwargs["kernel"] + self.assertEqual(kernel, BuildConstants.BUILD_DIR / "aarch64" / "Image") + self.assertEqual( + run.call_args.kwargs["initrd"], + BuildConstants.BUILD_DIR / "aarch64" / "initramfs.cpio.gz", + ) + self.assertNotIn( + BuildConstants.BUILD_DIR / "vmlinux", + [call.args[0] for call in required.call_args_list], + ) + with patch.object(microvm_tests, "run_sandbox_lifecycle") as public: + microvm_tests._run_scenarios( + MagicMock(), + args, + ["sandbox-lifecycle"], + descriptor=microvm_tests.guest_descriptor("alpine"), + executable=Path("openvmm.exe"), + kernel=kernel, + initrd=run.call_args.kwargs["initrd"], + output_dir=args.output_dir, + ) + public.assert_called_once_with( + "whp", + timeout=args.timeout, + output_dir=args.output_dir, + architecture="aarch64", + ) + args.scenario = None + with ( + patch.object(microvm_tests, "validate_openvmm_test_backend"), + self.assertRaisesRegex(build.ScriptError, "explicit --scenario"), + ): + microvm_tests.run(args) + if __name__ == "__main__": unittest.main() diff --git a/scripts/test_microvm_tests.py b/scripts/test_microvm_tests.py index bf276aaeb..389084176 100644 --- a/scripts/test_microvm_tests.py +++ b/scripts/test_microvm_tests.py @@ -2554,6 +2554,7 @@ def test_scripts_run_once_the_guest_has_consumed_them(self): time_abi_status=True, on_stderr=None, expected_exit_status=42, + architecture=None, ) self.assertEqual(result["output"], "NVX-OUT none\r\nNVX-DONE\r\n") self.assertEqual(result["text"], transcript) @@ -4289,6 +4290,10 @@ def _run_managed_lifecycle( console: bytes = b"NVX-GUEST-BOOT-OK: managed\n", report: dict[str, object] | None = None, rejected: int = 1, + processors: int = 1, + layered: microvm_tests.WorkloadArtifacts | None = None, + scratch: Path | None = None, + expected_previous_state: str | None = None, ) -> list[tuple[list[str], dict[str, str], object]]: """Runs the scenario with a stand-in for the nvx-rs test, which leaves `console`, `report`, and `returncode` as the real one would.""" @@ -4327,6 +4332,10 @@ def start_test( memory_mib=256, timeout=5, output_dir=output_dir, + processors=processors, + layered=layered, + scratch=scratch, + expected_previous_state=expected_previous_state, ) return runs @@ -4408,6 +4417,56 @@ def test_managed_lifecycle_runs_the_nvx_rs_test_and_checks_what_it_leaves(self): self.assertIn("--microvm-control-auth-stdin", openvmm) self.assertEqual(preserved, self._managed_report()) + def test_managed_layered_inputs_reuse_the_same_authenticated_test(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifacts = microvm_tests.WorkloadArtifacts( + root / "distro.erofs", "fixture-uuid", root / "template.ext4" + ) + for mode in ("scratch", "ram"): + output = root / mode + output.mkdir() + scratch = root / "scratch.ext4" if mode == "scratch" else None + runs = self._run_managed_lifecycle( + output, + processors=8, + layered=artifacts, + scratch=scratch, + expected_previous_state="present" if scratch else "absent", + ) + self.assertEqual(len(runs), 1) + command, environment, _ = runs[0] + self.assertIn("managed_lifecycle", command) + openvmm = json.loads(environment["NVX_MANAGED_LIFECYCLE_COMMAND"]) + self.assertEqual(openvmm[openvmm.index("--processors") + 1], "8") + blocks = [ + openvmm[index + 1] + for index, option in enumerate(openvmm) + if option == "--microvm-sandbox-block" + ] + self.assertIn(f"distro:file:{artifacts.distro.resolve()},ro", blocks) + if scratch: + self.assertIn(f"scratch:file:{scratch.resolve()}", blocks) + else: + self.assertEqual(len(blocks), 1) + cmdline = openvmm[openvmm.index("--cmdline") + 1] + self.assertIn("nvx_sandbox=1", cmdline) + self.assertIn( + "nvx_scratch=0xd0006000,ext4" + if scratch + else "nvx_overlay_upper=ramfs", + cmdline.split(), + ) + self.assertIn("--microvm-control-auth-stdin", openvmm) + self.assertEqual(environment["NVX_MANAGED_LIFECYCLE_PROCESSORS"], "8") + self.assertEqual( + environment["NVX_MANAGED_LIFECYCLE_ROOT_MODE"], "sandbox" + ) + self.assertEqual( + environment["NVX_MANAGED_LIFECYCLE_PREVIOUS_STATE"], + "present" if scratch else "absent", + ) + def test_managed_lifecycle_hands_the_test_an_absolute_pid_journal(self): # The nvx-rs test records OpenVMM in an adversarial campaign's journal # from the nvx-rs directory, so a relative journal resolves here. @@ -16153,7 +16212,10 @@ def require(path: Path, _description: str) -> Path: ): self.assertEqual(microvm_tests.run(args), 0) run.assert_called_once_with( - "whp", timeout=args.timeout, output_dir=output_dir + "whp", + timeout=args.timeout, + output_dir=output_dir, + architecture="x86_64", ) def test_runner_dispatches_public_sandbox_lifecycle(self): @@ -16184,7 +16246,10 @@ def require(path: Path, _description: str) -> Path: ): self.assertEqual(microvm_tests.run(args), 0) run.assert_called_once_with( - "mshv", timeout=args.timeout, output_dir=output_dir + "mshv", + timeout=args.timeout, + output_dir=output_dir, + architecture="x86_64", ) managed_exec.assert_not_called() @@ -16214,7 +16279,10 @@ def require(path: Path, _description: str) -> Path: ): self.assertEqual(microvm_tests.run(args), 0) run.assert_called_once_with( - "kvm", timeout=args.timeout, output_dir=output_dir + "kvm", + timeout=args.timeout, + output_dir=output_dir, + architecture="x86_64", ) lifecycle.assert_not_called() @@ -16244,7 +16312,10 @@ def require(path: Path, _description: str) -> Path: ): self.assertEqual(microvm_tests.run(args), 0) run.assert_called_once_with( - "whp", timeout=args.timeout, output_dir=output_dir + "whp", + timeout=args.timeout, + output_dir=output_dir, + architecture="x86_64", ) lifecycle.assert_not_called() @@ -16329,6 +16400,211 @@ def require(path: Path, description: str) -> Path: guest_boot.assert_not_called() lifecycle.assert_not_called() + def test_arm_missing_sandbox_inputs_recommend_matching_producer_outputs(self): + paths = microvm_tests.workload_artifact_paths(architecture="aarch64") + for missing, _ in paths: + with ( + self.subTest(missing=missing), + tempfile.TemporaryDirectory() as temporary, + ): + args = nvx.parse_args( + [ + "test-microvm", + "--backend", + "whp", + "--architecture", + "aarch64", + "--scenario", + "sandbox-lifecycle", + "--output-dir", + temporary, + ] + ) + + def require( + path: Path, description: str, missing: Path = missing + ) -> Path: + if path == missing: + raise common.ScriptError(f"{description} not found: {path}") + return path + + with ( + patch.object(microvm_tests, "validate_openvmm_test_backend"), + patch.object(microvm_tests, "require_file", side_effect=require), + patch.object(microvm_tests, "_run_scenarios") as run, + self.assertRaises(common.ScriptError) as raised, + ): + microvm_tests.run(args) + run.assert_not_called() + message = str(raised.exception) + self.assertNotIn("nvx-rs download", message) + self.assertNotIn("build-guest --guest all", message) + commands = re.findall(r"`([^`]+)`", message) + self.assertEqual( + commands, + [ + "build-distro-layer --guest ubuntu --architecture aarch64", + "build-scratch-template --output " + "build/aarch64/ubuntu-smoke-scratch.ext4", + ], + ) + with ( + patch.object(nvx, "build_distro_layer") as distro, + patch.object(nvx, "build_scratch_template") as scratch, + ): + for command in commands: + selected = nvx.parse_args(shlex.split(command)) + selected.handler(selected) + self.assertEqual(distro.call_args.args[0].architecture, "aarch64") + self.assertEqual(distro.call_args.args[0].output, paths[0][0]) + self.assertEqual( + BuildConstants.REPO_ROOT / scratch.call_args.args[0].output, + paths[-1][0], + ) + self.assertFalse( + (Path(temporary) / microvm_durations.REPORT_NAME).exists() + ) + + def test_arm_debug_boot_passes_selected_architecture_to_the_shared_monitor(self): + marker = ( + "NVX-TIME-ABI: v=1 phase=boot status=ok cpus=1 " + "tsc_hz=19200000 lapic_hz=0 generation=0 elapsed_us=1\n" + "NVX-TIME-STATUS-EXIT status=0\n" + ) + with tempfile.TemporaryDirectory() as temporary: + build = Path(temporary) / "build" + config = build / "aarch64" / "Image-debug.config" + config.parent.mkdir(parents=True) + config.write_text( + "\n".join(microvm_tests.KernelBuildConstants.DEBUG_WATCHDOG_CONFIG), + encoding="utf-8", + ) + args = nvx.parse_args( + [ + "test-microvm", + "--backend", + "whp", + "--architecture", + "aarch64", + "--debug-kernel", + "--scenario", + "guest-boot", + "--output-dir", + temporary, + ] + ) + + def require(path: Path, _description: str) -> Path: + return path + + def guest( + command: Sequence[str], _script: str, completion: bytes, **options: Any + ) -> benchmark.GuestCommandResult: + self.assertEqual( + Path(command[command.index("--kernel") + 1]).name, "Image-debug" + ) + self.assertEqual(options["architecture"], "aarch64") + monitor = time_abi.TimeAbiMonitor( + command, architecture=options["architecture"] + ) + monitor.feed(marker.encode()) + monitor.require_boot("ARM debug guest") + return benchmark.GuestCommandResult( + text=marker + + "NVX-GUEST-SCRIPT-START\n" + + completion.decode() + + "\n", + wall_ms=0.0, + peak_rss_bytes=0, + ) + + with ( + patch.object(BuildConstants, "BUILD_DIR", build), + patch.object(microvm_tests, "validate_openvmm_test_backend"), + patch.object(microvm_tests, "require_file", side_effect=require), + patch.object( + microvm_tests, "_run_guest_script", side_effect=guest + ) as run, + redirect_stdout(io.StringIO()), + ): + self.assertEqual(microvm_tests.run(args), 0) + run.assert_called_once() + + def test_arm_debug_kernel_requires_each_detector_before_running_scenarios(self): + detectors = microvm_tests.KernelBuildConstants.DEBUG_WATCHDOG_CONFIG + with tempfile.TemporaryDirectory() as temporary: + build = Path(temporary) / "build" + config = build / "aarch64" / "Image-debug.config" + config.parent.mkdir(parents=True) + for missing in (None, *detectors): + with self.subTest(missing=missing): + config.write_text( + "\n".join(option for option in detectors if option != missing), + encoding="utf-8", + ) + output = Path(temporary) / (missing or "valid") + args = nvx.parse_args( + [ + "test-microvm", + "--backend", + "whp", + "--architecture", + "aarch64", + "--debug-kernel", + "--scenario", + "guest-boot", + "--output-dir", + str(output), + ] + ) + + def require(path: Path, description: str) -> Path: + if description == "microVM debug kernel config": + self.assertEqual(path, config) + return common.require_file(path, description) + return path + + with ( + patch.object(BuildConstants, "BUILD_DIR", build), + patch.object(microvm_tests, "validate_openvmm_test_backend"), + patch.object( + microvm_tests, "require_file", side_effect=require + ) as required, + patch.object(microvm_tests, "_run_scenarios") as run, + redirect_stdout(io.StringIO()), + ): + if missing is None: + self.assertEqual(microvm_tests.run(args), 0) + run.assert_called_once() + self.assertEqual( + run.call_args.kwargs["kernel"], + config.with_suffix(""), + ) + else: + with self.assertRaises(common.ScriptError) as raised: + microvm_tests.run(args) + self.assertIn(f"lacks {missing}", str(raised.exception)) + run.assert_not_called() + self.assertFalse(output.exists()) + self.assertIn( + ((config, "microVM debug kernel config"), {}), + required.call_args_list, + ) + + def test_invalid_script_counter_architecture_fails_before_process_start(self): + with ( + patch.object(benchmark, "InteractiveProcess") as process, + self.assertRaisesRegex(ValueError, "unsupported counter architecture"), + ): + benchmark.run_guest_script( + ["openvmm"], + "nvx-exit 0\n", + b"done", + timeout=1, + architecture="unknown", + ) + process.assert_not_called() + def test_managed_executions_enter_through_the_agent_not_the_container_helpers(self): root = Path(__file__).resolve().parent.parent bootstrap = (root / "guest" / "common" / "nvx-init-agent").read_text() diff --git a/scripts/test_nvx_tools.py b/scripts/test_nvx_tools.py index 7e172198f..ab2e9bf74 100644 --- a/scripts/test_nvx_tools.py +++ b/scripts/test_nvx_tools.py @@ -16693,6 +16693,10 @@ def mkfs(command: Sequence[str], *, env: Mapping[str, str]) -> None: ) with image.open("r+b") as template: template.write(b"formatted") + template.seek(1024 + 0x38) + template.write(b"\x53\xef") + template.seek(1024 + 0x160) + template.write((1).to_bytes(4, "little")) with tempfile.TemporaryDirectory() as temporary: output = Path(temporary) / "build" / "scratch.ext4" @@ -16774,6 +16778,61 @@ def mkfs(command: Sequence[str], *, env: Mapping[str, str]) -> None: ) self.assertEqual(stderr.getvalue(), "") + def test_scratch_template_canonicalizes_unsigned_hash_with_e2fsprogs(self): + for corrected in (True, False): + with ( + self.subTest(corrected=corrected), + tempfile.TemporaryDirectory() as temporary, + ): + image = Path(temporary) / "scratch.ext4" + header = bytearray(2048) + header[1024 + 0x38 : 1024 + 0x3A] = b"\x53\xef" + header[1024 + 0x160 : 1024 + 0x164] = (2).to_bytes(4, "little") + image.write_bytes(header) + + def debugfs( + command: Sequence[str], + *, + env: Mapping[str, str], + input_bytes: bytes, + image: Path = image, + corrected: bool = corrected, + ) -> None: + self.assertEqual(command, ["debugfs", "-w", str(image)]) + self.assertEqual(env["E2FSPROGS_FAKE_TIME"], "1") + self.assertEqual( + input_bytes, b"set_super_value flags 1\nclose -a\n" + ) + if corrected: + with image.open("r+b") as volume: + volume.seek(1024 + 0x160) + volume.write((1).to_bytes(4, "little")) + + with ( + patch.object(build, "_e2fsprogs_tool", return_value="debugfs"), + patch.object(build, "run_checked", side_effect=debugfs), + ): + if corrected: + build._canonicalize_scratch_hash(image) + self.assertEqual(build._scratch_hash_flags(image), 1) + else: + with self.assertRaisesRegex( + common.ScriptError, "debugfs did not" + ): + build._canonicalize_scratch_hash(image) + + def test_scratch_template_keeps_signed_hash_bytes_unchanged(self): + with tempfile.TemporaryDirectory() as temporary: + image = Path(temporary) / "scratch.ext4" + header = bytearray(2048) + header[1024 + 0x38 : 1024 + 0x3A] = b"\x53\xef" + header[1024 + 0x160 : 1024 + 0x164] = (1).to_bytes(4, "little") + image.write_bytes(header) + with patch.object(build, "run_checked") as command: + build._canonicalize_scratch_hash(image) + command.assert_not_called() + self.assertEqual(image.read_bytes(), header) + def test_scratch_template_names_where_else_to_get_it_off_linux(self): with tempfile.TemporaryDirectory() as temporary: output = Path(temporary) / "scratch.ext4" @@ -18000,9 +18059,10 @@ def test_docker_distro_layer_targets_export_the_scratch_template(self): ) name = UbuntuBuildConstants.SCRATCH_TEMPLATE_NAME copy = f"COPY --from=ubuntu-scratch-template /repo/build/{name} /{name}" - self.assertEqual(dockerfile.count(copy), 2) + self.assertEqual(dockerfile.count(copy), 3) for target in ( "ubuntu-distro-layer-artifacts", + "arm64-ubuntu-distro-layer-artifacts", DockerBuildConstants.ALL_GUESTS_TARGET, ): with self.subTest(target=target): @@ -20810,7 +20870,7 @@ def test_port_check_requires_the_fault_and_the_refusal(self): f"return {read}; " '[ "$*" = "test port-probe snapshot grant" ] && ' f"return {grant}; exit 2; }}\n" - "check_ports\necho checked", + "check_ports x86_64\necho checked", {"/sbin/nvx-time": "nvx_time"}, ) if message is None: @@ -20820,6 +20880,34 @@ def test_port_check_requires_the_fault_and_the_refusal(self): self.assertEqual(result.returncode, 1, result.stdout) self.assertIn(message, result.stderr) + def test_arm_port_diagnostic_is_unsupported_and_raw_devices_are_absent(self): + for status, exposed, message in ( + (2, False, None), + (0, False, "an ARM x86-port diagnostic ended with status 0"), + (1, False, "an ARM x86-port diagnostic ended with status 1"), + (2, True, "the workload can reach raw device"), + ): + with self.subTest(status=status, exposed=exposed): + device = self.root / "raw-device" + if exposed: + device.write_bytes(b"forbidden device fixture") + result = self._run( + ("check_ports",), + f"nvx_time() {{ return {status}; }}\ncheck_ports aarch64\necho checked", + { + "/sbin/nvx-time": "nvx_time", + "/dev/mem": device.as_posix(), + "/dev/port": (self.root / "absent-port").as_posix(), + "/dev/nvx-platform": (self.root / "absent-platform").as_posix(), + }, + ) + if message is None: + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(result.stdout, "checked\n") + else: + self.assertEqual(result.returncode, 1, result.stdout) + self.assertIn(message, result.stderr) + def test_entry_check_requires_nodev_beyond_the_devices(self): nodev = "".join( line diff --git a/scripts/test_time_abi.py b/scripts/test_time_abi.py index 1d0a31a0e..25529ffcb 100644 --- a/scripts/test_time_abi.py +++ b/scripts/test_time_abi.py @@ -1141,6 +1141,29 @@ def test_option_reader_matches_exact_names_and_keeps_native_defaults(self): "aarch64", ) + def test_arm_debug_product_and_selected_counter_architecture(self): + arm_line = BOOT_LINE.replace("tsc_hz=2194804000", "tsc_hz=19200000").replace( + "lapic_hz=1000000000", "lapic_hz=0" + ) + command = ["openvmm", "--hypervisor", "kvm", "--kernel", "Image-debug"] + for architecture in (None, "aarch64"): + monitor = TimeAbiMonitor(command, architecture=architecture) + monitor.feed(arm_line.encode() + STATUS_OK.encode()) + self.assertEqual(monitor.require_boot("ARM debug")["lapic_hz"], "0") + refused = TimeAbiMonitor(command, architecture=architecture) + with self.assertRaises(TimeAbiFailure): + refused.feed( + arm_line.replace("lapic_hz=0", "lapic_hz=1").encode() + + STATUS_OK.encode() + ) + explicit_x86 = TimeAbiMonitor(command, architecture="x86_64") + explicit_x86.feed(BOOT_LINE.encode() + STATUS_OK.encode()) + self.assertEqual( + explicit_x86.require_boot("explicit x86")["lapic_hz"], "1000000000" + ) + with self.assertRaisesRegex(ValueError, "unsupported counter architecture"): + TimeAbiMonitor(command, architecture="unknown") + def test_validates_restore_marker_fields_against_the_backend(self): restore = time_abi.parse_marker(RESTORE_LINE) assert restore is not None diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index a19df21b7..1b7e1a270 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -4002,7 +4002,13 @@ fn runtime(guest_build: &str) -> Result { Artifacts::ENV_DIR ) })?; - Runtime::from_dir(directory, guest_build).map_err(|error| describe(&error)) + let architecture = if cfg!(target_arch = "aarch64") { + nvx::host::artifacts::Architecture::Aarch64 + } else { + nvx::host::artifacts::Architecture::X86_64 + }; + Runtime::from_dir_for_architecture(directory, guest_build, architecture) + .map_err(|error| describe(&error)) } /// Connects to the control endpoint of the VM `vmm`, and attaches a session with `capability`, diff --git a/tests/runtime-adapter/src/runtime.rs b/tests/runtime-adapter/src/runtime.rs index 3b1ef8409..8c967794a 100644 --- a/tests/runtime-adapter/src/runtime.rs +++ b/tests/runtime-adapter/src/runtime.rs @@ -19,7 +19,7 @@ use std::path::Path; use nvx::Result; -use nvx::host::artifacts::{ArtifactDir, Artifacts}; +use nvx::host::artifacts::{Architecture, ArtifactDir, Artifacts}; use nvx::host::store::{ArtifactIdentity, RuntimeIdentity}; /// NVX's artifacts, and the runtime identity of the adapter's VMs that boot them with the guest @@ -42,9 +42,18 @@ impl Runtime { /// [`ErrorKind::Unsupported`](nvx::ErrorKind::Unsupported) if the identity of the initramfs /// with `guest_build` is not an artifact identity. pub fn from_dir(directory: impl AsRef, guest_build: &str) -> Result { - let directory = directory.as_ref(); - let artifacts = Artifacts::from_dir(directory)?; - let nvx = ArtifactDir::from_dir(directory)?.runtime_identity()?; + Self::from_dir_for_architecture(directory, guest_build, Architecture::X86_64) + } + + /// Selects an architecture-qualified local runtime while keeping `from_dir`'s x86 default. + pub fn from_dir_for_architecture( + directory: impl AsRef, + guest_build: &str, + architecture: Architecture, + ) -> Result { + let directory = ArtifactDir::from_dir(directory)?.with_architecture(architecture)?; + let artifacts = Artifacts::from_artifact_dir(&directory)?; + let nvx = directory.runtime_identity()?; let identity = RuntimeIdentity::new( nvx.openvmm, nvx.kernel, diff --git a/tests/runtime-adapter/tests/common/mod.rs b/tests/runtime-adapter/tests/common/mod.rs index d35782fdd..2a19454f9 100644 --- a/tests/runtime-adapter/tests/common/mod.rs +++ b/tests/runtime-adapter/tests/common/mod.rs @@ -7,8 +7,8 @@ //! - `NVX_ARTIFACTS_DIR`: NVX's artifacts, a repository checkout or an extracted release, with //! the install metadata from which NVX gives their runtime identity, and the [`Fixtures`] of //! the layered roots; -//! - `RUNTIME_ADAPTER_GUEST`: the guest agent, a static `x86_64-unknown-linux-musl` build of -//! `runtime-adapter-guest`. +//! - `RUNTIME_ADAPTER_GUEST`: a static Linux musl build of `runtime-adapter-guest` for the +//! selected native artifact ISA, x86-64 or AArch64. // Each test crate uses its own part of this module. #![allow(dead_code)] @@ -72,7 +72,15 @@ impl Fixtures { pub(crate) fn find() -> Self { let directory = env::var_os(Artifacts::ENV_DIR) .unwrap_or_else(|| panic!("{} must name NVX's artifacts", Artifacts::ENV_DIR)); - let artifacts = ArtifactDir::from_dir(directory).unwrap(); + let architecture = if cfg!(target_arch = "aarch64") { + nvx::host::artifacts::Architecture::Aarch64 + } else { + nvx::host::artifacts::Architecture::X86_64 + }; + let artifacts = ArtifactDir::from_dir(directory) + .unwrap() + .with_architecture(architecture) + .unwrap(); let distro = artifacts.distro_layer(Guest::Ubuntu).unwrap(); let manifest = artifacts.distro_manifest(Guest::Ubuntu).unwrap(); let scratch = artifacts.artifact(SCRATCH_TEMPLATE); diff --git a/tests/runtime-adapter/tests/guest_session.rs b/tests/runtime-adapter/tests/guest_session.rs index fbf3b0b3b..61c9c9ef1 100644 --- a/tests/runtime-adapter/tests/guest_session.rs +++ b/tests/runtime-adapter/tests/guest_session.rs @@ -1213,6 +1213,7 @@ struct FakeRoot { workload: Option, /// The read end of a pipe whose write end the test holds, which the application takes as the /// announcer that it starts, if the test made one with [`ending`](Self::ending). + #[cfg(target_arch = "x86_64")] ending: Option, /// The announcer that a clone's application started, for a record whose network publishes /// the workload's listeners, until the worker collected its end: the read end of a pipe, @@ -1257,6 +1258,7 @@ impl FakeRoot { #[cfg(target_arch = "x86_64")] deadlines: Vec::new(), workload, + #[cfg(target_arch = "x86_64")] ending: None, announcer: None, } @@ -1286,6 +1288,7 @@ impl FakeRoot { /// Makes the announcer that a clone's application starts end once the test drops the returned /// write end of its pipe. + #[cfg(target_arch = "x86_64")] fn ending(mut self) -> (Self, OwnedFd) { let (read, write) = announcer_pipe(); self.ending = Some(read); @@ -1537,6 +1540,7 @@ const GROUP: &str = "root:x:0:\nstaff:x:50:\nworkload:x:1000:\n"; /// Returns a pipe that stands for the pidfd of a clone's announcer: its read end, which polls /// readable once the write end closes, and its write end. Both close on exec, so that no program /// that a test spawns holds the write end, and keeps the announcer from ending. +#[cfg(target_arch = "x86_64")] fn announcer_pipe() -> (OwnedFd, OwnedFd) { nix::unistd::pipe2(OFlag::O_CLOEXEC).unwrap() } diff --git a/tests/runtime-adapter/tests/layered_launch.rs b/tests/runtime-adapter/tests/layered_launch.rs index a867b0008..79edefe4f 100644 --- a/tests/runtime-adapter/tests/layered_launch.rs +++ b/tests/runtime-adapter/tests/layered_launch.rs @@ -21,7 +21,7 @@ use std::process::{Command, Output}; use std::time::{Duration, Instant}; use nvx::exec::{Key, Limits, Timers, Window}; -use nvx::host::artifacts::Artifacts; +use nvx::host::artifacts::{Architecture, ArtifactDir, Artifacts, Guest}; use nvx::host::filesystem::GUEST_EXPORT; use nvx::host::launch::{DEFAULT_MEMORY_MIB, PROCESS_COMMAND_LINE_LIMIT, VmmLaunch}; use nvx::host::network; @@ -852,21 +852,66 @@ fn a_launch_refuses_shares_that_it_cannot_export() { #[test] fn the_host_program_refuses_shares_before_it_touches_the_store() { let directory = directory("layered-share-command"); - // NVX's artifacts in a release's layout, which the launch takes, but which boot nothing. + // Local artifacts of the host ISA, which the launch takes, but which boot nothing. let artifacts = directory.join("artifacts"); - let openvmm = if cfg!(windows) { - "bin/openvmm.exe" - } else { - "bin/openvmm" - }; - let mut sums = String::new(); - for name in [openvmm, "guest/vmlinux", "guest/initramfs.cpio.gz"] { - let path = artifacts.join(name); + let runtime = ArtifactDir::repository(&artifacts) + .unwrap() + .with_architecture(if cfg!(target_arch = "aarch64") { + Architecture::Aarch64 + } else { + Architecture::X86_64 + }) + .unwrap(); + for path in [ + runtime.openvmm(), + runtime.kernel(), + runtime.initramfs(Guest::Alpine), + ] { fs::create_dir_all(path.parent().unwrap()).unwrap(); - fs::write(&path, name).unwrap(); - sums.push_str(&format!("{} {name}\n", "0".repeat(64))); + fs::write(&path, b"non-bootable fixture").unwrap(); + } + if cfg!(target_arch = "aarch64") { + let mut pe = [0_u8; 70]; + pe[..2].copy_from_slice(b"MZ"); + pe[60..64].copy_from_slice(&64_u32.to_le_bytes()); + pe[64..68].copy_from_slice(b"PE\0\0"); + pe[68..70].copy_from_slice(&0xaa64_u16.to_le_bytes()); + fs::write(runtime.openvmm(), pe).unwrap(); + let mut image = [0_u8; 64]; + image[16..24].copy_from_slice(&4096_u64.to_le_bytes()); + image[24..32].copy_from_slice(&10_u64.to_le_bytes()); + image[56..60].copy_from_slice(b"ARM\x64"); + fs::write(runtime.kernel(), image).unwrap(); + fs::write( + runtime + .initramfs(Guest::Alpine) + .with_extension("gz.packages.json"), + r#"{"architecture":"aarch64"}"#, + ) + .unwrap(); + } + for (path, member) in [ + ( + artifacts.join("build/openvmm.provenance.json"), + "executable_sha256", + ), + ( + runtime.kernel().with_extension("provenance.json"), + "kernel_sha256", + ), + ( + runtime + .initramfs(Guest::Alpine) + .with_file_name("initramfs.provenance.json"), + "initramfs_sha256", + ), + ] { + fs::write( + path, + format!(r#"{{"format":1,"{member}":"{}"}}"#, "0".repeat(64)), + ) + .unwrap(); } - fs::write(artifacts.join("SHA256SUMS"), sums).unwrap(); let manifest = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../SOURCE-MANIFEST.json"); fs::copy(manifest, artifacts.join("SOURCE-MANIFEST.json")).unwrap(); diff --git a/tests/runtime-adapter/tests/runtime.rs b/tests/runtime-adapter/tests/runtime.rs index 710d4a845..0ff662ebf 100644 --- a/tests/runtime-adapter/tests/runtime.rs +++ b/tests/runtime-adapter/tests/runtime.rs @@ -7,7 +7,7 @@ use std::fs; use std::path::{Path, PathBuf}; use nvx::ErrorKind; -use nvx::host::artifacts::ArtifactDir; +use nvx::host::artifacts::{Architecture, ArtifactDir}; use nvx::host::store::{ArtifactIdentity, IdentityPart, RuntimeIdentity}; use runtime_adapter::GUEST_BUILD; use runtime_adapter::runtime::Runtime; @@ -123,6 +123,34 @@ fn another_build_of_the_guest_agent_is_another_runtime() { } } +#[test] +fn an_explicit_arm_runtime_keeps_the_legacy_x86_identity_and_paths_separate() { + let root = directory("arm-qualified"); + lay_out(&root, false); + let build = root.join("build/aarch64"); + fs::create_dir_all(&build).unwrap(); + for (name, field, digest) in [ + ("Image.provenance.json", "kernel_sha256", KERNEL), + ("initramfs.provenance.json", "initramfs_sha256", INITRAMFS), + ] { + fs::write( + build.join(name), + format!("{{\"format\":1,\"{field}\":\"{digest}\"}}"), + ) + .unwrap(); + } + let arm = + Runtime::from_dir_for_architecture(&root, GUEST_BUILD, Architecture::Aarch64).unwrap(); + assert_eq!(arm.identity, identity(GUEST_BUILD)); + assert_eq!(arm.artifacts.kernel, build.join("Image")); + assert_eq!(arm.artifacts.initrd, build.join("initramfs.cpio.gz")); + let legacy = Runtime::from_dir(&root, GUEST_BUILD).unwrap(); + assert_eq!(legacy.artifacts.kernel, root.join("build/vmlinux")); + fs::remove_file(build.join("Image.provenance.json")).unwrap(); + assert!(Runtime::from_dir_for_architecture(&root, GUEST_BUILD, Architecture::Aarch64).is_err()); + assert!(Runtime::from_dir(&root, GUEST_BUILD).is_ok()); +} + #[test] fn missing_install_metadata_fails_as_nvx_reports_it() { for (name, release) in [("missing-provenance", false), ("missing-checksums", true)] {