Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions doc/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,16 @@ The normal `build-guest --architecture aarch64 --guest alpine|ubuntu`
command selects these same producers and namespaces, adding `--native` on
Linux to avoid Docker. ARM rejects the x86 debug-kernel and all-guests
distro-layer modes instead of mixing their outputs.
The separate maintained Ubuntu EROFS producer accepts
`build-distro-layer --guest ubuntu --architecture aarch64`. It uses the same
root builder, ARM package lock and input identity as the Ubuntu initramfs,
and writes `build/aarch64/ubuntu-distro.erofs` and its manifest. The existing
ISA-neutral scratch producer can target
`--output build/aarch64/ubuntu-smoke-scratch.ext4`. BuildKit's
`arm64-ubuntu-distro-layer-artifacts` target exports both through the existing
local-output mechanism; select `--platform linux/arm64` and
`--build-arg NVX_ARCHITECTURE=aarch64`. These are maintained producer inputs,
not a qualification-only filesystem fixture.
The existing Alpine one-shot container entry selects its control guest's
fixed x86-64 or AArch64 musl loader; it keeps the same namespace and
privilege-drop policy and refuses other control ISAs.
Expand All @@ -166,6 +176,10 @@ before any process is launched. Existing x86 paths and default selection
remain unchanged.
This native runtime default does not change the build commands' x86-64
default or the existing artifact API's explicit architecture selection.
The public `sandbox` CLI likewise selects native local artifacts and validates
the ARM Image, initramfs manifest and Windows OpenVMM PE identity. Its options,
saved formats and one-VP lifecycle contract are unchanged. ARM release
publication and snapshot modes remain outside this enablement.

**Deferred ARM cacheinfo metadata.** Phase 1 defers complete Linux sysfs
cache geometry and sharing metadata, an existing upstream Standard ARM
Expand Down
5 changes: 4 additions & 1 deletion doc/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -1652,7 +1652,10 @@ template, so it has the same artifact requirements. It checks that:
file survives `stop` and a new `start`, because the overlay's upper directory
is on scratch;
- `/sbin/nvx-sandbox-smoke` passes its security profile checks in a sandbox
provisioned with `--hostname`, after earlier requests;
provisioned with `--hostname`, after earlier requests. The x86 port check
still requires the fault and permission refusal. ARM instead requires the
x86-only diagnostic to remain unsupported and raw platform devices to be
absent; the capability, namespace and exact private-device checks stay common;
- a request that writes more than the 1 MiB output bound exits with status
125 and an `output-limit` outcome report after the host has received more
than 1 MiB less one 32 KiB read chunk and at most 1 MiB, and a request whose
Expand Down
20 changes: 17 additions & 3 deletions doc/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -970,13 +970,17 @@ and runtime qualification gates are not waived.
```text
python3 scripts/nvx.py build-distro-layer
--guest ubuntu
[--architecture {x86_64,aarch64}]
[--output PATH]
[--replace]
```

Builds the immutable Ubuntu EROFS `distro` layer directly on Linux. The default
output is `build/ubuntu-distro.erofs`. Existing output or manifest files are
rejected unless `--replace` is present.
output is `build/ubuntu-distro.erofs`. `--architecture` defaults to x86-64;
`aarch64` selects the pinned ARM64 Ubuntu Base and supplemental lock and defaults
to `build/aarch64/ubuntu-distro.erofs`, with its matching manifest beside it.
An explicit `--output` is honored for either ISA. Existing output or manifest
files are rejected unless `--replace` is present.

### `build-scratch-template`

Expand All @@ -1000,7 +1004,11 @@ fixes the clock of `mkfs.ext4` with `E2FSPROGS_FAKE_TIME` and its directory
hash seed, and ignores the caller's other `E2FSPROGS_*` and `MKE2FS_*`
variables, so e2fsprogs 1.46.5 and 1.47.0 of Debian 12, Ubuntu 22.04 and
24.04, and Azure Linux 3 format the same bytes, whose SHA-256
`UbuntuBuildConstants.SCRATCH_TEMPLATE_SHA256` pins. It warns when
`UbuntuBuildConstants.SCRATCH_TEMPLATE_SHA256` pins. When libext2fs's compiler
defaults to unsigned `char`, the producer uses the same package's `debugfs`
to select the existing signed-directory-hash format and update its checksums.
Already canonical x86 output is untouched; the UUID, layout and pinned hash
are unchanged. It warns when
`mkfs.ext4` formats other bytes. The guest Docker build formats the template
with the Ubuntu distro layer, in the pinned Debian image, for
[`build-guest`](#build-guest) and for CI, which caches, records, and hands it
Expand Down Expand Up @@ -1133,6 +1141,7 @@ OpenVMM's tests, and CI checks that it passed the pinned revision with
```text
python3 scripts/nvx.py test-microvm
--backend {kvm,mshv,whp}
[--architecture {x86_64,aarch64}]
[--guest {alpine,ubuntu,azurelinux}]
[--scenario SCENARIO]...
[--debug-kernel]
Expand Down Expand Up @@ -1167,6 +1176,11 @@ the CI debug kernel, `build/vmlinux-debug`, whose soft-lockup and hung-task
detectors the guest's time ABI watcher reports; without `--scenario`, it runs
only the same-host restore scenarios `smp`, `smp-snapshot`,
`restore-processors`, `restore-downtime`, and `snapshot-tiers`.
The architecture defaults to x86-64. ARM requires explicit `--scenario`
selection and uses `build/aarch64/Image`, matching initramfs and managed-layer
inputs; it does not qualify every scenario listed here for ARM. Public sandbox
scenarios retain the existing one-VP CLI contract; `--processors` does not add
a sandbox processor option or change saved configurations.
`--shard INDEX/COUNT`, such as `1/2`, runs shard INDEX of the COUNT shards of
the default Alpine suite on the default kernel, the scenarios of a run without
`--scenario` but the public sandbox scenarios, which CI runs in its guest and
Expand Down
7 changes: 6 additions & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ RUN python3 scripts/nvx.py build-initramfs --guest ubuntu --architecture "$NVX_A
FROM ubuntu-initramfs AS ubuntu-distro-layer
RUN python3 scripts/nvx.py build-distro-layer \
--guest ubuntu \
--output build/ubuntu-distro.erofs
--architecture "$NVX_ARCHITECTURE"

# The sandbox scratch template needs only the base image's mkfs.ext4, so it
# builds no guest binary.
Expand Down Expand Up @@ -296,6 +296,11 @@ COPY --from=kernel /repo/build/aarch64/Image.provenance.json /Image.provenance.j
COPY --from=ubuntu-initramfs /repo/build/aarch64/initramfs-ubuntu.cpio.gz /initramfs-ubuntu.cpio.gz
COPY --from=ubuntu-initramfs /repo/build/aarch64/initramfs-ubuntu.cpio.gz.packages.json /initramfs-ubuntu.cpio.gz.packages.json

FROM scratch AS arm64-ubuntu-distro-layer-artifacts
COPY --from=ubuntu-distro-layer /repo/build/aarch64/ubuntu-distro.erofs /ubuntu-distro.erofs
COPY --from=ubuntu-distro-layer /repo/build/aarch64/ubuntu-distro.erofs.manifest.json /ubuntu-distro.erofs.manifest.json
COPY --from=ubuntu-scratch-template /repo/build/ubuntu-smoke-scratch.ext4 /ubuntu-smoke-scratch.ext4

FROM scratch AS kernel-debug-artifacts
COPY --from=kernel-debug /repo/build/vmlinux-debug /vmlinux-debug
COPY --from=kernel-debug /repo/build/vmlinux-debug.config /vmlinux-debug.config
Expand Down
19 changes: 19 additions & 0 deletions guest/common/nvx-sandbox-smoke
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,25 @@ check_machine_id() {
# The workload can neither use the processor's I/O ports, whose read ends
# the reader with SIGSEGV without the permission, nor obtain the permission.
check_ports() {
case "${1:-$(uname -m)}" in
x86_64) ;;
aarch64)
for raw_device in /dev/mem /dev/port /dev/nvx-platform; do
[ ! -e "$raw_device" ] && [ ! -L "$raw_device" ] ||
fail "the workload can reach raw device $raw_device"
done
for probe in snapshot "snapshot grant"; do
port_status=0
# The ARM helper must refuse the x86-only diagnostic itself.
# shellcheck disable=SC2086
/sbin/nvx-time test port-probe $probe 2>/dev/null || port_status=$?
[ "$port_status" -eq 2 ] ||
fail "an ARM x86-port diagnostic ended with status $port_status"
done
return
;;
*) fail "unsupported workload architecture ${1:-$(uname -m)}" ;;
esac
port_status=0
# The shell reports the signal on standard error, so silence it.
{ /sbin/nvx-time test port-probe snapshot; } 2>/dev/null || port_status=$?
Expand Down
2 changes: 2 additions & 0 deletions nvx-rs/public-api/linux.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ const nvx::Artifacts::ENV_OPENVMM
method nvx::Artifacts::artifact_dir
method nvx::Artifacts::bundled
method nvx::Artifacts::discover
method nvx::Artifacts::from_artifact_dir
method nvx::Artifacts::from_dir
method nvx::Artifacts::from_release_dir
method nvx::Artifacts::from_repo_layout
Expand Down Expand Up @@ -1690,6 +1691,7 @@ const nvx::host::artifacts::Artifacts::ENV_OPENVMM
method nvx::host::artifacts::Artifacts::artifact_dir
method nvx::host::artifacts::Artifacts::bundled
method nvx::host::artifacts::Artifacts::discover
method nvx::host::artifacts::Artifacts::from_artifact_dir
method nvx::host::artifacts::Artifacts::from_dir
method nvx::host::artifacts::Artifacts::from_release_dir
method nvx::host::artifacts::Artifacts::from_repo_layout
Expand Down
2 changes: 2 additions & 0 deletions nvx-rs/public-api/windows.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ const nvx::Artifacts::ENV_OPENVMM
method nvx::Artifacts::artifact_dir
method nvx::Artifacts::bundled
method nvx::Artifacts::discover
method nvx::Artifacts::from_artifact_dir
method nvx::Artifacts::from_dir
method nvx::Artifacts::from_release_dir
method nvx::Artifacts::from_repo_layout
Expand Down Expand Up @@ -634,6 +635,7 @@ const nvx::host::artifacts::Artifacts::ENV_OPENVMM
method nvx::host::artifacts::Artifacts::artifact_dir
method nvx::host::artifacts::Artifacts::bundled
method nvx::host::artifacts::Artifacts::discover
method nvx::host::artifacts::Artifacts::from_artifact_dir
method nvx::host::artifacts::Artifacts::from_dir
method nvx::host::artifacts::Artifacts::from_release_dir
method nvx::host::artifacts::Artifacts::from_repo_layout
Expand Down
77 changes: 76 additions & 1 deletion nvx-rs/src/host/artifacts/architecture/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,82 @@ fn explicit_arm_paths_never_change_legacy_default_or_openvmm_location() {
assert_eq!(arm.openvmm(), legacy.openvmm());
assert_eq!(arm.openvmm().file_name().unwrap(), openvmm_executable());
assert_eq!(arm.paths().guests.len(), 2);
assert!(arm.distro_layer(Guest::Ubuntu).is_none());
assert_eq!(
arm.distro_layer(Guest::Ubuntu).unwrap(),
root.path().join("build/aarch64/ubuntu-distro.erofs")
);
assert!(arm.sandbox_artifacts().is_err());
}

#[test]
fn arm_managed_artifacts_require_matching_native_kernel_and_manifest() {
let root = tempfile::tempdir().unwrap();
let arm = ArtifactDir::repository(root.path())
.unwrap()
.with_architecture(Architecture::Aarch64)
.unwrap();
fs::create_dir_all(arm.openvmm().parent().unwrap()).unwrap();
let mut executable = [0; 70];
executable[..2].copy_from_slice(b"MZ");
executable[60..64].copy_from_slice(&64_u32.to_le_bytes());
executable[64..68].copy_from_slice(b"PE\0\0");
executable[68..70].copy_from_slice(&0xaa64_u16.to_le_bytes());
fs::write(arm.openvmm(), executable).unwrap();
fs::create_dir_all(arm.kernel().parent().unwrap()).unwrap();
fs::write(arm.kernel(), image()).unwrap();
fs::write(
arm.initramfs(Guest::Alpine),
b"unit fixture, not a bootable CPIO",
)
.unwrap();
assert!(arm.sandbox_artifacts().is_err());
fs::write(
arm.package_manifest(Guest::Alpine),
r#"{"architecture":"x86_64"}"#,
)
.unwrap();
assert!(arm.sandbox_artifacts().is_err());
fs::write(
arm.package_manifest(Guest::Alpine),
r#"{"architecture":"aarch64"}"#,
)
.unwrap();
let artifacts = arm.sandbox_artifacts().unwrap();
assert_eq!(artifacts.kernel, arm.kernel());
assert_eq!(artifacts.initrd, arm.initramfs(Guest::Alpine));
#[cfg(any(target_os = "linux", windows))]
{
artifacts.probe().unwrap();
assert!(artifacts.runtime_identity().is_err());
let replacement = root.path().join("foreign-artifact");
fs::write(&replacement, b"foreign artifact").unwrap();
for description in ["OpenVMM executable", "guest kernel", "guest initramfs"] {
let mut replaced = artifacts.clone();
let path = match description {
"OpenVMM executable" => &mut replaced.openvmm,
"guest kernel" => &mut replaced.kernel,
_ => &mut replaced.initrd,
};
*path = replacement.clone();
let error = replaced.probe().unwrap_err();
assert_eq!(error.code(), crate::ErrorCode::Unavailable);
assert_eq!(
error.message(),
format!(
"the {description} {} is not the one of {}, whose install metadata \
identifies the artifacts",
replacement.display(),
arm.root().display()
)
);
}
fs::remove_file(arm.kernel()).unwrap();
assert_eq!(
artifacts.probe().unwrap_err().code(),
crate::ErrorCode::ArtifactMissing
);
}
fs::write(arm.kernel(), b"foreign kernel").unwrap();
assert!(arm.sandbox_artifacts().is_err());
}

Expand Down
24 changes: 11 additions & 13 deletions nvx-rs/src/host/artifacts/dir.rs
Original file line number Diff line number Diff line change
Expand Up @@ -157,17 +157,11 @@ impl ArtifactDir {

/// Returns the path of a guest's distro layer, if the guest provides one.
pub fn distro_layer(&self, guest: Guest) -> Option<PathBuf> {
if self.architecture == Architecture::Aarch64 {
return None;
}
guest.distro_layer_name().map(|name| self.artifact(name))
}

/// Returns the path of the manifest of a guest's distro layer, if the guest provides one.
pub fn distro_manifest(&self, guest: Guest) -> Option<PathBuf> {
if self.architecture == Architecture::Aarch64 {
return None;
}
guest
.distro_manifest_name()
.map(|name| self.artifact(&name))
Expand All @@ -180,7 +174,12 @@ impl ArtifactDir {
///
/// [`ErrorCode::ArtifactMissing`]: crate::ErrorCode::ArtifactMissing
pub fn require_openvmm(&self) -> Result<PathBuf> {
required(self.openvmm(), "OpenVMM release binary")
let path = required(self.openvmm(), "OpenVMM release binary")?;
#[cfg(windows)]
if self.architecture == Architecture::Aarch64 {
super::check_arm64_pe(&path)?;
}
Ok(path)
}

/// Checks that the guest kernel is a regular file and returns its path.
Expand Down Expand Up @@ -257,14 +256,13 @@ impl ArtifactDir {
///
/// [`ErrorCode::ArtifactMissing`]: crate::ErrorCode::ArtifactMissing
pub fn sandbox_artifacts(&self) -> Result<Artifacts> {
if self.architecture == Architecture::Aarch64 {
return Err(Error::unsupported(
"ARM64 managed sandboxes are not qualified by the Phase 1 cold-boot contract",
));
}
self.require_openvmm()?;
self.require_kernel()?;
required(self.initramfs(Guest::Alpine), "initramfs")?;
if self.architecture == Architecture::Aarch64 {
self.require_initramfs(Guest::Alpine)?;
} else {
required(self.initramfs(Guest::Alpine), "initramfs")?;
}
Ok(Artifacts::located(self))
}

Expand Down
2 changes: 1 addition & 1 deletion nvx-rs/src/host/artifacts/identity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ pub(super) fn provenance_identity(
}

/// Rejects a Windows OpenVMM executable from another ISA before any process or WHP call.
#[cfg(all(windows, feature = "cli"))]
#[cfg(windows)]
pub(crate) fn check_arm64_pe(path: &Path) -> Result<()> {
use std::io::{Seek, SeekFrom};

Expand Down
23 changes: 20 additions & 3 deletions nvx-rs/src/host/artifacts/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ use crate::host::store::RuntimeIdentity;
pub use self::architecture::Architecture;
pub use self::dir::{ArtifactDir, ArtifactPaths, GuestPaths, Layout};
pub use self::guest::Guest;
#[cfg(all(windows, feature = "cli"))]
#[cfg(windows)]
pub(crate) use self::identity::check_arm64_pe;
#[cfg(feature = "cli")]
pub(crate) use self::require::require_artifact;
Expand Down Expand Up @@ -128,6 +128,12 @@ impl Artifacts {
them in an NVX release or repository checkout, or supply their runtime identity",
));
};
self.check_directory_paths(directory)?;
directory.runtime_identity()
}

/// Checks that the public paths still name the artifacts of `directory`.
fn check_directory_paths(&self, directory: &ArtifactDir) -> Result<()> {
let located = Self::located(directory);
for (path, expected, description) in [
(&self.openvmm, &located.openvmm, "OpenVMM executable"),
Expand All @@ -143,11 +149,14 @@ impl Artifacts {
)));
}
}
directory.runtime_identity()
Ok(())
}

/// Checks that this host can run NVX VMs and that the three files exist.
///
/// Located ARM artifacts must still name their directory's files, whose native format and
/// initramfs architecture are checked without reading runtime identity metadata.
///
/// Fails with [`ErrorKind::Unavailable`](crate::ErrorKind::Unavailable) otherwise, with the
/// code [`ErrorCode::ArtifactMissing`] for a missing file. It does not check the hypervisor;
/// [`probe_hypervisor`](super::platform::probe_hypervisor) does.
Expand All @@ -169,6 +178,14 @@ impl Artifacts {
));
}
}
if let Some(directory) = self.artifact_dir()
&& directory.architecture() == Architecture::Aarch64
{
self.check_directory_paths(directory)?;
directory.require_openvmm()?;
directory.require_kernel()?;
directory.require_initramfs(Guest::Alpine)?;
}
Ok(())
}

Expand Down Expand Up @@ -202,7 +219,7 @@ impl Artifacts {
///
/// Fails with [`ErrorKind::Unavailable`](crate::ErrorKind::Unavailable) as
/// [`check_manifest`] does.
fn from_artifact_dir(directory: &ArtifactDir) -> Result<Self> {
pub fn from_artifact_dir(directory: &ArtifactDir) -> Result<Self> {
check_manifest(&directory.root().join(MANIFEST_NAME))?;
Ok(Self::located(directory))
}
Expand Down
Loading
Loading