Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1800,6 +1800,24 @@ dependencies = [
"windows-sys 0.61.0",
]

[[package]]
name = "disk_zerofill"
version = "0.0.0"
dependencies = [
"anyhow",
"async-trait",
"disk_backend",
"disk_backend_resources",
"disklayer_ram",
"guestmem",
"inspect",
"pal_async",
"scsi_buffers",
"storage_tests",
"thiserror 2.0.16",
"vm_resource",
]

[[package]]
name = "disklayer_ram"
version = "0.0.0"
Expand Down Expand Up @@ -6282,6 +6300,7 @@ dependencies = [
"disk_prwrap",
"disk_vhd1",
"disk_vhdmp",
"disk_zerofill",
"disklayer_ram",
"disklayer_sqlite",
"disklayer_vhdx",
Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,7 @@ disk_striped = { path = "vm/devices/storage/disk_striped" }
disk_vhd1 = { path = "vm/devices/storage/disk_vhd1" }
disklayer_vhdx = { path = "vm/devices/storage/disklayer_vhdx" }
disk_vhdmp = { path = "vm/devices/storage/disk_vhdmp" }
disk_zerofill = { path = "vm/devices/storage/disk_zerofill" }
disklayer_ram = { path = "vm/devices/storage/disklayer_ram" }
disklayer_sqlite = { path = "vm/devices/storage/disklayer_sqlite" }
floppy = { path = "vm/devices/storage/floppy" }
Expand Down
5 changes: 4 additions & 1 deletion Guide/src/dev_guide/snapshot_format.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,10 @@ an available snapshot. A missing restore-mode field retains `private-copy`
behavior.

A pre-mount capture instead records the `fresh` scratch policy and geometry,
contains no `scratch.img`, and requires a new matching scratch file on restore.
contains no `scratch.img`, and requires a new scratch file of that geometry on
restore, unless `--restore-absent-scratch` declares the scratch absent:
OpenVMM then attaches a writable scratch device of that geometry with no file,
which reads zeros and fails every write.

## Code references

Expand Down
2 changes: 1 addition & 1 deletion Guide/src/reference/architecture/devices/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ A decorator is a `DiskIo` implementation that wraps another `Disk` and transform
└── BlockDeviceDisk
```

Three decorators exist: [`CryptDisk`](https://openvmm.dev/rustdoc/linux/disk_crypt/struct.CryptDisk.html) (XTS-AES-256 encryption), [`DelayDisk`](https://openvmm.dev/rustdoc/linux/disk_delay/struct.DelayDisk.html) (injected latency), and [`DiskWithReservations`](https://openvmm.dev/rustdoc/linux/disk_prwrap/struct.DiskWithReservations.html) (in-memory persistent reservation emulation). All three forward metadata (sector count, sector size, disk ID, `wait_resize`) to the inner disk unchanged. See the [storage backends](../../backends/storage.md) page for the decorator catalog.
Four decorators exist: [`CryptDisk`](https://openvmm.dev/rustdoc/linux/disk_crypt/struct.CryptDisk.html) (XTS-AES-256 encryption), [`DelayDisk`](https://openvmm.dev/rustdoc/linux/disk_delay/struct.DelayDisk.html) (injected latency), [`DiskWithReservations`](https://openvmm.dev/rustdoc/linux/disk_prwrap/struct.DiskWithReservations.html) (in-memory persistent reservation emulation), and [`ZeroFillDisk`](https://openvmm.dev/rustdoc/linux/disk_zerofill/struct.ZeroFillDisk.html) (a fixed geometry over a shorter inner disk, or none). The first three forward metadata (sector count, sector size, disk ID, `wait_resize`) to the inner disk unchanged. `ZeroFillDisk` presents its own sector count, which never changes: reads past the inner disk's end return zeros, and writes fail without reaching it. See the [storage backends](../../backends/storage.md) page for the decorator catalog.

## The layered disk model

Expand Down
1 change: 1 addition & 0 deletions Guide/src/reference/backends/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ without modifying the backends underneath.
| CryptDisk | [`disk_crypt`](https://openvmm.dev/rustdoc/linux/disk_crypt/index.html) | XTS-AES-256 encryption. Encrypts on write, decrypts on read. |
| DelayDisk | [`disk_delay`](https://openvmm.dev/rustdoc/linux/disk_delay/index.html) | Adds configurable latency to each I/O operation. |
| DiskWithReservations | [`disk_prwrap`](https://openvmm.dev/rustdoc/linux/disk_prwrap/index.html) | In-memory SCSI persistent reservation emulation. |
| ZeroFillDisk | [`disk_zerofill`](https://openvmm.dev/rustdoc/linux/disk_zerofill/index.html) | Presents a fixed sector count over a shorter disk, or none. Reads past the inner disk's end return zeros; writes fail. |

## Layered disks

Expand Down
22 changes: 19 additions & 3 deletions Guide/src/reference/openvmm/management/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,8 @@ describes the source definitions.
mounted, it freezes the workload cgroup with a bounded wait, syncs, freezes
the scratch filesystem, and asks OpenVMM to drain queues and atomically
publish `scratch.img`. `/sbin/nvx-snapshot --fresh-scratch` is for a
pre-mount boundary and records that restore must supply a fresh scratch.
pre-mount boundary and records that restore must supply a fresh scratch,
or declare it absent with `--restore-absent-scratch`.
* `--microvm-workload-identity <UID:GID>`: Add a fixed non-root numeric
workload identity to the host-owned microVM command line. UID and GID zero
are rejected. The guest workload supervisor must resolve both values in the
Expand Down Expand Up @@ -450,10 +451,15 @@ describes the source definitions.

For sandbox-block snapshots, restore repeats each read-only
`--microvm-sandbox-block` argument. Its role, access, geometry, and SHA-256
must match the manifest. A paired snapshot supplies scratch internally from
must match the manifest, with two exceptions for a platform snapshot: its
read-only layers are unbound, and its `distro` block keeps the captured
capacity over an image that may be shorter. Such an image must have the
captured logical and physical block sizes, reads past its end return zeros,
and OpenVMM never resizes, pads, or writes it; a longer image is rejected.
A paired snapshot supplies scratch internally from
a verified process-private copy of `scratch.img`; passing another scratch is
rejected. A fresh-scratch snapshot instead requires a writable scratch
argument with matching geometry.
argument with matching geometry, or `--restore-absent-scratch`.

`--restore-memory <SIZE>` selects the total RAM for this launch. It requires
an expansion-capable snapshot and a 128-MiB-aligned value from the exact
Expand Down Expand Up @@ -503,6 +509,16 @@ describes the source definitions.
validating the full saved VP inventory; that reduced-prefix process cannot
be saved again. MSHV restores without this option, and KVM and WHP restores,
instantiate the full VP capacity.
* `--restore-absent-scratch`: Restore a platform snapshot's fresh scratch
without a file. OpenVMM attaches the scratch device at the length and block
sizes that the manifest records. The device stays writable, as the guest
negotiated it and as the restore contract requires, but has nowhere to keep
data: reads return zeros, and every write fails with an I/O error, so no
write is dropped or kept in host memory. Only a platform snapshot's fresh
scratch may be absent. OpenVMM rejects the option before any vCPU starts
for a cold boot, for a paired snapshot or one without sandbox blocks, and
together with a scratch `--microvm-sandbox-block`. A scratch file, when given
instead, keeps the exact captured length.
* `--restore-gate-timeout-ms <MILLISECONDS>`: Bound microVM guest repair and
gate acknowledgement after restore. The default is 60000 milliseconds.
* `--cpu-profile <ID>`: Select the microVM's pinned CPU profile, `auto`
Expand Down
14 changes: 13 additions & 1 deletion Guide/src/user_guide/openvmm/snapshots.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,19 @@ host can connect while input is gated, but OpenVMM reads its authentication
only after the release, so its `--microvm-control-auth-timeout-ms` deadline
starts over when the gate releases. Platform manifests
leave read-only layer identities unbound, while later tiers require exact image
identities. An instance-checkpoint restore attempt atomically creates
identities. A platform restore also presents the captured capacity of its
`distro` block before any vCPU runs, so one platform snapshot serves images of
any length up to that capacity. An image with the captured logical and
physical block sizes may be shorter, and reads past its end, including the tail
of a request that crosses it, return zeros. OpenVMM rejects a longer image and
never resizes, pads, or writes the image file. A platform restore may also
declare its fresh scratch absent with `--restore-absent-scratch`, for a clone
that never writes it, such as one that keeps its upper layer in RAM. OpenVMM
then attaches the scratch device with no file, at the captured length and
block sizes. The device stays writable, as the guest negotiated it, but reads
return zeros and writes fail with an I/O error: OpenVMM never drops a write or
keeps one in host memory. Only a platform snapshot's fresh scratch may be
absent, and a scratch file, when given, keeps the exact captured length. An instance-checkpoint restore attempt atomically creates
`resume.claim`; subsequent restores are rejected. The claim is committed after
artifact and configuration validation but before worker construction, so the
restore attempt remains consumed if later worker startup fails.
Expand Down
72 changes: 71 additions & 1 deletion openvmm/openvmm_entry/src/cli_args/microvm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,14 @@ pub struct MicrovmCli {
#[clap(long, value_name = "SIZE", requires = "restore_snapshot")]
pub restore_memory: Option<vmm_cli::MemorySize>,

/// Attach a platform snapshot's fresh scratch with no file.
///
/// The scratch device keeps the captured length and block sizes, and stays
/// writable, as the guest negotiated it: reads return zeros, and writes
/// fail with an I/O error. Do not pass a scratch block with it.
#[clap(long, requires = "restore_snapshot")]
pub restore_absent_scratch: bool,

/// Maximum time allowed for a microVM guest to complete post-restore repair.
#[clap(long, value_name = "MILLISECONDS", default_value_t = 60000)]
pub restore_gate_timeout_ms: u64,
Expand Down Expand Up @@ -768,13 +776,14 @@ impl Options {
&& self.microvm.microvm_report.is_none()
&& self.microvm.restore_processors.is_none()
&& self.microvm.restore_memory.is_none()
&& !self.microvm.restore_absent_scratch
&& self.microvm.memory_capacity.is_none()
&& self.microvm.snapshot_block_identity.is_none()
&& self.microvm.snapshot_generation_id.is_none()
&& self.microvm.snapshot_scratch_restore_mode.is_none()
&& self.microvm.microvm_control_console.is_none()
&& !self.microvm.microvm_control_auth_stdin,
"--network-profile, --net-tap, --mount, --mount-aggregate, --microvm-sandbox-block, --microvm-workload-identity, --microvm-lifecycle, --microvm-control-console, --microvm-control-auth-stdin, --restore-processors, --restore-memory, --memory-capacity, snapshot block policy, and microVM network policy require a microVM machine"
"--network-profile, --net-tap, --mount, --mount-aggregate, --microvm-sandbox-block, --microvm-workload-identity, --microvm-lifecycle, --microvm-control-console, --microvm-control-auth-stdin, --restore-processors, --restore-memory, --restore-absent-scratch, --memory-capacity, snapshot block policy, and microVM network policy require a microVM machine"
);
return Ok(());
}
Expand Down Expand Up @@ -889,6 +898,13 @@ impl Options {
);
}
}
// Clap's `requires` does not hold when an option that conflicts with
// --restore-snapshot, such as --kernel, is present, so a cold boot
// would otherwise ignore the declaration.
anyhow::ensure!(
self.restore_snapshot.is_some() || !self.microvm.restore_absent_scratch,
"--restore-absent-scratch requires --restore-snapshot"
);
if self.restore_snapshot.is_some() {
anyhow::ensure!(
self.microvm.microvm_workload_identity.is_none(),
Expand Down Expand Up @@ -1576,6 +1592,60 @@ mod tests {
assert!(unaligned.validate_microvm_options().is_err());
}

#[cfg(guest_arch = "x86_64")]
#[test]
fn test_microvm_restore_absent_scratch_requires_a_microvm_restore() {
let restore = Options::try_parse_from([
"openvmm",
"--machine",
"microvm",
"--restore-snapshot",
"snapshot",
"--restore-absent-scratch",
])
.unwrap();
restore.validate_microvm_options().unwrap();
assert!(restore.microvm.restore_absent_scratch);

// A cold boot has no captured scratch to declare absent.
assert!(
Options::try_parse_from([
"openvmm",
"--machine",
"microvm",
"--restore-absent-scratch",
])
.is_err()
);
// Clap accepts it when --kernel, which conflicts with --restore-snapshot,
// lifts the requirement, so the validation refuses it.
let cold_boot = Options::try_parse_from([
"openvmm",
"--machine",
"microvm",
"--kernel",
"vmlinux",
"--restore-absent-scratch",
])
.unwrap();
assert_eq!(
cold_boot
.validate_microvm_options()
.unwrap_err()
.to_string(),
"--restore-absent-scratch requires --restore-snapshot"
);
// Nor does a standard machine.
let standard = Options::try_parse_from([
"openvmm",
"--restore-snapshot",
"snapshot",
"--restore-absent-scratch",
])
.unwrap();
assert!(standard.validate_microvm_options().is_err());
}

#[cfg(guest_arch = "x86_64")]
#[test]
fn test_microvm_sandbox_block_parser_and_validation() {
Expand Down
10 changes: 9 additions & 1 deletion openvmm/openvmm_entry/src/microvm/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ use openvmm_defs::config::Config;
use openvmm_defs::config::DeviceVtl;
use openvmm_defs::config::LoadMode;
use openvmm_defs::microvm::MachineProfile;
use openvmm_defs::microvm::MicrovmSandboxBlockRole;
use openvmm_defs::microvm::build_microvm_command_line;
use openvmm_defs::microvm::build_microvm_control_command_line;
use pal_async::DefaultDriver;
Expand Down Expand Up @@ -367,7 +368,8 @@ impl<'a> MicrovmConfigBuilder<'a> {
Ok(virtio_console_backend)
}

/// Adds the fixed-role sandbox block devices.
/// Adds the fixed-role sandbox block devices, with a platform restore's
/// absent fresh scratch.
pub(crate) async fn add_sandbox_blocks(
&self,
storage: &mut StorageBuilder,
Expand All @@ -392,9 +394,15 @@ impl<'a> MicrovmConfigBuilder<'a> {
&disk.kind,
disk.read_only,
opt.microvm.snapshot_destination.is_some() || opt.restore_snapshot.is_some(),
self.restore
.distro_capacity
.filter(|_| block.role == MicrovmSandboxBlockRole::Distro),
)
.await?;
}
if let Some(geometry) = self.restore.absent_scratch {
storage.add_microvm_absent_scratch(geometry);
}
Ok(())
}

Expand Down
Loading
Loading