diff --git a/Guide/src/reference/devices/virtio/virtio-fs.md b/Guide/src/reference/devices/virtio/virtio-fs.md index 2bb778f82be..95113e1a992 100644 --- a/Guide/src/reference/devices/virtio/virtio-fs.md +++ b/Guide/src/reference/devices/virtio/virtio-fs.md @@ -47,7 +47,7 @@ fixes the remaining guest-visible configuration: | Transport | virtio-mmio at `0xd0001000` | | Interrupt | IRQ 6 | | Queues | One high-priority and one request queue | -| Features | Indirect descriptors, event index, version 1, access-platform | +| Features | Indirect descriptors, event index, version 1 | | DAX window | None | | Cache policy | Zero entry and attribute lifetimes | | File policy | Direct I/O | diff --git a/Guide/src/reference/openvmm/management/cli.md b/Guide/src/reference/openvmm/management/cli.md index 1edf5873923..c995220c21e 100644 --- a/Guide/src/reference/openvmm/management/cli.md +++ b/Guide/src/reference/openvmm/management/cli.md @@ -64,12 +64,18 @@ describes the source definitions. `nvx_control_tty=hvc2` token. The reserved slot and resource identity do not expose a live control endpoint. Both consoles remain virtio-console devices from the guest's perspective. Ordinary - `--virtio-blk` is rejected. All use split rings. Firmware, ACPI, SMBIOS, PCI, - VMBus, UARTs, graphics, isolation, nested virtualization, and other devices - are rejected. Linux discovers contiguous APIC IDs and the IOAPIC from Intel - MP 1.4 tables at `0x0` and `0x400`; `boot_params` is at `0x2000`, the command - line starts at `0x20000`, and no ACPI or SMBIOS data is exposed. Host-driven - save/restore, pulse-save/restore, and worker restart remain unavailable. + `--virtio-blk` is rejected. All use split rings, and each offers an exact + feature mask of virtio 1.0 features: the NIC its MAC address; the block + roles a segment limit, block size, flush, and topology, and `distro` + read-only; and every device but the NIC indirect descriptors and event-index + notification suppression. No device offers `VIRTIO_F_ACCESS_PLATFORM`, since + the microVM has no IOMMU, and no console offers its size. Firmware, ACPI, + SMBIOS, PCI, VMBus, UARTs, graphics, isolation, nested virtualization, and + other devices are rejected. Linux discovers contiguous APIC IDs and the + IOAPIC from Intel MP 1.4 tables at `0x0` and `0x400`; `boot_params` is at + `0x2000`, the command line starts at `0x20000`, and no ACPI or SMBIOS data is + exposed. Host-driven save/restore, pulse-save/restore, and worker restart + remain unavailable. `microvm` may also expose a dedicated control virtio-console at MMIO `0xd0007000`, IRQ 3. It requires the boot virtio-console, preserves @@ -94,7 +100,10 @@ describes the source definitions. ```admonish warning title="microVM migration" The canonical `microvm` spelling now selects the contract formerly exposed as `microvm-v2`; the `microvm-v2` selector and the former ABI-v1 behavior are - removed. Snapshot ABI and boot layout remain numeric value 2. + removed. Snapshots record microVM ABI value 3 and boot layout value 2. ABI + value 3 offers no device `VIRTIO_F_ACCESS_PLATFORM` and no console + `VIRTIO_CONSOLE_F_SIZE`, which value 2 offered, so restore refuses a + snapshot of value 2, which must be recaptured. ``` * `--net `: With `--machine microvm`, attach one virtio-net NIC at MMIO `0xd0000000`. KVM and MSHV use IRQ 10; WHP uses IRQ 5. Prefixes diff --git a/Guide/src/reference/openvmm/management/grpc.md b/Guide/src/reference/openvmm/management/grpc.md index c329c6cc543..caeb7d9bb5d 100644 --- a/Guide/src/reference/openvmm/management/grpc.md +++ b/Guide/src/reference/openvmm/management/grpc.md @@ -260,9 +260,11 @@ opened. Clients must regenerate bindings because `machine_profile` moved from field 15 to field 17 when upstream assigned field 15 to `iommufds`; the colliding legacy encoding is rejected. Clients that used the former `MICROVM_V2` source name must also update their bindings. RPC construction is -currently blockless; role-bearing sandbox blocks remain CLI-only. Snapshot ABI -and boot layout remain value 2, which identifies the Linux-direct MP-table -layout. Value 1 snapshots are unsupported. +currently blockless; role-bearing sandbox blocks remain CLI-only. Snapshots +record microVM ABI value 3 and boot layout value 2, which identifies the +Linux-direct MP-table layout. Snapshots of ABI values 1 and 2 are unsupported: +value 2 offered `VIRTIO_F_ACCESS_PLATFORM` on every device but the NIC, and the +console size on the consoles. `VMConfig.cpu_profile` selects the microVM's CPU profile as [`--cpu-profile`](cli.md) does, with the same values and checks: `auto`, the diff --git a/Guide/src/user_guide/openvmm/run.md b/Guide/src/user_guide/openvmm/run.md index 5a740dff10a..0da0ac7efde 100644 --- a/Guide/src/user_guide/openvmm/run.md +++ b/Guide/src/user_guide/openvmm/run.md @@ -92,7 +92,9 @@ NUMA, xAPIC mode, and contiguous APIC IDs `0..N-1`; APIC ID 0 is the BSP. Custom socket, SMT, APIC, x2APIC, and NUMA options are rejected. The microVM uses fixed virtio device slots and sandbox block roles. Its -persisted ABI and boot layout remain value 2. The +persisted ABI is value 3 and its boot layout value 2. Each fixed virtio device +offers an exact feature mask, which never includes `VIRTIO_F_ACCESS_PLATFORM`, +since the microVM has no IOMMU, nor packed virtqueues. The MP floating pointer begins at `0x0`, the MP configuration table at `0x400`, the boot GDT at `0x1000`, and the Linux zero page at `0x2000`. No ACPI MADT or SMBIOS data is exposed. diff --git a/Guide/src/user_guide/openvmm/snapshots.md b/Guide/src/user_guide/openvmm/snapshots.md index 67040cdfc9b..0b2b07665bf 100644 --- a/Guide/src/user_guide/openvmm/snapshots.md +++ b/Guide/src/user_guide/openvmm/snapshots.md @@ -604,8 +604,9 @@ immediately with a clear error if any active device does not support it. - VMs using PCAT firmware do not support save/restore - Standard-machine restore still requires matching `--memory` and `--processors`. MicroVM restore reads them authoritatively from the manifest - and rejects overrides. Persisted microVM ABI and boot-layout value 2 are - supported. + and rejects overrides. Persisted microVM ABI value 3 and boot-layout value 2 + are supported; a snapshot of ABI value 2, whose devices offered + `VIRTIO_F_ACCESS_PLATFORM`, must be recaptured. - MicroVM restore requires the backend, CPU generation, and CPU profile of the capture, and an OpenVMM build that computes the same effective CPUID. A host profile's snapshot, including one of a cold boot whose `auto` fell diff --git a/openvmm/openvmm_core/src/worker/dispatch/microvm.rs b/openvmm/openvmm_core/src/worker/dispatch/microvm.rs index 80b7a411dc1..11c680f22bd 100644 --- a/openvmm/openvmm_core/src/worker/dispatch/microvm.rs +++ b/openvmm/openvmm_core/src/worker/dispatch/microvm.rs @@ -472,6 +472,8 @@ impl LoadedVm { } } +/// Returns the fixed slot of the microVM virtio device `id`: its MMIO window, +/// interrupt, exact guest-visible feature mask, and interrupt-status delivery. fn virtio_mmio_config( id: &str, sandbox_blocks: &[openvmm_defs::microvm::MicrovmSandboxBlockConfig], @@ -479,10 +481,11 @@ fn virtio_mmio_config( filesystem_index: &mut usize, chipset_mmio: ChipsetMmioRanges, ) -> anyhow::Result<(u64, u64, u32, u64, VirtioMmioInterruptMode)> { - let (start, irq) = match id { + let (start, irq, features) = match id { "virtio-net" => ( openvmm_defs::microvm::MICROVM_VIRTIO_NET_MMIO_BASE, openvmm_defs::microvm::microvm_virtio_net_irq(None)?, + openvmm_defs::microvm::MICROVM_VIRTIO_NET_FEATURES, ), // The configuration orders virtio-fs devices by slot. "virtiofs" => { @@ -490,22 +493,32 @@ fn virtio_mmio_config( .get(*filesystem_index) .context("microVM has more virtio-fs devices than fixed slots")?; *filesystem_index += 1; - (slot.mmio_base, slot.irq) + ( + slot.mmio_base, + slot.irq, + openvmm_defs::microvm::MICROVM_VIRTIO_FS_FEATURES, + ) } "virtio-console" => ( openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_MMIO_BASE, openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_IRQ, + openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_FEATURES, ), openvmm_defs::microvm::MICROVM_VIRTIO_CONTROL_CONSOLE_ID => ( openvmm_defs::microvm::MICROVM_VIRTIO_CONTROL_CONSOLE_MMIO_BASE, openvmm_defs::microvm::MICROVM_VIRTIO_CONTROL_CONSOLE_IRQ, + openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_FEATURES, ), "virtio-blk" => { let block = sandbox_blocks .get(*sandbox_block_index) .context("microVM virtio-blk device has no sandbox role")?; *sandbox_block_index += 1; - (block.role.mmio_base(), block.role.irq()) + ( + block.role.mmio_base(), + block.role.irq(), + openvmm_defs::microvm::microvm_sandbox_block_features(block.role), + ) } _ => anyhow::bail!("unsupported microVM virtio device '{id}' reached worker construction"), }; @@ -517,23 +530,11 @@ fn virtio_mmio_config( .is_some_and(|end| end <= chipset_mmio.low.end()), "microVM virtio slot for '{id}' is outside the fixed low-MMIO aperture" ); - let disabled_features = match id { - "virtio-net" => !openvmm_defs::microvm::MICROVM_VIRTIO_NET_FEATURES, - "virtiofs" => !openvmm_defs::microvm::MICROVM_VIRTIO_FS_FEATURES, - "virtio-blk" => { - let block = sandbox_blocks - .iter() - .find(|block| block.role.mmio_base() == start) - .context("microVM block slot has no role")?; - !openvmm_defs::microvm::microvm_sandbox_block_features(block.role) - } - _ => 1 << 34, - }; let interrupt_mode = VirtioMmioInterruptMode::SharedStatus { status_gpa: openvmm_defs::microvm::microvm_virtio_status_gpa(start) .context("microVM slot has no shared-status word")?, }; - Ok((start, len, irq, disabled_features, interrupt_mode)) + Ok((start, len, irq, features, interrupt_mode)) } /// The guest memory that the microVM e820 map reserves inside RAM besides the @@ -602,7 +603,7 @@ impl<'a> VirtioMmioSlots<'a> { id: &str, device: ResolvedVirtioDevice, ) -> anyhow::Result<()> { - let (mmio_start, mmio_len, irq, disabled_features, interrupt_mode) = virtio_mmio_config( + let (mmio_start, mmio_len, irq, features, interrupt_mode) = virtio_mmio_config( id, self.sandbox_blocks, &mut self.sandbox_block_index, @@ -625,7 +626,7 @@ impl<'a> VirtioMmioSlots<'a> { partition.clone().into_doorbell_registration(Vtl::Vtl0), mmio_start, mmio_len, - disabled_features, + !features, interrupt_mode, ) .context("failed to create the virtio-mmio transport")?; @@ -803,3 +804,161 @@ pub(super) fn x86_topology_builder( Ok(vm_topology::processor::TopologyBuilder::from_host_topology()?) } } + +// The microVM runs on Linux and Windows hosts alone, where its NIC has an +// interrupt. +#[cfg(all(test, any(target_os = "linux", windows)))] +mod tests { + use super::virtio_mmio_config; + use crate::worker::memory_layout::ChipsetMmioRanges; + use chipset_device::io::IoResult; + use chipset_device::mmio::MmioIntercept; + use guestmem::GuestMemory; + use inspect::InspectMut; + use memory_range::MemoryRange; + use openvmm_defs::microvm::MicrovmSandboxBlockConfig; + use openvmm_defs::microvm::MicrovmSandboxBlockRole; + use pal_async::DefaultDriver; + use pal_async::async_test; + use virtio::DeviceTraits; + use virtio::QueueResources; + use virtio::VirtioDevice; + use virtio::VirtioMmioDevice; + use virtio::queue::QueueState; + use virtio::spec::VirtioDeviceFeatures; + use virtio::spec::mmio::VirtioMmioRegister; + use vmcore::line_interrupt::LineInterrupt; + use vmcore::vm_task::SingleDriverBackend; + use vmcore::vm_task::VmTaskDriverSource; + + /// A device that offers every feature, so that its transport offers the + /// guest exactly the features that the microVM configuration admits. + #[derive(InspectMut)] + struct EveryFeatureDevice; + + impl VirtioDevice for EveryFeatureDevice { + fn traits(&self) -> DeviceTraits { + DeviceTraits { + device_features: VirtioDeviceFeatures::from_bits(u64::MAX), + max_queues: 1, + ..Default::default() + } + } + + async fn read_registers_u32(&mut self, _offset: u16) -> u32 { + 0 + } + + async fn write_registers_u32(&mut self, _offset: u16, _val: u32) {} + + async fn start_queue( + &mut self, + _idx: u16, + _resources: QueueResources, + _features: &VirtioDeviceFeatures, + _initial_state: Option, + ) -> anyhow::Result<()> { + anyhow::bail!("the test never starts a queue") + } + + async fn stop_queue(&mut self, _idx: u16) -> Option { + None + } + } + + /// Returns the numbers of the feature bits that the transport at + /// `mmio_base` offers the guest, read as a driver reads them, one 32-bit + /// bank at a time. + fn offered_feature_bits(device: &mut VirtioMmioDevice, mmio_base: u64) -> Vec { + let select = mmio_base + u64::from(VirtioMmioRegister::DEVICE_FEATURES_SEL.0); + let register = mmio_base + u64::from(VirtioMmioRegister::DEVICE_FEATURES.0); + let mut features = 0; + for bank in 0..2u32 { + assert!(matches!( + device.mmio_write(select, &bank.to_le_bytes()), + IoResult::Ok + )); + let mut value = [0; 4]; + assert!(matches!( + device.mmio_read(register, &mut value), + IoResult::Ok + )); + features |= u64::from(u32::from_le_bytes(value)) << (32 * bank); + } + (0..u64::BITS) + .filter(|bit| features & (1 << bit) != 0) + .collect() + } + + #[async_test] + async fn microvm_devices_offer_exactly_their_feature_masks(driver: DefaultDriver) { + let driver_source = VmTaskDriverSource::new(SingleDriverBackend::new(driver)); + // The memory holds the page of the slots' shared interrupt-status + // words. + let gm = GuestMemory::allocate( + (openvmm_defs::microvm::MICROVM_SHARED_STATUS_PAGE_GPA + + openvmm_defs::microvm::MICROVM_SHARED_STATUS_PAGE_SIZE) as usize, + ); + let chipset_mmio = ChipsetMmioRanges { + low: MemoryRange::new(0xc000_0000..0x1_0000_0000), + high: MemoryRange::EMPTY, + vtl2: MemoryRange::EMPTY, + }; + let blocks = [ + MicrovmSandboxBlockConfig { + role: MicrovmSandboxBlockRole::Distro, + read_only: true, + }, + MicrovmSandboxBlockConfig { + role: MicrovmSandboxBlockRole::Scratch, + read_only: false, + }, + ]; + let mut sandbox_block_index = 0; + let mut filesystem_index = 0; + // Each device in the order of the slots of its kind, with its window + // and the features that the guest sees: never access-platform, which + // the transport offers every device, nor the console size, which the + // virtio-console device offers. + let devices: [(&str, u64, &[u32]); 6] = [ + ("virtio-net", 0xd000_0000, &[5, 32]), + ("virtiofs", 0xd000_1000, &[28, 29, 32]), + ("virtio-console", 0xd000_2000, &[28, 29, 32]), + ("virtio-blk", 0xd000_3000, &[2, 5, 6, 9, 10, 28, 29, 32]), + ("virtio-blk", 0xd000_6000, &[2, 6, 9, 10, 28, 29, 32]), + ( + openvmm_defs::microvm::MICROVM_VIRTIO_CONTROL_CONSOLE_ID, + 0xd000_7000, + &[28, 29, 32], + ), + ]; + for (id, mmio_base, bits) in devices { + let (start, len, _irq, features, interrupt_mode) = virtio_mmio_config( + id, + &blocks, + &mut sandbox_block_index, + &mut filesystem_index, + chipset_mmio, + ) + .unwrap(); + assert_eq!(start, mmio_base, "{id}"); + let mut device = VirtioMmioDevice::new_with_disabled_features_and_interrupt_mode( + Box::new(EveryFeatureDevice), + &driver_source.simple(), + gm.clone(), + LineInterrupt::detached(), + None, + start, + len, + !features, + interrupt_mode, + ) + .unwrap(); + assert_eq!( + offered_feature_bits(&mut device, start), + bits, + "{id} at {start:#x}" + ); + } + } +} diff --git a/openvmm/openvmm_defs/src/microvm.rs b/openvmm/openvmm_defs/src/microvm.rs index 6d2d1432a63..9a7980ba685 100644 --- a/openvmm/openvmm_defs/src/microvm.rs +++ b/openvmm/openvmm_defs/src/microvm.rs @@ -22,7 +22,12 @@ use std::fmt::Write as _; use vmotherboard::options::BaseChipsetManifest; /// The persisted microVM ABI version. -pub const MICROVM_ABI_VERSION_2: u32 = 2; +/// +/// Version 3 offers no device `VIRTIO_F_ACCESS_PLATFORM`, since the microVM +/// has no IOMMU, and no console `VIRTIO_CONSOLE_F_SIZE`, both of which +/// version 2 offered. A snapshot of version 2 records those feature masks, so +/// restore refuses it by its version, before it compares the devices. +pub const MICROVM_ABI_VERSION_3: u32 = 3; /// Linux memory-block granularity used for microVM restore-time expansion. pub const MICROVM_MEMORY_BLOCK_SIZE_BYTES: u64 = 128 * 1024 * 1024; @@ -69,11 +74,55 @@ pub const MICROVM_VIRTIO_FS_IRQ: u32 = 6; pub const MICROVM_VIRTIO_NET_KVM_IRQ: u32 = 10; /// Fixed microVM virtio-net interrupt on WHP. pub const MICROVM_VIRTIO_NET_WHP_IRQ: u32 = 5; +// The virtio 1.2 feature bits that the exact microVM feature masks name. The +// masks leave out `VIRTIO_F_ACCESS_PLATFORM` (bit 33), which the virtio +// transport offers every device, since the microVM has no IOMMU to translate a +// device's DMA, and the console size, `VIRTIO_CONSOLE_F_SIZE` (bit 0), which +// the virtio-console device offers with a size of 0 columns and 0 rows. +/// Indirect descriptors. +const VIRTIO_F_INDIRECT_DESC: u64 = 1 << 28; +/// Event-index notification suppression. +const VIRTIO_F_EVENT_IDX: u64 = 1 << 29; +/// A virtio 1.0 device. +const VIRTIO_F_VERSION_1: u64 = 1 << 32; +/// The NIC's MAC address. +const VIRTIO_NET_F_MAC: u64 = 1 << 5; +/// A per-request segment limit. +const VIRTIO_BLK_F_SEG_MAX: u64 = 1 << 2; +/// A read-only block device. +const VIRTIO_BLK_F_RO: u64 = 1 << 5; +/// The block device's logical block size. +const VIRTIO_BLK_F_BLK_SIZE: u64 = 1 << 6; +/// A write-back cache with explicit flush. +const VIRTIO_BLK_F_FLUSH: u64 = 1 << 9; +/// The block device's topology. +const VIRTIO_BLK_F_TOPOLOGY: u64 = 1 << 10; +/// The ring features of every microVM virtio device but the NIC: split +/// virtqueues with indirect descriptors and event-index notification +/// suppression, of a virtio 1.0 device. +const MICROVM_VIRTIO_SPLIT_RING_FEATURES: u64 = + VIRTIO_F_INDIRECT_DESC | VIRTIO_F_EVENT_IDX | VIRTIO_F_VERSION_1; /// Exact microVM virtio-net feature mask: MAC and virtio version 1. -pub const MICROVM_VIRTIO_NET_FEATURES: u64 = (1 << 5) | (1 << 32); +pub const MICROVM_VIRTIO_NET_FEATURES: u64 = VIRTIO_NET_F_MAC | VIRTIO_F_VERSION_1; /// Exact microVM virtio-fs feature mask: indirect descriptors, event index, -/// virtio version 1, and access-platform. -pub const MICROVM_VIRTIO_FS_FEATURES: u64 = (1 << 28) | (1 << 29) | (1 << 32) | (1 << 33); +/// and virtio version 1. +pub const MICROVM_VIRTIO_FS_FEATURES: u64 = MICROVM_VIRTIO_SPLIT_RING_FEATURES; +/// Exact feature mask of both microVM virtio consoles, the boot console and +/// the control console: indirect descriptors, event index, and virtio version +/// 1. +pub const MICROVM_VIRTIO_CONSOLE_FEATURES: u64 = MICROVM_VIRTIO_SPLIT_RING_FEATURES; +/// Exact feature mask of the writable `scratch` virtio-blk role: the segment +/// limit, block size, flush, topology, indirect descriptors, event index, and +/// virtio version 1. +pub const MICROVM_VIRTIO_SCRATCH_BLK_FEATURES: u64 = VIRTIO_BLK_F_SEG_MAX + | VIRTIO_BLK_F_BLK_SIZE + | VIRTIO_BLK_F_FLUSH + | VIRTIO_BLK_F_TOPOLOGY + | MICROVM_VIRTIO_SPLIT_RING_FEATURES; +/// Exact feature mask of the read-only `distro` virtio-blk role: the +/// `scratch` role's features, and read-only. +pub const MICROVM_VIRTIO_DISTRO_BLK_FEATURES: u64 = + MICROVM_VIRTIO_SCRATCH_BLK_FEATURES | VIRTIO_BLK_F_RO; /// MicroVM client console reconnect timeout. pub const MICROVM_CONSOLE_RECONNECT_TIMEOUT_MS: u64 = 5_000; /// Resource identity of the microVM control virtio-console device. @@ -218,31 +267,12 @@ impl MicrovmSandboxBlockRole { } } -/// Returns the fixed virtio-blk feature mask for a sandbox role. +/// Returns the exact virtio-blk feature mask of a sandbox role. pub const fn microvm_sandbox_block_features(role: MicrovmSandboxBlockRole) -> u64 { - const RING_INDIRECT_DESC: u64 = 1 << 28; - const RING_EVENT_IDX: u64 = 1 << 29; - const VERSION_1: u64 = 1 << 32; - const ACCESS_PLATFORM: u64 = 1 << 33; - const BLK_SEG_MAX: u64 = 1 << 2; - const BLK_READ_ONLY: u64 = 1 << 5; - const BLK_SIZE: u64 = 1 << 6; - const BLK_FLUSH: u64 = 1 << 9; - const BLK_TOPOLOGY: u64 = 1 << 10; - - RING_INDIRECT_DESC - | RING_EVENT_IDX - | VERSION_1 - | ACCESS_PLATFORM - | BLK_SEG_MAX - | BLK_SIZE - | BLK_FLUSH - | BLK_TOPOLOGY - | if role.is_read_only() { - BLK_READ_ONLY - } else { - 0 - } + match role { + MicrovmSandboxBlockRole::Distro => MICROVM_VIRTIO_DISTRO_BLK_FEATURES, + MicrovmSandboxBlockRole::Scratch => MICROVM_VIRTIO_SCRATCH_BLK_FEATURES, + } } /// The immutable role and access mode of a microVM sandbox block device. @@ -2673,6 +2703,48 @@ mod tests { assert_eq!(microvm_virtio_status_gpa(0xd000_9000), None); } + /// Returns the numbers of the feature bits that `features` sets. + fn feature_bits(features: u64) -> Vec { + (0..u64::BITS) + .filter(|bit| features & (1 << bit) != 0) + .collect() + } + + #[test] + fn microvm_virtio_feature_masks_are_exact() { + let masks = [ + (MICROVM_VIRTIO_NET_FEATURES, vec![5, 32]), + (MICROVM_VIRTIO_FS_FEATURES, vec![28, 29, 32]), + (MICROVM_VIRTIO_CONSOLE_FEATURES, vec![28, 29, 32]), + ( + microvm_sandbox_block_features(MicrovmSandboxBlockRole::Distro), + vec![2, 5, 6, 9, 10, 28, 29, 32], + ), + ( + microvm_sandbox_block_features(MicrovmSandboxBlockRole::Scratch), + vec![2, 6, 9, 10, 28, 29, 32], + ), + ]; + for (features, bits) in masks { + assert_eq!(feature_bits(features), bits); + // Every device is a virtio 1.0 device with split virtqueues, and + // the microVM has no IOMMU. + assert_ne!(features & VIRTIO_F_VERSION_1, 0); + assert_eq!(features & (1 << 34), 0, "packed virtqueues are offered"); + assert_eq!(features & (1 << 33), 0, "access-platform is offered"); + } + // The consoles report no size. + assert_eq!(MICROVM_VIRTIO_CONSOLE_FEATURES & 1, 0); + assert_eq!( + microvm_sandbox_block_features(MicrovmSandboxBlockRole::Distro), + MICROVM_VIRTIO_DISTRO_BLK_FEATURES + ); + assert_eq!( + microvm_sandbox_block_features(MicrovmSandboxBlockRole::Scratch), + MICROVM_VIRTIO_SCRATCH_BLK_FEATURES + ); + } + #[test] fn microvm_filesystem_slot_is_stable() { assert_eq!( diff --git a/openvmm/openvmm_entry/src/ttrpc/microvm.rs b/openvmm/openvmm_entry/src/ttrpc/microvm.rs index d73cd40e52e..700ca718e00 100644 --- a/openvmm/openvmm_entry/src/ttrpc/microvm.rs +++ b/openvmm/openvmm_entry/src/ttrpc/microvm.rs @@ -24,7 +24,7 @@ use openvmm_defs::config::NumaNode; use openvmm_defs::config::NumaTopology; use openvmm_defs::config::VpAssignment; use openvmm_defs::config::X86TopologyConfig; -use openvmm_defs::microvm::MICROVM_ABI_VERSION_2; +use openvmm_defs::microvm::MICROVM_ABI_VERSION_3; use openvmm_defs::microvm::MachineProfile; use openvmm_defs::microvm::build_microvm_command_line; use openvmm_defs::worker::SharedMemoryFd; @@ -1443,8 +1443,8 @@ fn openvmm_machine_profile(profile: vmservice::vm_config::MachineProfile) -> Mac fn ttrpc_machine_profile(abi_version: u32) -> anyhow::Result { anyhow::ensure!( - abi_version == MICROVM_ABI_VERSION_2, - "unsupported microVM ABI version {abi_version}; this OpenVMM supports version {MICROVM_ABI_VERSION_2}" + abi_version == MICROVM_ABI_VERSION_3, + "unsupported microVM ABI version {abi_version}; this OpenVMM supports version {MICROVM_ABI_VERSION_3}" ); Ok(vmservice::vm_config::MachineProfile::Microvm) } @@ -1773,20 +1773,24 @@ mod tests { #[test] fn ttrpc_microvm_profile_preserves_wire_identity_and_processor_policy() { - assert_eq!( - vmservice::vm_config::MachineProfile::Microvm as i32, - MICROVM_ABI_VERSION_2 as i32 - ); + // The profile keeps its numeric value 2, which clients send, whatever + // microVM ABI version it selects; the ABI version is a snapshot's. + assert_eq!(vmservice::vm_config::MachineProfile::Microvm as i32, 2); assert_eq!( openvmm_machine_profile(vmservice::vm_config::MachineProfile::Microvm), OpenvmmMachineProfile::Microvm ); assert_eq!( - ttrpc_machine_profile(MICROVM_ABI_VERSION_2).unwrap(), + ttrpc_machine_profile(MICROVM_ABI_VERSION_3).unwrap(), vmservice::vm_config::MachineProfile::Microvm ); assert!(vmservice::vm_config::MachineProfile::from_i32(1).is_none()); + // Snapshots of microVM ABI versions 1 and 2 are unsupported. assert!(ttrpc_machine_profile(1).is_err()); + assert_eq!( + ttrpc_machine_profile(2).unwrap_err().to_string(), + "unsupported microVM ABI version 2; this OpenVMM supports version 3" + ); for processor_count in [1, 2, 4, 8] { assert!(openvmm_defs::microvm::microvm_processor_count_supported( diff --git a/openvmm/openvmm_helpers/src/snapshot/microvm.rs b/openvmm/openvmm_helpers/src/snapshot/microvm.rs index 76fff91ee62..62af75d279d 100644 --- a/openvmm/openvmm_helpers/src/snapshot/microvm.rs +++ b/openvmm/openvmm_helpers/src/snapshot/microvm.rs @@ -987,6 +987,11 @@ fn memory_expansion_prefix( Ok(prefix) } +/// Returns the feature banks of an exact virtio feature mask, in bank order. +fn feature_banks(features: u64) -> Vec { + vec![features as u32, (features >> 32) as u32] +} + /// Returns the snapshot device of a fixed virtio-fs slot. fn microvm_filesystem_device( slot: &openvmm_defs::microvm::MicrovmFilesystemSlot, @@ -1004,10 +1009,7 @@ fn microvm_filesystem_device( }], irq: Some(slot.irq), transport: "virtio-mmio".to_owned(), - feature_banks: vec![ - openvmm_defs::microvm::MICROVM_VIRTIO_FS_FEATURES as u32, - (openvmm_defs::microvm::MICROVM_VIRTIO_FS_FEATURES >> 32) as u32, - ], + feature_banks: feature_banks(openvmm_defs::microvm::MICROVM_VIRTIO_FS_FEATURES), queue_count: 2, queue_max_sizes: vec![256, 256], } @@ -1300,10 +1302,7 @@ pub fn microvm_machine_contract( )], irq: Some(irq), transport: "virtio-mmio".to_owned(), - feature_banks: vec![ - openvmm_defs::microvm::MICROVM_VIRTIO_NET_FEATURES as u32, - (openvmm_defs::microvm::MICROVM_VIRTIO_NET_FEATURES >> 32) as u32, - ], + feature_banks: feature_banks(openvmm_defs::microvm::MICROVM_VIRTIO_NET_FEATURES), queue_count: 2, queue_max_sizes: vec![256, 256], }); @@ -1418,7 +1417,7 @@ pub fn microvm_machine_contract( )], irq: Some(openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_IRQ), transport: "virtio-mmio".to_owned(), - feature_banks: vec![0x3000_0001, 0x0000_0003], + feature_banks: feature_banks(openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_FEATURES), queue_count: 2, queue_max_sizes: vec![256, 256], }); @@ -1444,7 +1443,6 @@ pub fn microvm_machine_contract( "microVM sandbox block '{}' is missing from the effective command line", block.role ); - let features = openvmm_defs::microvm::microvm_sandbox_block_features(role); devices.push(SnapshotDevice { stable_id: format!("blk:sandbox:{}", role.as_str()), state_unit_name: format!("virtio-blk-{}", role.mmio_base()), @@ -1456,7 +1454,9 @@ pub fn microvm_machine_contract( )], irq: Some(role.irq()), transport: "virtio-mmio".to_owned(), - feature_banks: vec![features as u32, (features >> 32) as u32], + feature_banks: feature_banks(openvmm_defs::microvm::microvm_sandbox_block_features( + role, + )), queue_count: 1, queue_max_sizes: vec![256], }); @@ -1516,7 +1516,7 @@ pub fn microvm_machine_contract( )], irq: Some(openvmm_defs::microvm::MICROVM_VIRTIO_CONTROL_CONSOLE_IRQ), transport: "virtio-mmio".to_owned(), - feature_banks: vec![0x3000_0001, 0x0000_0003], + feature_banks: feature_banks(openvmm_defs::microvm::MICROVM_VIRTIO_CONSOLE_FEATURES), queue_count: 2, queue_max_sizes: vec![256, 256], }); @@ -1525,7 +1525,7 @@ pub fn microvm_machine_contract( let mut contract = SnapshotMachineContract { machine_profile: "microvm".to_owned(), - microvm_abi_version: openvmm_defs::microvm::MICROVM_ABI_VERSION_2, + microvm_abi_version: openvmm_defs::microvm::MICROVM_ABI_VERSION_3, source_hypervisor: source_hypervisor.to_owned(), effective_command_line: String::new(), effective_command_line_sha256: Vec::new(), @@ -1564,7 +1564,7 @@ pub fn requires_post_restore_gate(manifest: &SnapshotManifest) -> bool { manifest.machine_contract.as_ref().is_some_and(|contract| { matches!( contract.microvm_abi_version, - openvmm_defs::microvm::MICROVM_ABI_VERSION_2 + openvmm_defs::microvm::MICROVM_ABI_VERSION_3 ) }) && !manifest.snapshot_tier.is_empty() } @@ -1585,10 +1585,10 @@ pub fn validate_supported_microvm_contract( contract: &SnapshotMachineContract, ) -> anyhow::Result<()> { anyhow::ensure!( - contract.microvm_abi_version == openvmm_defs::microvm::MICROVM_ABI_VERSION_2, + contract.microvm_abi_version == openvmm_defs::microvm::MICROVM_ABI_VERSION_3, "snapshot microVM ABI version {} is unsupported; this OpenVMM supports version {}", contract.microvm_abi_version, - openvmm_defs::microvm::MICROVM_ABI_VERSION_2, + openvmm_defs::microvm::MICROVM_ABI_VERSION_3, ); anyhow::ensure!( contract.boot_layout_version == MICROVM_BOOT_LAYOUT_VERSION, @@ -2486,7 +2486,7 @@ pub(super) fn paired_scratch_manifest(scratch: &[u8]) -> SnapshotManifest { pub(super) fn test_machine_contract() -> SnapshotMachineContract { let mut contract = SnapshotMachineContract { machine_profile: "microvm".to_owned(), - microvm_abi_version: openvmm_defs::microvm::MICROVM_ABI_VERSION_2, + microvm_abi_version: openvmm_defs::microvm::MICROVM_ABI_VERSION_3, source_hypervisor: "whp".to_owned(), effective_command_line: String::new(), effective_command_line_sha256: Vec::new(), @@ -3404,7 +3404,9 @@ mod tests { assert_eq!(console.ranges[0].length, 0x1000); assert_eq!(console.irq, Some(7)); assert_eq!(console.transport, "virtio-mmio"); - assert_eq!(console.feature_banks, [0x3000_0001, 0x0000_0003]); + // Indirect descriptors, event index, and virtio version 1: no console + // size and no access-platform. + assert_eq!(console.feature_banks, [0x3000_0000, 0x0000_0001]); assert_eq!(console.queue_max_sizes, [256, 256]); assert_eq!(contract.attachments, [microvm_console_attachment()]); } @@ -3431,6 +3433,22 @@ mod tests { assert_eq!(control.transport, "virtio-mmio"); assert_eq!(control.feature_banks, boot.feature_banks); assert_eq!(control.queue_max_sizes, [256, 256]); + // The sandbox blocks between the consoles offer the segment limit, + // block size, flush, topology, the ring features, and virtio version + // 1, and the distro layer is read-only: no access-platform. + let blocks = contract + .devices + .iter() + .filter(|device| device.kind == "virtio-blk") + .map(|device| (device.stable_id.as_str(), device.feature_banks.as_slice())) + .collect::>(); + assert_eq!( + blocks, + [ + ("blk:sandbox:distro", &[0x3000_0664, 0x0000_0001][..]), + ("blk:sandbox:scratch", &[0x3000_0644, 0x0000_0001][..]), + ] + ); assert_eq!( contract.attachments, [ @@ -3767,7 +3785,7 @@ mod tests { assert_eq!(filesystem_device.ranges[0].length, 0x1000); assert_eq!(filesystem_device.irq, Some(6)); assert_eq!(filesystem_device.transport, "virtio-mmio"); - assert_eq!(filesystem_device.feature_banks, [0x3000_0000, 0x0000_0003]); + assert_eq!(filesystem_device.feature_banks, [0x3000_0000, 0x0000_0001]); assert_eq!(filesystem_device.queue_count, 2); assert_eq!(filesystem_device.queue_max_sizes, [256, 256]); assert_eq!( @@ -4025,6 +4043,29 @@ mod tests { assert!(err.to_string().contains("ABI version 1 is unsupported")); } + #[test] + fn abi_v2_contracts_are_refused_by_their_version() { + // Version 2 offered access-platform on every device but the NIC, and + // the console size, so a console of version 2 records other feature + // banks than one of version 3. Restore refuses such a contract by its + // version, before it compares the devices. + const REFUSAL: &str = + "snapshot microVM ABI version 2 is unsupported; this OpenVMM supports version 3"; + let mut contract = generated_console_contract(); + contract.microvm_abi_version = 2; + contract.devices.last_mut().unwrap().feature_banks = vec![0x3000_0001, 0x0000_0003]; + let err = validate_supported_microvm_contract(&contract).unwrap_err(); + assert_eq!(err.to_string(), REFUSAL); + + let mut manifest = test_manifest(); + let expected = test_machine_contract(); + let mut contract = expected.clone(); + contract.microvm_abi_version = 2; + manifest.machine_contract = Some(contract); + let err = validate_microvm_machine_contract(&manifest, &expected).unwrap_err(); + assert_eq!(err.to_string(), REFUSAL); + } + #[test] fn validate_microvm_machine_contract_rejects_backend() { let mut manifest = test_manifest(); @@ -4418,7 +4459,7 @@ mod tests { } #[test] - fn abi_v2_snapshot_tier_contract_is_canonical() { + fn abi_v3_snapshot_tier_contract_is_canonical() { let scratch = vec![0x5a; 512]; let mut manifest = paired_scratch_manifest(&scratch); validate_manifest_contents(&manifest).unwrap(); @@ -4770,7 +4811,7 @@ mod tests { } #[test] - fn abi_v2_contract_rejects_scratch_without_a_lower_layer() { + fn abi_v3_contract_rejects_scratch_without_a_lower_layer() { let scratch = vec![0x5a_u8; 1024]; let mut manifest = paired_scratch_manifest(&scratch); manifest @@ -4790,7 +4831,7 @@ mod tests { } #[test] - fn abi_v2_contract_rejects_the_retired_runtime_and_custom_layers() { + fn abi_v3_contract_rejects_the_retired_runtime_and_custom_layers() { let scratch = vec![0x5a_u8; 1024]; for role in ["runtime", "custom"] { // In place of the distro layer.