Repository navigation
[DO NOT REVIEW] cpu_profile: add NVIDIA N1X WHP profile - #5096
Thans (QThans) wants to merge 284 commits into
Conversation
…e is stopped The virtio transports save and restore only transport state: feature negotiation, queue configuration, and queue progress. A device whose state lives outside its queues, such as buffered console or network data, has nowhere to save it, so such devices cannot support save/restore. Restore also trusted the saved queue layout and progress. Add device-private saved state to `VirtioDevice`. After all queues stop, `save_device` returns an optional typed `SavedStateBlob`, which the PCI and MMIO transports save as a new `device_state` field; `restore_device` receives it, and `device_state_validator` returns an immutable validator that the transport keeps after the device moves into its task. The defaults preserve the current behavior: devices that support save/restore have no private state, and other devices cannot be saved. The new `device::saved_state` module holds the validator type and a helper that reads the front restored descriptor without consuming it. Restore now validates the saved state before it changes the transport: exactly two feature banks, queue sizes within each queue's device maximum, enabled rings aligned, inside guest RAM, and not overlapping, available and used progress consistent with the rings, progress present for every enabled queue of a DRIVER_OK device, and private state accepted by the device's validator. The transport then hands the private state to the stopped device task and waits for the device to restore it, so that a device that cannot restore, such as one that cannot reopen its host resources, fails the restore itself, before any VP runs, rather than a later start. The MMIO transport's save and restore become asynchronous, as the PCI transport's already are. Restoring starts no queue: a save before the first start returns the restored state, and queues that were never started keep their restored progress. A guest reset of a device that is not DRIVER_OK resets the device, which discards the private state that the restore gave it. `ChangeDeviceState::start` cannot wait for the device task, so the transport keeps the task's answer and applies it the next time it polls or stops the device. If a queue fails to start, the device task stops the queues that did start, and the transport sets DEVICE_NEEDS_RESET, notifying a driver that has set DRIVER_OK, as virtio 1.2 section 2.1.2 requires of a device in an error state, rather than leave the guest waiting on a device that processes nothing. The device task traces queue starts and private-state restores under the `virtio_restore` target. `SavedStateBlob` becomes `Clone`. The virtio-blk persistence test now shuts the guest down with an enlightened shutdown after its save/restore pulse, avoiding the pipette shutdown delay, which depends on the guest timer state that the test just restored. It also includes these changes: - virtio: await the PCI transport's save and restore in saved-state tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A snapshot of a running VM first gates host input, so that no new external input reaches a device while its state is captured, and resumes the input if the save transaction fails. `ChangeDeviceState` provides `quiesce_input` and `resume_input` for this, but the virtio transports did not implement them, so a virtio device could not take part. Add `quiesce_input` and `resume_input` to `VirtioDevice`, defaulting to success for devices without host input. The PCI and MMIO transports forward both through the device task as the new `QuiesceInput` and `ResumeInput` commands, so they are ordered with the device's other commands, and return the device's error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
When a write to the connected socket failed with BrokenPipe, the socket backend reported the write as successful and kept the dead socket. Any other write error was returned, but the socket also stayed attached. Either way the backend kept writing to a connection that could no longer carry data, and it never reported the disconnect or accepted a new client. Frontends that keep guest output for the next client need to know which bytes a write failed to send: - A virtio-console with the `Retain` disconnect policy, which the next commit adds, leaves a transmit descriptor queued, with the offset of its bytes already written, only when the write fails. Were the write reported as successful, the worker could complete the guest's output before its receive side noticed the hangup, and that output would be lost. - The microVM portb console, which a later commit adds, keeps its buffered output for the next peer only when a write fails with `BrokenPipe`. Drop the current connection on any write error and return the error. Writes are then discarded, as they are while no client is connected, and the backend reports itself disconnected so the next connect accepts a new client. Say so next to the code. Upstream reports a broken pipe on a write as successful and keeps the socket. microsoft#4605 first proposed this change to detect disconnects, but review settled disconnect detection in virtio-console, which polls `poll_disconnect` without receive buffers since microsoft#4778, and microsoft#4605 merged narrowed to the read path. A test checks that a write to a Unix socket peer that has closed fails with `BrokenPipe`, disconnects the backend, and lets the listener accept a replacement client. It runs on Linux only: Windows accepts the first write after the peer closes and fails the next with `ConnectionReset`, so a console that has not yet noticed the hangup loses that one write's bytes there. It also includes these changes: - serial_socket: document and test why a failed write drops the connection Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
…t policies A virtio-console always discarded guest output while its host backend was disconnected, and a console resource could not describe how its host endpoint is attached or reconnected. A host that must not lose console output, or that reattaches endpoints after a restore, has no way to ask for that, and the resolver cannot reject an inconsistent endpoint description. Add `VirtioConsoleDisconnectPolicy` and an optional `VirtioConsoleAttachment` to `VirtioConsoleHandle`. With the `Retain` policy, the console keeps pending guest transmit descriptors while the backend is disconnected and forwards them after it reconnects; `Discard` keeps the current behavior and remains the default of `VirtioConsoleDevice::new`. The new `new_with_policy` constructor selects the policy, and the worker now holds its backend in a `ConsoleWorkerMode`. The attachment records a stable ID, the backend kind, the attachment mode, the endpoint identity, and a reconnect policy with its requirement and timeout. The resolver rejects an attachment whose identifiers are empty or oversized, whose mode and reconnect policy conflict, or whose requirement and timeout do not match the policy: a recreated listener or a discarding attachment is optional without a timeout, a reconnecting client is required with a bounded timeout, and an inherited attachment is required without a timeout. Disconnects are also handled consistently: after a read error or end-of-file, the worker completes the backend's disconnect before it waits for a new connection, and a backend write that accepts no bytes is treated as a disconnect instead of a panic. All existing console resources use `Discard` without an attachment. It also includes these changes: - petri: adapt upstream's Incubator changes to the chain's petri Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A virtio-console could not be saved: host input already read from the backend lived only in a local buffer of the receive loop, the worker state was removed once both queues stopped, and the progress within the current transmit descriptor was lost with it. Restoring a console would therefore drop accepted input or replay output. Implement device-private save and restore for the direct console mode. The worker state is now created with the device and kept while the queues are stopped, so a stopped device still holds its private state. Host input is staged in the worker state before it is copied to a guest receive buffer, so input accepted from the backend survives worker restarts and snapshots; the receive loop also reads the backend before the guest posts a buffer, completing zero-length buffers as it goes. The saved state, schema version 1, records the console size, the bytes already written from the current transmit descriptor, the staged input, and the disconnect policy. Restore validates the encoded size, the schema version, the policy, and the bounds of every field before decoding further, and requires a restored transmit offset to lie within the front descriptor of the enabled transmit queue, both before the queues restart and when the transmit queue starts. Reset clears the private state. The console also implements `quiesce_input` and `resume_input`: while input is gated, the worker reads no host input, and output continues. `SavedStateBlob::encoded_len` returns the encoded payload length without decoding it, which the console uses to bound its private state. Document that virtio-console now supports save and restore through the transport's device-private payload. A console with the `Retain` disconnect policy keeps the guest's transmit descriptors while its backend is disconnected and forwards them after the backend reconnects. The case that matters is a peer that leaves before the worker notices: the worker then writes the guest's next output to it, and keeps that output only because the write fails, as the socket backend's does. A mock control makes the peer leave without waking the worker, so that its next write fails with `BrokenPipe` and disconnects the backend, and two tests of the `Retain` policy check that: - output that the guest writes after its peer left stays queued while the worker waits for the next peer, which then receives it (`retain_keeps_output_whose_write_fails_for_the_next_peer`); - when the peer leaves after taking part of a descriptor, the next peer receives the rest of it, and not what the departed peer already received (`retain_sends_the_rest_of_a_partial_write_to_the_next_peer`). With the `Discard` policy, both fail at their check that the descriptor is still queued, and the first fails there too when the departed peer's write reports success, as upstream's socket backend reports a broken pipe. It also includes these changes: - virtio_console: test that Retain keeps output for the next peer Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
To save a network device, its frontend must stop exchanging packets with the endpoint at a well-defined point: every receive and transmit already accepted by the endpoint must complete, and the endpoint must not keep any guest buffer. The `Queue` trait had no operation for this, so a frontend could only drop its queues, losing work in flight. Add `Queue::quiesce`, which finishes the work that a queue already accepted without accepting new work. On success, all resulting receive and transmit completions are available through `rx_poll` and `tx_poll`, the queue retains no buffer IDs or guest-memory segments, and the returned `QueueQuiesceResult` reports how many completions of each kind remain to be polled. `Queue::resume` resumes host receive admission after a rolled-back quiesce. Queues do not support quiesce by default, and resume succeeds by default. `BufferAccess` now requires `Send`, so a quiesce future can hold the frontend's buffer pool across await points; every existing implementation already is. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A TAP queue wrote guest packets synchronously and completed every packet, including the ones that the TAP interface dropped because its buffer was full or the interface was down. A packet dropped under backpressure was therefore lost, and the queue could not implement the endpoint quiesce contract, because it could not report which transmits were still in flight. Each `tx_avail` call now writes the packets that it is given until the TAP interface applies backpressure, because the frontends submit the rest of a batch only when `poll_ready` reports that the queue can accept it. The queue keeps the packet that meets backpressure, and with it the ownership of its descriptors, until `poll_ready` writes it through the new `AsyncWrite` implementation of `PolledTap`; the call then completes asynchronously, and `tx_poll` returns the packets that it wrote before that packet and then that packet. A partial write or a write error is reported as a fatal transmit error instead of being ignored. Implement `Queue::quiesce` for TAP queues: a quiesced queue drops its free receive buffers, stops reading from the interface and admitting receive buffers, finishes writing its pending packet, and reports the receive and transmit completions that it still holds. `Queue::resume` resumes receiving after a rolled-back quiesce. The TAP backpressure test now checks that every packet is completed exactly once instead of being dropped. It also includes these changes: - net_tap: write every packet of a transmit batch - net: enable no receive offloads in the chain's endpoint tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A Consomme queue could not stop exchanging packets at a well-defined point for a save, because it always polled the Consomme stack and its control channels and kept accepting receive buffers. Implement `Queue::quiesce` for Consomme queues. A quiesced queue processes the frames that the guest already queued for transmit, drops its posted receive buffers, ignores new ones, and stops polling the Consomme stack and the control channels; `poll_ready` then only reports the completions still pending. Quiesce reports the receive and transmit completions that the queue still holds, and `Queue::resume` restores normal polling after a rolled-back quiesce. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
Consomme derives the guest and gateway addresses from a CIDR, so a host cannot give a guest an exact, preconfigured IPv4 identity: a specific guest address and prefix, and a gateway whose address and MAC address stay fixed across snapshots and restores. Consomme also always treats traffic to the gateway as ordinary egress, with no way to map it onto host loopback, which a guest with a static identity uses to reach host services. Add `ConsommeParams::set_static_ipv4`, which sets an exact guest address, prefix length, gateway address, and gateway MAC address. The identity must be consistent: a prefix of 1 to 30 bits, a guest address that is neither the network nor the broadcast address, and a gateway at the first host address of the network, distinct from the guest. A static identity stops advertising a routable IPv6 prefix and enables the new `map_gateway_to_host_loopback` parameter, which translates guest TCP and UDP flows addressed to the gateway onto host loopback. With the mapping disabled, such flows are dropped, except TCP to the single port named by the new `gateway_loopback_proxy_port` parameter. Expose the identity to resources: `ConsommeHandle` gains an optional `StaticIpv4Config`, which is mutually exclusive with `cidr`, and optional overrides of host-local access, the gateway mapping, and the gateway proxy port. The resolver rejects a handle with both a CIDR and a static identity, or with an inconsistent identity. Existing handles set none of the new fields. This changes one behavior of every Consomme endpoint: a guest TCP or UDP flow addressed to the gateway, other than the gateway's own DNS and DHCP services, used to be sent to the gateway's address on the host network, where no gateway exists. Such flows are now dropped unless the gateway mapping or the proxy port allows them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
virtio-net claimed to support save and restore unconditionally, but the transport saved only queue progress: packets that the endpoint had accepted, or that the device held for the guest, were lost, and nothing tied a restored guest's view of its network identity to the endpoint. Saving a VM with virtio-net therefore silently corrupted the guest's network state. Save and restore are now opt-in. `NicBuilder::save_restore` binds a device to an exact static IPv4 identity and a fixed effective feature contract, and `VirtioNetHandle` gains `save_restore`, `static_ipv4`, and `effective_features` fields for resources; a resource that opts in must supply both. Other devices report that they do not support save and restore. When the transport stops an active queue pair of such a device, the device quiesces the endpoint queues, drains every completion the endpoint still holds so that no descriptor stays in flight, and captures the progress of both queues, which the transport then collects queue by queue. The device-private saved state, schema version 1, records the identity, the guest MAC address, the link state, the effective and negotiated feature banks, the endpoint offloads, the queue-pair lifecycle, and an endpoint generation that each restore advances. Restore validates it against the destination device: the identity must match and use MAC addresses derived from its IPv4 addresses, the effective features and endpoint offloads must be unchanged, the negotiated features must stay within the contract and match the saved banks, and the device must have a single queue pair whose restored queues own no descriptors. Restarting a stopped pair requires the same negotiated features. The device also implements `quiesce_input` and `resume_input`: while input is gated, the endpoint queues finish the work they already accepted but admit no new host receive traffic, and the gate is re-established whenever the queues restart. Workers are now started only by the coordinator, after it establishes the gate. Petri records the PCIe ports of virtio-net devices added without a save and restore contract and skips its startup save/restore test while any of them is configured. Every existing virtio-net resource leaves `save_restore` unset. Document that virtio-net supports save and restore only with a static identity and feature contract. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A FUSE session negotiates its protocol version, feature flags, and limits with the guest during FUSE_INIT and keeps them only in memory. A file system device built on it could not be saved and restored, because a restored session would be uninitialized while the guest driver continued with its negotiated protocol. Add `SessionState`, a serializable snapshot of whether a session is initialized and of its negotiated values, which contains no file handle, task, or native resource. `Session::save_state` captures it, and `Session::restore_state` restores it after validating it: an uninitialized state must carry no negotiated values, and an initialized one must use a supported protocol version (7.27 through the implementation's minor version), accept only feature flags that the guest advertised, keep its extended-init flags consistent, and have nonzero write and time-granularity limits. The types are re-exported from the crate root. Two related fixes make the saved values well-defined. FUSE_INIT now records the negotiated minor version as the lower of the guest's and the implementation's, instead of the guest's value, so a session never claims a protocol newer than it implements. Destroying a session also resets its negotiated information, so a later session starts from defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A virtio-fs inode records a single host path, which it uses for every later operation on the node. That path went stale in several cases. A hard link gave the inode a second name that nothing recorded, so unlinking the original name left the inode pointing at a missing path even though it was still reachable through the link. Renaming a directory left every open inode below it pointing at the old location. The dedup map, which lets repeated lookups of a host file return one FUSE node, only evicted the renamed or removed path, without updating the inodes that referred to it. Each inode now keeps the set of host paths through which the guest knows it. Lookups and hard links add a path. After an unlink, the paths at or below the removed name are dropped from every inode of the volume, and an inode whose primary path was removed switches to one of its remaining aliases. After a rename, the paths at or below the old name are rewritten under the new name, after first dropping the aliases of any overwritten destination. The dedup map is then rebuilt from the surviving inodes, and a path-keyed inode with no remaining alias is left out of it, so a later create at a vacated path still gets a fresh node. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The x86 ACPI builder describes legacy ISA IRQs with their bus defaults, edge-triggered and active-high, and a VM whose chipset devices signal a legacy IRQ by level had no way to say so in the MADT. The MP tables that the loader now builds already take such a list, so the two descriptions of a machine could not agree. Add `level_triggered_irqs` to the x86 `AcpiArchConfig`. The MADT gains an active-high, level-triggered interrupt source override for each listed IRQ. Every existing configuration, in OpenVMM, OpenHCL, and igvmfilegen, passes an empty list, so the generated tables are unchanged. A test checks that the MADT and the MP configuration table enumerate the same APIC IDs for 1, 2, 4, and 8 processors, and another checks the override for a level-triggered IRQ. It also includes these changes: - vmm_core: build the ACPI builder tests with level-triggered IRQs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
OpenVMM builds one kind of guest-visible machine. A second, minimal machine needs a way to be selected in the VM configuration that is independent of the hypervisor backend, and a place for its own settings. Add the `microvm` module to `openvmm_defs` with `MachineProfile`, which selects the standard machine or the microVM machine, and a `MicrovmConfig` for settings specific to the microVM machine, initially empty. `Config` gains `machine_profile` and `microvm` fields. The worker keeps the profile in its manifest and in the loaded VM, and carries it into the manifest of a restarted worker. Every existing configuration selects the standard machine, and nothing acts on the profile yet. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The x86 processor topology of a VM is derived from the host's topology, so the same configuration can present different APIC IDs and cache topologies on different hosts. A microVM whose snapshots must restore on any host needs a topology that depends only on its configuration. The worker now builds the x86 topology through the new `dispatch::microvm::x86_topology_builder`: a microVM starts from the fixed default topology instead of the host's, and other VMs are unchanged. The partition-wide topology CPUID leaves also report the bootstrap processor's APIC identity, with the fields included in each leaf's mask: the initial APIC ID in leaf 01h, the x2APIC ID in leaves 0Bh and 1Fh, and the extended APIC, compute-unit, and node IDs in leaf 8000001Eh. Backends replace these values for each processor, so the reported identity of every processor is unchanged, but the partition-wide leaves now describe a real processor. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The CMOS RTC has a single behavior, fixed at construction, and the base chipset builder offers no way to choose another. A machine profile that needs a different guest-visible RTC behavior has nowhere to select it. Add `RtcMode` to the RTC and `GenericCmosRtcMode` to the generic CMOS RTC dependencies of the base chipset, each with a single `Standard` mode, the programmable MC146818-compatible behavior. `Rtc::new_with_mode` and `Rtc::new_with_mode_and_utc_clock`, which also takes a fallible UTC clock source, construct an RTC in a given mode, and a reset returns the RTC to its mode's initial register state. Every mode currently shares the standard state, and OpenVMM and OpenHCL select the standard mode, so the guest-visible behavior is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
Every base chipset type includes firmware-facing or bus infrastructure that a minimal machine does not want, such as VMBus, PCI, the PIIX4 devices, or ACPI power management. A microVM needs a base chipset that contains only the architectural devices its guest requires. Add `BaseChipsetType::Microvm`, which is supported only on x86-64. Its chipset contains exactly a generic CMOS RTC, a generic I/O APIC, the PIC, and the PIT, without VMBus, PCI, ISA DMA, or ACPI power management. Its layout reserves 1 GiB of chipset MMIO below 4 GiB and none above, so guest RAM ends at 3 GiB and resumes at 4 GiB; a memory-layout test checks that placement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A guest can ask to be powered off, but not with a result: a guest that runs a single workload and then powers off has no way to hand its exit status to the host process that launched it. Add `PowerRequest::PowerOffWithStatus` and `HaltReason::PowerOffWithStatus`, which carry an 8-bit guest-provided status, and forward the request as that halt reason. Every consumer treats the new reason like a power-off: the debugger stop report, the OpenVMM VM controller, OpenHCL, and Petri. No device issues the request yet. A test checks that the status survives a mesh round trip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A microVM has no UART, no VMBus, and no firmware console, so it needs a debug console that costs the guest a single port access per byte and the host no emulated serial state machine. Add `MicrovmPortb`, a raw bidirectional byte channel at I/O ports 0xe9 (data) and 0xea (status), backed by any `SerialIo` endpoint: - A guest write to the data port queues every byte of the access; the device drains the queue to the endpoint from `poll_device`. Output is binary-transparent, so a guest can stream arbitrary bytes. - A guest read from the data port pops one received byte (0 when the queue is empty) and zero-fills the rest of the access. Bit 0 of the status port reports that input is available. - Each direction buffers at most 1 MiB. Output beyond that drops the newest bytes with a rate-limited warning, and input stops being read from the endpoint until the guest consumes some of it. - An endpoint write error other than a broken pipe drops the buffered output and re-arms the output wakeup, so the next guest write still reaches a recovered endpoint. - `stop` flushes whatever the endpoint accepts immediately. Bytes still owned by the device are saved in `MicrovmPortbSavedState` and resent to the reconstructed endpoint after restore; restore rejects buffers larger than the device limit. Add `MicrovmPortbHandle` and `MicrovmPortbResolver`, and register the resolver for x86_64 guests. Nothing instantiates the device yet. Tests cover binary output and zero-filled reads, polling a newly accepted connection immediately, and preserving the output wakeup across an endpoint error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A microVM guest that runs one workload needs to end the VM and report the workload's result to the host with a single port access, without ACPI, a PM1 block, or a firmware power-off path. Add `MicrovmShutdown`, a write-only port at 0x604. A write of any size requests `PowerRequest::PowerOffWithStatus` with the first byte of the access as the status (0 for an empty access). Reads return all ones. The device is stateless, so it saves `NoSavedState`. Add `MicrovmShutdownHandle` and `MicrovmShutdownResolver`, which obtains the power request client from the platform, and register the resolver for x86_64 guests. Nothing instantiates the device yet. A test checks that the first byte of a wide or single-byte write becomes the power-off status. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The microVM profile has no firmware and no ACPI. Linux instead discovers the processors and interrupt routing from Intel MP 1.4 tables, which the loader can already build. The worker needs a boot mode that selects that loader, and a machine contract to check before it builds anything. Add `LinuxDirectBootMode::MpTable` (x86-64 only). The worker loads the kernel through `vm_loaders::microvm::load_linux_x86_mptable`, which builds MP tables from the VP APIC IDs and the profile's level-triggered IRQ list (currently empty). MP-table boot rejects isolation. An embedded NUL in a Linux command line is now a load error instead of a panic. Define the profile's command-line ABI in `openvmm_defs::microvm`: - The profile owns the leading tokens `earlycon=xe9 console=hvc0 reboot=t panic=-1`, which use the patched guest kernel's port-0xe9 console. The command line is limited to 64 KiB. - `build_microvm_command_line` appends user arguments and rejects any that override `earlycon=`, `console=`, or `nr_cpus=`. - `append_microvm_processor_limit` appends the host-owned `nr_cpus=` for a supported processor count (1, 2, 4, or 8). Add `validate_machine_config`, which checks the fixed machine contract: - Boot and backend: MP-table Linux boot without isolation, emulated serial, or SMBIOS overrides, and a KVM, MSHV, or WHP backend. - Processors and memory: 1, 2, 4, or 8 vCPUs in one socket without SMT or x2APIC, and one NUMA node with RAM. - Features: no Hyper-V enlightenments, VTL2, isolation, or nested virtualization. - Devices: exactly the microVM base chipset with the IOAPIC, PIC, PIT, portb, and shutdown devices. No floppy, IDE, PCI, PCIe, VPCI, ISA DMA, VMBus, graphics, VMGS, firmware or debugger channel, RTC offset, or virtio device. - Layout: the fixed 1-GiB low MMIO layout. For the standard profile, the validator only rejects MP-table boot. The worker validates every configuration before it constructs the VM. On a microVM cold boot (not a restore), the worker adds `lapic_timer_hz=` with the backend-reported LAPIC timer frequency, so the patched guest kernel skips timer calibration. A user-supplied value is canonicalized when it matches the backend and rejected when it does not. Parsing follows the kernel's rules: hyphens and underscores are interchangeable, quoting is honored, values may be decimal, octal, or hexadecimal, and tokens after `--` belong to init. The token is inserted before `--` and before any virtio-mmio discovery tokens. `HvlitePartition` gains a `MicrovmPartition` supertrait that exposes the backend LAPIC frequency. The microVM chipset does not trace unknown port I/O: Linux probes many legacy ports that the profile deliberately omits. Nothing configures a microVM yet. Tests cover the boot-mode matrix, the host-owned processor limit, and LAPIC frequency propagation. It also includes these changes: - openvmm_core: reject MP-table boot for aarch64 guests - openvmm_core: gate the microVM's x86_64-only items for aarch64 guests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
…t for microVMs The microVM profile's persistent state is defined by its own snapshot contract, which later commits establish. Until then, the generic host-driven paths would produce state that nothing can restore: a saved microVM cannot be restored, and a worker restart would rebuild the VM from a manifest that does not describe it. Filter management RPCs in the worker loop before dispatch. For a microVM: - `VmRpc::Save` fails with "save is unavailable for microVM". - `VmRpc::PulseSaveRestore` fails with the new `PulseSaveRestoreError::UnsupportedMachineProfile`. - `WorkerRpc::Restart` fails with "worker restart is unavailable for microVM". The filters complete a rejected RPC in place and return every other RPC unchanged, so the standard machine is unaffected. Petri's `verify_save_restore` treats the new error like a missing reset: it logs a warning and skips the save/restore check. It also includes these changes: - openvmm_core: gate the microVM's x86_64-only items for aarch64 guests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The worker can boot the microVM profile, but nothing selects it. Add
`--machine <standard|microvm>`, which defaults to `standard`.
With `--machine microvm`, the entry point does the following:
- Validates the options before it opens any host resource:
- The guest is x86-64 with 1, 2, 4, or 8 processors and no custom
NUMA, socket, SMT, APIC-offset, or x2APIC settings.
- The guest boots Linux directly, without firmware options.
- Hyper-V, VTL2, isolation, nested virtualization, GET, VMBus, and
the debugger are disabled.
- A named backend is KVM or MSHV on Linux, or WHP on Windows.
- There are no UART, debugcon, storage, network, virtio, legacy,
PCIe, IOMMU, or assigned devices.
- Snapshot restore is rejected.
- Builds the microVM base chipset and fixed layout, and adds the portb
console and shutdown devices. Portb owns the interactive console
(`hvc0` in the guest), so COM1 defaults to none.
- Loads the kernel and the optional initrd in MP-table boot mode with
the profile-owned command line: the base tokens, then the user
arguments, then the host-owned `nr_cpus=`. Hyper-V enlightenments,
VMGS, and SMBIOS overrides are disabled.
- Fixes the topology to one socket with one core per vCPU, no SMT, and
xAPIC with contiguous APIC IDs.
- Auto-selects the backend with `choose_microvm_hypervisor`, which uses
the usual probe order on Linux and Windows and fails on other hosts.
Every configuration, for either profile, is then checked against the
machine contract, including the named hypervisor.
For now, a status-bearing power-off through port 0x604 follows
`--guest-shutdown-action` like any power-off. A later change propagates
the status to the process exit code.
Document `--machine` in the CLI reference and the MP-table boot layout
in the Linux direct boot reference. Tests cover option parsing,
processor and topology validation, command-line ownership, and the
rejection of unsupported combinations.
It also includes these changes:
- openvmm_entry: gate the microVM checks of x86_64-only options
- microVM: fix the macOS clippy build
- openvmm_entry: run the microVM option tests only for x86_64 guests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
… test Nothing exercises a microVM end to end yet. Teach Petri to build one and add a cold-boot lifecycle test. `PetriVmBuilder::with_microvm_machine(n)` selects the microVM profile with a deterministic topology: n vCPUs in one socket, no SMT, and xAPIC. The builder also sets: - Minimal mode, with no VMBus and no Hyper-V enlightenments. - No storage controllers, NVMe or virtio-blk drives, or agent images. - Linux direct boot, with pipette not running as init. `PetriVmConfig` carries the machine profile, and the Hyper-V backend rejects the microVM profile. For a microVM, the OpenVMM backend does the following: - Builds the microVM chipset. - Switches the Linux load mode to MP-table boot with the profile-owned command line: the base tokens, the test's init arguments, and `nr_cpus=`. - Drops VMGS, the firmware event channel, VMBus, and Hyper-V enlightenments. - Attaches the portb and shutdown devices, and validates the configuration against the machine contract. - Chooses the backend with `choose_microvm_hypervisor`. Portb replaces the emulated serial ports. Its output is logged and also forwarded raw to the test through the new `log_task_with_output`, and the test can write raw input. New test APIs: - `wait_for_microvm_portb_output`, `wait_for_microvm_portb_bytes`, and `write_microvm_portb_input` for the portb console. - `pulse_save_restore`, which performs one pulse save/restore. - `with_linux_command_line`, which edits the effective Linux command line. - `prepare_initrd_with_file`, which injects one file into the Linux direct initrd. `prepare_initrd` now uses it. - `linux_direct_boot_files`, which returns the kernel and initrd paths. A builder-supplied initrd now replaces the artifact initrd for any Linux direct VM, not only with pipette as init, and Linux direct firmware may omit the initrd. `x86_64::microvm::phase_1_lifecycle` boots a 1-vCPU microVM whose init script drives portb through `/dev/port`. The test: 1. Waits for the boot marker. 2. Checks that host save and pulse save/restore are rejected. 3. Round-trips a ping and a binary-transparent echo (NUL, CR, LF, DEL, and 0xff) through portb. 4. Requests a status-37 shutdown through port 0x604, and checks that the resulting power-off halt carries the status. It also includes these changes: - petri: adapt upstream's Incubator changes to the chain's petri Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A machine profile with a fixed guest ABI must control which virtio features a device offers. The microVM profile, for example, supports only split rings. The MMIO transport currently offers every feature the device reports. Add `VirtioDeviceFeatures::without_bits`, which clears the features selected by a 64-bit mask, and `VirtioMmioDevice::new_with_disabled_features`, which masks the guest-visible device features before the guest reads them. `VirtioMmioDevice::new` delegates to it with an empty mask, so existing devices are unchanged. Saved-state validation already requires restored driver features to be a subset of the transport's device features, so a saved state that negotiated a masked feature fails to restore. A test checks that masking the packed-ring bit leaves the other features in place. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The standard machine assigns virtio-mmio windows from a dynamic region of the memory layout in device order, so a device's address and IRQ depend on which other devices are present. A microVM guest instead discovers its devices from a fixed map that does not change with the device set, which keeps the guest ABI stable. Define the microVM virtio-mmio map in `openvmm_defs::microvm`: eight 0x1000-byte windows from 0xd0000000 in stable device order (network, filesystem, boot console, four sandbox blocks, control console), their IRQs, and a check that the windows are ordered, disjoint, and inside the 3-to-4-GiB aperture. IRQ 8 stays with the RTC, and no slot IRQ is level triggered. Wire the first slot, the boot virtio-console: - `--virtio-console` is now accepted with `--machine microvm` using `listen=PATH`, `listen=tcp:IP:PORT`, `console`, or `none`. A PCIe port is rejected. The console appears in the guest as `hvc1` at 0xd0002000, IRQ 7, and the profile's base command line switches to `console=hvc1`. - `append_microvm_virtio_discovery` appends a host-owned `virtio_mmio.device=` token for each present slot, in slot order. `build_microvm_command_line` now rejects user `virtio_mmio.device=` tokens. The machine contract requires exactly one discovery token per device, in fixed-address order at the end of the command line. - The machine contract admits at most eight virtio-mmio devices, and at most one of them may be a virtio-console. - The worker places microVM virtio-mmio devices at their fixed slots through `VirtioMmioSlots` instead of the layout's dynamic region. The layout reserves no dynamic windows for a microVM. The fixed slots expose split rings only: the packed-ring feature is masked. - When the console uses the terminal, portb output moves to stderr. A listener retains guest output until a client connects, and `none` discards it. Tests cover the console command line, discovery tokens, reserved tokens, the console option matrix, and the IRQ assignments. It also includes these changes: - openvmm_core: gate the microVM's x86_64-only items for aarch64 guests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A virtio-mmio interrupt costs the guest two trapping register accesses:
it reads INTERRUPT_STATUS to learn why the line fired, then writes
INTERRUPT_ACK to clear it. Both accesses exit to the VMM.
Add two ways for the MMIO transport to avoid these exits.
`VirtioMmioInterruptMode::SharedStatus { status_gpa }` keeps the
interrupt status in a naturally aligned 32-bit word of guest memory
instead of the INTERRUPT_STATUS register. The device sets status bits
with an atomic compare-exchange, and it pulses the interrupt line only
when the word goes from zero to nonzero. A guest that supports the mode
reads and clears the word without exiting. INTERRUPT_STATUS reads return
the word. The transport rejects a misaligned or inaccessible word and
zeroes it when the transport is created, reset, or dropped.
In the default `Legacy` mode on Linux, the transport registers an
accelerated doorbell on INTERRUPT_ACK for the used-buffer bit. A
used-buffer acknowledgement then no longer exits to user mode. A
used-buffer generation count keeps a deferred acknowledgement from
clearing a completion that the guest has not read yet.
`VirtioDevice::supports_accelerated_doorbells` (true by default) lets a
device opt out of accelerated queue-notify and INTERRUPT_ACK doorbells.
Its notifications are then emulated by the transport.
`VirtioMmioDevice::new_with_disabled_features_and_interrupt_mode`
selects the mode. `new` and `new_with_disabled_features` keep the legacy
mode, so existing devices are unchanged. The MMIO constructors now
require a driver that can wait on the doorbell event on Linux.
Saving reads the effective status from the register or the shared word.
Restore resets the interrupt state, then applies the saved status to
the same place.
Tests cover the following:
- Used-buffer acknowledgement through the doorbell, with and without
registered doorbells.
- An acknowledgement that races a new completion.
- Shared-status save, restore, and reset.
- Bit coalescing and zero-transition pulses, and a completion that
races the guest's exchange.
- Teardown, and the doorbell opt-out.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The fixed microVM virtio-mmio slots still deliver interrupts through the legacy INTERRUPT_STATUS and INTERRUPT_ACK registers, so every virtio interrupt costs the guest two exits. Give every fixed slot a word on a shared interrupt-status page and deliver its interrupts through `VirtioMmioInterruptMode::SharedStatus`: - The page is at GPA 0x30000 and holds one naturally aligned 32-bit word per slot, in slot order from offset 0x00 (network) to 0x1c (control console). `microvm_virtio_status_gpa` maps a slot to its word. The reservation check now also requires every slot to have a distinct, ordered word inside the page. `MICROVM_VIRTIO_SANDBOX_BLOCK_MMIO_BASES` names the four sandbox-block slots. - The worker creates each fixed-slot device in shared-status mode, and the MP-table loader reserves the page in the guest's e820 map so the guest never allocates it as RAM. The guest must use the patched kernel's shared-status virtio-mmio support. In this mode an INTERRUPT_ACK write does not clear the word, so a stock driver would never see the zero-to-nonzero transition that raises the next interrupt. A test pins the word of every slot and rejects an unknown slot. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A guest can request a snapshot by writing to an I/O port. For the snapshot to be exact, every VP must be stopped after that write has been accepted but before the issuing VP resumes guest execution: capturing earlier loses the request, and letting the VP run past the write lets the guest change state that the snapshot claims to contain. Add `PartitionUnit::temporarily_stop_vps_at_io_boundary`. The caller passes a sender that releases the deferred I/O completion and a receiver that reports that the completion has run. The partition unit: 1. Queues a stop event for every running VP, and marks the set stopped. 2. Releases the deferred I/O. Every VP has already observed its stop event, so the VP that issued the I/O cannot re-enter the guest. 3. Waits for the I/O completion, then for every VP to stop, even when the completion channel closed or a VP runner was dropped. The returned `StopGuard` holds the VPs stopped. Failure is terminal: the stop reference is kept, so a failed boundary cannot be restarted before the partition is torn down. `VpSet::stop` now marks the set stopped as soon as it has queued the stop events. Dropping a stop mid-wait therefore no longer leaves the set marked running, and a later stop does not queue duplicate stop events. The partition unit's snapshot request is no longer x86-64 only; the TSC advance remains x86-64 only. Tests cover a successful and a failed boundary, the ordering of the I/O completion and the VP stops, a dropped VP runner, and a boundary on already-stopped VPs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A microVM guest requests a snapshot of itself by writing to port 0x605. The snapshot must reflect the instant just after that write, so the write cannot complete until the VMM has stopped every vCPU; the issuing vCPU must not execute another guest instruction first. Add `MicrovmSnapshotRequest`, a write-only port at 0x605. When it has a notification target, a write: - Defers the port write and sends a `MicrovmSnapshotBoundaryRequest` to the target. The request carries a sender that releases the write, a receiver that reports that the write has completed, and an RPC that the target completes once the snapshot transaction's outcome is final. - Completes the deferred write when the target releases it, and signals that completion. - Coalesces duplicate writes while a transaction is pending, with a rate-limited warning. A completed transaction is reaped before the next write, so the guest can request another snapshot even before the device's poll routine runs. Without a notification target, writes complete immediately and are ignored. Reads return all ones. Stop and reset fail a deferred write that has not been released, and keep a released transaction pending until it completes. The device has no saved state. Add `MicrovmSnapshotRequestHandle`, its resolver, and the resolver's registration for x86_64 guests. Nothing instantiates the device yet. Tests cover coalescing until acknowledgement, reuse after a completed or closed transaction, an early completion, the poll waker across duplicate requests, stop and reset with and without a released write, and the nonblocking behavior without a target. Co-authored-by: Ruize Tang <1466040111@qq.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
In debug builds, the ttrpc VM service overflowed the stack of its worker thread on Linux as it created a microVM, so test_ttrpc_microvm_linux_direct_lifecycle_and_snapshot and test_ttrpc_microvm_device_snapshots failed in the x64 Linux vmm-tests of the OpenVMM PR workflow, which builds debug, at feature/microvm/dev 73cf403 as with microsoft#5077. microvm_options, which builds the options of a CreateVM request, parsed them with clap through on_big_stack, which moves the parse to a thread with a 4 MiB stack only on Windows, since other hosts parse the command line on the main thread, whose stack is large. microvm_options runs on the ttrpc worker, whose stack is 2 MiB, below the request's async call chain, and in debug builds clap's construction of the Command for Options needs more stack than is left there (clap-rs/clap#5134). Release builds, which the push CI runs, were not affected. microvm_options now always parses on a thread with a 4 MiB stack; parse_options, which runs on the main thread, keeps the Windows-only workaround. Tests: with cargo xflowey vmm-tests-run, a debug build, on a Linux KVM host, both tests fail with "thread 'worker-TtrpcWorker' has overflowed its stack" at 73cf403 and at 67ac3e2, and pass with this change. With cargo nextest on Windows x64, the 171 cli_args and ttrpc tests of openvmm_entry pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the profile-owned ARM64 microVM boot, fixed device and platform-control integration using the existing loaders, device tree and shared device abstractions. Add governed ARM CPU/time admission with native WHP programming and readback, retaining the legacy fingerprint API and x86 behavior. Keep the qualified PA36 contract and cold-boot-only boundary; ARM snapshot capture and restore remain unavailable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extend the existing --x-time-abi-verify lifecycle to ARM64 using one shared TimeAbiReport and the existing native admission checks. Report the selected profile, effective register checks and actual WHP address, feature-bank and counter-frequency readbacks without executing the guest. Keep x86 output compatible, reject unsupported ARM fault-injection hooks, and fail closed on missing evidence. This does not establish UTC synchronization or clock authority. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the shared CPU/time report and sample-field conversion within the architecture-selected adapter. Restore native type and clock-status import ownership so target-branch clock changes merge without conflict, retaining one shared report and unchanged field conversion and runtime behavior. No target commits or time-policy changes are incorporated. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A restore of an RTC saved state whose clock_time_millis lies near either end of i64, or a snapshot downtime that advances the RTC's time there, overflowed the RTC's clock source. SystemTimeClock, the source that OpenVMM gives the RTC, keeps the guest-visible time as an i64 millisecond offset from the host's SystemTime: set_time computes new - now, and get_time now + offset. For such a time, the RTC's start or its next sample of the clock overflows: debug builds abort, and release builds, which do not check overflow, wrap the guest's time. Fields::validate checked the saved state's flags and timers but not its time, and advance_clock checked only that the advanced time fits in an i64. The RTC now holds the time that a restore or an advance sets to CLOCK_TIME_MILLIS: the times that the guest can write to the calendar registers, from 0000-01-01T00:00:00Z to 9999-12-30T22:00:00Z, the last whole second that jiff represents in UTC, widened on both sides by KTIME_SEC_MAX seconds, the range of the Linux host clock, over which the guest's time runs with the host's. Fields::validate rejects a saved time outside it with RestoreError::InvalidSavedState before the restore changes any state, and advance_clock fails without changing the time when the advanced time falls outside it. For a time in the range and a host time in the Linux host clock's range, the offset and the sum stay well within i64. The Guide's timekeeping reference describes the bound. Tests: four tests in cmos_rtc::tests::snapshot restore and advance the RTC, held and running, on the mock clock and on the host's clock, to times just outside the range, near both ends of i64, and at both ends of the range. At feature/microvm/dev fb2c8b7, all four fail, since the restore and the advance accept every such time; there, on the host's clock, a restore of i64::MIN panics with "attempt to subtract with overflow" at support/local_clock/src/lib.rs:236 as the RTC starts, and an advance that leaves the time a second short of i64::MAX panics with "attempt to add with overflow" at :255 as the RTC saves a second later. With this change, the 131 chipset tests pass with cargo nextest on Windows x64. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A guest read of the IOAPIC's index or data register wider than 4 bytes, such as a 64-bit mov from 0xfec00000, panicked the VP thread with "copy_from_slice: source slice length (4) does not match destination slice length (8)", which ended OpenVMM and the VM. IoApicDevice::mmio_read means to accept a read of any size, but it copied the register's min(len, 4) low bytes into the whole buffer, so only reads of at most 4 bytes matched the slice lengths. mmio_read now copies those low bytes into the start of the buffer and zeroes the rest, so a wider read returns the 32-bit register zero-extended, as the RTC already does for a wide read of its ports, and a narrower one still returns the register's low bytes. Tests: the new test ioapic::tests:: mmio_reads_of_any_size_are_zero_extended reads the index and data registers, for the ID, the version, and both halves of a redirection entry, with every length from 1 to 8 bytes into a buffer of stale bytes. At feature/microvm/dev 09cd771, it fails with the panic above at vm/devices/chipset/src/ioapic/mod.rs:724:14. NVX f02bdcd53 pins 09cd771; there, an Alpine guest on 1 processor that maps /dev/mem at 0xfec00000 and reads both registers with 32-bit and then 64-bit loads panics OpenVMM at the same line on its first 64-bit read, on KVM, MSHV, and WHP. With this change, the same guest reads the registers' 32-bit values zero-extended on all three, on 1 and 2 processors, and the 132 chipset tests pass with cargo nextest on Windows x64 and Linux x64. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
IoApicDevice::restore accepted any u8 as the saved IOAPIC id, but the ID register's id field has 4 bits. A saved id above 15 restored without error, and the next read of the ID register panicked with "value out of bounds [0, 15]": read_register builds the register with IoApicId::with_id, whose setter asserts that the id fits its field, in release builds too. The device never saves such an id, since a write to the ID register keeps only bits 27:24, so only a damaged or crafted saved state reaches it. The restore appeared to succeed, and the failure surfaced later, at the guest's first read of the register. restore now rejects a saved id above IoApicId::MAX_ID, 15, with RestoreError::InvalidSavedState before it changes any state or any route, as it does for a wrong number of redirection entries or line levels. Tests: restore_rejects_an_id_wider_than_the_id_register restores the ids 0x10, 0x80, and 0xff over a device whose id is 5, and checks that each restore fails with InvalidSavedState and leaves the saved state, the routes, and the ID register's value unchanged. restore_accepts_every_id_that_the_guest_can_write writes each id from 0 to 15 to the ID register through MMIO, with the register's reserved bits set, and checks that the saved state restores and that the register reads back the id. At feature/microvm/dev e2458a6, the first test fails, since the restore accepts every id, and restoring the id 16 and then reading the ID register panics with "value out of bounds [0, 15]" at vm/devices/chipset/src/ioapic/mod.rs:99:5. With this change, the 134 chipset tests pass with cargo nextest on Windows x64. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…licy A snapshot of a host profile (--cpu-profile host, or auto's fallback) carries the only copy of the profile's document. host_profile_for_restore accepted any document that decodes to a valid host profile with the ID and digest that the snapshot recorded, and whoever crafts a snapshot can record the digest of an edited document and the effective CPUID that it gives. Nothing compared the document with the derivation's policy, which clears the features that a VM must not see, so a document with CPUID.1:ECX[5], VMX, or CPUID.0x80000001:ECX[2], SVM, restored: the effective CPUID, the backend's support check, and VP 0's CPUID all follow the document. On KVM, where a time ABI partition's CPUID is the profile's effective CPUID, the VMX or SVM strip that new_partition adds for !nested_virt does not reach it, so the guest could enable VMX or SVM and reach KVM's nested-virtualization code, host attack surface that nested_virt false keeps closed. host_profile_for_restore now also refuses a host profile that the policy would change (E_PROFILE_DIGEST), naming at most eight of the differences between the document and the policy. Both the controller's restore preflight and the VM worker's profile selection call it, so a crafted snapshot fails before the worker starts, and again if a worker is ever handed one. The check is derive::policy_violations. It applies apply_policy and clear_stateless_features, the derivation's own rules, to the profile's CPUID table, with the profile's XCR0, and compares the result with the table; and it checks the XSAVE features against ALLOWED_XCR0, ALLOWED_XSS, and ALLOWED_XSAVE_EXTENSIONS. A profile that the derivation produced is a fixed point of its policy, which only sets and clears bits and zeroes leaves, so the check accepts it, and a rule that the derivation gains is checked without a second copy of it. What the derivation fixes besides features, the XSAVE layout, the pinned MSRs, and the physical address width, stays with the support check of the destination backend, which bounds a profile by what the backend presents. The derivation itself is unchanged, and so are the pinned profiles and POLICY, the label that provenance records, since no profile or rule changes; pinned profiles are constants that OpenVMM verifies, and need no check. No version is created or bumped, and restore does not read the policy label that a profile's provenance names: it applies the rules of its own build. A host profile that an OpenVMM with another policy derived, if this policy would change it, therefore no longer restores, and the snapshot needs a recapture, which the error says. Host profiles are for development and not pinned, and a build that computes another effective CPUID already rejects their snapshots (E_CPU_SURFACE). The Guide says so. Tests: restore_refuses_a_host_profile_outside_the_derivation_policy restores an Intel and an AMD host profile with VMX, and AMD's with SVM, each recorded with its own digest, and restore_names_only_the_first_violations_of_the_policy bounds the error. The derive tests check that every pinned profile, the host profile of every recorded host, and the host profile of random mutations of each host's CPUID follow the policy, so that no snapshot that this OpenVMM captured is refused (a run of 18,000 mutations found none); that each feature that the policy withholds, as the Intel SDM and the AMD APM define it, is a violation alone; and that the policy's other parts are checked. The restore preflight's and the worker's selection's tests restore a crafted VMX profile with the effective CPUID that it gives. At feature/microvm/dev efe6bc2, the three tests of the restore fail, since each restore succeeds; with this change, the 132 cpu_profile, 138 openvmm_helpers, and 130 openvmm_core tests pass with cargo nextest on Windows x64, and the 132 cpu_profile and 276 openvmm_helpers and openvmm_core tests on Linux x64. On KVM, an Azure VM with an Intel Xeon Platinum 8573C on Linux 6.18, a script cold boots a microVM with --cpu-profile host, captures its snapshot, and restores a copy whose host profile has VMX set, recorded with the edited document's own digest and the effective CPUID that it gives. At efe6bc2, the release build that CI published restores it, and the guest reads IA32_VMX_BASIC, 0x0098100011e57ed0, through /dev/cpu/0/msr without a #GP, which the unedited snapshot's guest cannot. With this change, the restore fails with [E_PROFILE_DIGEST], naming CPUID 0x1 ECX bit 5. On MSHV and WHP, Azure VMs too, efe6bc2 refuses the edited snapshot with E_PROFILE_UNSUPPORTED, since their partitions lack VMX, and this change with E_PROFILE_DIGEST. The guest-boot, cpu-profile-fallback, and snapshot-core scenarios of NVX, which capture and restore a real host profile, pass with this change on KVM, MSHV, and WHP. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
XsaveCapabilities::standard_len_for and compact_len_for added and aligned u32 lengths without checking them, and X86PartitionCapabilities::from_cpuid calls both on the components that its CPUID describes. A host's CPUID never describes components that large, but a restored CPU profile can: it is the snapshot's own document, whose leaf 0xd only has to agree with itself, and the KVM backend builds a time ABI partition's capabilities from it, before the profile support check runs. A component that ends past 2^32 bytes, components whose compact places add up past 2^32, or an alignment past the last multiple of 64 panicked a debug build with "attempt to add with overflow", and wrapped a release build to a length that no component fits in. Both lengths now saturate at u32::MAX, and so does rounding a length up to a multiple of 64 past 0xffffffc0, since masking the rounded sum would bring a saturated length back below u32::MAX. A length that fits is unchanged. Tests: xsave_lengths_saturate_at_the_largest_length checks that a length of 2^32 - 1 fits and one of 2^32 saturates, for the standard end and for the compact sum, and each bound of the alignment. from_cpuid_saturates_xsave_lengths_that_overflow checks the three overflows through from_cpuid, and that it still compares the saturated standard length with the one that the CPU advertises. With cargo nextest run --profile agent -p virt on Windows x64, both fail at feature/microvm/dev efe6bc2 with "attempt to add with overflow" at vmm_core/virt/src/x86/mod.rs:327:27, and pass with this change, as do the other 103 tests that run. On nvx-kvm-test, in a debug build, a KVM time ABI partition whose CPUID has component 2 end at 2^32, or components 5 and 6 of 2^31 bytes each, panics the same way when it is built at efe6bc2, and builds with this change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
KvmProtoPartition::build derives the partition's capabilities from its CPUID, which for a time ABI partition is the CPU profile's effective CPUID, and a restore takes that profile from the snapshot. This backend converts a VP's XSAVE state between the compact area of virt::x86::vp::Xsave, which it builds from compact_len in whole words, and the 4096 bytes of the buffer of KVM_GET_XSAVE and KVM_SET_XSAVE. A profile can describe a layout that this conversion cannot hold: a component that ends past that buffer, or a compact length that is not in whole words, can make it panic, and a saturated compact length is the 4 GiB that Xsave::at_reset would allocate. Only the profile support check, which the worker runs after the build, refuses such a layout (E_PROFILE_UNSUPPORTED), since it requires each component to match KVM's own, so until then the partition holds capabilities that its own VP state code cannot handle. partition_capabilities now derives the capabilities for the build, and check_xsave_layout refuses them with XSaveLengthTooLarge, new, for a standard length past KVM_XSAVE_LEN, the size of kvm_xsave, and with XSaveLengthTooSmall for a compact length that is short of its components or not in whole words. A host's own CPUID passes, and so does the layout of every pinned profile. A refused build fails with KvmError::Capabilities before any VP binds. That error did not have the PartitionCapabilitiesError as its source, so its message, "host does not support required cpu capabilities", named no reason; it does now. Tests: partitions_hold_the_xsave_layouts_of_the_pinned_profiles builds the capabilities of the effective CPUID of every pinned profile, partitions_hold_the_xsave_layouts_that_cpus_describe those of a CPU without components and of one whose component ends at the end of KVM's buffer, and partitions_refuse_the_xsave_layouts_that_kvm_cannot_hold the three refusals, with and without the time ABI. xsave_layout_needs_a_compact_length_that_holds_its_components checks each bound of the compact length. On nvx-kvm-test (Linux 6.18.57, nested KVM on an Intel Xeon Platinum 8573C), the unit tests of virt and virt_kvm pass with cargo nextest, and so do the /dev/kvm tests of virt_kvm and kvm with --run-ignored all. In a debug build there, a KVM time ABI partition built from the CPUID of the Skylake-SP profile builds, and with component 2 ending at 2^32, or components 5 and 6 of 2^31 bytes each, the build fails with the XSaveLengthTooLarge errors for 4294967295 and 2147484800 bytes. At efe6bc2 both panic with "attempt to add with overflow", and with only the previous commit both build and leave the refusal to the later profile support check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Admit the two observed N1X CPU part numbers through a pinned ARM64 profile while retaining the existing WHP feature and state checks. A single-part profile fails when a retained VP exposes the other core type on this heterogeneous host. Validate the configured ID-register surface on all 20 logical CPUs of a MECHREVO N1X Spark with NVIDIA N1X, and document the hardware observations and cold-boot qualification limits. Test both MIDRs, exclude every other part number, and verify the WHP feature-bank translation and unavailable-feature rejection. Depends on the ARM cold-boot implementation in microsoft#5045.
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
|
@microsoft-github-policy-service agree company="MECHREVO" |
Follow-up on the unsafe-code review noticeI checked the Unsafe Check run and the commit boundaries. The workflow completed successfully; its warning marks files for additional review rather than reporting a failed safety test. The workflow scanned 43 Rust files in the full dependent PR and reported:
Both files are inherited from the prerequisite ARM implementation in #5045. They are unchanged between For context, the prerequisite's changes in these two files remove the x86-only conditional on The N1X-specific commit adds the CPU profile, catalog registration, focused tests and hardware notes. It introduces no unsafe blocks or unsafe functions, and does not change the existing WHP FFI implementation or bypass CPU feature validation. I am retaining the Separately, the corporate CLA declaration has been submitted with |
Pin the expected canonical digest and model mask, reject a different implementer or architecture, and use the measured 39-bit native WHP GPA limit in the backend test fixture. The guest address contract remains 36 bits. This changes tests only.
fe61f9c to
1b25c5b
Compare
|
Following Pedro's direction in #5045 (comment), the N1X-only follow-up is now open at ahuja-gautam#1, targeting The new draft retains This original PR remains open as a draft for continuity and tracking; its branch history has not been rewritten. Please review the focused follow-up rather than the prerequisite-inclusive diff here. |
Refs #5045 and microsoft/nvx#1516.
Blocked on #5045; not merge-ready. The full GitHub diff currently spans 49 files because the ARM prerequisite has not landed. The N1X adaptation itself touches four files. Please use the N1X-only comparison to inspect that contribution; this PR must be rebased and revalidated after the prerequisite lands.
What and why
Add a pinned
nvidia.n1x.v1ARM64 CPU profile for NVIDIA N1X, validated on a MECHREVO N1X Spark running Windows with Windows Hypervisor Platform (WHP).The existing ARM catalog does not admit this host. A profile restricted to the first observed MIDR also fails when a later virtual processor exposes the other core type:
ARM model mismatch: VP 0 MIDR=0x410fd871.A thread-affinity scan of all 20 logical processors observed ten instances each of
0x410fd851and0x410fd871. After applying the same restricted WHP feature banks and 36-bit address contract, all 15 configured architectural ID registers matched across the scan. The new model mask admits exactly Arm parts0xd85and0xd87for implementer0x41, architecture0xf, while ignoring variant/revision. An exhaustive test excludes the other 4,094 part numbers.The change adds the profile, registers it for
autoand explicit selection, and tests its model bounds, fixed canonical digest, digest/state validation, WHP feature translation and missing-feature rejection. The backend test records the measured 39-bit native WHP GPA limit while retaining the 36-bit guest contract. It retains the existing backend checks and state policy; it neither spoofs MIDR nor bypasses feature verification. Hardware observations and qualification limits are documented next to the profile.Dependency and review scope
This PR depends on #5045. Its parent is
b09c58843ae310dfff9ae68e9e822f9c5846b914, the ARM implementation tested with NVX24bdced073a24418c1ebbf1d0775bb5ba42266ba. The target isfeature/microvm/dev, as required by NVX's OpenVMM contribution workflow.The branch includes the three prerequisite commits from #5045. The N1X-specific contribution consists of commit
2ff56a61b,cpu_profile: add NVIDIA N1X WHP profile, and the test-only follow-up229ecb91b, touching four files in total. It is not a replacement submission of the ARM implementation. This PR must be rebased and revalidated after the prerequisite lands; it is not ready to merge or promote into NVX's submodule pin.Submission containing materials of a third party: Microsoft Corporation and Gautam Ahuja, under the MIT License. The three prerequisite commits from #5045 retain their original authorship and license notices.
Current-head qualification:
9e138af90489e7ba76011b7b1f90667d805eba87mergesfeature/microvm/devinto the topic branch. The four N1X-specific files are unchanged from229ecb91b, but the integrated head has not been rebuilt or hardware-qualified by the validation reported below. The passing CLA, dependency, label and unsafe checks are not full build/runtime CI. The N1X-only comparison above is intentionally pinned to the two adaptation commits, excluding the later upstream merge.Tested hardware
NVIDIA RTX Spark N1X10.0.28000.3231Validation
Executed on the native Windows ARM64 host with Rust 1.97.1:
All commands completed successfully. The check, Clippy, rustdoc, test and formatting commands were rerun for the test-only follow-up
229ecb91b. The release build and runtime results apply to the original adaptation; the follow-up changes no production code. Package tests: 135 CPU-profile tests and 11 WHP tests passed; two existing explicitly ignored native diagnostic tests were not run. MSVC emitted linker/unused delay-load diagnostics; no Clippy code lint failures were reported. Formatting ran last, including repository lints, fuzzer verification and Flowey verification, without unrelated file changes.Hardware execution completed 18 successful boots, each ending with guest exit status zero:
auto: 128/256 MiB with 1 vCPU; 512 MiB with 2/4/8 vCPUs.autobatches of 4 and 8 guests, each 256 MiB / 1 vCPU, held at a common barrier before workload execution.Each of the 17
autoruns checked a private temporary-file marker, 16 MiB file I/O with SHA-256, synchronized time status and clean shutdown. These are short functional smoke tests, not adversarial isolation or maximum-capacity tests.Remaining qualification
Full upstream CI, other host architectures/backends, managed sandbox lifecycle, snapshot/restore, long-duration load, and production reliability are not established by this patch. E2B API compatibility is outside its scope. Native
hostprofile derivation has not been extended for N1X; useautoornvidia.n1x.v1. The product name identifies the tested machine, not a vendor certification or an SMBIOS admission rule.