Skip to content

[DO NOT REVIEW] cpu_profile: add NVIDIA N1X WHP profile - #5096

Draft
Thans (QThans) wants to merge 284 commits into
microsoft:feature/microvm/devfrom
QThans:cpu-profile/nvidia-n1x-whp
Draft

Thans (QThans) wants to merge 284 commits into
microsoft:feature/microvm/devfrom
QThans:cpu-profile/nvidia-n1x-whp

Conversation

@QThans

@QThans Thans (QThans) commented Nov 14, 2026 •

Copy link
Copy Markdown

Refs #5045 and microsoft/nvx#1516.

Blocked on #5045; not merge-ready. The full GitHub diff currently spans 49 files because the ARM prerequisite has not landed. The N1X adaptation itself touches four files. Please use the N1X-only comparison to inspect that contribution; this PR must be rebased and revalidated after the prerequisite lands.

What and why

Add a pinned nvidia.n1x.v1 ARM64 CPU profile for NVIDIA N1X, validated on a MECHREVO N1X Spark running Windows with Windows Hypervisor Platform (WHP).

The existing ARM catalog does not admit this host. A profile restricted to the first observed MIDR also fails when a later virtual processor exposes the other core type: ARM model mismatch: VP 0 MIDR=0x410fd871.

A thread-affinity scan of all 20 logical processors observed ten instances each of 0x410fd851 and 0x410fd871. After applying the same restricted WHP feature banks and 36-bit address contract, all 15 configured architectural ID registers matched across the scan. The new model mask admits exactly Arm parts 0xd85 and 0xd87 for implementer 0x41, architecture 0xf, while ignoring variant/revision. An exhaustive test excludes the other 4,094 part numbers.

The change adds the profile, registers it for auto and explicit selection, and tests its model bounds, fixed canonical digest, digest/state validation, WHP feature translation and missing-feature rejection. The backend test records the measured 39-bit native WHP GPA limit while retaining the 36-bit guest contract. It retains the existing backend checks and state policy; it neither spoofs MIDR nor bypasses feature verification. Hardware observations and qualification limits are documented next to the profile.

Dependency and review scope

This PR depends on #5045. Its parent is b09c58843ae310dfff9ae68e9e822f9c5846b914, the ARM implementation tested with NVX 24bdced073a24418c1ebbf1d0775bb5ba42266ba. The target is feature/microvm/dev, as required by NVX's OpenVMM contribution workflow.

The branch includes the three prerequisite commits from #5045. The N1X-specific contribution consists of commit 2ff56a61b, cpu_profile: add NVIDIA N1X WHP profile, and the test-only follow-up 229ecb91b, touching four files in total. It is not a replacement submission of the ARM implementation. This PR must be rebased and revalidated after the prerequisite lands; it is not ready to merge or promote into NVX's submodule pin.

Submission containing materials of a third party: Microsoft Corporation and Gautam Ahuja, under the MIT License. The three prerequisite commits from #5045 retain their original authorship and license notices.

Current-head qualification: 9e138af90489e7ba76011b7b1f90667d805eba87 merges feature/microvm/dev into the topic branch. The four N1X-specific files are unchanged from 229ecb91b, but the integrated head has not been rebuilt or hardware-qualified by the validation reported below. The passing CLA, dependency, label and unsafe checks are not full build/runtime CI. The N1X-only comparison above is intentionally pinned to the two adaptation commits, excluding the later upstream merge.

Tested hardware

Item Observed configuration
Product MECHREVO N1X Spark
CPU NVIDIA N1X; Windows reports NVIDIA RTX Spark N1X
Logical processors / RAM 20 / approximately 32 GiB
Host OS Windows ARM64, build 10.0.28000.3231
Hypervisor Native Windows WHP; no nested KVM dependency
Guest ARM64 Linux 6.18.38, Alpine 3.24.1

Validation

Executed on the native Windows ARM64 host with Rust 1.97.1:

cargo check --locked -p cpu_profile -p virt_whp
cargo clippy --locked --all-targets -p cpu_profile -p virt_whp
cargo doc --locked --no-deps -p cpu_profile -p virt_whp
cargo test --locked -p cpu_profile -p virt_whp
cargo build --release --locked -p openvmm --bin openvmm
cargo xtask fmt --fix

All commands completed successfully. The check, Clippy, rustdoc, test and formatting commands were rerun for the test-only follow-up 229ecb91b. The release build and runtime results apply to the original adaptation; the follow-up changes no production code. Package tests: 135 CPU-profile tests and 11 WHP tests passed; two existing explicitly ignored native diagnostic tests were not run. MSVC emitted linker/unused delay-load diagnostics; no Clippy code lint failures were reported. Formatting ran last, including repository lints, fuzzer verification and Flowey verification, without unrelated file changes.

Hardware execution completed 18 successful boots, each ending with guest exit status zero:

  • Explicit profile: 512 MiB / 1 vCPU, shell execution and synchronized time status.
  • auto: 128/256 MiB with 1 vCPU; 512 MiB with 2/4/8 vCPUs.
  • Concurrent auto batches of 4 and 8 guests, each 256 MiB / 1 vCPU, held at a common barrier before workload execution.

Each of the 17 auto runs checked a private temporary-file marker, 16 MiB file I/O with SHA-256, synchronized time status and clean shutdown. These are short functional smoke tests, not adversarial isolation or maximum-capacity tests.

Remaining qualification

Full upstream CI, other host architectures/backends, managed sandbox lifecycle, snapshot/restore, long-duration load, and production reliability are not established by this patch. E2B API compatibility is outside its scope. Native host profile derivation has not been extended for N1X; use auto or nvidia.n1x.v1. The product name identifies the tested machine, not a vendor certification or an SMBIOS admission rule.

…e is stopped

The virtio transports save and restore only transport state: feature
negotiation, queue configuration, and queue progress. A device whose
state lives outside its queues, such as buffered console or network
data, has nowhere to save it, so such devices cannot support
save/restore. Restore also trusted the saved queue layout and progress.

Add device-private saved state to `VirtioDevice`. After all queues stop,
`save_device` returns an optional typed `SavedStateBlob`, which the PCI
and MMIO transports save as a new `device_state` field; `restore_device`
receives it, and `device_state_validator` returns an immutable validator
that the transport keeps after the device moves into its task. The
defaults preserve the current behavior: devices that support
save/restore have no private state, and other devices cannot be saved.
The new `device::saved_state` module holds the validator type and a
helper that reads the front restored descriptor without consuming it.

Restore now validates the saved state before it changes the transport:
exactly two feature banks, queue sizes within each queue's device
maximum, enabled rings aligned, inside guest RAM, and not overlapping,
available and used progress consistent with the rings, progress present
for every enabled queue of a DRIVER_OK device, and private state accepted
by the device's validator. The transport then hands the private state to
the stopped device task and waits for the device to restore it, so that
a device that cannot restore, such as one that cannot reopen its host
resources, fails the restore itself, before any VP runs, rather than a
later start. The MMIO transport's save and restore become asynchronous,
as the PCI transport's already are. Restoring starts no queue: a save
before the first start returns the restored state, and queues that were
never started keep their restored progress. A guest reset of a device
that is not DRIVER_OK resets the device, which discards the private
state that the restore gave it.

`ChangeDeviceState::start` cannot wait for the device task, so the
transport keeps the task's answer and applies it the next time it polls
or stops the device. If a queue fails to start, the device task stops
the queues that did start, and the transport sets DEVICE_NEEDS_RESET,
notifying a driver that has set DRIVER_OK, as virtio 1.2 section 2.1.2
requires of a device in an error state, rather than leave the guest
waiting on a device that processes nothing. The device task traces
queue starts and private-state restores under the `virtio_restore`
target.

`SavedStateBlob` becomes `Clone`.

The virtio-blk persistence test now shuts the guest down with an
enlightened shutdown after its save/restore pulse, avoiding the pipette
shutdown delay, which depends on the guest timer state that the test
just restored.

It also includes these changes:

- virtio: await the PCI transport's save and restore in saved-state tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A snapshot of a running VM first gates host input, so that no new
external input reaches a device while its state is captured, and resumes
the input if the save transaction fails. `ChangeDeviceState` provides
`quiesce_input` and `resume_input` for this, but the virtio transports
did not implement them, so a virtio device could not take part.

Add `quiesce_input` and `resume_input` to `VirtioDevice`, defaulting to
success for devices without host input. The PCI and MMIO transports
forward both through the device task as the new `QuiesceInput` and
`ResumeInput` commands, so they are ordered with the device's other
commands, and return the device's error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
When a write to the connected socket failed with BrokenPipe, the socket
backend reported the write as successful and kept the dead socket. Any
other write error was returned, but the socket also stayed attached.
Either way the backend kept writing to a connection that could no longer
carry data, and it never reported the disconnect or accepted a new
client.

Frontends that keep guest output for the next client need to know which
bytes a write failed to send:

- A virtio-console with the `Retain` disconnect policy, which the next
  commit adds, leaves a transmit descriptor queued, with the offset of
  its bytes already written, only when the write fails. Were the write
  reported as successful, the worker could complete the guest's output
  before its receive side noticed the hangup, and that output would be
  lost.
- The microVM portb console, which a later commit adds, keeps its
  buffered output for the next peer only when a write fails with
  `BrokenPipe`.

Drop the current connection on any write error and return the error.
Writes are then discarded, as they are while no client is connected,
and the backend reports itself disconnected so the next connect accepts
a new client. Say so next to the code.

Upstream reports a broken pipe on a write as successful and keeps the
socket. microsoft#4605 first proposed this change to detect
disconnects, but review settled disconnect detection in virtio-console,
which polls `poll_disconnect` without receive buffers since
microsoft#4778, and microsoft#4605 merged narrowed to the read path.

A test checks that a write to a Unix socket peer that has closed fails
with `BrokenPipe`, disconnects the backend, and lets the listener accept
a replacement client. It runs on Linux only: Windows accepts the first
write after the peer closes and fails the next with `ConnectionReset`,
so a console that has not yet noticed the hangup loses that one write's
bytes there.

It also includes these changes:

- serial_socket: document and test why a failed write drops the
  connection

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
…t policies

A virtio-console always discarded guest output while its host backend
was disconnected, and a console resource could not describe how its host
endpoint is attached or reconnected. A host that must not lose console
output, or that reattaches endpoints after a restore, has no way to ask
for that, and the resolver cannot reject an inconsistent endpoint
description.

Add `VirtioConsoleDisconnectPolicy` and an optional
`VirtioConsoleAttachment` to `VirtioConsoleHandle`. With the `Retain`
policy, the console keeps pending guest transmit descriptors while the
backend is disconnected and forwards them after it reconnects; `Discard`
keeps the current behavior and remains the default of
`VirtioConsoleDevice::new`. The new `new_with_policy` constructor selects
the policy, and the worker now holds its backend in a
`ConsoleWorkerMode`. The attachment records a stable ID, the backend
kind, the attachment mode, the endpoint identity, and a reconnect policy
with its requirement and timeout. The resolver rejects an attachment
whose identifiers are empty or oversized, whose mode and reconnect policy
conflict, or whose requirement and timeout do not match the policy: a
recreated listener or a discarding attachment is optional without a
timeout, a reconnecting client is required with a bounded timeout, and an
inherited attachment is required without a timeout.

Disconnects are also handled consistently: after a read error or
end-of-file, the worker completes the backend's disconnect before it
waits for a new connection, and a backend write that accepts no bytes is
treated as a disconnect instead of a panic.

All existing console resources use `Discard` without an attachment.

It also includes these changes:

- petri: adapt upstream's Incubator changes to the chain's petri

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A virtio-console could not be saved: host input already read from the
backend lived only in a local buffer of the receive loop, the worker
state was removed once both queues stopped, and the progress within the
current transmit descriptor was lost with it. Restoring a console would
therefore drop accepted input or replay output.

Implement device-private save and restore for the direct console mode.
The worker state is now created with the device and kept while the
queues are stopped, so a stopped device still holds its private state.
Host input is staged in the worker state before it is copied to a guest
receive buffer, so input accepted from the backend survives worker
restarts and snapshots; the receive loop also reads the backend before
the guest posts a buffer, completing zero-length buffers as it goes. The
saved state, schema version 1, records the console size, the bytes
already written from the current transmit descriptor, the staged input,
and the disconnect policy. Restore validates the encoded size, the
schema version, the policy, and the bounds of every field before
decoding further, and requires a restored transmit offset to lie within
the front descriptor of the enabled transmit queue, both before the
queues restart and when the transmit queue starts. Reset clears the
private state.

The console also implements `quiesce_input` and `resume_input`: while
input is gated, the worker reads no host input, and output continues.

`SavedStateBlob::encoded_len` returns the encoded payload length without
decoding it, which the console uses to bound its private state.

Document that virtio-console now supports save and restore through the
transport's device-private payload.

A console with the `Retain` disconnect policy keeps the guest's transmit
descriptors while its backend is disconnected and forwards them after
the backend reconnects. The case that matters is a peer that leaves
before the worker notices: the worker then writes the guest's next
output to it, and keeps that output only because the write fails, as
the socket backend's does. A mock control makes the peer leave without
waking the worker, so that its next write fails with `BrokenPipe` and
disconnects the backend, and two tests of the `Retain` policy check
that:

- output that the guest writes after its peer left stays queued while
  the worker waits for the next peer, which then receives it
  (`retain_keeps_output_whose_write_fails_for_the_next_peer`);
- when the peer leaves after taking part of a descriptor, the next peer
  receives the rest of it, and not what the departed peer already
  received (`retain_sends_the_rest_of_a_partial_write_to_the_next_peer`).

With the `Discard` policy, both fail at their check that the descriptor
is still queued, and the first fails there too when the departed peer's
write reports success, as upstream's socket backend reports a broken
pipe.

It also includes these changes:

- virtio_console: test that Retain keeps output for the next peer

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
To save a network device, its frontend must stop exchanging packets with
the endpoint at a well-defined point: every receive and transmit already
accepted by the endpoint must complete, and the endpoint must not keep
any guest buffer. The `Queue` trait had no operation for this, so a
frontend could only drop its queues, losing work in flight.

Add `Queue::quiesce`, which finishes the work that a queue already
accepted without accepting new work. On success, all resulting receive
and transmit completions are available through `rx_poll` and `tx_poll`,
the queue retains no buffer IDs or guest-memory segments, and the
returned `QueueQuiesceResult` reports how many completions of each kind
remain to be polled. `Queue::resume` resumes host receive admission after
a rolled-back quiesce. Queues do not support quiesce by default, and
resume succeeds by default.

`BufferAccess` now requires `Send`, so a quiesce future can hold the
frontend's buffer pool across await points; every existing
implementation already is.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A TAP queue wrote guest packets synchronously and completed every
packet, including the ones that the TAP interface dropped because its
buffer was full or the interface was down. A packet dropped under
backpressure was therefore lost, and the queue could not implement the
endpoint quiesce contract, because it could not report which transmits
were still in flight.

Each `tx_avail` call now writes the packets that it is given until the
TAP interface applies backpressure, because the frontends submit the
rest of a batch only when `poll_ready` reports that the queue can accept
it. The queue keeps the packet that meets backpressure, and with it the
ownership of its descriptors, until `poll_ready` writes it through the
new `AsyncWrite` implementation of `PolledTap`; the call then completes
asynchronously, and `tx_poll` returns the packets that it wrote before
that packet and then that packet. A partial write or a write error is
reported as a fatal transmit error instead of being ignored.

Implement `Queue::quiesce` for TAP queues: a quiesced queue drops its
free receive buffers, stops reading from the interface and admitting
receive buffers, finishes writing its pending packet, and reports the
receive and transmit completions that it still holds. `Queue::resume`
resumes receiving after a rolled-back quiesce.

The TAP backpressure test now checks that every packet is completed
exactly once instead of being dropped.

It also includes these changes:

- net_tap: write every packet of a transmit batch
- net: enable no receive offloads in the chain's endpoint tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A Consomme queue could not stop exchanging packets at a well-defined
point for a save, because it always polled the Consomme stack and its
control channels and kept accepting receive buffers.

Implement `Queue::quiesce` for Consomme queues. A quiesced queue
processes the frames that the guest already queued for transmit, drops
its posted receive buffers, ignores new ones, and stops polling the
Consomme stack and the control channels; `poll_ready` then only reports
the completions still pending. Quiesce reports the receive and transmit
completions that the queue still holds, and `Queue::resume` restores
normal polling after a rolled-back quiesce.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
Consomme derives the guest and gateway addresses from a CIDR, so a host
cannot give a guest an exact, preconfigured IPv4 identity: a specific
guest address and prefix, and a gateway whose address and MAC address
stay fixed across snapshots and restores. Consomme also always treats
traffic to the gateway as ordinary egress, with no way to map it onto
host loopback, which a guest with a static identity uses to reach host
services.

Add `ConsommeParams::set_static_ipv4`, which sets an exact guest address,
prefix length, gateway address, and gateway MAC address. The identity
must be consistent: a prefix of 1 to 30 bits, a guest address that is
neither the network nor the broadcast address, and a gateway at the
first host address of the network, distinct from the guest. A static
identity stops advertising a routable IPv6 prefix and enables the new
`map_gateway_to_host_loopback` parameter, which translates guest TCP and
UDP flows addressed to the gateway onto host loopback. With the mapping
disabled, such flows are dropped, except TCP to the single port named by
the new `gateway_loopback_proxy_port` parameter.

Expose the identity to resources: `ConsommeHandle` gains an optional
`StaticIpv4Config`, which is mutually exclusive with `cidr`, and optional
overrides of host-local access, the gateway mapping, and the gateway
proxy port. The resolver rejects a handle with both a CIDR and a static
identity, or with an inconsistent identity. Existing handles set none of
the new fields.

This changes one behavior of every Consomme endpoint: a guest TCP or UDP
flow addressed to the gateway, other than the gateway's own DNS and DHCP
services, used to be sent to the gateway's address on the host network,
where no gateway exists. Such flows are now dropped unless the gateway
mapping or the proxy port allows them.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
virtio-net claimed to support save and restore unconditionally, but the
transport saved only queue progress: packets that the endpoint had
accepted, or that the device held for the guest, were lost, and nothing
tied a restored guest's view of its network identity to the endpoint.
Saving a VM with virtio-net therefore silently corrupted the guest's
network state.

Save and restore are now opt-in. `NicBuilder::save_restore` binds a
device to an exact static IPv4 identity and a fixed effective feature
contract, and `VirtioNetHandle` gains `save_restore`, `static_ipv4`, and
`effective_features` fields for resources; a resource that opts in must
supply both. Other devices report that they do not support save and
restore.

When the transport stops an active queue pair of such a device, the
device quiesces the endpoint queues, drains every completion the
endpoint still holds so that no descriptor stays in flight, and captures
the progress of both queues, which the transport then collects queue by
queue. The device-private saved state, schema version 1, records the
identity, the guest MAC address, the link state, the effective and
negotiated feature banks, the endpoint offloads, the queue-pair
lifecycle, and an endpoint generation that each restore advances.
Restore validates it against the destination device: the identity must
match and use MAC addresses derived from its IPv4 addresses, the
effective features and endpoint offloads must be unchanged, the
negotiated features must stay within the contract and match the saved
banks, and the device must have a single queue pair whose restored
queues own no descriptors. Restarting a stopped pair requires the same
negotiated features.

The device also implements `quiesce_input` and `resume_input`: while
input is gated, the endpoint queues finish the work they already accepted
but admit no new host receive traffic, and the gate is re-established
whenever the queues restart. Workers are now started only by the
coordinator, after it establishes the gate.

Petri records the PCIe ports of virtio-net devices added without a save
and restore contract and skips its startup save/restore test while any
of them is configured. Every existing virtio-net resource leaves
`save_restore` unset.

Document that virtio-net supports save and restore only with a static
identity and feature contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A FUSE session negotiates its protocol version, feature flags, and
limits with the guest during FUSE_INIT and keeps them only in memory. A
file system device built on it could not be saved and restored, because
a restored session would be uninitialized while the guest driver
continued with its negotiated protocol.

Add `SessionState`, a serializable snapshot of whether a session is
initialized and of its negotiated values, which contains no file handle,
task, or native resource. `Session::save_state` captures it, and
`Session::restore_state` restores it after validating it: an
uninitialized state must carry no negotiated values, and an initialized
one must use a supported protocol version (7.27 through the
implementation's minor version), accept only feature flags that the
guest advertised, keep its extended-init flags consistent, and have
nonzero write and time-granularity limits. The types are re-exported
from the crate root.

Two related fixes make the saved values well-defined. FUSE_INIT now
records the negotiated minor version as the lower of the guest's and the
implementation's, instead of the guest's value, so a session never
claims a protocol newer than it implements. Destroying a session also
resets its negotiated information, so a later session starts from
defaults.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A virtio-fs inode records a single host path, which it uses for every
later operation on the node. That path went stale in several cases. A
hard link gave the inode a second name that nothing recorded, so
unlinking the original name left the inode pointing at a missing path
even though it was still reachable through the link. Renaming a
directory left every open inode below it pointing at the old location.
The dedup map, which lets repeated lookups of a host file return one
FUSE node, only evicted the renamed or removed path, without updating
the inodes that referred to it.

Each inode now keeps the set of host paths through which the guest
knows it. Lookups and hard links add a path. After an unlink, the paths
at or below the removed name are dropped from every inode of the volume,
and an inode whose primary path was removed switches to one of its
remaining aliases. After a rename, the paths at or below the old name
are rewritten under the new name, after first dropping the aliases of
any overwritten destination. The dedup map is then rebuilt from the
surviving inodes, and a path-keyed inode with no remaining alias is left
out of it, so a later create at a vacated path still gets a fresh node.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The x86 ACPI builder describes legacy ISA IRQs with their bus defaults,
edge-triggered and active-high, and a VM whose chipset devices signal a
legacy IRQ by level had no way to say so in the MADT. The MP tables that
the loader now builds already take such a list, so the two descriptions
of a machine could not agree.

Add `level_triggered_irqs` to the x86 `AcpiArchConfig`. The MADT gains
an active-high, level-triggered interrupt source override for each
listed IRQ. Every existing configuration, in OpenVMM, OpenHCL, and
igvmfilegen, passes an empty list, so the generated tables are
unchanged.

A test checks that the MADT and the MP configuration table enumerate the
same APIC IDs for 1, 2, 4, and 8 processors, and another checks the
override for a level-triggered IRQ.

It also includes these changes:

- vmm_core: build the ACPI builder tests with level-triggered IRQs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
OpenVMM builds one kind of guest-visible machine. A second, minimal
machine needs a way to be selected in the VM configuration that is
independent of the hypervisor backend, and a place for its own settings.

Add the `microvm` module to `openvmm_defs` with `MachineProfile`, which
selects the standard machine or the microVM machine, and a
`MicrovmConfig` for settings specific to the microVM machine, initially
empty. `Config` gains `machine_profile` and `microvm` fields. The worker
keeps the profile in its manifest and in the loaded VM, and carries it
into the manifest of a restarted worker.

Every existing configuration selects the standard machine, and nothing
acts on the profile yet.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The x86 processor topology of a VM is derived from the host's topology,
so the same configuration can present different APIC IDs and cache
topologies on different hosts. A microVM whose snapshots must restore on
any host needs a topology that depends only on its configuration.

The worker now builds the x86 topology through the new
`dispatch::microvm::x86_topology_builder`: a microVM starts from the
fixed default topology instead of the host's, and other VMs are
unchanged.

The partition-wide topology CPUID leaves also report the bootstrap
processor's APIC identity, with the fields included in each leaf's mask:
the initial APIC ID in leaf 01h, the x2APIC ID in leaves 0Bh and 1Fh,
and the extended APIC, compute-unit, and node IDs in leaf 8000001Eh.
Backends replace these values for each processor, so the reported
identity of every processor is unchanged, but the partition-wide leaves
now describe a real processor.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The CMOS RTC has a single behavior, fixed at construction, and the base
chipset builder offers no way to choose another. A machine profile that
needs a different guest-visible RTC behavior has nowhere to select it.

Add `RtcMode` to the RTC and `GenericCmosRtcMode` to the generic CMOS RTC
dependencies of the base chipset, each with a single `Standard` mode,
the programmable MC146818-compatible behavior. `Rtc::new_with_mode` and
`Rtc::new_with_mode_and_utc_clock`, which also takes a fallible UTC clock
source, construct an RTC in a given mode, and a reset returns the RTC to
its mode's initial register state. Every mode currently shares the
standard state, and OpenVMM and OpenHCL select the standard mode, so the
guest-visible behavior is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
Every base chipset type includes firmware-facing or bus infrastructure
that a minimal machine does not want, such as VMBus, PCI, the PIIX4
devices, or ACPI power management. A microVM needs a base chipset that
contains only the architectural devices its guest requires.

Add `BaseChipsetType::Microvm`, which is supported only on x86-64. Its
chipset contains exactly a generic CMOS RTC, a generic I/O APIC, the PIC,
and the PIT, without VMBus, PCI, ISA DMA, or ACPI power management. Its
layout reserves 1 GiB of chipset MMIO below 4 GiB and none above, so
guest RAM ends at 3 GiB and resumes at 4 GiB; a memory-layout test checks
that placement.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A guest can ask to be powered off, but not with a result: a guest that
runs a single workload and then powers off has no way to hand its exit
status to the host process that launched it.

Add `PowerRequest::PowerOffWithStatus` and `HaltReason::PowerOffWithStatus`,
which carry an 8-bit guest-provided status, and forward the request as
that halt reason. Every consumer treats the new reason like a power-off:
the debugger stop report, the OpenVMM VM controller, OpenHCL, and Petri.
No device issues the request yet. A test checks that the status survives
a mesh round trip.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A microVM has no UART, no VMBus, and no firmware console, so it needs
a debug console that costs the guest a single port access per byte and
the host no emulated serial state machine.

Add `MicrovmPortb`, a raw bidirectional byte channel at I/O ports 0xe9
(data) and 0xea (status), backed by any `SerialIo` endpoint:

- A guest write to the data port queues every byte of the access; the
  device drains the queue to the endpoint from `poll_device`. Output is
  binary-transparent, so a guest can stream arbitrary bytes.
- A guest read from the data port pops one received byte (0 when the
  queue is empty) and zero-fills the rest of the access. Bit 0 of the
  status port reports that input is available.
- Each direction buffers at most 1 MiB. Output beyond that drops the
  newest bytes with a rate-limited warning, and input stops being read
  from the endpoint until the guest consumes some of it.
- An endpoint write error other than a broken pipe drops the buffered
  output and re-arms the output wakeup, so the next guest write still
  reaches a recovered endpoint.
- `stop` flushes whatever the endpoint accepts immediately. Bytes still
  owned by the device are saved in `MicrovmPortbSavedState` and resent
  to the reconstructed endpoint after restore; restore rejects buffers
  larger than the device limit.

Add `MicrovmPortbHandle` and `MicrovmPortbResolver`, and register the
resolver for x86_64 guests. Nothing instantiates the device yet.

Tests cover binary output and zero-filled reads, polling a newly
accepted connection immediately, and preserving the output wakeup
across an endpoint error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A microVM guest that runs one workload needs to end the VM and report
the workload's result to the host with a single port access, without
ACPI, a PM1 block, or a firmware power-off path.

Add `MicrovmShutdown`, a write-only port at 0x604. A write of any size
requests `PowerRequest::PowerOffWithStatus` with the first byte of the
access as the status (0 for an empty access). Reads return all ones.
The device is stateless, so it saves `NoSavedState`.

Add `MicrovmShutdownHandle` and `MicrovmShutdownResolver`, which obtains
the power request client from the platform, and register the resolver
for x86_64 guests. Nothing instantiates the device yet.

A test checks that the first byte of a wide or single-byte write
becomes the power-off status.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The microVM profile has no firmware and no ACPI. Linux instead discovers
the processors and interrupt routing from Intel MP 1.4 tables, which the
loader can already build. The worker needs a boot mode that selects that
loader, and a machine contract to check before it builds anything.

Add `LinuxDirectBootMode::MpTable` (x86-64 only). The worker loads the
kernel through `vm_loaders::microvm::load_linux_x86_mptable`, which
builds MP tables from the VP APIC IDs and the profile's level-triggered
IRQ list (currently empty). MP-table boot rejects isolation. An embedded
NUL in a Linux command line is now a load error instead of a panic.

Define the profile's command-line ABI in `openvmm_defs::microvm`:

- The profile owns the leading tokens `earlycon=xe9 console=hvc0
  reboot=t panic=-1`, which use the patched guest kernel's port-0xe9
  console. The command line is limited to 64 KiB.
- `build_microvm_command_line` appends user arguments and rejects any
  that override `earlycon=`, `console=`, or `nr_cpus=`.
- `append_microvm_processor_limit` appends the host-owned `nr_cpus=` for
  a supported processor count (1, 2, 4, or 8).

Add `validate_machine_config`, which checks the fixed machine contract:

- Boot and backend: MP-table Linux boot without isolation, emulated
  serial, or SMBIOS overrides, and a KVM, MSHV, or WHP backend.
- Processors and memory: 1, 2, 4, or 8 vCPUs in one socket without SMT
  or x2APIC, and one NUMA node with RAM.
- Features: no Hyper-V enlightenments, VTL2, isolation, or nested
  virtualization.
- Devices: exactly the microVM base chipset with the IOAPIC, PIC, PIT,
  portb, and shutdown devices. No floppy, IDE, PCI, PCIe, VPCI, ISA DMA,
  VMBus, graphics, VMGS, firmware or debugger channel, RTC offset, or
  virtio device.
- Layout: the fixed 1-GiB low MMIO layout.

For the standard profile, the validator only rejects MP-table boot. The
worker validates every configuration before it constructs the VM.

On a microVM cold boot (not a restore), the worker adds
`lapic_timer_hz=` with the backend-reported LAPIC timer frequency, so
the patched guest kernel skips timer calibration. A user-supplied value
is canonicalized when it matches the backend and rejected when it does
not. Parsing follows the kernel's rules: hyphens and underscores are
interchangeable, quoting is honored, values may be decimal, octal, or
hexadecimal, and tokens after `--` belong to init. The token is
inserted before `--` and before any virtio-mmio discovery tokens.
`HvlitePartition` gains a `MicrovmPartition` supertrait that exposes the
backend LAPIC frequency.

The microVM chipset does not trace unknown port I/O: Linux probes many
legacy ports that the profile deliberately omits.

Nothing configures a microVM yet. Tests cover the boot-mode matrix, the
host-owned processor limit, and LAPIC frequency propagation.

It also includes these changes:

- openvmm_core: reject MP-table boot for aarch64 guests
- openvmm_core: gate the microVM's x86_64-only items for aarch64 guests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
…t for microVMs

The microVM profile's persistent state is defined by its own snapshot
contract, which later commits establish. Until then, the generic
host-driven paths would produce state that nothing can restore: a saved
microVM cannot be restored, and a worker restart would rebuild the VM
from a manifest that does not describe it.

Filter management RPCs in the worker loop before dispatch. For a microVM:

- `VmRpc::Save` fails with "save is unavailable for microVM".
- `VmRpc::PulseSaveRestore` fails with the new
  `PulseSaveRestoreError::UnsupportedMachineProfile`.
- `WorkerRpc::Restart` fails with "worker restart is unavailable for
  microVM".

The filters complete a rejected RPC in place and return every other RPC
unchanged, so the standard machine is unaffected. Petri's
`verify_save_restore` treats the new error like a missing reset: it logs
a warning and skips the save/restore check.

It also includes these changes:

- openvmm_core: gate the microVM's x86_64-only items for aarch64 guests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The worker can boot the microVM profile, but nothing selects it. Add
`--machine <standard|microvm>`, which defaults to `standard`.

With `--machine microvm`, the entry point does the following:

- Validates the options before it opens any host resource:
  - The guest is x86-64 with 1, 2, 4, or 8 processors and no custom
    NUMA, socket, SMT, APIC-offset, or x2APIC settings.
  - The guest boots Linux directly, without firmware options.
  - Hyper-V, VTL2, isolation, nested virtualization, GET, VMBus, and
    the debugger are disabled.
  - A named backend is KVM or MSHV on Linux, or WHP on Windows.
  - There are no UART, debugcon, storage, network, virtio, legacy,
    PCIe, IOMMU, or assigned devices.
  - Snapshot restore is rejected.
- Builds the microVM base chipset and fixed layout, and adds the portb
  console and shutdown devices. Portb owns the interactive console
  (`hvc0` in the guest), so COM1 defaults to none.
- Loads the kernel and the optional initrd in MP-table boot mode with
  the profile-owned command line: the base tokens, then the user
  arguments, then the host-owned `nr_cpus=`. Hyper-V enlightenments,
  VMGS, and SMBIOS overrides are disabled.
- Fixes the topology to one socket with one core per vCPU, no SMT, and
  xAPIC with contiguous APIC IDs.
- Auto-selects the backend with `choose_microvm_hypervisor`, which uses
  the usual probe order on Linux and Windows and fails on other hosts.

Every configuration, for either profile, is then checked against the
machine contract, including the named hypervisor.

For now, a status-bearing power-off through port 0x604 follows
`--guest-shutdown-action` like any power-off. A later change propagates
the status to the process exit code.

Document `--machine` in the CLI reference and the MP-table boot layout
in the Linux direct boot reference. Tests cover option parsing,
processor and topology validation, command-line ownership, and the
rejection of unsupported combinations.

It also includes these changes:

- openvmm_entry: gate the microVM checks of x86_64-only options
- microVM: fix the macOS clippy build
- openvmm_entry: run the microVM option tests only for x86_64 guests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
… test

Nothing exercises a microVM end to end yet. Teach Petri to build one and
add a cold-boot lifecycle test.

`PetriVmBuilder::with_microvm_machine(n)` selects the microVM profile
with a deterministic topology: n vCPUs in one socket, no SMT, and xAPIC.
The builder also sets:

- Minimal mode, with no VMBus and no Hyper-V enlightenments.
- No storage controllers, NVMe or virtio-blk drives, or agent images.
- Linux direct boot, with pipette not running as init.

`PetriVmConfig` carries the machine profile, and the Hyper-V backend
rejects the microVM profile.

For a microVM, the OpenVMM backend does the following:

- Builds the microVM chipset.
- Switches the Linux load mode to MP-table boot with the profile-owned
  command line: the base tokens, the test's init arguments, and
  `nr_cpus=`.
- Drops VMGS, the firmware event channel, VMBus, and Hyper-V
  enlightenments.
- Attaches the portb and shutdown devices, and validates the
  configuration against the machine contract.
- Chooses the backend with `choose_microvm_hypervisor`.

Portb replaces the emulated serial ports. Its output is logged and also
forwarded raw to the test through the new `log_task_with_output`, and
the test can write raw input.

New test APIs:

- `wait_for_microvm_portb_output`, `wait_for_microvm_portb_bytes`, and
  `write_microvm_portb_input` for the portb console.
- `pulse_save_restore`, which performs one pulse save/restore.
- `with_linux_command_line`, which edits the effective Linux command
  line.
- `prepare_initrd_with_file`, which injects one file into the Linux
  direct initrd. `prepare_initrd` now uses it.
- `linux_direct_boot_files`, which returns the kernel and initrd paths.

A builder-supplied initrd now replaces the artifact initrd for any Linux
direct VM, not only with pipette as init, and Linux direct firmware may
omit the initrd.

`x86_64::microvm::phase_1_lifecycle` boots a 1-vCPU microVM whose init
script drives portb through `/dev/port`. The test:

1. Waits for the boot marker.
2. Checks that host save and pulse save/restore are rejected.
3. Round-trips a ping and a binary-transparent echo (NUL, CR, LF, DEL,
   and 0xff) through portb.
4. Requests a status-37 shutdown through port 0x604, and checks that
   the resulting power-off halt carries the status.

It also includes these changes:

- petri: adapt upstream's Incubator changes to the chain's petri

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A machine profile with a fixed guest ABI must control which virtio
features a device offers. The microVM profile, for example, supports
only split rings. The MMIO transport currently offers every feature
the device reports.

Add `VirtioDeviceFeatures::without_bits`, which clears the features
selected by a 64-bit mask, and
`VirtioMmioDevice::new_with_disabled_features`, which masks the
guest-visible device features before the guest reads them.
`VirtioMmioDevice::new` delegates to it with an empty mask, so existing
devices are unchanged.

Saved-state validation already requires restored driver features to be
a subset of the transport's device features, so a saved state that
negotiated a masked feature fails to restore.

A test checks that masking the packed-ring bit leaves the other
features in place.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The standard machine assigns virtio-mmio windows from a dynamic region
of the memory layout in device order, so a device's address and IRQ
depend on which other devices are present. A microVM guest instead
discovers its devices from a fixed map that does not change with the
device set, which keeps the guest ABI stable.

Define the microVM virtio-mmio map in `openvmm_defs::microvm`: eight
0x1000-byte windows from 0xd0000000 in stable device order (network,
filesystem, boot console, four sandbox blocks, control console), their
IRQs, and a check that the windows are ordered, disjoint, and inside the
3-to-4-GiB aperture. IRQ 8 stays with the RTC, and no slot IRQ is level
triggered.

Wire the first slot, the boot virtio-console:

- `--virtio-console` is now accepted with `--machine microvm` using
  `listen=PATH`, `listen=tcp:IP:PORT`, `console`, or `none`. A PCIe port
  is rejected. The console appears in the guest as `hvc1` at 0xd0002000,
  IRQ 7, and the profile's base command line switches to
  `console=hvc1`.
- `append_microvm_virtio_discovery` appends a host-owned
  `virtio_mmio.device=` token for each present slot, in slot order.
  `build_microvm_command_line` now rejects user `virtio_mmio.device=`
  tokens. The machine contract requires exactly one discovery token
  per device, in fixed-address order at the end of the command line.
- The machine contract admits at most eight virtio-mmio devices, and
  at most one of them may be a virtio-console.
- The worker places microVM virtio-mmio devices at their fixed slots
  through `VirtioMmioSlots` instead of the layout's dynamic region.
  The layout reserves no dynamic windows for a microVM. The fixed slots
  expose split rings only: the packed-ring feature is masked.
- When the console uses the terminal, portb output moves to stderr.
  A listener retains guest output until a client connects, and `none`
  discards it.

Tests cover the console command line, discovery tokens, reserved
tokens, the console option matrix, and the IRQ assignments.

It also includes these changes:

- openvmm_core: gate the microVM's x86_64-only items for aarch64 guests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A virtio-mmio interrupt costs the guest two trapping register accesses:
it reads INTERRUPT_STATUS to learn why the line fired, then writes
INTERRUPT_ACK to clear it. Both accesses exit to the VMM.

Add two ways for the MMIO transport to avoid these exits.

`VirtioMmioInterruptMode::SharedStatus { status_gpa }` keeps the
interrupt status in a naturally aligned 32-bit word of guest memory
instead of the INTERRUPT_STATUS register. The device sets status bits
with an atomic compare-exchange, and it pulses the interrupt line only
when the word goes from zero to nonzero. A guest that supports the mode
reads and clears the word without exiting. INTERRUPT_STATUS reads return
the word. The transport rejects a misaligned or inaccessible word and
zeroes it when the transport is created, reset, or dropped.

In the default `Legacy` mode on Linux, the transport registers an
accelerated doorbell on INTERRUPT_ACK for the used-buffer bit. A
used-buffer acknowledgement then no longer exits to user mode. A
used-buffer generation count keeps a deferred acknowledgement from
clearing a completion that the guest has not read yet.

`VirtioDevice::supports_accelerated_doorbells` (true by default) lets a
device opt out of accelerated queue-notify and INTERRUPT_ACK doorbells.
Its notifications are then emulated by the transport.

`VirtioMmioDevice::new_with_disabled_features_and_interrupt_mode`
selects the mode. `new` and `new_with_disabled_features` keep the legacy
mode, so existing devices are unchanged. The MMIO constructors now
require a driver that can wait on the doorbell event on Linux.

Saving reads the effective status from the register or the shared word.
Restore resets the interrupt state, then applies the saved status to
the same place.

Tests cover the following:

- Used-buffer acknowledgement through the doorbell, with and without
  registered doorbells.
- An acknowledgement that races a new completion.
- Shared-status save, restore, and reset.
- Bit coalescing and zero-transition pulses, and a completion that
  races the guest's exchange.
- Teardown, and the doorbell opt-out.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
The fixed microVM virtio-mmio slots still deliver interrupts through the
legacy INTERRUPT_STATUS and INTERRUPT_ACK registers, so every virtio
interrupt costs the guest two exits.

Give every fixed slot a word on a shared interrupt-status page and
deliver its interrupts through `VirtioMmioInterruptMode::SharedStatus`:

- The page is at GPA 0x30000 and holds one naturally aligned 32-bit
  word per slot, in slot order from offset 0x00 (network) to 0x1c
  (control console). `microvm_virtio_status_gpa` maps a slot to its
  word. The reservation check now also requires every slot to have a
  distinct, ordered word inside the page.
  `MICROVM_VIRTIO_SANDBOX_BLOCK_MMIO_BASES` names the four sandbox-block
  slots.
- The worker creates each fixed-slot device in shared-status mode, and
  the MP-table loader reserves the page in the guest's e820 map so the
  guest never allocates it as RAM.

The guest must use the patched kernel's shared-status virtio-mmio
support. In this mode an INTERRUPT_ACK write does not clear the word,
so a stock driver would never see the zero-to-nonzero transition that
raises the next interrupt.

A test pins the word of every slot and rejects an unknown slot.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A guest can request a snapshot by writing to an I/O port. For the
snapshot to be exact, every VP must be stopped after that write has
been accepted but before the issuing VP resumes guest execution:
capturing earlier loses the request, and letting the VP run past the
write lets the guest change state that the snapshot claims to contain.

Add `PartitionUnit::temporarily_stop_vps_at_io_boundary`. The caller
passes a sender that releases the deferred I/O completion and a
receiver that reports that the completion has run. The partition unit:

1. Queues a stop event for every running VP, and marks the set stopped.
2. Releases the deferred I/O. Every VP has already observed its stop
   event, so the VP that issued the I/O cannot re-enter the guest.
3. Waits for the I/O completion, then for every VP to stop, even when
   the completion channel closed or a VP runner was dropped.

The returned `StopGuard` holds the VPs stopped. Failure is terminal:
the stop reference is kept, so a failed boundary cannot be restarted
before the partition is torn down.

`VpSet::stop` now marks the set stopped as soon as it has queued the
stop events. Dropping a stop mid-wait therefore no longer leaves the
set marked running, and a later stop does not queue duplicate stop
events.

The partition unit's snapshot request is no longer x86-64 only; the
TSC advance remains x86-64 only.

Tests cover a successful and a failed boundary, the ordering of the
I/O completion and the VP stops, a dropped VP runner, and a boundary
on already-stopped VPs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
A microVM guest requests a snapshot of itself by writing to port 0x605.
The snapshot must reflect the instant just after that write, so the
write cannot complete until the VMM has stopped every vCPU; the
issuing vCPU must not execute another guest instruction first.

Add `MicrovmSnapshotRequest`, a write-only port at 0x605. When it has a
notification target, a write:

- Defers the port write and sends a `MicrovmSnapshotBoundaryRequest`
  to the target. The request carries a sender that releases the write,
  a receiver that reports that the write has completed, and an RPC
  that the target completes once the snapshot transaction's outcome is
  final.
- Completes the deferred write when the target releases it, and
  signals that completion.
- Coalesces duplicate writes while a transaction is pending, with a
  rate-limited warning. A completed transaction is reaped before the
  next write, so the guest can request another snapshot even before
  the device's poll routine runs.

Without a notification target, writes complete immediately and are
ignored. Reads return all ones. Stop and reset fail a deferred write
that has not been released, and keep a released transaction pending
until it completes. The device has no saved state.

Add `MicrovmSnapshotRequestHandle`, its resolver, and the resolver's
registration for x86_64 guests. Nothing instantiates the device yet.

Tests cover coalescing until acknowledgement, reuse after a completed
or closed transaction, an early completion, the poll waker across
duplicate requests, stop and reset with and without a released write,
and the nonblocking behavior without a target.

Co-authored-by: Ruize Tang <1466040111@qq.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ef640993-0003-4e49-b1ed-1b8b4776d0ef
In debug builds, the ttrpc VM service overflowed the stack of its
worker thread on Linux as it created a microVM, so
test_ttrpc_microvm_linux_direct_lifecycle_and_snapshot and
test_ttrpc_microvm_device_snapshots failed in the x64 Linux vmm-tests
of the OpenVMM PR workflow, which builds debug, at feature/microvm/dev
73cf403 as with microsoft#5077. microvm_options, which
builds the options of a CreateVM request, parsed them with clap
through on_big_stack, which moves the parse to a thread with a 4 MiB
stack only on Windows, since other hosts parse the command line on the
main thread, whose stack is large. microvm_options runs on the ttrpc
worker, whose stack is 2 MiB, below the request's async call chain,
and in debug builds clap's construction of the Command for Options
needs more stack than is left there (clap-rs/clap#5134). Release
builds, which the push CI runs, were not affected.

microvm_options now always parses on a thread with a 4 MiB stack;
parse_options, which runs on the main thread, keeps the Windows-only
workaround.

Tests: with cargo xflowey vmm-tests-run, a debug build, on a Linux KVM
host, both tests fail with "thread 'worker-TtrpcWorker' has overflowed
its stack" at 73cf403 and at 67ac3e2, and pass with this change.
With cargo nextest on Windows x64, the 171 cli_args and ttrpc tests of
openvmm_entry pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the profile-owned ARM64 microVM boot, fixed device and platform-control integration using the existing loaders, device tree and shared device abstractions. Add governed ARM CPU/time admission with native WHP programming and readback, retaining the legacy fingerprint API and x86 behavior.

Keep the qualified PA36 contract and cold-boot-only boundary; ARM snapshot capture and restore remain unavailable.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extend the existing --x-time-abi-verify lifecycle to ARM64 using one shared TimeAbiReport and the existing native admission checks. Report the selected profile, effective register checks and actual WHP address, feature-bank and counter-frequency readbacks without executing the guest. Keep x86 output compatible, reject unsupported ARM fault-injection hooks, and fail closed on missing evidence. This does not establish UTC synchronization or clock authority.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the shared CPU/time report and sample-field conversion within the architecture-selected adapter. Restore native type and clock-status import ownership so target-branch clock changes merge without conflict, retaining one shared report and unchanged field conversion and runtime behavior. No target commits or time-policy changes are incorporated.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A restore of an RTC saved state whose clock_time_millis lies near
either end of i64, or a snapshot downtime that advances the RTC's time
there, overflowed the RTC's clock source. SystemTimeClock, the source
that OpenVMM gives the RTC, keeps the guest-visible time as an i64
millisecond offset from the host's SystemTime: set_time computes
new - now, and get_time now + offset. For such a time, the RTC's start
or its next sample of the clock overflows: debug builds abort, and
release builds, which do not check overflow, wrap the guest's time.
Fields::validate checked the saved state's flags and timers but not
its time, and advance_clock checked only that the advanced time fits
in an i64.

The RTC now holds the time that a restore or an advance sets to
CLOCK_TIME_MILLIS: the times that the guest can write to the calendar
registers, from 0000-01-01T00:00:00Z to 9999-12-30T22:00:00Z, the last
whole second that jiff represents in UTC, widened on both sides by
KTIME_SEC_MAX seconds, the range of the Linux host clock, over which
the guest's time runs with the host's. Fields::validate rejects a
saved time outside it with RestoreError::InvalidSavedState before the
restore changes any state, and advance_clock fails without changing
the time when the advanced time falls outside it. For a time in the
range and a host time in the Linux host clock's range, the offset and
the sum stay well within i64. The Guide's timekeeping reference
describes the bound.

Tests: four tests in cmos_rtc::tests::snapshot restore and advance
the RTC, held and running, on the mock clock and on the host's clock,
to times just outside the range, near both ends of i64, and at both
ends of the range. At feature/microvm/dev fb2c8b7, all four fail,
since the restore and the advance accept every such time; there, on
the host's clock, a restore of i64::MIN panics with "attempt to
subtract with overflow" at support/local_clock/src/lib.rs:236 as the
RTC starts, and an advance that leaves the time a second short of
i64::MAX panics with "attempt to add with overflow" at :255 as the RTC
saves a second later. With this change, the 131 chipset tests pass
with cargo nextest on Windows x64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A guest read of the IOAPIC's index or data register wider than 4
bytes, such as a 64-bit mov from 0xfec00000, panicked the VP thread
with "copy_from_slice: source slice length (4) does not match
destination slice length (8)", which ended OpenVMM and the VM.
IoApicDevice::mmio_read means to accept a read of any size, but it
copied the register's min(len, 4) low bytes into the whole buffer, so
only reads of at most 4 bytes matched the slice lengths.

mmio_read now copies those low bytes into the start of the buffer and
zeroes the rest, so a wider read returns the 32-bit register
zero-extended, as the RTC already does for a wide read of its ports,
and a narrower one still returns the register's low bytes.

Tests: the new test ioapic::tests::
mmio_reads_of_any_size_are_zero_extended reads the index and data
registers, for the ID, the version, and both halves of a redirection
entry, with every length from 1 to 8 bytes into a buffer of stale
bytes. At feature/microvm/dev 09cd771, it fails with the panic above
at vm/devices/chipset/src/ioapic/mod.rs:724:14. NVX f02bdcd53 pins
09cd771; there, an Alpine guest on 1 processor that maps /dev/mem at
0xfec00000 and reads both registers with 32-bit and then 64-bit loads
panics OpenVMM at the same line on its first 64-bit read, on KVM,
MSHV, and WHP. With this change, the same guest reads the registers'
32-bit values zero-extended on all three, on 1 and 2 processors, and
the 132 chipset tests pass with cargo nextest on Windows x64 and Linux
x64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
IoApicDevice::restore accepted any u8 as the saved IOAPIC id, but the
ID register's id field has 4 bits. A saved id above 15 restored
without error, and the next read of the ID register panicked with
"value out of bounds [0, 15]": read_register builds the register with
IoApicId::with_id, whose setter asserts that the id fits its field, in
release builds too. The device never saves such an id, since a write
to the ID register keeps only bits 27:24, so only a damaged or
crafted saved state reaches it. The restore appeared to succeed, and
the failure surfaced later, at the guest's first read of the register.

restore now rejects a saved id above IoApicId::MAX_ID, 15, with
RestoreError::InvalidSavedState before it changes any state or any
route, as it does for a wrong number of redirection entries or line
levels.

Tests: restore_rejects_an_id_wider_than_the_id_register restores the
ids 0x10, 0x80, and 0xff over a device whose id is 5, and checks that
each restore fails with InvalidSavedState and leaves the saved state,
the routes, and the ID register's value unchanged.
restore_accepts_every_id_that_the_guest_can_write writes each id from
0 to 15 to the ID register through MMIO, with the register's reserved
bits set, and checks that the saved state restores and that the
register reads back the id. At feature/microvm/dev e2458a6, the
first test fails, since the restore accepts every id, and restoring
the id 16 and then reading the ID register panics with "value out of
bounds [0, 15]" at vm/devices/chipset/src/ioapic/mod.rs:99:5. With
this change, the 134 chipset tests pass with cargo nextest on Windows
x64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…licy

A snapshot of a host profile (--cpu-profile host, or auto's fallback)
carries the only copy of the profile's document.
host_profile_for_restore accepted any document that decodes to a valid
host profile with the ID and digest that the snapshot recorded, and
whoever crafts a snapshot can record the digest of an edited document
and the effective CPUID that it gives. Nothing compared the document
with the derivation's policy, which clears the features that a VM must
not see, so a document with CPUID.1:ECX[5], VMX, or
CPUID.0x80000001:ECX[2], SVM, restored: the effective CPUID, the
backend's support check, and VP 0's CPUID all follow the document. On
KVM, where a time ABI partition's CPUID is the profile's effective
CPUID, the VMX or SVM strip that new_partition adds for !nested_virt
does not reach it, so the guest could enable VMX or SVM and reach KVM's
nested-virtualization code, host attack surface that nested_virt false
keeps closed.

host_profile_for_restore now also refuses a host profile that the policy
would change (E_PROFILE_DIGEST), naming at most eight of the differences
between the document and the policy. Both the controller's restore
preflight and the VM worker's profile selection call it, so a crafted
snapshot fails before the worker starts, and again if a worker is ever
handed one. The check is derive::policy_violations. It applies
apply_policy and clear_stateless_features, the derivation's own rules,
to the profile's CPUID table, with the profile's XCR0, and compares the
result with the table; and it checks the XSAVE features against
ALLOWED_XCR0, ALLOWED_XSS, and ALLOWED_XSAVE_EXTENSIONS. A profile that
the derivation produced is a fixed point of its policy, which only sets
and clears bits and zeroes leaves, so the check accepts it, and a rule
that the derivation gains is checked without a second copy of it. What
the derivation fixes besides features, the XSAVE layout, the pinned
MSRs, and the physical address width, stays with the support check of
the destination backend, which bounds a profile by what the backend
presents. The derivation itself is unchanged, and so are the pinned
profiles and POLICY, the label that provenance records, since no profile
or rule changes; pinned profiles are constants that OpenVMM verifies,
and need no check.

No version is created or bumped, and restore does not read the policy
label that a profile's provenance names: it applies the rules of its own
build. A host profile that an OpenVMM with another policy derived, if
this policy would change it, therefore no longer restores, and the
snapshot needs a recapture, which the error says. Host profiles are for
development and not pinned, and a build that computes another effective
CPUID already rejects their snapshots (E_CPU_SURFACE). The Guide says
so.

Tests: restore_refuses_a_host_profile_outside_the_derivation_policy
restores an Intel and an AMD host profile with VMX, and AMD's with SVM,
each recorded with its own digest, and
restore_names_only_the_first_violations_of_the_policy bounds the error.
The derive tests check that every pinned profile, the host profile of
every recorded host, and the host profile of random mutations of each
host's CPUID follow the policy, so that no snapshot that this OpenVMM
captured is refused (a run of 18,000 mutations found none); that each
feature that the policy withholds, as the Intel SDM and the AMD APM
define it, is a violation alone; and that the policy's other parts are
checked. The restore preflight's and the worker's selection's tests
restore a crafted VMX profile with the effective CPUID that it gives. At
feature/microvm/dev efe6bc2, the three tests of the restore fail,
since each restore succeeds; with this change, the 132 cpu_profile, 138
openvmm_helpers, and 130 openvmm_core tests pass with cargo nextest on
Windows x64, and the 132 cpu_profile and 276 openvmm_helpers and
openvmm_core tests on Linux x64.

On KVM, an Azure VM with an Intel Xeon Platinum 8573C on Linux 6.18, a
script cold boots a microVM with --cpu-profile host, captures its
snapshot, and restores a copy whose host profile has VMX set, recorded
with the edited document's own digest and the effective CPUID that it
gives. At efe6bc2, the release build that CI published restores it,
and the guest reads IA32_VMX_BASIC, 0x0098100011e57ed0, through
/dev/cpu/0/msr without a #GP, which the unedited snapshot's guest
cannot. With this change, the restore fails with [E_PROFILE_DIGEST],
naming CPUID 0x1 ECX bit 5. On MSHV and WHP, Azure VMs too, efe6bc2
refuses the edited snapshot with E_PROFILE_UNSUPPORTED, since their
partitions lack VMX, and this change with E_PROFILE_DIGEST. The
guest-boot, cpu-profile-fallback, and snapshot-core scenarios of NVX,
which capture and restore a real host profile, pass with this change on
KVM, MSHV, and WHP.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
XsaveCapabilities::standard_len_for and compact_len_for added and
aligned u32 lengths without checking them, and
X86PartitionCapabilities::from_cpuid calls both on the components that
its CPUID describes. A host's CPUID never describes components that
large, but a restored CPU profile can: it is the snapshot's own
document, whose leaf 0xd only has to agree with itself, and the KVM
backend builds a time ABI partition's capabilities from it, before the
profile support check runs. A component that ends past 2^32 bytes,
components whose compact places add up past 2^32, or an alignment past
the last multiple of 64 panicked a debug build with "attempt to add
with overflow", and wrapped a release build to a length that no
component fits in.

Both lengths now saturate at u32::MAX, and so does rounding a length up
to a multiple of 64 past 0xffffffc0, since masking the rounded sum
would bring a saturated length back below u32::MAX. A length that fits
is unchanged.

Tests: xsave_lengths_saturate_at_the_largest_length checks that a
length of 2^32 - 1 fits and one of 2^32 saturates, for the standard end
and for the compact sum, and each bound of the alignment.
from_cpuid_saturates_xsave_lengths_that_overflow checks the three
overflows through from_cpuid, and that it still compares the saturated
standard length with the one that the CPU advertises. With cargo
nextest run --profile agent -p virt on Windows x64, both fail at
feature/microvm/dev efe6bc2 with "attempt to add with overflow" at
vmm_core/virt/src/x86/mod.rs:327:27, and pass with this change, as do
the other 103 tests that run. On nvx-kvm-test, in a debug build, a KVM
time ABI partition whose CPUID has component 2 end at 2^32, or
components 5 and 6 of 2^31 bytes each, panics the same way when it is
built at efe6bc2, and builds with this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
KvmProtoPartition::build derives the partition's capabilities from its
CPUID, which for a time ABI partition is the CPU profile's effective
CPUID, and a restore takes that profile from the snapshot. This backend
converts a VP's XSAVE state between the compact area of
virt::x86::vp::Xsave, which it builds from compact_len in whole words,
and the 4096 bytes of the buffer of KVM_GET_XSAVE and KVM_SET_XSAVE. A
profile can describe a layout that this conversion cannot hold: a
component that ends past that buffer, or a compact length that is not
in whole words, can make it panic, and a saturated compact length is
the 4 GiB that Xsave::at_reset would allocate. Only the profile support
check, which the worker runs after the build, refuses such a layout
(E_PROFILE_UNSUPPORTED), since it requires each component to match
KVM's own, so until then the partition holds capabilities that its own
VP state code cannot handle.

partition_capabilities now derives the capabilities for the build, and
check_xsave_layout refuses them with XSaveLengthTooLarge, new, for a
standard length past KVM_XSAVE_LEN, the size of kvm_xsave, and with
XSaveLengthTooSmall for a compact length that is short of its
components or not in whole words. A host's own CPUID passes, and so
does the layout of every pinned profile. A refused build fails with
KvmError::Capabilities before any VP binds. That error did not have the
PartitionCapabilitiesError as its source, so its message, "host does
not support required cpu capabilities", named no reason; it does now.

Tests: partitions_hold_the_xsave_layouts_of_the_pinned_profiles builds
the capabilities of the effective CPUID of every pinned profile,
partitions_hold_the_xsave_layouts_that_cpus_describe those of a CPU
without components and of one whose component ends at the end of KVM's
buffer, and partitions_refuse_the_xsave_layouts_that_kvm_cannot_hold
the three refusals, with and without the time ABI.
xsave_layout_needs_a_compact_length_that_holds_its_components checks
each bound of the compact length. On nvx-kvm-test (Linux 6.18.57,
nested KVM on an Intel Xeon Platinum 8573C), the unit tests of virt and
virt_kvm pass with cargo nextest, and so do the /dev/kvm tests of
virt_kvm and kvm with --run-ignored all. In a debug build there, a KVM
time ABI partition built from the CPUID of the Skylake-SP profile
builds, and with component 2 ending at 2^32, or components 5 and 6 of
2^31 bytes each, the build fails with the XSaveLengthTooLarge errors
for 4294967295 and 2147484800 bytes. At efe6bc2 both panic with
"attempt to add with overflow", and with only the previous commit both
build and leave the refusal to the later profile support check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Admit the two observed N1X CPU part numbers through a pinned ARM64
profile while retaining the existing WHP feature and state checks.
A single-part profile fails when a retained VP exposes the other
core type on this heterogeneous host.

Validate the configured ID-register surface on all 20 logical CPUs
of a MECHREVO N1X Spark with NVIDIA N1X, and document the hardware
observations and cold-boot qualification limits. Test both MIDRs,
exclude every other part number, and verify the WHP feature-bank
translation and unavailable-feature rejection.

Depends on the ARM cold-boot implementation in microsoft#5045.
@github-actions github-actions Bot added Guide release-nvx Targets a branch under feature/microvm/. unsafe Related to unsafe code labels Nov 14, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Unsafe Code Detected

This PR modifies files containing unsafe Rust code. Extra scrutiny is required during review.

For more on why we check whole files, instead of just diffs, check out the Rustonomicon

@QThans

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="MECHREVO"

@QThans

Copy link
Copy Markdown
Author

Follow-up on the unsafe-code review notice

I checked the Unsafe Check run and the commit boundaries. The workflow completed successfully; its warning marks files for additional review rather than reporting a failed safety test.

The workflow scanned 43 Rust files in the full dependent PR and reported:

  • tmk/tmk_vmm/src/host_vmm.rs
  • vm/whp/src/lib.rs

Both files are inherited from the prerequisite ARM implementation in #5045. They are unchanged between b09c58843ae310dfff9ae68e9e822f9c5846b914 and the N1X adaptation commit 2ff56a61b.

For context, the prerequisite's changes in these two files remove the x86-only conditional on time_abi: None and add two property-read wrappers using the existing get_property abstraction. The workflow examines whole file contents, so existing unsafe code elsewhere in these files triggers the notice even though these particular hunks introduce no unsafe blocks.

The N1X-specific commit adds the CPU profile, catalog registration, focused tests and hardware notes. It introduces no unsafe blocks or unsafe functions, and does not change the existing WHP FFI implementation or bypass CPU feature validation.

I am retaining the unsafe label: the full prerequisite stack still requires the normal maintainer safety review. This explanation is not a safety certification or a request to waive review. After #5045 lands, this draft needs to be rebased and revalidated so the PR diff contains only the N1X contribution.

Separately, the corporate CLA declaration has been submitted with company="MECHREVO", and the license/cla check now passes.

Thans (QThans) and others added 2 commits November 14, 2026 18:17
Pin the expected canonical digest and model mask, reject a different implementer or architecture, and use the measured 39-bit native WHP GPA limit in the backend test fixture. The guest address contract remains 36 bits. This changes tests only.
@QThans

Copy link
Copy Markdown
Author

Following Pedro's direction in #5045 (comment), the N1X-only follow-up is now open at ahuja-gautam#1, targeting openvmm-arm64-microvm-dev-20261111.

The new draft retains [DO NOT REVIEW] and contains exactly the two existing N1X commits (2ff56a61b, 229ecb91b) and four changed files on b09c58843. It excludes the later upstream merge and introduces no additional code changes.

This original PR remains open as a draft for continuity and tracking; its branch history has not been rewritten. Please review the focused follow-up rather than the prerequisite-inclusive diff here.

@QThans Thans (QThans) changed the title cpu_profile: add NVIDIA N1X WHP profile [DO NOT REVIEW] cpu_profile: add NVIDIA N1X WHP profile Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Guide release-nvx Targets a branch under feature/microvm/. unsafe Related to unsafe code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants