Repository navigation
[DO NOT REVIEW] virtio: keep a held input gate across a device reset - #5022
Merged
Pedro Henrique Penna (ppenna) merged 1 commit intoNov 10, 2026
Conversation
A snapshot boundary gates host input before it stops the vCPUs, and a gated microVM restore keeps host input gated while the vCPUs run the guest's repair, until the guest acknowledges the restore. While the gate was held, the guest could reset a virtio device by writing 0 to its device status, and the device's reset cleared its own gate: virtio-net cleared the latch under which its endpoint queues start, the console its input_gated flag, and virtio-fs reopened its request admission. Nothing gated the device again, so once the guest set DRIVER_OK again, the device delivered host packets, console input, or virtio-fs replies that the gate held back, while the VM worker still recorded host input as gated (microsoft/nvx#1416). In a capture, the reopened virtio-fs admission also failed the save, which rolled the capture back. A reset now keeps the gate, and only resume_input releases it. The gate is the VM worker's: the worker takes it, and releases it when it releases the snapshot boundary or the guest acknowledges the restore, and no reset changes the worker's record of it. So a reset of the VM keeps the gate too, as does the reset that follows a failed DRIVER_OK. Each device's reset keeps its gate. virtio-net keeps the latch, from which the coordinator that the next start of the queues creates gates its new endpoint queues, as it already does for a gate taken before the queues first start. The console keeps input_gated, so its worker reads no host input and its broker does not time the host's authentication. virtio-fs leaves admission closed, so the guest's requests stay on their virtqueues. VirtioDevice::reset and quiesce_input, ChangeDeviceState::quiesce_input and reset, and the snapshot guide state it. Tests: on each device, while input is gated, a reset made as the transport makes the guest's, and a new start of the queues: the console's direct worker and its broker read no host input, the host's attachment included, until input resumes; virtio-net quiesces the endpoint queues that the new start creates; and virtio-fs leaves a FUSE_INIT on its virtqueue until input resumes. Through the virtio-mmio transport: a guest that writes 0 to the status of the running console and starts it again gets no host input until input resumes, and after a status write of 0 to an inactive virtio-fs device, or after a reset of the VM, while input is gated, the save that follows succeeds. Without the fix, all seven fail: host input reaches the guest, the new endpoint queue is never quiesced, the FUSE_INIT is answered, and the save fails with "virtio-fs save was not preceded by input quiesce". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
requested review from
a team and
Anatol Belski (weltling)
as code owners
November 10, 2026 17:22
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
Contributor
Author
|
This pull request adds no |
Pedro Henrique Penna (ppenna)
merged commit Nov 10, 2026
61cdd63
into
feature/microvm/dev
73 of 74 checks passed
syeedali
pushed a commit
to syeedali/nvx
that referenced
this pull request
Nov 11, 2026
A guest that reset a virtio device while OpenVMM held host input gated let host input past the gate (microsoft#1416): between a snapshot boundary's input gate and the vCPU stop, or during a gated restore before the guest acknowledged it, the device's reset cleared its gate, and once the guest set DRIVER_OK again, the device delivered the host input that the gate held back, virtio-net's packets, a console's input, and virtio-fs's replies, while the VM worker still recorded input as gated. A capture whose virtio-fs device the guest reset failed its save and rolled back. Pin 61cdd63978ea58ce045d159208ed4ca22f30cbb8, the head of microsoft/openvmm#5022, one commit on cd5cd3814, the head of feature/microvm/dev: - 61cdd6397 virtio: keep a held input gate across a device reset The revision makes each device's reset keep its input gate, so that only resuming input releases it, also for a reset of the VM: the VM worker holds the gate and releases it when it releases the boundary or the guest acknowledges the restore, and no reset changes its record. Evidence: the platform tier of the snapshot-tiers scenario, whose restore gates host input until the guest acknowledges it, with a runtime post-restore hook, which runs before the acknowledgement, that resets the guest's virtio-net device by unbinding and binding its driver, configures it, and pings the gateway, whose reply is host input. The reply arrived before the acknowledgement in every run on nvx-kvm-test (KVM), nvx-mshv-test (MSHV), and nvx-whp-test (WHP), all virtual machines, with the OpenVMM of the releases that CI built from 46bfe02ed, 4 of 4 runs each, and from a8a0e3517, which differs from cd5cd3814 only in tests, 2 of 2 runs each. With OpenVMM built on each host from 61cdd6397, it arrived in none of 3 runs each, every run passed the scenario, whose host input reached the guest only after the acknowledgement, and the affected microVM scenarios passed. At the pinned revision, the OpenVMM tests of the changed packages pass, 515 on Windows and 520 on Linux; without the fix, the 7 new ones fail. NVX needs no coordinated change: the control-session vectors, the source excerpts, the contract versions, and the CPU profiles that nvx-rs copies from OpenVMM match the revision's. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
syeedali
pushed a commit
to syeedali/nvx
that referenced
this pull request
Nov 11, 2026
Maps NVX's new pin, 61cdd6397, the head of microsoft/openvmm#5022, at position 247, after position 246, cd5cd3814, the head of feature/microvm/dev, which microsoft/openvmm#5016 merged. The pin is 116 commits past the mapped tip. Position 247 is a PR13 fixup. It changes the input gates that PR13's positions 043 and 048 and PR18's position 088 give the console, virtio-net, and virtio-fs with their saved state: each device's reset cleared its gate, and now keeps it. It states that rule on the gate's methods of VirtioDevice and ChangeDeviceState, which PR13's position 040 and PR09's position 018 add, extends the snapshot guide's paragraph on the post-restore gate that PR20's position 095 adds, and adds tests beside those of the devices, which extend the broker's test harness of PR23's position 113 and the gated-queue tests of position 172. It folds into positions 043 and 048, its virtio-fs change into PR18's position 088, and its other parts into the positions that add what they change, as the fold sentences after the DAG and the commit mapping say; the Scope, the DAG, the dependency graph, the PR summary, and the commit mapping record it with PR13. Replaying it on upstream main conflicts in 13 of its 15 files, all but a new module of console tests and the development dependency that virtio-fs takes for its transport tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
syeedali
pushed a commit
to syeedali/nvx
that referenced
this pull request
Nov 11, 2026
Maps NVX's new pin, 58d3c3b1d, the head of microsoft/openvmm#5026, at position 248, after position 247, 61cdd6397, the head of feature/microvm/dev, which microsoft/openvmm#5022 merged. The pin is 117 commits past the mapped tip. Position 248 is a PR18 fixup. It checks the prefix of the network identity that PR18's position 083 adds, in validate_machine_config, which PR15's position 062 adds, before the command-line check of position 083 derives the identity's netmask, and gives that position's parser the same range. It folds into position 083, and its test's virtio-fs device into position 202, which reserves the virtio-fs slot on every command line, as the fold sentence after the DAG and the commit mapping say; the Scope, the DAG, the dependency graph, the PR summary, and the commit mapping record it with PR18. Replaying it on upstream main conflicts in its only file: upstream main lacks the microVM module of PR15's position 055. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Keeps the host-input gate that the VM worker holds across a guest's reset of a virtio device, for microsoft/nvx#1416. A guest that reset a virtio device while OpenVMM held host input gated, between a snapshot boundary's input gate and the vCPU stop, or during a gated restore before the guest acknowledged it, cleared the device's gate: once the guest set
DRIVER_OKagain, the device delivered host packets, console input, and virtio-fs replies that the gate held back, while the VM worker still recorded input as gated, and a capture whose virtio-fs device the guest reset failed its save and rolled back.One commit on
cd5cd3814, the head offeature/microvm/dev. microsoft/nvx#1452 pins this head.Keep a held input gate across a device reset (
61cdd6397)VirtioDevice::resetreturned to its initial value: virtio-net's latch under which its endpoint queues start, the console'sinput_gated, and virtio-fs's request admission, which its reset reopened. The transport resets the device when the guest writes 0 to its status (Disablefor aDRIVER_OKdevice,Resetfor another), whenDRIVER_OKfails, and onChangeDeviceState::reset, so a reset, which the guest controls, opened the gate.resume_inputreleases it. virtio-net keeps the latch, from which the coordinator that the next start of the queues creates gates its new endpoint queues, as it already does for a gate taken before the queues first start; the console keepsinput_gated, so its direct worker reads no host input and its broker neither reads the host's records nor times its authentication; virtio-fs leaves admission closed, so the guest's requests stay on their virtqueues.VirtioDevice::resetandquiesce_input,ChangeDeviceState::quiesce_inputandreset, and the snapshot guide state the rule.FUSE_INITon its virtqueue until input resumes. Through the virtio-mmio transport: a guest that writes 0 to the status of the running console and starts it again receives no host input until input resumes; after a status write of 0 to an inactive virtio-fs device, or a reset of the VM, the save succeeds. Without the fix, all seven fail: host input reaches the guest, the new endpoint queue is never quiesced, theFUSE_INITis answered, and the save fails with "virtio-fs save was not preceded by input quiesce".Validation
At this head,
61cdd6397, with Rust 1.97.1. It is the commit that NVX's phase-1 validation tested,7870c09e2on46bfe02ed, rebased ontocd5cd3814:git range-diffshows the patch identical, and the commits from46bfe02edtocd5cd3814change none of its files but the snapshot guide, in another section.cargo clippy --all-targetsandcargo doc --no-depsforvmcore,virtio,virtio_net,virtio_console, andvirtiofs, without warnings;cargo nextest run --profile agentfor the five, 515 passed and 2 skipped;cargo xtask fmt, clean.46bfe02edand froma8a0e3517, which differs from this base only in tests, the reply, which is host input, arrived before the acknowledgement in every run, 4 and 2 runs on each backend; with OpenVMM built from7870c09e2and from this head, in none of 3 runs each, and NVX's affected microVM scenarios, the snapshot, restore, network, console, and filesystem ones among them, passed on all three. See openvmm: keep a held input gate across a guest's device reset nvx#1452 for the table.