Repository navigation
[DO NOT REVIEW] virt_kvm: reject a restored APIC timer whose delivery mode is not Fixed - #5102
Merged
Pedro Henrique Penna (ppenna) merged 1 commit intoNov 14, 2026
Conversation
KvmVpStateAccess::set_apic gave KVM_SET_LAPIC the saved LAPIC page as it was. KVM starts the timer again from the page's counts and delivers an expiry through the delivery mode of the LVT timer entry, in kvm_apic_local_deliver. A page whose entry is unmasked and has a mode other than Fixed therefore makes KVM interrupt the restored VP in that mode once the timer has expired, as a one-shot timer whose current count is 0 has: inside KVM_SET_LAPIC when APICv is active, and at the next KVM_RUN otherwise. Depending on the mode, the VP takes an INIT, a startup IPI, an NMI, or an SMI, or wakes from halt (remote read), none of which the saved state holds. The restore appeared to succeed, since restore_all does not read KVM's LAPIC state back: Apic::can_compare is false while the backend cannot freeze time. A guest cannot set the entry's delivery mode, because KVM masks its writes to the entry (apic_lvt_mask in kvm_lapic_reg_write), so a save never holds such a page. Only a damaged or crafted saved state does. set_apic now rejects such a page with KvmError::InvalidState as its first step, before it forgets the timer that the VP stopped with or changes the APIC base or the page, so a rejected restore leaves the APIC as it was. It refuses every unmasked entry with a mode other than Fixed, whatever the timer's mode, lowest priority and ExtINT included, which KVM delivers as a fixed interrupt or ignores, since no save holds them either. A masked entry is accepted: it delivers nothing, and a guest write that unmasks it rewrites its delivery mode too. Every page that OpenVMM saves restores as before, and so does the reset page, whose timer is masked. The Guide's KVM timekeeping section describes the check. Tests: an_unmasked_timer_that_is_not_fixed_is_refused checks the refusal for the delivery modes 1 to 7 of a one-shot, a periodic, and a TSC-deadline timer, and that a masked entry, the Fixed mode, and the reset value are accepted. The backend contract test restore_refuses_a_non_fixed_apic_timer, which only KVM owes and which registers with the `kvm_only` form of backend_test!, sets an expired one-shot timer in each of those modes on a real KVM VP, and requires that every restore is refused and leaves the APIC state unchanged, and that an expired Fixed timer and masked timers in every mode restore. The Guide's page of the backend contract tests lists the refusal among the contracts that only KVM owes. At feature/microvm/dev b723cbe, on nvx-kvm-test (Linux 6.18.57, nested KVM on an Intel Xeon Platinum 8573C without APICv), the contract test fails with "the restore accepted an expired timer in the delivery modes [1, 2, 3, 4, 5, 6, 7]". A scratch test that gave KVM_SET_LAPIC the same page and ran a real-mode BSP that spins with interrupts enabled showed what KVM then does at KVM_RUN: INIT resets the VP (an emulation failure at rip 0xfff0), NMI injects an NMI, SMI makes KVM_RUN fail and every later ioctl on the VM return EIO, and lowest priority takes the vector as a fixed interrupt; remote read, startup IPI, and ExtINT have no visible effect on a running BSP. With this change both tests pass. Mutants of the check fail them: refusing nothing, or refusing a masked entry too, fails both; accepting lowest priority and ExtINT fails the unit test; and a set_apic that ignores the refusal fails the contract test. On nvx-kvm-test, cargo check, clippy --all-targets (gnu and musl), doc, and nextest pass for virt_kvm, as do clippy and doc for vmm_tests, all 84 tests of virt_kvm and kvm with those that need /dev/kvm, and the two contract tests that KVM runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
requested a review
from a team
as a code owner
November 14, 2026 16:32
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Nov 14, 2026
Restoring a VP whose saved APIC page held an unmasked LVT timer entry with a delivery mode other than Fixed succeeded on KVM, and KVM then fired the timer in that mode once it had expired, as a one-shot timer whose current count is 0 has: inside KVM_SET_LAPIC when APICv is active, and at the next KVM_RUN otherwise. The restored VP took an INIT, a startup IPI, an NMI, or an SMI, or woke from halt, none of which the snapshot holds (#1365). Pin 30b7554069182a5262099f902c4a203d54c8bb88, the head of microsoft/openvmm#5102, one commit on b723cbed3, the head of feature/microvm/dev: - 30b755406 virt_kvm: reject a restored APIC timer whose delivery mode is not Fixed set_apic now rejects such a page with KvmError::InvalidState as its first step, before it forgets the timer that the VP stopped with or changes the APIC base or the page, so the restore fails with the apic element's StateError, and set_apic itself leaves the VP's APIC as it was. The elements that restore_all restores before the APIC, the registers among them, have run: KVM's write of CR8 can change the LAPIC's task priority. It refuses every unmasked entry with a mode other than Fixed, lowest priority and ExtINT included, and accepts a masked entry. A guest cannot write the entry's delivery mode, since KVM masks its writes, so a save never holds such a page, a snapshot that a VP saved restores as before, and only a crafted snapshot is refused. MSHV and WHP are unchanged. No version is created or bumped, and no snapshot or wire format changes. Tests: on nvx-kvm-test (Intel Xeon Platinum 8573C, Linux 6.18.57, nested KVM without APICv), the revision's contract test restore_refuses_a_non_fixed_apic_timer fails at b723cbed3 with "the restore accepted an expired timer in the delivery modes [1, 2, 3, 4, 5, 6, 7]" and passes at the pin, as does its unit test an_unmasked_timer_that_is_not_fixed_is_refused. A scratch test that gave KVM_SET_LAPIC the same page showed what KVM does with it at KVM_RUN: INIT resets the VP, NMI injects an NMI, SMI kills the VM, and lowest priority is taken as a fixed interrupt. Mutants of the check fail the tests. At the pin, the 84 tests of virt_kvm and kvm pass with cargo nextest, the ignored ones included. With OpenVMM built from the pin's tree, which replaced the packaged binary (SHA-256 193d1063 to 04915a93), and the artifacts of v0.1.0-dev.9e41ebd10be8, the microVM scenarios guest-boot (1 processor), interrupt-map, smp, smp-snapshot, snapshot-core, snapshot-rollback, restore-downtime, restore-processors, snapshot-tiers, time-abi-conformance, and time-host-correction pass at 1 and 2 processors, and smp-lapic at 1, 2, 4, and 8, on nvx-kvm-test. The change is KVM-only, so nvx-mshv-test and nvx-whp-test were not run. NVX needs no coordinated change: the control-session vectors, the source excerpts, the contract versions, the activation bound, and the CPU profiles that nvx-rs copies from OpenVMM match the revision's. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Nov 14, 2026
The NVX pin moves to 30b755406, position 282, 158 commits past the mapped tip. Position 282, the one commit of microsoft/openvmm#5102 on b723cbed3, the head of feature/microvm/dev, is not yet assigned to a PR. Position 282 makes the KVM backend's set_apic reject a LAPIC page whose unmasked timer entry has a delivery mode other than Fixed, which KVM would fire as an INIT, a startup IPI, an NMI, or an SMI, or as a wake of a halted VP, none of which the saved state holds, in code that upstream main has. It adds check_restorable_apic_timer, which set_apic calls first, a unit test, a KVM-only contract test of virt_backend_tests in a module of its own, and a paragraph and a sentence of the Guide. It changes files that PR01's position 002, PR05's positions 007 and 008, PR10's positions 022, 271, and 273, and positions 140, 248, 250, 251, 253, 279, and 281 also change, and adds to no line that a mapped PR adds. Replaying it on upstream main, 7d90c4104, conflicts in vp_state.rs, at the start of set_apic, whose timer reset position 253 adds, and from the end of the AccessVpState implementation on, which positions 250, 251, 253, and 279 add; in the Guide's KVM timekeeping section, which positions 250 and 253 extend; in the Guide's page of the contract tests, whose kvm_only sentence is position 279's; and in the contract tests' main.rs, beside the attribute that position 008 drops. So it follows positions 008, 250, 251, 253, and 279 upstream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
merged commit Nov 14, 2026
30b7554
into
feature/microvm/dev
73 of 74 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Keeps a restore from giving KVM an APIC timer that delivers in a mode other than Fixed, for microsoft/nvx#1365.
One commit on
b723cbed3, the head offeature/microvm/dev. microsoft/nvx#1590 pins it for microsoft/nvx#1365; this pull request lands first, by fast-forwardingfeature/microvm/devto30b755406.Reject a restored APIC timer whose delivery mode is not Fixed (
30b755406)KvmVpStateAccess::set_apicgaveKVM_SET_LAPICthe saved LAPIC page unchanged. KVM starts the timer again from the page's counts and delivers each expiry through the delivery mode of the LVT timer entry (kvm_apic_local_deliver). A timer that has expired, as a one-shot timer whose current count is 0 has, fires at once: insideKVM_SET_LAPICwhen APICv is active, and at the nextKVM_RUNotherwise. A page whose entry is unmasked and has a mode other than Fixed therefore gives the restored VP an INIT, a startup IPI, an NMI, or an SMI, or wakes it from halt (remote read), none of which the saved state holds. The restore appears to succeed, sincerestore_alldoes not read KVM's LAPIC state back:Apic::can_compareis false while the backend cannot freeze time. A guest cannot set the entry's delivery mode, because KVM masks its writes to the entry (apic_lvt_maskinkvm_lapic_reg_write), so a save never holds such a page, and only a damaged or crafted saved state does. MSHV and WHP hand the page to their hypervisor, whose APIC is not OpenVMM's, and are unchanged and not examined.set_apiccallscheck_restorable_apic_timeras its first step, before it forgets the timer that the VP stopped with or changes the APIC base or the page. The check fails withKvmError::InvalidStatefor an unmasked entry whose delivery mode is not Fixed, sorestore_allfails with theStateErrorof the apic element, as it does for a pending ExtINT, andset_apicleaves the VP's APIC as it was. The elements thatrestore_allrestores before the APIC, the registers among them, have run: KVM's write of CR8 can change the LAPIC's task priority. It refuses every such mode, whatever the timer's mode and whether or not the timer has expired, lowest priority and ExtINT included, which KVM delivers as a fixed interrupt or ignores, since no save holds them either.an_unmasked_timer_that_is_not_fixed_is_refusedchecks the refusal for the delivery modes 1 to 7 of a one-shot, a periodic, and a TSC-deadline timer, and that a masked entry, the Fixed mode, and the reset value are accepted. The backend contract testapic::kvm::restore_refuses_a_non_fixed_apic_timerregisters with thekvm_onlyform ofbackend_test!that [DO NOT REVIEW] virt_kvm: refuse a restored EFER.SVME for a guest whose CPUID has no SVM #5097 added, sets an expired one-shot timer in each of those modes on a real KVM VP through the publicvirtinterfaces, and requires thatset_apicrefuses each one and leaves the APIC state unchanged, and that it accepts an expired Fixed timer and masked timers in every mode. Thevirt_backend_testsexecutable runs inrun vmm-tests [x64-linux-amd-kvm], whereas no CI job runs the ignored/dev/kvmtests ofvirt_kvm.No version is created or bumped, and no snapshot, saved-state, or wire format changes.
Validation
With Rust 1.97.1 and cargo-nextest 0.9.133, on a KVM virtual machine (Intel Xeon Platinum 8573C, Linux 6.18.57.1-1.azl3, nested KVM without APICv or the VMX preemption timer):
b723cbed3with only the new test files,cargo nextest run -p vmm_tests --test virt_backend_tests -E 'test(/restore_refuses/)'fails with "the restore accepted an expired timer in the delivery modes [1, 2, 3, 4, 5, 6, 7]"; with this change it passes. A scratch test, which is not part of this change, gaveKVM_SET_LAPICthe same page for each delivery mode and ran a real-mode BSP that spins with interrupts enabled. INIT resets the VP (an emulation failure at rip0xfff0), NMI injects an NMI, SMI makesKVM_RUNfail and every later ioctl on the VM return EIO, and lowest priority takes the vector as a fixed interrupt; remote read, startup IPI, and ExtINT have no visible effect on a running BSP. The host has no APICv, so delivery inside the ioctl was not observed.cargo check,cargo clippy --all-targetsfor the gnu and musl targets, andcargo doc --no-depsonvirt_kvmhave no warnings, andcargo clippy --all-targetsandcargo doc --no-depsonvmm_testshave none.cargo nextest run --profile agent --run-ignored all -p virt_kvm -p kvmpasses 84 tests.cargo nextest run -p vmm_tests --test virt_backend_testsruns the two contract tests that KVM runs, and both pass.cargo xtask fmt --fix, run on Windows, changes nothing.set_apicthat ignores the refusal fails the contract test.vmm_testsare left to this pull request's CI.