From 30b7554069182a5262099f902c4a203d54c8bb88 Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Fri, 13 Nov 2026 20:06:29 -0800 Subject: [PATCH] virt_kvm: reject a restored APIC timer whose delivery mode is not Fixed KvmVpStateAccess::set_apic gave KVM_SET_LAPIC the saved LAPIC page as it was. KVM starts the timer again from the page's counts and delivers an expiry through the delivery mode of the LVT timer entry, in kvm_apic_local_deliver. A page whose entry is unmasked and has a mode other than Fixed therefore makes KVM interrupt the restored VP in that mode once the timer has expired, as a one-shot timer whose current count is 0 has: inside KVM_SET_LAPIC when APICv is active, and at the next KVM_RUN otherwise. Depending on the mode, the VP takes an INIT, a startup IPI, an NMI, or an SMI, or wakes from halt (remote read), none of which the saved state holds. The restore appeared to succeed, since restore_all does not read KVM's LAPIC state back: Apic::can_compare is false while the backend cannot freeze time. A guest cannot set the entry's delivery mode, because KVM masks its writes to the entry (apic_lvt_mask in kvm_lapic_reg_write), so a save never holds such a page. Only a damaged or crafted saved state does. set_apic now rejects such a page with KvmError::InvalidState as its first step, before it forgets the timer that the VP stopped with or changes the APIC base or the page, so a rejected restore leaves the APIC as it was. It refuses every unmasked entry with a mode other than Fixed, whatever the timer's mode, lowest priority and ExtINT included, which KVM delivers as a fixed interrupt or ignores, since no save holds them either. A masked entry is accepted: it delivers nothing, and a guest write that unmasks it rewrites its delivery mode too. Every page that OpenVMM saves restores as before, and so does the reset page, whose timer is masked. The Guide's KVM timekeeping section describes the check. Tests: an_unmasked_timer_that_is_not_fixed_is_refused checks the refusal for the delivery modes 1 to 7 of a one-shot, a periodic, and a TSC-deadline timer, and that a masked entry, the Fixed mode, and the reset value are accepted. The backend contract test restore_refuses_a_non_fixed_apic_timer, which only KVM owes and which registers with the `kvm_only` form of backend_test!, sets an expired one-shot timer in each of those modes on a real KVM VP, and requires that every restore is refused and leaves the APIC state unchanged, and that an expired Fixed timer and masked timers in every mode restore. The Guide's page of the backend contract tests lists the refusal among the contracts that only KVM owes. At feature/microvm/dev b723cbed3, on nvx-kvm-test (Linux 6.18.57, nested KVM on an Intel Xeon Platinum 8573C without APICv), the contract test fails with "the restore accepted an expired timer in the delivery modes [1, 2, 3, 4, 5, 6, 7]". A scratch test that gave KVM_SET_LAPIC the same page and ran a real-mode BSP that spins with interrupts enabled showed what KVM then does at KVM_RUN: INIT resets the VP (an emulation failure at rip 0xfff0), NMI injects an NMI, SMI makes KVM_RUN fail and every later ioctl on the VM return EIO, and lowest priority takes the vector as a fixed interrupt; remote read, startup IPI, and ExtINT have no visible effect on a running BSP. With this change both tests pass. Mutants of the check fail them: refusing nothing, or refusing a masked entry too, fails both; accepting lowest priority and ExtINT fails the unit test; and a set_apic that ignores the refusal fails the contract test. On nvx-kvm-test, cargo check, clippy --all-targets (gnu and musl), doc, and nextest pass for virt_kvm, as do clippy and doc for vmm_tests, all 84 tests of virt_kvm and kvm with those that need /dev/kvm, and the two contract tests that KVM runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Guide/src/dev_guide/tests/vmm.md | 5 +- .../src/reference/architecture/timekeeping.md | 10 +++ vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs | 67 +++++++++++++++ .../tests/virt_backend_tests/apic.rs | 84 +++++++++++++++++++ .../tests/virt_backend_tests/main.rs | 2 + 5 files changed, 166 insertions(+), 2 deletions(-) create mode 100644 vmm_tests/vmm_tests/tests/virt_backend_tests/apic.rs diff --git a/Guide/src/dev_guide/tests/vmm.md b/Guide/src/dev_guide/tests/vmm.md index 95822a12ef..cefd5fa481 100644 --- a/Guide/src/dev_guide/tests/vmm.md +++ b/Guide/src/dev_guide/tests/vmm.md @@ -66,8 +66,9 @@ the partition and bind its single VP before invoking the function. Cases have names such as `time::kvm::initial_frozen` and `time::whp::clock_restore`. Fixtures default to unenlightened; `hv1: true` enables Hv1, using native enlightenment offload on WHP. A contract that only KVM owes, such as its refusal -of a restored `EFER.SVME` for a guest whose CPUID has no SVM, registers with -`kvm_only` and has a case for KVM alone. +of a restored `EFER.SVME` for a guest whose CPUID has no SVM, or of a restored +APIC timer whose delivery mode is not Fixed, registers with `kvm_only` and has a +case for KVM alone. The clock contracts also have separately named Hv1-enabled cases: `initial_frozen_hv1`, `freeze_thaw_hv1`, `reset_clocks_hv1`, and `clock_restore_hv1`. These reuse the unenlightened contracts to check reference diff --git a/Guide/src/reference/architecture/timekeeping.md b/Guide/src/reference/architecture/timekeeping.md index d6362a8660..e2b1e147b4 100644 --- a/Guide/src/reference/architecture/timekeeping.md +++ b/Guide/src/reference/architecture/timekeeping.md @@ -341,6 +341,16 @@ no expiry while the VP is outside guest mode, so the backend saves a timer that had expired at the read as KVM reports it, and a restored timer that the guest already took an interrupt from fires once more. +KVM also delivers the expiry of an unmasked timer in the delivery mode +of its LVT entry, which a guest cannot set, since KVM masks its writes +to the entry. So the backend refuses to restore a LAPIC state whose +unmasked timer entry has a delivery mode other than Fixed. Five of the +seven other modes would give the VP an INIT, a startup IPI, an NMI, or +an SMI, or wake it from halt, none of which the saved state holds, and +lowest priority and ExtINT, which KVM delivers as a fixed interrupt or +ignores, are refused too, since no save holds them either. A masked +entry delivers nothing, and is accepted. + KVM's synthetic-timer snapshot path saves config/count MSRs, but cannot preserve timer adjustment or undelivered expiration-message state. diff --git a/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs b/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs index 55c1c0d79f..0001c5e88a 100644 --- a/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs +++ b/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs @@ -483,6 +483,10 @@ impl AccessVpState for KvmVpStateAccess<'_, '_> { } fn set_apic(&mut self, value: &vp::Apic) -> Result<(), Self::Error> { + // Refuse the page before any state changes, so that a refusal leaves + // the VP's APIC, and the timer that it stopped with, as they were. + check_restorable_apic_timer(value.registers().lvt_timer)?; + // The VP's timer is no longer the one that it stopped with. self.vp.timer_at_stop = None; @@ -765,8 +769,34 @@ pub(super) fn kvm_lapic_page(registers: &vp::ApicRegisters) -> [u8; 1024] { registers.as_page() } +/// Refuses an APIC page whose LVT timer entry is unmasked and has a delivery +/// mode other than Fixed. +/// +/// `KVM_SET_LAPIC` starts the timer again from the page's counts, and KVM +/// delivers its expiries through the entry's delivery mode. A timer that has +/// expired, as a one-shot timer whose current count is 0 has, fires at once: +/// inside the ioctl when APICv is active, and at the next `KVM_RUN` otherwise. +/// Most modes other than Fixed would give the restored VP an INIT, a startup +/// IPI, an NMI, or an SMI, or wake it from halt (remote read), none of which +/// the saved state holds. A guest cannot write the timer's delivery mode, since +/// KVM masks its writes to the entry, so a save never holds one, and only a +/// damaged or crafted saved state does. Lowest priority and ExtINT, which KVM +/// delivers as a fixed interrupt or ignores, are refused too. A masked entry +/// delivers nothing until the guest writes it, which clears its delivery mode, +/// so it is accepted. +fn check_restorable_apic_timer(lvt_timer: u32) -> Result<(), KvmError> { + let lvt = x86defs::apic::Lvt::from(lvt_timer); + if !lvt.masked() && lvt.delivery_mode() != x86defs::apic::DeliveryMode::FIXED.0 { + return Err(KvmError::InvalidState( + "an unmasked APIC timer whose delivery mode is not Fixed", + )); + } + Ok(()) +} + #[cfg(test)] mod tests { + use super::check_restorable_apic_timer; use super::check_restored_efer; use super::kvm_lapic_page; use crate::KvmError; @@ -779,6 +809,9 @@ mod tests { use x86defs::X64_EFER_NXE; use x86defs::X64_EFER_SCE; use x86defs::X64_EFER_SVME; + use x86defs::apic::DeliveryMode; + use x86defs::apic::Lvt; + use x86defs::apic::TimerMode; use zerocopy::FromZeros; const VECTOR: u32 = 0x40; @@ -874,6 +907,40 @@ mod tests { } assert_eq!(kvm_initial_count(VECTOR, (0, 0), true), 0); } + + #[test] + fn an_unmasked_timer_that_is_not_fixed_is_refused() { + // KVM masks a guest's writes to the entry, so the guest never sets a + // delivery mode, whatever the timer's mode. + for timer_mode in [ + TimerMode::ONE_SHOT, + TimerMode::PERIODIC, + TimerMode::TSC_DEADLINE, + ] { + let fixed = Lvt::new() + .with_vector(VECTOR as u8) + .with_timer_mode(timer_mode.0); + assert_eq!(fixed.delivery_mode(), DeliveryMode::FIXED.0); + check_restorable_apic_timer(fixed.into()).unwrap(); + check_restorable_apic_timer(fixed.with_masked(true).into()).unwrap(); + + for mode in 1..=7 { + let lvt = fixed.with_delivery_mode(mode); + assert!( + matches!( + check_restorable_apic_timer(lvt.into()), + Err(KvmError::InvalidState(_)) + ), + "timer mode {}, delivery mode {mode}", + timer_mode.0 + ); + // A masked entry delivers nothing. + check_restorable_apic_timer(lvt.with_masked(true).into()).unwrap(); + } + } + // The reset value is masked. + check_restorable_apic_timer(MASKED).unwrap(); + } } /// Restores of one-shot LAPIC timers through `KVM_SET_LAPIC`, with a real-mode diff --git a/vmm_tests/vmm_tests/tests/virt_backend_tests/apic.rs b/vmm_tests/vmm_tests/tests/virt_backend_tests/apic.rs new file mode 100644 index 0000000000..82286f925f --- /dev/null +++ b/vmm_tests/vmm_tests/tests/virt_backend_tests/apic.rs @@ -0,0 +1,84 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Guest-free contract tests of the APIC state that a backend restores. + +use super::native::backend_test; +use hvdef::Vtl; +use virt::Partition; +use virt::Processor; +use virt::vp; +use virt::vp::AccessVpState as _; +use x86defs::apic::ApicBase; +use x86defs::apic::DeliveryMode; +use x86defs::apic::Lvt; +use x86defs::apic::TimerMode; + +const TIMER_VECTOR: u8 = 0x40; +const INITIAL_COUNT: u32 = 1_000_000; + +/// Returns `apic` with a one-shot timer that has expired: its initial count is +/// not 0, and its current count is. KVM fires such a timer as soon as it +/// restores the state, in the delivery mode of the timer's LVT entry, unless +/// the entry is masked. +fn expired_timer(apic: &vp::Apic, delivery_mode: DeliveryMode, masked: bool) -> vp::Apic { + let mut registers = *apic.registers(); + registers.svr = 0x1ff; + registers.lvt_timer = Lvt::new() + .with_vector(TIMER_VECTOR) + .with_delivery_mode(delivery_mode.0) + .with_masked(masked) + .with_timer_mode(TimerMode::ONE_SHOT.0) + .into(); + registers.timer_dcr = 0xb; + registers.timer_icr = INITIAL_COUNT; + registers.timer_ccr = 0; + vp::Apic::new( + ApicBase::from(apic.apic_base).with_enable(true), + registers, + apic.auto_eoi, + ) +} + +// KVM delivers the expiry of an unmasked timer in the delivery mode of its LVT +// entry, so most modes other than Fixed would give a restored VP an INIT, a +// startup IPI, an NMI, or an SMI, or wake it from halt, which the saved state +// does not hold; lowest priority and ExtINT, which KVM delivers as a fixed +// interrupt or ignores, would not. The guest cannot write the mode, so a save +// never holds an unmasked timer in any mode but Fixed, and the restore must +// refuse one. +backend_test!(restore_refuses_a_non_fixed_apic_timer, kvm_only, requires: []); +fn restore_refuses_a_non_fixed_apic_timer( + _partition: &impl Partition, + processor: &mut impl Processor, +) -> anyhow::Result<()> { + let mut state = processor.access_state(Vtl::Vtl0); + let reset = state.apic()?; + + let mut accepted = Vec::new(); + let mut changed = Vec::new(); + for mode in (1..=7).map(DeliveryMode) { + if state.set_apic(&expired_timer(&reset, mode, false)).is_ok() { + accepted.push(mode.0); + state.set_apic(&reset)?; + } else if state.apic()? != reset { + changed.push(mode.0); + } + } + anyhow::ensure!( + accepted.is_empty(), + "the restore accepted an expired timer in the delivery modes {accepted:?}" + ); + anyhow::ensure!( + changed.is_empty(), + "the refused restore changed the APIC state, for the delivery modes {changed:?}" + ); + + // What a save holds restores: an expired Fixed timer, and masked timers, + // which deliver nothing. + state.set_apic(&expired_timer(&reset, DeliveryMode::FIXED, false))?; + for mode in (0..=7).map(DeliveryMode) { + state.set_apic(&expired_timer(&reset, mode, true))?; + } + Ok(()) +} diff --git a/vmm_tests/vmm_tests/tests/virt_backend_tests/main.rs b/vmm_tests/vmm_tests/tests/virt_backend_tests/main.rs index e225796362..efdbdb5efb 100644 --- a/vmm_tests/vmm_tests/tests/virt_backend_tests/main.rs +++ b/vmm_tests/vmm_tests/tests/virt_backend_tests/main.rs @@ -3,6 +3,8 @@ //! Native hypervisor contract tests without VM boot or guest artifacts. +#[cfg(all(guest_arch = "x86_64", guest_is_native, target_os = "linux"))] +mod apic; #[cfg(all( guest_arch = "x86_64", guest_is_native,