Skip to content

openvmm: on KVM a guest can kill the whole VM by sending itself an SMI #1408

Closed

Description

@tangruize

On a host whose KVM supports SMM (Linux built with CONFIG_KVM_SMM, and on Intel with unrestricted guest), a guest can kill the whole VM by sending itself an SMI: KVM takes the vCPU into SMM and switches it to address space 1 to save its state at SMBASE, OpenVMM maps RAM only into address space 0, the save fails, and KVM marks the VM dead, after which every vCPU ioctl on the partition returns EIO. No device is needed: a write to the x2APIC ICR MSR with SMI delivery and the self shorthand does it, as does any IOAPIC redirection or MSI with SMI delivery.

Reproduction

The patch's test a_guest_smi_enters_smm_instead_of_killing_the_vm (virt_kvm, arch::x86_64::smm_ram_tests) reproduces it on real KVM: it maps RAM into the partition through PartitionMemoryMap, delivers an SMI to the VP as an MSI with DeliveryMode::SMI, and runs the VP. Without the fix the first KVM_RUN returns EIO and the VM is dead; with it the VP enters SMM, executing the stub at SMBASE + 0x8000, and the VM stays alive. The test needs /dev/kvm and is #[ignore]d; it was run for this report.

Cause

KvmPartitionInner::map_region (vmm_core/virt_kvm/src/memory.rs) passes its dense slot index to KVM as the slot id, and KVM takes the address space from the slot id's upper 16 bits, so every RAM slot lands in address space 0 and address space 1, the address space a vCPU uses while in SMM, stays empty:

// vmm_core/virt_kvm/src/memory.rs
// ...
self.kvm.set_user_memory_region(
slot_to_use as u32,
data,
size,
addr,
readonly,
)?

On the SMI, KVM enters SMM and switches the vCPU to address space 1 before writing the 512-byte state-save area at SMBASE + 0xfe00 (the default SMBASE, 0x30000, is in RAM). The write finds no memslot, fails, and KVM marks the VM dead; every later vCPU ioctl returns EIO.

Fix

fix.patch also maps every userspace-backed RAM slot into address space 1, setting the address-space bits of the slot id, on hosts that report KVM_CAP_X86_SMM and for partitions without private memory (KVM does not support SMM with private memory, and rejects address space 1 slots there). The mirror is cleared when the slot is cleared or reused. It mirrors all of RAM rather than a fixed 64 KiB SMRAM window, since a guest SMM handler can relocate SMBASE, and KVM gives userspace no way to refuse SMIs. It adds the test above.

Related: #1158.

Activity

  1. tangruize commented on Nov 9, 2026

    @tangruize
    ContributorAuthor

    Proposed patch

    Patch against OpenVMM 881619911d1a6ec2fec7bebdad7cfe69d09018f1. Save the contents below as fix.patch and run git apply fix.patch.

    fix.patch
    diff --git a/Cargo.lock b/Cargo.lock
    index cc6797f..6396cc3 100644
    --- a/Cargo.lock
    +++ b/Cargo.lock
    @@ -10496,6 +10496,7 @@ dependencies = [
      "libc",
      "memory_range",
      "open_enum",
    + "pal_async",
      "pal_event",
      "parking_lot",
      "pci_core",
    diff --git a/vmm_core/virt_kvm/Cargo.toml b/vmm_core/virt_kvm/Cargo.toml
    index 23e5aaf..66a6845 100644
    --- a/vmm_core/virt_kvm/Cargo.toml
    +++ b/vmm_core/virt_kvm/Cargo.toml
    @@ -36,6 +36,7 @@ zerocopy.workspace = true
     
     [dev-dependencies]
     test_with_tracing.workspace = true
    +pal_async.workspace = true
     
     [target.'cfg(all(target_os = "linux", target_arch = "aarch64"))'.dependencies]
     aarch64defs.workspace = true
    diff --git a/vmm_core/virt_kvm/src/arch/x86_64/mod.rs b/vmm_core/virt_kvm/src/arch/x86_64/mod.rs
    index 517d4a4..0f80c3a 100644
    --- a/vmm_core/virt_kvm/src/arch/x86_64/mod.rs
    +++ b/vmm_core/virt_kvm/src/arch/x86_64/mod.rs
    @@ -12,6 +12,8 @@
     pub(crate) mod snp;
     mod time_abi;
     mod vm_state;
    +#[cfg(test)]
    +mod smm_ram_tests;
     mod vp_state;
     
     pub(crate) use time_abi::KvmTimeAbi;
    @@ -677,6 +679,14 @@ fn build(
                 }
             };
     
    +        // A vCPU in SMM uses address space 1, so RAM is mirrored into it
    +        // when KVM can enter SMM at all: without a slot at SMBASE, which is
    +        // in RAM, SMM entry makes KVM mark the whole VM dead. KVM does not
    +        // support SMM with private memory, and rejects address space 1
    +        // slots entirely there.
    +        let smm_supported = matches!(memory_backing_mode, KvmMemoryBackingMode::Userspace)
    +            && self.vm.check_extension(kvm::KVM_CAP_X86_SMM).unwrap_or(0) > 0;
    +
             let partition = Arc::new(KvmPartitionInner {
                 kvm: self.vm,
                 sev: self.sev,
    @@ -710,6 +720,7 @@ fn build(
                 cpuid,
                 reserved_vps_per_socket: self.config.processor_topology.reserved_vps_per_socket(),
                 mce_cmci_supported: x86defs::McgCap::from(self.supported_mce_cap).cmci_p(),
    +            smm_supported,
                 synic_ports: Default::default(),
                 time_abi,
             });
    diff --git a/vmm_core/virt_kvm/src/arch/x86_64/smm_ram_tests.rs b/vmm_core/virt_kvm/src/arch/x86_64/smm_ram_tests.rs
    new file mode 100644
    index 0000000..c3955b1
    --- /dev/null
    +++ b/vmm_core/virt_kvm/src/arch/x86_64/smm_ram_tests.rs
    @@ -0,0 +1,128 @@
    +// Copyright (c) Microsoft Corporation.
    +// Licensed under the MIT License.
    +
    +use super::*;
    +use guestmem::GuestMemory;
    +use pal_async::DefaultPool;
    +use test_with_tracing::test;
    +use virt::BindProcessor;
    +use virt::Hypervisor;
    +use virt::PartitionConfig;
    +use virt::PartitionMemoryMapper;
    +use virt::ProtoPartitionConfig;
    +use vm_topology::memory::MemoryLayout;
    +use vm_topology::processor::TopologyBuilder;
    +use vmcore::vmtime::VmTime;
    +use vmcore::vmtime::VmTimeKeeper;
    +use x86defs::apic::DeliveryMode;
    +
    +/// The RAM mapped into the partition under test: enough to cover KVM's
    +/// default SMBASE (0x30000) and the 64 KiB of SMRAM above it.
    +const RAM_LEN: usize = 0x400000;
    +
    +/// Builds a KVM partition with `vp_count` VPs and calls `f` with it and its
    +/// VP binders. No memory is mapped into the partition.
    +fn with_partition<R>(
    +    vp_count: u32,
    +    f: impl FnOnce(&KvmPartition, Vec<KvmProcessorBinder>) -> R,
    +) -> R {
    +    DefaultPool::run_with(async |driver| {
    +        let keeper = VmTimeKeeper::new(&driver, VmTime::from_100ns(0));
    +        let vmtime = keeper.builder().build(&driver).await.unwrap();
    +        let topology = TopologyBuilder::new_x86().build(vp_count).unwrap();
    +        let mem_layout = MemoryLayout::new(RAM_LEN as u64, &[], &[], &[], None).unwrap();
    +        let guest_memory = GuestMemory::allocate(mem_layout.end_of_ram() as usize);
    +        let (partition, binders) = Kvm::new()
    +            .unwrap()
    +            .new_partition(ProtoPartitionConfig {
    +                processor_topology: &topology,
    +                hv_config: None,
    +                vmtime: &vmtime,
    +                isolation: virt::ProtoPartitionIsolation::None,
    +                nested_virt: false,
    +                user_mode_memory_faults: false,
    +                lazy_memory_registration: false,
    +                time_abi: None,
    +            })
    +            .unwrap()
    +            .build(PartitionConfig {
    +                mem_layout: &mem_layout,
    +                guest_memory: &guest_memory,
    +                cpuid: &[],
    +                vtl0_alias_map: None,
    +                fault_resolver: None,
    +            })
    +            .unwrap();
    +        f(&partition, binders)
    +    })
    +}
    +
    +/// A device's MSI to `apic_id` (physical destination, edge).
    +fn msi(mode: DeliveryMode, apic_id: u32, vector: u8) -> MsiRequest {
    +    MsiRequest::new_x86(mode, apic_id, false, vector, false)
    +}
    +
    +/// A guest SMI enters SMM on a partition whose RAM is mapped through
    +/// `PartitionMemoryMap`: the VP runs in SMM, executing RAM at SMBASE, and
    +/// the VM stays alive. Before, every memslot was in address space 0, so
    +/// SMM entry could not write its saved state at SMBASE through address
    +/// space 1, and KVM marked the whole VM dead: the run failed and every
    +/// later vCPU ioctl returned EIO.
    +#[test]
    +#[ignore = "requires /dev/kvm"]
    +fn a_guest_smi_enters_smm_instead_of_killing_the_vm() {
    +    with_partition(1, |partition, mut binders| {
    +        // Map RAM into the partition as the partition mapper does.
    +        let ram = unsafe {
    +            libc::mmap(
    +                std::ptr::null_mut(),
    +                RAM_LEN,
    +                libc::PROT_READ | libc::PROT_WRITE,
    +                libc::MAP_PRIVATE | libc::MAP_ANONYMOUS,
    +                -1,
    +                0,
    +            )
    +        };
    +        assert_ne!(ram, libc::MAP_FAILED, "mmap failed");
    +        let ram = unsafe { std::slice::from_raw_parts_mut(ram.cast::<u8>(), RAM_LEN) };
    +
    +        // KVM enters SMM at SMBASE + 0x8000. The stub reports itself on
    +        // port 0x80 and loops, parking the VP in SMM.
    +        ram[0x38000..0x38004].copy_from_slice(&[0xe7, 0x80, 0xeb, 0xfc]);
    +
    +        // SAFETY: the mapping stays valid (the test never unmaps it) for
    +        // the lifetime of the partition.
    +        unsafe {
    +            partition
    +                .memory_mapper(Vtl::Vtl0)
    +                .map_range(ram.as_mut_ptr(), RAM_LEN, 0, true, false)
    +        }
    +        .unwrap();
    +
    +        let mut vp = binders[0].bind().unwrap();
    +        let apic_id = vp.inner.vp_info.apic_id;
    +        {
    +            partition.request_msi(Vtl::Vtl0, msi(DeliveryMode::SMI, apic_id, 0));
    +        }
    +        assert_eq!(
    +            vp.kvm.get_vcpu_events().unwrap().smi.pending,
    +            1,
    +            "KVM has the SMI pending"
    +        );
    +
    +        // The SMI enters SMM at the stub, which outs to port 0x80. Before
    +        // the fix this run failed with EIO, and the VM was dead.
    +        match vp.runner.run().unwrap() {
    +            kvm::Exit::IoOut { port, .. } => assert_eq!(port, 0x80),
    +            exit => panic!("unexpected first exit: {exit:?}"),
    +        }
    +        // The VM is alive: the stub's loop runs again, and a state ioctl
    +        // still works, reporting the VP in SMM.
    +        match vp.runner.run().unwrap() {
    +            kvm::Exit::IoOut { port, .. } => assert_eq!(port, 0x80),
    +            exit => panic!("unexpected second exit: {exit:?}"),
    +        }
    +        let smi = vp.kvm.get_vcpu_events().unwrap().smi;
    +        assert_eq!(smi.smm, 1, "the VP is in SMM");
    +    })
    +}
    diff --git a/vmm_core/virt_kvm/src/lib.rs b/vmm_core/virt_kvm/src/lib.rs
    index 5ad901a..e7cb89a 100644
    --- a/vmm_core/virt_kvm/src/lib.rs
    +++ b/vmm_core/virt_kvm/src/lib.rs
    @@ -150,6 +150,15 @@ struct KvmPartitionInner {
         #[cfg(guest_arch = "x86_64")]
         mce_cmci_supported: bool,
     
    +    /// Whether RAM slots are mirrored into KVM address space 1, the address
    +    /// space a vCPU uses while in SMM, so that an SMI a guest sends itself
    +    /// enters SMM instead of making KVM mark the VM dead when SMM entry
    +    /// cannot write its saved state at SMBASE, which is in RAM. Enabled only
    +    /// when the host supports SMM and the partition has no private memory,
    +    /// with which KVM does not support SMM.
    +    #[cfg(guest_arch = "x86_64")]
    +    smm_supported: bool,
    +
         /// The GIC device fd, kept alive for the VM lifetime.
         #[cfg(guest_arch = "aarch64")]
         #[inspect(skip)]
    diff --git a/vmm_core/virt_kvm/src/memory.rs b/vmm_core/virt_kvm/src/memory.rs
    index 27448c8..f89165f 100644
    --- a/vmm_core/virt_kvm/src/memory.rs
    +++ b/vmm_core/virt_kvm/src/memory.rs
    @@ -19,6 +19,12 @@
     use std::sync::Arc;
     use thiserror::Error;
     
    +/// The slot-id bits that select KVM address space 1, the address space a
    +/// vCPU uses while in SMM: KVM derives the address-space id from the slot
    +/// id's upper 16 bits.
    +#[cfg(guest_arch = "x86_64")]
    +const SMM_MEMSLOT_ADDRESS_SPACE: u32 = 1 << 16;
    +
     #[derive(Debug, Error)]
     pub enum MemoryError {
         #[error("kvm memory operation failed")]
    @@ -204,6 +210,14 @@ unsafe fn map_region(
                     // SAFETY: clearing a slot removes the memory reference.
                     unsafe { self.clear_slot(slot_to_use, true)? };
                     state.ranges[slot_to_use] = None;
    +            } else {
    +                // Clear this slot's address space 1 mirror: the mapping
    +                // below re-creates it for a userspace-backed range, and a
    +                // slot reused with a guest_memfd backing must leave no
    +                // mirror behind.
    +                // SAFETY: clearing a slot removes the memory reference.
    +                #[cfg(guest_arch = "x86_64")]
    +                unsafe { self.clear_smm_twin(slot_to_use)? };
                 }
             }
             let (guest_memfd_offset, private_attributes_set) = match backing {
    @@ -220,6 +234,28 @@ unsafe fn map_region(
                             readonly,
                         )?
                     };
    +                // Also map the range into address space 1, which a vCPU uses
    +                // while in SMM: without it, an SMI a guest sends itself makes
    +                // KVM mark the whole VM dead when SMM entry cannot write its
    +                // saved state at SMBASE, which is in RAM.
    +                #[cfg(guest_arch = "x86_64")]
    +                if let Err(err) =
    +                    unsafe { self.map_smm_twin(slot_to_use, data, size, addr, readonly) }
    +                {
    +                    // Leave the range unmapped rather than half-mapped.
    +                    // SAFETY: clearing a slot removes the memory reference.
    +                    unsafe {
    +                        self.kvm.set_user_memory_region(
    +                            slot_to_use as u32,
    +                            std::ptr::null_mut(),
    +                            0,
    +                            0,
    +                            false,
    +                        )?
    +                    };
    +                    state.ranges[slot_to_use] = None;
    +                    return Err(err.into());
    +                }
                     (None, false)
                 }
                 #[cfg(guest_arch = "x86_64")]
    @@ -303,13 +339,79 @@ unsafe fn clear_slot(&self, slot: usize, guest_memfd_backed: bool) -> Result<(),
                         false,
                         None,
                     )
    -            }
    +            }?;
             } else {
                 // SAFETY: the caller ensures clearing this slot is valid.
                 unsafe {
                     self.kvm
                         .set_user_memory_region(slot as u32, std::ptr::null_mut(), 0, 0, false)
    -            }
    +            }?;
    +            // SAFETY: the caller ensures clearing this slot is valid, and a
    +            // userspace slot's address space 1 mirror is cleared with it.
    +            #[cfg(guest_arch = "x86_64")]
    +            unsafe { self.clear_smm_twin(slot)? };
    +        }
    +        Ok(())
    +    }
    +
    +    /// Maps the slot `slot`'s range into address space 1, the address space a
    +    /// vCPU uses while in SMM, mirroring the slot's address space 0 mapping.
    +    ///
    +    /// A no-op when SMM mirroring is off (`smm_supported`).
    +    ///
    +    /// # Safety
    +    ///
    +    /// `data..data+size` must be and remain an allocated VA range until the
    +    /// partition is destroyed or the region is unmapped, as for the address
    +    /// space 0 mapping this mirrors.
    +    #[cfg(guest_arch = "x86_64")]
    +    unsafe fn map_smm_twin(
    +        &self,
    +        slot: usize,
    +        data: *mut u8,
    +        size: usize,
    +        addr: u64,
    +        readonly: bool,
    +    ) -> Result<(), kvm::Error> {
    +        if !self.smm_supported {
    +            return Ok(());
    +        }
    +        // SAFETY: the caller keeps `data..data+size` valid for as long as
    +        // KVM may reference the slot.
    +        unsafe {
    +            self.kvm.set_user_memory_region(
    +                (slot as u32) | SMM_MEMSLOT_ADDRESS_SPACE,
    +                data,
    +                size,
    +                addr,
    +                readonly,
    +            )
    +        }
    +    }
    +
    +    /// Clears the slot `slot`'s mirror in address space 1, the address space
    +    /// a vCPU uses while in SMM.
    +    ///
    +    /// A no-op when SMM mirroring is off (`smm_supported`).
    +    ///
    +    /// # Safety
    +    ///
    +    /// The caller must ensure that clearing the target slot's mirror is
    +    /// valid, as for `clear_slot`.
    +    #[cfg(guest_arch = "x86_64")]
    +    unsafe fn clear_smm_twin(&self, slot: usize) -> Result<(), kvm::Error> {
    +        if !self.smm_supported {
    +            return Ok(());
    +        }
    +        // SAFETY: clearing a slot removes the memory reference.
    +        unsafe {
    +            self.kvm.set_user_memory_region(
    +                (slot as u32) | SMM_MEMSLOT_ADDRESS_SPACE,
    +                std::ptr::null_mut(),
    +                0,
    +                0,
    +                false,
    +            )
             }
         }
     
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area: guestInitial title-keyword triage suggestionarea: openvmmInitial title-keyword triage suggestionplatform: kvmInitial title-keyword triage suggestion

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions