Repository navigation
Closed
Copy link
Description
When restoring a snapshot into fresh KVM vCPUs, OpenVMM only saves and restores the VP state elements listed in vmm_core/virt/src/x86/vp.rs and the 8 syscall MSRs in VirtualMsrs::names:
(8, "pat", pat, set_pat, Pat),
(9, "msrs", virtual_msrs, set_virtual_msrs, VirtualMsrs),
(10, "drs", debug_regs, set_debug_regs, DebugRegisters),
(11, "tsc", tsc, set_tsc, Tsc),
(12, "cet", cet, set_cet, Cet),
(13, "cet_ss", cet_ss, set_cet_ss, CetSs),
(14, "tsc_aux", tsc_aux, set_tsc_aux, TscAux),
// The deadline must be restored after APIC mode and guest TSC.
(15, "tsc_deadline", tsc_deadline, set_tsc_deadline, TscDeadline),Beyond IA32_SPEC_CTRL (tracked separately in P0-urgent/run-1-part-1-spec-ctrl), several non-mitigation guest-writable MSRs are neither included in VpSavedState nor carried by VirtualMsrs, causing fresh destination vCPUs to revert to reset/binder defaults after restore:
IA32_UMWAIT_CONTROL(0xe1,run-1-part-2): Exposed on profiles advertisingWAITPKG(CPUID.(7,0):ECX[5], e.g., Alder Lake). Guest-configuredUMWAIT/TPAUSEtime limits are lost on restore.- Machine-check configuration and bank status (
MCG_STATUS,MCi_CTL/STATUS/ADDR/MISC/CTL2,run-1-part-3): Restoring the LAPICCMCILVT entry does not restore MCA bank controls or pending error telemetry. - Miscellaneous feature/power MSRs (
IA32_MISC_ENABLE,MISC_FEATURES_ENABLES,POWER_CTL, AMDVM_HSAVE_PA,run-1-part-4):KvmProcessorBinder::bindreinitializesIA32_MISC_ENABLEtohv1_emulator::x86::MISC_ENABLEon fresh vCPUs, discarding any guest modifications permitted by KVM. IA32_DEBUGCTL(0x1d9,run-1-part-5): ArchitecturalDR0–DR7registers are saved inDebugRegisters, butIA32_DEBUGCTL(branch-step, LBR, RTM debug controls where supported) is omitted.IA32_BNDCFGS(0xd90,run-1-part-6): Exposed on Skylake-SP (CPUID.(7,0):EBX[14]);Xsavesaves MPX bounds registers, but theIA32_BNDCFGSsupervisor configuration MSR is omitted.- AMD legacy PMU MSRs (
PerfEvtSel0–3,PerfCtr0–3,run-1-part-7): Although architectural PMU leaf0xAis cleared in built-in profiles, AMD legacy PMU MSRs remain accessible unless filtered by KVM and are not saved across restore.
Proposed Fix
For each MSR family above, either:
- Add capability-gated optional state elements to
VpSavedState(with a defined fallback when restoring older snapshots that lack those fields), or - Filter/disallow guest writes (or hide the corresponding CPUID feature bits, such as
WAITPKGorMPX, on snapshot-supported profiles) when OpenVMM intentionally keeps those registers fixed.
Metadata
Metadata
Assignees
Labels
No labels
Activity
openvmm: pin the save and restore of a guest's user-wait, debug, feat…
ppenna commentedon Nov 17, 2026
Correction, from #1182's audit: two of the reasons that #1638 recorded for state without an element, in Guest-Writable Processor State and OpenVMM's Guide, do not hold.
IA32_BNDCFGS. It said that KVM has not presented MPX since Linux 5.6, because the kernel does not enable its state components. Linux 5.6 dropped MPX for the kernel's own processes, but still enables MPX's state components for KVM's guests:XFEATURE_MASK_USER_SUPPORTEDkeepsBNDREGSandBNDCSR,KVM_SUPPORTED_XCR0includes them,vmx_set_cpu_caps()setsX86_FEATURE_MPXwherekvm_mpx_supported()finds them in XCR0, andvmx_set_msr()takes a guest's write of the MSR where its CPUID has MPX. So a guest of the Skylake-SP profile can write it, on every backend that supports the profile.MSR_VM_HSAVE_PA. It said that no guest runs the instructions that use it, since no CPU profile enumerates SVM. That holds for the instructions, but KVM on an AMD host keeps a guest's writes ofVM_CRandVM_HSAVE_PAwhatever its CPUID says (svm_set_msr()), and keeps the OSVW MSRs for every guest whose CPUID enumerates OSVW, as every AMD profile does, so a restore lost them.#1732, with microsoft/openvmm#5134, adds elements for these MSRs (
bndcfgs,svm_msrs, andosvw, fields 21-23 of the VP saved state, with no version change), has WHP and MSHV hold theumwait_controlelement that this issue gave KVM, and corrects the design.openvmm: pin the processor state that snapshots lost
doc: describe the processor state that snapshots keep on each backend