Repository navigation
Description
On a host whose kernel uses neither XSAVEC nor XSAVES, KVM_GET_XSAVE and KVM_GET_XSAVE2 report MXCSR as 0x1f80 whenever the guest's XMM registers and the upper halves of its YMM registers are in their init state, whatever the guest's real MXCSR. Such hosts are Intel before Skylake, AMD before Zen, an L1 host whose hypervisor hides both instructions, or a kernel with both disabled on its command line. A snapshot taken then restores the guest with the default MXCSR: its rounding control, FTZ, DAZ and exception masks are lost. Without a snapshot the guest keeps its MXCSR. This is a Linux regression that is not fixed upstream; hosts whose kernel uses XSAVEC or XSAVES, which is most current ones, are not affected.
Affected host kernels
| Host kernel | MXCSR that KVM_GET_XSAVE reports when XSTATE_BV[1] and XSTATE_BV[2] are clear |
|---|---|
| using XSAVEC or XSAVES, any version | the guest's (the CPU sets XSTATE_BV[1] whenever MXCSR is not 0x1f80) |
| using XSAVE or XSAVEOPT, 5.16 and later (7.3 included) | 0x1f80 |
| using XSAVE or XSAVEOPT, before 5.16 | the guest's (KVM copied the legacy area as it was) |
ptrace (NT_X86_XSTATE, NT_PRFPREG) and core dumps have reported 0x1f80 the same way since 5.14.
Reproduction
Not reproduced; it needs a host kernel without XSAVEC and XSAVES. On one, run a guest that sets MXCSR to a value other than 0x1f80 (for example 0x7f80, rounding toward zero), with its XMM registers zero and its AVX upper halves in their init state, then snapshot and restore it and read MXCSR in the restored guest. Expected: 0x7f80. Actual: 0x1f80.
Cause
The kernel builds the uabi copy's header from the fpstate's XSTATE_BV and copies MXCSR only when it has SSE or YMM set:
// arch/x86/kernel/fpu/xstate.c (Linux 7.3)
/* Copy MXCSR when SSE or YMM are set in the feature mask */
copy_feature(header.xfeatures & (XFEATURE_MASK_SSE | XFEATURE_MASK_YMM),
&to, &xsave->i387.mxcsr, &xinit->i387.mxcsr,
MXCSR_AND_FLAGS_SIZE);
That rule holds for the compacted format, where XSAVEC and XSAVES set XSTATE_BV[1] whenever MXCSR is not 0x1f80. In the standard format, which the kernel uses without XSAVEC and XSAVES, XSTATE_BV[1] tracks the XMM registers only, while XSAVE stores MXCSR and the standard form of XRSTOR loads it in any case. So the fpstate holds the live MXCSR and the copy reports 0x1f80. KVM_GET_XSAVE and KVM_GET_XSAVE2 go through this copy (fpu_copy_guest_fpstate_to_uabi). It regressed when KVM's own copy of the legacy area was replaced by it (bf5d00470787, 5.16), and ptrace and core dumps when they moved to it (3a3351126ee8 and adc997b3d66d, 5.14).
OpenVMM cannot work around it: KvmVpStateAccess::xsave (vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs) saves the area that get_xsave returns, and no other KVM interface reports MXCSR (KVM_GET_FPU leaves struct kvm_fpu's mxcsr unset).
Fix
A patch for the x86/fpu code is attached below; it has not been sent upstream yet. On kernels that use the standard format only, it marks SSE as in use in the uabi header when MXCSR is not 0x1f80, as XSAVEC and XSAVES do in hardware. The XMM registers are still taken from the fpstate only if its own XSTATE_BV[1] is set, because XSAVEOPT leaves stale values in an area whose XMM registers are in their init state. A standard-format kernel then exports what a compacted-format kernel already exports for the same state, and KVM_SET_XSAVE takes MXCSR back from it. The change is +20/−2 in __copy_xstate_to_uabi_buf. It applies to tip master, v7.3, 6.18.y and 6.12.y; it is compile-tested only.
Related: #1360.
Activity
tangruize commentedon Nov 8, 2026
Proposed Linux patch
Patch for the x86/fpu code (tip tree), against tip master 4f08fc2dfebccc28b7a7645dd81d28a755815e47; it also applies to v7.3. Compile-tested only, and not yet sent upstream. Save the contents below as
linux.patchand rungit apply linux.patchin a kernel tree.linux.patchppenna commentedon Nov 15, 2026
Closing as not planned. This is not only a failed reproduction: the kernel defect is confirmed in the Linux source (v5.15, v5.16, v6.18, and master at fa2933aac170, 2026-11-15), where the uabi export drops MXCSR when XSTATE_BV has neither SSE nor AVX set; it lies outside NVX and OpenVMM, since no KVM interface reports the value and no refusal in
check_saveablecan be exact; and on the pinned-profile CPUs that I could measure, a guest's own instructions do not leave SSE clear with a non-default MXCSR. The fix belongs in the kernel. Two things need adding to the issue's analysis: which guests can reach the case on real CPUs, and which hosts use the standard format.The kernel
KVM_GET_XSAVEandKVM_GET_XSAVE2both end inkvm_vcpu_ioctl_x86_get_xsave2(),fpu_copy_guest_fpstate_to_uabi(), and__copy_xstate_to_uabi_buf(..., XSTATE_COPY_XSAVE). It copies the fpstate's MXCSR onlyif (header.xfeatures & (XFEATURE_MASK_SSE | XFEATURE_MASK_YMM))and writes 0x1f80 otherwise. Linux 5.16 moved KVM onto this path (bf5d00470787); before it,fill_xsave()copied the legacy area as it was. master has the same code.kvm_arch_vcpu_ioctl_get_fpu()fillsfpr,fcw,fsw,ftwx,last_opcode,last_ip,last_dp, andxmm, and nevermxcsr; the ioctlkzallocsstruct kvm_fpu, so itsmxcsrreads 0 (the same in 5.15).KVM_GET_ONE_REGtakes MSRs only, and MXCSR is not an MSR.KVM_GET_REGS,KVM_GET_SREGS,KVM_GET_SREGS2,KVM_GET_MSRS,KVM_GET_XCRS,KVM_GET_DEBUGREGS,KVM_GET_VCPU_EVENTS,KVM_GET_LAPIC, andKVM_GET_NESTED_STATEhold none either. The live MXCSR exists only in the kernel's fpstate, behind the copy above, soKvmVpStateAccess::xsavehas no second source to read it from.X86_FEATURE_XCOMPACTEDis set when either is). Bothxsavecandxsavesare then missing from the flags in/proc/cpuinfo, and the kernel logs its choice at boot:x86/fpu: Enabled xstate features ..., using 'standard' format.The kernel has used XSAVEC since 5.19 (8ad7e8f69695). In 5.16 to 5.18 the compacted format meant XSAVES only, so a CPU with XSAVEC but not XSAVES ran the standard format there.Which guests lose it
The issue's table is right about what the kernel exports when XSTATE_BV[1] and [2] are clear. Whether a guest reaches that case with a non-default MXCSR depends on the CPU. XSAVE and XSAVEOPT write XSTATE_BV[1] from XINUSE[1], which the SDM defines for the XMM registers only ("XSTATE_BV[1] may be written with 0 even if MXCSR does not have its RESET value of 1F80H", Vol. 1 section 13.9), while XSAVEC and XSAVES set it for a non-default MXCSR (sections 13.10 and 13.11). The first program below changes MXCSR in user space, saves with each instruction, and prints what it wrote. It ran on a Core i9-12900H (Alder Lake, Windows 11), a Xeon Platinum 8370C (Ice Lake-SP, Windows build 26100), and a Xeon Platinum 8573C (Emerald Rapids, Linux 6.18), the CPUs of three pinned profiles, with the same result on all three.
xsave,xsaveopt, andxsavecwrote this to the area's XSTATE_BV after MXCSR became 0x7f80 with the XMM registers zero:xsaveandxsaveoptwrite XSTATE_BVxsavecwritesLDMXCSR,VLDMXCSRFXRSTORXRSTOR, standard form, XSTATE_BV=0XRSTOR, standard form, XSTATE_BV=1On these CPUs the instructions that a guest ordinarily uses to change MXCSR mark SSE in use, so a standard-format kernel exports the guest's MXCSR. The SSE bit stays clear only when MXCSR came from an
XRSTORof an area whose SSE bit is clear, which those instructions do not do. KVM does it on a standard-format kernel whenKVM_SET_XSAVEtakes an area with none of x87, SSE, and AVX: it keeps the MXCSR that its vCPU held, and the standard form ofXRSTORloads it. A VM reset left a stale MXCSR that way (#1360) until #1641 changed OpenVMM's restore. I could not measure AMD CPUs or Intel CPUs before Skylake-SP, so the issue's list of standard-format hosts is an upper bound on the affected ones that I can neither confirm nor exclude.The issue's reproduction, on
nvx-kvm-test(Xeon 8573C, Linux 6.18.57, XSAVES), with the second program below, which uses raw KVM ioctls: a real-mode guest runsLDMXCSRwith 0x7f80 and leaves XMM and AVX alone;KVM_GET_XSAVEreturns XSTATE_BV=0x2 and MXCSR=0x7f80; a guest in a fresh vCPU that the area is restored into reads 0x7f80. As the issue says, a host that uses XSAVES is not affected, and no available host uses the standard format.Why OpenVMM cannot work around it
Xsave::normalizeclears it otherwise, andwrite_standardrestores 0x1f80), the same contract as the kernel's export, so the value is gone before OpenVMM reads the area./proc/cpuinfoflags and dmesg, and the format does not decide the outcome: the CPU's XINUSE tracking does. Only a probe guest in a scratch VM would separate the hosts.NVX's pinned CPU profiles cover Intel Skylake-SP, Ice Lake-SP, Emerald Rapids, Granite Rapids, and Alder Lake, and AMD Milan, Genoa, and Turin. Those CPUs have both instructions, and every profile enumerates them to the guest (
CPUID.(0xd,1).EAXis 0xf), so a standard-format host among them needs a hypervisor that hides both instructions or a kernel command line that disables both. Any other Intel or AMD CPU runs on a host profile, which the host checks ofdoctordo not qualify.Where the fix belongs
__copy_xstate_to_uabi_buf()inarch/x86/kernel/fpu/xstate.c, andcopy_uabi_to_xstate()for the write direction. The x86/fpu code goes through the tip tree: x86@kernel.org and linux-kernel@vger.kernel.org, with kvm@vger.kernel.org and the KVM/x86 maintainers on cc, sinceKVM_GET_XSAVEis their ABI. Intel's Chang S. Bae posted the same format-dependent correction in 2022, x86/fpu: Fix MXCSR handling and SSE component definition (v3), after gVisor's report of the write direction on hosts without XSAVES. It did not reach mainline: master still has the code above. The patch in this issue and that series fix the same dependence on the format, so a report to the kernel should cite both.The restore direction differs by format too, and there user space can shape what it gives
KVM_SET_XSAVE. OpenVMM now does: #1641 pins OpenVMM23fb678876c9(microsoft/openvmm#5118), whosexsave::restorezeroes the x87 and XMM bytes of a legacy component that the area's XSTATE_BV leaves out and, where KVM would keep the vCPU's MXCSR or PKRU through the area, writes a seed buffer first, so that KVM reads the area's clear bits as init state on every host. That fixes #1360. The description of #1641 also records a case that it leaves alone and that belongs to this family: an area with AVX set, SSE clear, and a non-default MXCSR, which only a standard-format host's save can hold, loses its MXCSR when it is restored onto a host that uses the compacted format.What would reopen this
I could not measure AMD CPUs or Intel CPUs before Skylake-SP, and no such host is available, so this half stays open to evidence. Reopen the issue with the output of both programs from a host on which the first writes XSTATE_BV 0x0 in its
LDMXCSRrows, or on which the second reports XSTATE_BV=0x0 and MXCSR=0x1f80 for the guest that ranLDMXCSRand 0x1f80 for the restored guest, together with that host'slscpu,uname -r, thex86/fpu:line ofdmesg, andCPUID.(0xd,1).EAX. A refusal incheck_saveablewould then be the way to fail closed: the capture rolls back, and the saved state does not change. The host test would have to follow the CPU's behavior, not the kernel's format.xsave_mxcsr_probe.rs
kvm_mxcsr_e2e.rs