diff --git a/Guide/src/dev_guide/snapshot_format.md b/Guide/src/dev_guide/snapshot_format.md index 6fb51b6526..5e20952021 100644 --- a/Guide/src/dev_guide/snapshot_format.md +++ b/Guide/src/dev_guide/snapshot_format.md @@ -254,19 +254,26 @@ or group of MSRs that a VP holds and a fresh VP does not: | 18 | `debug_ctl` | `IA32_DEBUGCTL` | the backend has it (KVM) | | 19 | `misc_msrs` | `IA32_MISC_ENABLE`, `MSR_MISC_FEATURES_ENABLES` (CPUID faulting), and `IA32_POWER_CTL` | the backend has them (KVM) | | 20 | `machine_check` | `IA32_MCG_STATUS`, `IA32_MCG_CTL`, and the control, status, address, miscellaneous, and CMCI control registers of each machine-check bank | the backend has them (KVM) | +| 21 | `bndcfgs` | `IA32_BNDCFGS` | the guest's CPUID enumerates MPX | +| 22 | `svm_msrs` | AMD's `VM_CR` and `VM_HSAVE_PA` | the backend has them (KVM on an AMD host) | +| 23 | `osvw` | AMD's `OSVW_ID_LENGTH` and `OSVW_STATUS` | the guest's CPUID enumerates OSVW, and the backend keeps them (KVM on an AMD host) | A partition has an element only where the guest can have the state and the backend can access it, which the partition's capabilities record. The guest's -CPUID decides the first for `spec_ctrl` and `umwait_control`; no CPUID bit -decides it for the other elements, whose MSRs every CPU has. For the KVM -backend: - -- It can access an MSR only if the host's `KVM_GET_MSR_INDEX_LIST` lists it, so - a host without the MSR, such as one without WAITPKG, holds no element for it - instead of failing every save. +CPUID decides the first for `spec_ctrl`, `umwait_control`, `bndcfgs`, and +`osvw`; no CPUID bit decides it for the other elements, whose MSRs every CPU of +the vendor has. For the KVM backend: + +- It can access an MSR only if the host's `KVM_GET_MSR_INDEX_LIST` lists it, + except for the ones below that KVM keeps without listing them, so a host + without the MSR, such as one without WAITPKG, holds no element for it instead + of failing every save. KVM lists `IA32_BNDCFGS` where it exposes MPX, + which it does on a host with MPX: the kernel still enables MPX's state + components in the host's XCR0 for KVM's guests, although it dropped MPX for + its own processes in Linux 5.6. - It can access none of them for an SNP guest, whose registers live in an encrypted VMSA. -- `IA32_DEBUGCTL` is the exception to the list. Every KVM virtualizes it, and +- `IA32_DEBUGCTL` is an exception to the list. Every KVM virtualizes it, and `KVM_GET_MSRS` and `KVM_SET_MSRS` reach it on every kernel that the time ABI supports, Linux 5.16 and later: the vendor code fails a write only for a bit that KVM does not support (VMX, which drops BTF and LBR instead) or that is @@ -282,9 +289,36 @@ backend: - KVM does not list the registers of the machine-check banks, so `machine_check` follows `IA32_MCG_STATUS`, and the number of banks is the one that the VP's `IA32_MCG_CAP` reports. - -The MSHV, WHP, and OpenHCL backends do not have elements 17 through 20, and -clear their capabilities. +- On an AMD host, KVM keeps a guest's `VM_CR` and `VM_HSAVE_PA` whatever the + guest's CPUID says, though no CPU profile enumerates SVM. It lists + `VM_HSAVE_PA`, and not `VM_CR`, but it lists `VM_HSAVE_PA` wherever the host + can read it, as an Intel host can under a hypervisor that reads it as 0, + where KVM has no `VM_CR`. So `svm_msrs` needs KVM to run on SVM, as the vendor + of its supported CPUID shows, as well as the MSR in its list. A guest can + only write a page address that it has to `VM_HSAVE_PA`, while KVM takes any + value from the host, and cannot set `VM_CR`'s `SVMDIS` while `EFER.SVME` is + set, which KVM refuses from the host only after it has set `SVMDIS`, so a + restore refuses either before it writes either MSR. Once a guest sets + `VM_CR`'s `SVMDIS`, KVM keeps `SVMDIS` and `LOCK` for the life of the vCPU, + so a VM reset leaves them set, as it did before the element; a restore into + a new VP sets them as the guest saved them. +- KVM keeps the OSVW MSRs for a vCPU whose CPUID enumerates OSVW, as every AMD + CPU profile does, where KVM supports OSVW, as it does on an AMD host, and + lists neither, so `osvw` follows the guest's CPUID and KVM's supported CPUID. + A new VP holds values that KVM derives from the host's, which a snapshot + replaces with the ones that the guest saw, and which a VM reset puts back. + +The MSHV and WHP backends hold `umwait_control` and `bndcfgs` as registers of +the VP, as they hold `spec_ctrl`, where the guest's CPUID enumerates WAITPKG or +MPX, which the hypervisor then presents to the guest; MSHV clears them for an +SNP guest, as KVM does. They do not have elements 18 through 20, and clear their +capabilities: the Microsoft hypervisor keeps none of those registers for a guest +to write (it drops the guest's writes of `IA32_DEBUGCTL` and `IA32_MISC_ENABLE`, +raises #GP for `MSR_MISC_FEATURES_ENABLES` and `IA32_POWER_CTL`, and gives the +guest no machine-check banks). They do not have elements 22 and 23 either: on +WHP, OpenVMM serves a guest's accesses of the OSVW MSRs itself, reading them as +0 and refusing writes, and neither backend reads AMD's SVM MSRs. OpenHCL does +not have elements 17 through 23. A saved state restores only into a partition that has every element that it holds. A destination whose backend lacks one, such as a host whose KVM does not @@ -306,26 +340,19 @@ also fails for a state that holds a value that KVM refuses, such as a bank control other than 0 or all ones (bit 0 and bit 10 may be clear), which no guest writes, after it has written the registers that precede it. -Some guest-writable state has no element: - -- `IA32_BNDCFGS` belongs to MPX, which only the Skylake-SP CPU profile - enumerates. Since Linux 5.6, the kernel does not enable the MPX state - components in the host's XCR0, which KVM needs to expose MPX, so that CPU - profile cannot run on a KVM host that the time ABI supports, which needs - Linux 5.16. -- `MSR_VM_HSAVE_PA` belongs to SVM, which no CPU profile enumerates. A guest - without SVM cannot run VMRUN, so nothing uses the MSR. -- AMD's legacy performance counters, `0xc0010000` through `0xc0010007`, hold - state only on an AMD host whose KVM virtual PMU is on. A time ABI guest has no - PMU, so the KVM backend raises #GP for them instead (see [Time and CPU - compatibility](../user_guide/openvmm/snapshots.md#time-and-cpu-compatibility)), - and a snapshot of another guest does not hold their state. - -A snapshot from an earlier build holds none of elements 17 through 20, so a +Some guest-writable state has no element: AMD's legacy performance counters, +`0xc0010000` through `0xc0010007`, hold state only on an AMD host whose KVM +virtual PMU is on. A time ABI guest has no PMU, so the KVM backend raises #GP +for them instead (see [Time and CPU +compatibility](../user_guide/openvmm/snapshots.md#time-and-cpu-compatibility)), +and a snapshot of another guest does not hold their state. + +A snapshot from an earlier build holds none of elements 17 through 23, so a restore leaves those MSRs as `bind` set them, as it did before: `IA32_MISC_ENABLE` at the Microsoft hypervisor's value, the machine-check banks -as a new VP has them, and the other MSRs at 0. An earlier build ignores the -elements of a later snapshot in the same way. +as a new VP has them, the OSVW MSRs at KVM's values for the host, and the other +MSRs at 0. An earlier build ignores the elements of a later snapshot in the same +way. ### Fields of an element @@ -372,6 +399,15 @@ save records it as the activity's interruption alone and holds no event for it: A restore refuses any other pending exception, which KVM would drop, before it changes the activity. Only a damaged or crafted snapshot holds one. +Nor does a software interruption need a field, since no save holds one. A +Hyper-V VP that the hypervisor stops while it delivers a software interrupt +(`INT n`), a software exception (`INT3` or `INTO`), or a privileged software +exception (`INT1`) holds the event with the length of the instruction that +raised it, which the activity has no place for, so MSHV and WHP refuse to save +it rather than drop it, as they refuse a pending event other than an exception +or an extint, or one whose vector is not 8 bits. KVM reports no such event: it +has the guest run the instruction again. + KVM on an Intel CPU can hold an interrupt shadow of either kind, which the Boolean `interrupt_shadow` cannot say: the shadow that a `MOV SS` or `POP SS` leaves also blocks NMIs and suppresses certain debug exceptions at the next diff --git a/Guide/src/dev_guide/tests/vmm.md b/Guide/src/dev_guide/tests/vmm.md index f07f0ebd45..ffaf82d119 100644 --- a/Guide/src/dev_guide/tests/vmm.md +++ b/Guide/src/dev_guide/tests/vmm.md @@ -60,11 +60,17 @@ The other guest-writable MSRs that the VP state holds, where the backend can access them, have cases of their own in `msr_state.rs`: `misc_msrs_survive_a_save_and_restore`, `umwait_control_survives_a_save_and_restore` (which needs a host with WAITPKG), +`bndcfgs_survives_a_save_and_restore` (which needs a host with MPX), +`svm_msrs_survive_a_save_and_restore` and `osvw_survives_a_save_and_restore` +(which need an AMD host), `debug_ctl_survives_a_save_and_restore`, and `machine_check_state_survives_a_save_and_restore`, each of which sets the state, -saves it, changes the VP, and restores it, and two that check that a reset -returns the feature MSRs and the machine-check state to what a new VP holds. -MSHV and WHP do not hold these elements, so these cases report ignored there. +saves it, changes the VP, and restores it, and three that check that a reset +returns the feature MSRs, the machine-check state, and the OSVW MSRs to what a +new VP holds. +MSHV and WHP hold `IA32_UMWAIT_CONTROL` and `IA32_BNDCFGS`, whose cases run +there on a host with WAITPKG or MPX, and none of the other elements, so their +cases report ignored there. A restore refuses a saved state that holds an element that the partition lacks, such as SynIC state for a partition without Hyper-V enlightenments: it fails with an error that names the element, before it changes any state. diff --git a/Guide/src/user_guide/openvmm/snapshots.md b/Guide/src/user_guide/openvmm/snapshots.md index 0dea62c681..165b2870b8 100644 --- a/Guide/src/user_guide/openvmm/snapshots.md +++ b/Guide/src/user_guide/openvmm/snapshots.md @@ -172,6 +172,15 @@ SEV-SNP VM, keeps its RAM in the normal address space alone, since KVM does not support SMM with private memory. A capture that meets a vCPU in its handler fails, and the guest can request it again once the handler has executed `RSM`. +A save on MSHV or WHP fails for a vCPU that the hypervisor stopped while it +delivered a software interrupt (`INT n`), a software exception (`INT3` or +`INTO`), or a privileged software exception (`INT1`): the hypervisor holds the +event with the length of the instruction that raised it, which the saved state +has no place for, so a restore would lose it. KVM holds no such event, and runs +the instruction again instead. A guest-requested capture that fails this way +rolls back, so the guest continues, takes the event, and can request the +capture again. + While a guest-requested snapshot boundary is held, VM-worker management RPCs that can change VM state are rejected rather than queued. This includes memory writes, pause/resume, reset, interrupt injection, hotplug, and state dumps. @@ -831,7 +840,11 @@ immediately with a clear error if any active device does not support it. elements](../../dev_guide/snapshot_format.md#vp-state-elements) for the list. A snapshot from a build that predates one of them leaves that state as a new VP has it, and a build that predates it ignores it in a later snapshot, so - neither restore fails, and neither keeps what the guest wrote. Only a host - whose KVM can access an MSR saves it, and a snapshot restores only on a host - that can access every MSR that it holds: on another host, the restore is - refused, with an error that names the element, before it changes the VP. + neither restore fails, and neither keeps what the guest wrote. Only a backend + that can access an MSR saves it: KVM where the host lists the MSR, or, for + AMD's `VM_CR` and OSVW MSRs, which KVM does not list, where it keeps them, and + MSHV and WHP only `IA32_SPEC_CTRL`, `IA32_UMWAIT_CONTROL`, and + `IA32_BNDCFGS`, the only ones of these MSRs that their hypervisor keeps for a + guest; a snapshot restores only on a host that can access every MSR that it + holds: on another host, the restore is refused, with an error that names the + element, before it changes the VP. diff --git a/openhcl/virt_mshv_vtl/src/lib.rs b/openhcl/virt_mshv_vtl/src/lib.rs index e6b5f2d41e..4c2c4afdbf 100755 --- a/openhcl/virt_mshv_vtl/src/lib.rs +++ b/openhcl/virt_mshv_vtl/src/lib.rs @@ -2508,10 +2508,12 @@ impl UhPartition { // Hardware-isolated guests use our software APIC, which does not // implement the architectural TSC-deadline MSR. caps.tsc_deadline &= !isolation.is_hardware_isolated(); - // The VP state access of OpenHCL does not hold SPEC_CTRL or - // UMWAIT_CONTROL. + // The VP state access of OpenHCL does not hold SPEC_CTRL, + // UMWAIT_CONTROL, BNDCFGS, or AMD's OSVW MSRs. caps.spec_ctrl = false; caps.umwait_control = false; + caps.bndcfgs = false; + caps.osvw = false; match isolation { IsolationType::Tdx => { assert_eq!(caps.vtom.is_some(), !hide_isolation); diff --git a/openhcl/virt_mshv_vtl/src/processor/mshv/x64.rs b/openhcl/virt_mshv_vtl/src/processor/mshv/x64.rs index 7301f00574..e4f166e58c 100644 --- a/openhcl/virt_mshv_vtl/src/processor/mshv/x64.rs +++ b/openhcl/virt_mshv_vtl/src/processor/mshv/x64.rs @@ -1717,6 +1717,30 @@ impl AccessVpState for UhVpStateAccess<'_, '_, HypervisorBackedX86> { Err(vp_state::Error::Unimplemented("machine_check")) } + fn bndcfgs(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("bndcfgs")) + } + + fn set_bndcfgs(&mut self, _value: &vp::Bndcfgs) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("bndcfgs")) + } + + fn svm_msrs(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("svm_msrs")) + } + + fn set_svm_msrs(&mut self, _value: &vp::SvmMsrs) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("svm_msrs")) + } + + fn osvw(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("osvw")) + } + + fn set_osvw(&mut self, _value: &vp::Osvw) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("osvw")) + } + fn synic_msrs(&mut self) -> Result { self.get_register_state() } diff --git a/openhcl/virt_mshv_vtl/src/processor/snp/mod.rs b/openhcl/virt_mshv_vtl/src/processor/snp/mod.rs index ae242ff2b3..ae9cc8c257 100755 --- a/openhcl/virt_mshv_vtl/src/processor/snp/mod.rs +++ b/openhcl/virt_mshv_vtl/src/processor/snp/mod.rs @@ -3007,6 +3007,30 @@ impl AccessVpState for UhVpStateAccess<'_, '_, SnpBacked> { Err(vp_state::Error::Unimplemented("machine_check")) } + fn bndcfgs(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("bndcfgs")) + } + + fn set_bndcfgs(&mut self, _value: &vp::Bndcfgs) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("bndcfgs")) + } + + fn svm_msrs(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("svm_msrs")) + } + + fn set_svm_msrs(&mut self, _value: &vp::SvmMsrs) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("svm_msrs")) + } + + fn osvw(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("osvw")) + } + + fn set_osvw(&mut self, _value: &vp::Osvw) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("osvw")) + } + fn cet(&mut self) -> Result { let vmsa = self.vp.runner.vmsa(self.vtl); Ok(vp::Cet { scet: vmsa.s_cet() }) diff --git a/openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs b/openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs index b8df7aa5a9..545292b540 100644 --- a/openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs +++ b/openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs @@ -4266,6 +4266,30 @@ impl AccessVpState for UhVpStateAccess<'_, '_, TdxBacked> { Err(vp_state::Error::Unimplemented("machine_check")) } + fn bndcfgs(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("bndcfgs")) + } + + fn set_bndcfgs(&mut self, _value: &vp::Bndcfgs) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("bndcfgs")) + } + + fn svm_msrs(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("svm_msrs")) + } + + fn set_svm_msrs(&mut self, _value: &vp::SvmMsrs) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("svm_msrs")) + } + + fn osvw(&mut self) -> Result { + Err(vp_state::Error::Unimplemented("osvw")) + } + + fn set_osvw(&mut self, _value: &vp::Osvw) -> Result<(), Self::Error> { + Err(vp_state::Error::Unimplemented("osvw")) + } + fn cet(&mut self) -> Result { Err(vp_state::Error::Unimplemented("cet")) } diff --git a/vm/x86/x86defs/src/lib.rs b/vm/x86/x86defs/src/lib.rs index 205242990d..0cc686a075 100644 --- a/vm/x86/x86defs/src/lib.rs +++ b/vm/x86/x86defs/src/lib.rs @@ -302,6 +302,8 @@ pub const X86X_MSR_MTRR_DEF_TYPE: u32 = 0x2ff; /// has four, `CTL`, `STATUS`, `ADDR`, and `MISC`, in that order. pub const X86X_MSR_MC0_CTL: u32 = 0x400; +/// `IA32_BNDCFGS`, MPX's supervisor-mode configuration. +pub const X86X_MSR_BNDCFGS: u32 = 0xd90; pub const X86X_MSR_XSS: u32 = 0xda0; pub const X86X_IA32_MSR_RAPL_POWER_UNIT: u32 = 0x606; @@ -346,6 +348,8 @@ pub const X86X_AMD_MSR_HW_CFG: u32 = 0xC0010015; pub const X86X_AMD_MSR_NB_CFG: u32 = 0xC001001F; pub const X86X_AMD_MSR_VM_CR: u32 = 0xC0010114; pub const X86X_AMD_MSR_IGNNE: u32 = 0xC0010115; +/// `VM_HSAVE_PA`, the physical address of SVM's host save area. +pub const X86X_AMD_MSR_VM_HSAVE_PA: u32 = 0xC0010117; pub const X86X_AMD_MSR_GHCB: u32 = 0xC0010130; pub const X86X_AMD_MSR_SEV: u32 = 0xC0010131; pub const X86X_AMD_MSR_SECURE_AVIC_CONTROL: u32 = 0xc0010138; diff --git a/vmm_core/virt/src/x86/mod.rs b/vmm_core/virt/src/x86/mod.rs index 66f70ca9de..4706adba09 100644 --- a/vmm_core/virt/src/x86/mod.rs +++ b/vmm_core/virt/src/x86/mod.rs @@ -86,6 +86,21 @@ pub struct X86PartitionCapabilities { /// without access to the MSR must clear this even if guest CPUID /// enumerates WAITPKG. pub umwait_control: bool, + /// The guest's CPUID enumerates MPX, so the guest can write + /// `IA32_BNDCFGS`, which the VP's saved state then holds. Backends without + /// access to the MSR must clear this even if guest CPUID enumerates MPX. + pub bndcfgs: bool, + /// The guest's CPUID enumerates OSVW, so the guest can write AMD's + /// `OSVW_ID_LENGTH` and `OSVW_STATUS`, which the VP's saved state then + /// holds. Backends that do not keep what the guest writes must clear this + /// even if guest CPUID enumerates OSVW. + pub osvw: bool, + /// The backend holds the guest's `VM_CR` and `VM_HSAVE_PA`, AMD's SVM + /// control and host save area MSRs, in the VP's saved state. No CPUID bit + /// decides whether the guest can write them: KVM on an AMD host takes a + /// guest's writes of both whether or not its CPUID enumerates SVM. So + /// `from_cpuid` leaves this clear, and a backend that keeps them sets it. + pub svm_msrs: bool, /// The backend holds the guest's `IA32_DEBUGCTL` in the VP's saved state. /// No CPUID bit decides whether the guest can write the MSR, which every /// CPU has, so `from_cpuid` leaves this clear and a backend that can @@ -177,6 +192,9 @@ impl X86PartitionCapabilities { tsc_aux: false, spec_ctrl: false, umwait_control: false, + bndcfgs: false, + osvw: false, + svm_msrs: false, debug_ctl: false, misc_msrs: false, machine_check: false, @@ -314,17 +332,19 @@ impl X86PartitionCapabilities { } let extended_max_function = f(CpuidFunction::ExtendedMaxFunction.0, 0)[0]; - - this.tsc_aux = { - let rdtscp = if extended_max_function >= CpuidFunction::ExtendedVersionAndFeatures.0 { - x86defs::cpuid::ExtendedVersionAndFeaturesEdx::from( - f(CpuidFunction::ExtendedVersionAndFeatures.0, 0)[3], - ) - .rdtscp() + let extended_version_and_features = + if extended_max_function >= CpuidFunction::ExtendedVersionAndFeatures.0 { + f(CpuidFunction::ExtendedVersionAndFeatures.0, 0) } else { - false + Default::default() }; + this.tsc_aux = { + let rdtscp = x86defs::cpuid::ExtendedVersionAndFeaturesEdx::from( + extended_version_and_features[3], + ) + .rdtscp(); + let rdpid = x86defs::cpuid::ExtendedFeatureSubleaf0Ecx::from(extended_features[2]).rd_pid(); @@ -341,6 +361,8 @@ impl X86PartitionCapabilities { }; this.spec_ctrl = enumerates_spec_ctrl(extended_features, address_space_sizes); this.umwait_control = enumerates_waitpkg(extended_features); + this.bndcfgs = enumerates_mpx(extended_features); + this.osvw = enumerates_osvw(extended_version_and_features); Ok(this) } @@ -635,6 +657,25 @@ pub fn enumerates_waitpkg(extended_features: [u32; 4]) -> bool { x86defs::cpuid::ExtendedFeatureSubleaf0Ecx::from(extended_features[2]).umwait_tpause() } +/// Returns whether CPUID enumerates MPX, so that a guest can write +/// `IA32_BNDCFGS`: `CPUID.(7,0):EBX[14]`. +/// +/// `extended_features` is the result of `CPUID.(7,0)`, zero if the CPU has no +/// such leaf. +pub fn enumerates_mpx(extended_features: [u32; 4]) -> bool { + x86defs::cpuid::ExtendedFeatureSubleaf0Ebx::from(extended_features[1]).mpx() +} + +/// Returns whether CPUID enumerates OSVW, AMD's OS-visible workarounds, so +/// that a guest can write `OSVW_ID_LENGTH` and `OSVW_STATUS`: +/// `CPUID.0x80000001:ECX[9]`. +/// +/// `extended_version_and_features` is the result of `CPUID.0x80000001`, zero +/// if the CPU has no such leaf. +pub fn enumerates_osvw(extended_version_and_features: [u32; 4]) -> bool { + x86defs::cpuid::ExtendedVersionAndFeaturesEcx::from(extended_version_and_features[2]).osvw() +} + #[cfg(test)] mod tests { use super::*; @@ -1008,6 +1049,123 @@ mod tests { assert!(!caps.debug_ctl); assert!(!caps.misc_msrs); assert!(!caps.machine_check); + // KVM on an AMD host takes a guest's writes of VM_CR and VM_HSAVE_PA + // whether or not its CPUID enumerates SVM. + let topology = vm_topology::processor::TopologyBuilder::new_x86() + .build(1) + .unwrap(); + let x2apic = topology.apic_mode() != ApicMode::XApic; + let with_svm = X86PartitionCapabilities::from_cpuid( + &topology, + &mut |leaf, subleaf| match (leaf, subleaf) { + (0, _) => [0xd, 0x6874_7541, 0x444d_4163, 0x6974_6e65], + (1, _) => [0, 0, u32::from(x2apic) << 21, 0], + (0x8000_0000, _) => [0x8000_0001, 0, 0, 0], + (0x8000_0001, _) => [0, 0, !0, !0], + _ => [0; 4], + }, + ) + .unwrap(); + assert!(!with_svm.svm_msrs); + } + + #[test] + fn mpx_is_bit_14_of_the_extended_features_ebx() { + for bit in 0..32 { + assert_eq!( + enumerates_mpx([0, 1 << bit, 0, 0]), + bit == 14, + "CPUID.(7,0):EBX[{bit}]" + ); + } + // The other registers of the leaf enumerate none. + assert!(!enumerates_mpx([!0, 0, !0, !0])); + } + + #[test] + fn bndcfgs_state_needs_mpx_in_a_leaf_that_the_cpu_has() { + const MPX: u32 = 1 << 14; + let has_bndcfgs = |max_function, leaf7_ebx| { + let topology = vm_topology::processor::TopologyBuilder::new_x86() + .build(1) + .unwrap(); + let x2apic = topology.apic_mode() != ApicMode::XApic; + X86PartitionCapabilities::from_cpuid(&topology, &mut |leaf, subleaf| match ( + leaf, subleaf, + ) { + (0, _) => [max_function, 0x756e_6547, 0x6c65_746e, 0x4965_6e69], + (1, _) => [0, 0, u32::from(x2apic) << 21, 0], + (7, 0) => [0, leaf7_ebx, 0, 0], + _ => [0; 4], + }) + .unwrap() + .bndcfgs + }; + assert!(!has_bndcfgs(0xd, 0)); + assert!(!has_bndcfgs(0xd, !MPX)); + assert!(has_bndcfgs(0xd, MPX)); + // A leaf above the highest is not CPUID's to enumerate. + assert!(!has_bndcfgs(6, MPX)); + } + + /// The pinned profiles whose CPUID enumerates MPX: the Skylake-SP CPUs + /// have it, and every later Intel generation and every AMD one lacks it. + #[test] + #[cfg(guest_arch = "x86_64")] + fn bndcfgs_state_is_present_for_the_profiles_that_enumerate_mpx() { + for (id, caps) in pinned_profile_capabilities() { + assert_eq!(caps.bndcfgs, id == "intel.skylake-sp.v1", "{id}"); + } + } + + #[test] + fn osvw_is_bit_9_of_the_extended_version_and_features_ecx() { + for bit in 0..32 { + assert_eq!( + enumerates_osvw([0, 0, 1 << bit, 0]), + bit == 9, + "CPUID.0x80000001:ECX[{bit}]" + ); + } + // The other registers of the leaf enumerate none. + assert!(!enumerates_osvw([!0, !0, 0, !0])); + } + + #[test] + fn osvw_state_needs_osvw_in_a_leaf_that_the_cpu_has() { + const OSVW: u32 = 1 << 9; + let has_osvw = |max_extended, ext1_ecx| { + let topology = vm_topology::processor::TopologyBuilder::new_x86() + .build(1) + .unwrap(); + let x2apic = topology.apic_mode() != ApicMode::XApic; + X86PartitionCapabilities::from_cpuid(&topology, &mut |leaf, subleaf| match ( + leaf, subleaf, + ) { + (0, _) => [0xd, 0x6874_7541, 0x444d_4163, 0x6974_6e65], + (1, _) => [0, 0, u32::from(x2apic) << 21, 0], + (0x8000_0000, _) => [max_extended, 0, 0, 0], + (0x8000_0001, 0) => [0, 0, ext1_ecx, 0], + _ => [0; 4], + }) + .unwrap() + .osvw + }; + assert!(!has_osvw(0x8000_0008, 0)); + assert!(!has_osvw(0x8000_0008, !OSVW)); + assert!(has_osvw(0x8000_0008, OSVW)); + // A leaf above the highest is not CPUID's to enumerate. + assert!(!has_osvw(0x8000_0000, OSVW)); + } + + /// The pinned profiles whose CPUID enumerates OSVW: every AMD profile, and + /// no Intel one. + #[test] + #[cfg(guest_arch = "x86_64")] + fn osvw_state_is_present_for_the_profiles_that_enumerate_it() { + for (id, caps) in pinned_profile_capabilities() { + assert_eq!(caps.osvw, id.starts_with("amd."), "{id}"); + } } } diff --git a/vmm_core/virt/src/x86/vp.rs b/vmm_core/virt/src/x86/vp.rs index 9f7bfcfb73..bc11c0f63b 100644 --- a/vmm_core/virt/src/x86/vp.rs +++ b/vmm_core/virt/src/x86/vp.rs @@ -627,10 +627,30 @@ impl HvRegisterState for Activity { } fn set_values(&mut self, mut it: impl Iterator) { - let activity = HvInternalActivityRegister::from(it.next().unwrap().as_u64()); - let interruption = HvX64PendingInterruptionRegister::from(it.next().unwrap().as_u64()); - let interrupt_state = HvX64InterruptStateRegister::from(it.next().unwrap().as_u64()); - let event = HvX64PendingEventReg0::from(it.next().unwrap().as_u128()); + let values = std::array::from_fn(|_| it.next().unwrap()); + *self = Self::from_hv_values(values) + .unwrap_or_else(|unsupported| panic!("unhandled activity: {unsupported}")); + } +} + +impl Activity { + /// Returns the activity that the values of the Hyper-V registers that + /// [`HvRegisterState::names`] lists for it hold, in that order, or names + /// what of them it cannot hold, which a save refuses rather than drop. + /// + /// The activity holds a pending interruption that is an external + /// interrupt, an NMI, or a hardware exception, and a pending event that is + /// an exception or an extint, each with an 8-bit vector. A Hyper-V VP can + /// also hold, while it delivers one, a software interrupt (`INT n`), a + /// privileged software exception (`INT1`), or a software exception (`INT3` + /// or `INTO`), which the hypervisor delivers with the length of the + /// instruction that raised it, and which the activity has no way to hold. + pub fn from_hv_values(values: [HvRegisterValue; 4]) -> Result { + let [activity, interruption, interrupt_state, event] = values; + let activity = HvInternalActivityRegister::from(activity.as_u64()); + let interruption = HvX64PendingInterruptionRegister::from(interruption.as_u64()); + let interrupt_state = HvX64InterruptStateRegister::from(interrupt_state.as_u64()); + let event = HvX64PendingEventReg0::from(event.as_u128()); let mp_state = if activity.startup_suspend() { MpState::WaitForSipi @@ -642,61 +662,81 @@ impl HvRegisterState for Activity { MpState::Running }; - let pending_event = event.event_pending().then(|| match event.event_type() { - hvdef::HV_X64_PENDING_EVENT_EXCEPTION => { - let event = HvX64PendingExceptionEvent::from(u128::from(event)); - PendingEvent::Exception { - vector: event.vector().try_into().expect("exception code is 8 bits"), - error_code: event.deliver_error_code().then(|| event.error_code()), - parameter: event.exception_parameter(), - } - } - hvdef::HV_X64_PENDING_EVENT_EXT_INT => { - let event = HvX64PendingExtIntEvent::from(u128::from(event)); - PendingEvent::ExtInt { - vector: event.vector(), - } - } - ty => panic!("unhandled event type: {}", ty), - }); - - let pending_interruption = interruption.interruption_pending().then(|| { - match HvX64PendingInterruptionType(interruption.interruption_type()) { - HvX64PendingInterruptionType::HV_X64_PENDING_INTERRUPT => { - PendingInterruption::Interrupt { - vector: interruption - .interruption_vector() - .try_into() - .expect("x86 vector is 8 bits"), - } - } - HvX64PendingInterruptionType::HV_X64_PENDING_NMI => PendingInterruption::Nmi, - HvX64PendingInterruptionType::HV_X64_PENDING_EXCEPTION => { - PendingInterruption::Exception { - vector: interruption - .interruption_vector() - .try_into() - .expect("exception code is 8 bits"), - error_code: interruption - .deliver_error_code() - .then(|| interruption.error_code()), - } - } - ty => panic!("unhandled interruption type: {ty:?}"), - } - }); - - *self = Self { + Ok(Self { mp_state, nmi_pending: false, nmi_masked: interrupt_state.nmi_masked(), interrupt_shadow: interrupt_state.interrupt_shadow(), - pending_event, - pending_interruption, + pending_event: event + .event_pending() + .then(|| pending_event_from_hv(event)) + .transpose()?, + pending_interruption: interruption + .interruption_pending() + .then(|| pending_interruption_from_hv(interruption)) + .transpose()?, nmi_pending_count: None, // The interrupt state register has one bit for both kinds. interrupt_shadow_kind: None, - }; + }) + } +} + +/// Returns the pending event that `event`, which is pending, holds, or names +/// what of it the activity cannot hold. +fn pending_event_from_hv(event: HvX64PendingEventReg0) -> Result { + match event.event_type() { + hvdef::HV_X64_PENDING_EVENT_EXCEPTION => { + let event = HvX64PendingExceptionEvent::from(u128::from(event)); + Ok(PendingEvent::Exception { + vector: event + .vector() + .try_into() + .map_err(|_| "a pending exception event whose vector is not 8 bits")?, + error_code: event.deliver_error_code().then(|| event.error_code()), + parameter: event.exception_parameter(), + }) + } + hvdef::HV_X64_PENDING_EVENT_EXT_INT => { + let event = HvX64PendingExtIntEvent::from(u128::from(event)); + Ok(PendingEvent::ExtInt { + vector: event.vector(), + }) + } + _ => Err("a pending event other than an exception or an extint"), + } +} + +/// Returns the pending interruption that `interruption`, which is pending, +/// holds, or names what of it the activity cannot hold. +fn pending_interruption_from_hv( + interruption: HvX64PendingInterruptionRegister, +) -> Result { + use HvX64PendingInterruptionType as Type; + + let vector = || { + interruption + .interruption_vector() + .try_into() + .map_err(|_| "a pending interruption whose vector is not 8 bits") + }; + match Type(interruption.interruption_type()) { + Type::HV_X64_PENDING_INTERRUPT => Ok(PendingInterruption::Interrupt { vector: vector()? }), + Type::HV_X64_PENDING_NMI => Ok(PendingInterruption::Nmi), + Type::HV_X64_PENDING_EXCEPTION => Ok(PendingInterruption::Exception { + vector: vector()?, + error_code: interruption + .deliver_error_code() + .then(|| interruption.error_code()), + }), + Type::HV_X64_PENDING_SOFTWARE_INTERRUPT => Err("a pending software interrupt (INT n)"), + Type::HV_X64_PENDING_PRIVILEGED_SOFTWARE_EXCEPTION => { + Err("a pending privileged software exception (INT1)") + } + Type::HV_X64_PENDING_SOFTWARE_EXCEPTION => { + Err("a pending software exception (INT3 or INTO)") + } + _ => Err("a pending interruption of a type that Hyper-V does not define"), } } @@ -2220,6 +2260,130 @@ impl StateElement for MachineCheck { } } +/// The architectural IA32_BNDCFGS MSR, which holds MPX's supervisor-mode +/// configuration: whether MPX is enabled at CPL 0, whether `BNDPRESERVE` +/// keeps the bound registers across a branch, and the base of the bound +/// directory. +/// +/// The guest can write it only if its CPUID enumerates MPX, which +/// [`X86PartitionCapabilities::bndcfgs`] records, so the saved state holds it +/// only then. The user-mode configuration, BNDCFGU, and the bound registers +/// are part of the XSAVE state. A VP that never saw a write holds 0, the reset +/// value. +#[repr(C)] +#[derive(Default, Debug, Copy, Clone, PartialEq, Eq, Protobuf, Inspect)] +#[mesh(package = "virt.x86")] +pub struct Bndcfgs { + #[mesh(1)] + #[inspect(hex)] + pub value: u64, +} + +impl HvRegisterState for Bndcfgs { + fn names(&self) -> &'static [HvX64RegisterName; 1] { + &[HvX64RegisterName::Bndcfgs] + } + + fn get_values<'a>(&self, it: impl Iterator) { + for (dest, src) in it.zip([self.value]) { + *dest = src.into(); + } + } + + fn set_values(&mut self, it: impl Iterator) { + for (src, dest) in it.zip([&mut self.value]) { + *dest = src.as_u64(); + } + } +} + +impl StateElement for Bndcfgs { + fn is_present(caps: &X86PartitionCapabilities) -> bool { + caps.bndcfgs + } + + fn at_reset(_caps: &X86PartitionCapabilities, _vp_info: &X86VpInfo) -> Self { + Self::default() + } +} + +/// AMD's OS-visible workaround MSRs, `OSVW_ID_LENGTH` and `OSVW_STATUS`, which +/// say which errata, by their OSVW IDs, a processor has. +/// +/// Firmware writes them on a CPU, and so can the guest, if its CPUID enumerates +/// OSVW, which [`X86PartitionCapabilities::osvw`] records, so the saved state +/// holds them only then. Their reset value is 0, as on a CPU before its +/// firmware runs, but a backend gives a new VP values of its own, which it puts +/// back after a VM reset, as a reboot's firmware would write them again: KVM +/// derives them from the host's. +#[derive(Default, Debug, Copy, Clone, PartialEq, Eq, Protobuf, Inspect)] +#[mesh(package = "virt.x86")] +#[inspect(hex)] +pub struct Osvw { + /// `OSVW_ID_LENGTH`. + #[mesh(1)] + pub id_length: u64, + /// `OSVW_STATUS`. + #[mesh(2)] + pub status: u64, +} + +impl StateElement for Osvw { + fn is_present(caps: &X86PartitionCapabilities) -> bool { + caps.osvw + } + + fn at_reset(_caps: &X86PartitionCapabilities, _vp_info: &X86VpInfo) -> Self { + Self::default() + } + + // A new VP holds the backend's values, which KVM derives from the host's, + // not the reset value. + fn can_compare(_caps: &X86PartitionCapabilities) -> bool { + false + } +} + +/// AMD's SVM control MSR, `VM_CR`, and the address of SVM's host save area, +/// `VM_HSAVE_PA`. +/// +/// A backend that lets a guest write them holds them in the VP, and +/// [`X86PartitionCapabilities::svm_msrs`] records that the saved state holds +/// them. KVM on an AMD host does, whether or not the guest's CPUID enumerates +/// SVM. A VP that never saw a write holds 0 in each, the reset value. A guest +/// can only write a page-aligned `VM_HSAVE_PA`, and cannot set `VM_CR`'s +/// `SVMDIS` while `EFER.SVME` is set, and a restore refuses either, which only +/// a damaged or crafted saved state holds. A restore into a new VP sets both as +/// the guest saved them. +#[derive(Default, Debug, Copy, Clone, PartialEq, Eq, Protobuf, Inspect)] +#[mesh(package = "virt.x86")] +#[inspect(hex)] +pub struct SvmMsrs { + /// `VM_CR`. + #[mesh(1)] + pub vm_cr: u64, + /// `VM_HSAVE_PA`. + #[mesh(2)] + pub vm_hsave_pa: u64, +} + +impl StateElement for SvmMsrs { + fn is_present(caps: &X86PartitionCapabilities) -> bool { + caps.svm_msrs + } + + fn at_reset(_caps: &X86PartitionCapabilities, _vp_info: &X86VpInfo) -> Self { + Self::default() + } + + // Once a guest sets VM_CR's SVMDIS, KVM keeps SVMDIS and LOCK for the life + // of the vCPU, so neither a reset nor a restore into that VP can set + // another value of them. + fn can_compare(_caps: &X86PartitionCapabilities) -> bool { + false + } +} + #[repr(C)] #[derive(Default, Debug, PartialEq, Eq, Protobuf, Inspect)] #[mesh(package = "virt.x86")] @@ -2607,6 +2771,9 @@ state_trait! { (18, "debug_ctl", debug_ctl, set_debug_ctl, DebugCtl), (19, "misc_msrs", misc_msrs, set_misc_msrs, MiscMsrs), (20, "machine_check", machine_check, set_machine_check, MachineCheck), + (21, "bndcfgs", bndcfgs, set_bndcfgs, Bndcfgs), + (22, "svm_msrs", svm_msrs, set_svm_msrs, SvmMsrs), + (23, "osvw", osvw, set_osvw, Osvw), // Synic state (100, "synic", synic_msrs, set_synic_msrs, SyntheticMsrs), @@ -3029,6 +3196,118 @@ mod tests { } } + #[test] + fn bndcfgs_needs_no_version_to_coexist_with_earlier_builds() { + let earlier = || EarlierVpSavedState { + tsc: Some(Tsc { value: 7 }), + tsc_aux: Some(TscAux { value: 3 }), + }; + let current = |bndcfgs| VpSavedState { + tsc: Some(Tsc { value: 7 }), + tsc_aux: Some(TscAux { value: 3 }), + bndcfgs, + ..Default::default() + }; + + // A saved state of an earlier build holds no IA32_BNDCFGS, so + // restoring it sets none. + assert_eq!( + mesh_protobuf::decode::(&mesh_protobuf::encode(earlier())).unwrap(), + current(None) + ); + // An earlier build reads the saved state of this one and ignores the + // element. + assert_eq!( + mesh_protobuf::decode::(&mesh_protobuf::encode(current(Some( + Bndcfgs { value: 0x1003 } + )))) + .unwrap(), + earlier() + ); + + // The element is field 21 of the saved state, a message with the value + // in its field 1: bytes that every build that holds it reads, and that + // no later change may alter. + let bytes = [0xaa, 0x01, 0x03, 0x08, 0x83, 0x20]; + let state = VpSavedState { + bndcfgs: Some(Bndcfgs { value: 0x1003 }), + ..Default::default() + }; + assert_eq!(mesh_protobuf::encode(state), bytes); + assert_eq!( + mesh_protobuf::decode::(&bytes).unwrap(), + VpSavedState { + bndcfgs: Some(Bndcfgs { value: 0x1003 }), + ..Default::default() + } + ); + } + + #[test] + fn the_amd_msr_elements_need_no_version_to_coexist_with_earlier_builds() { + let earlier = || EarlierVpSavedState { + tsc: Some(Tsc { value: 7 }), + tsc_aux: Some(TscAux { value: 3 }), + }; + let svm_msrs = SvmMsrs { + vm_cr: 0x5, + vm_hsave_pa: 0x5000, + }; + let osvw = Osvw { + id_length: 5, + status: 2, + }; + let current = |svm_msrs, osvw| VpSavedState { + tsc: Some(Tsc { value: 7 }), + tsc_aux: Some(TscAux { value: 3 }), + svm_msrs, + osvw, + ..Default::default() + }; + + // A saved state of an earlier build holds neither element, so + // restoring it sets neither. + assert_eq!( + mesh_protobuf::decode::(&mesh_protobuf::encode(earlier())).unwrap(), + current(None, None) + ); + // An earlier build reads the saved state of this one and ignores them. + assert_eq!( + mesh_protobuf::decode::(&mesh_protobuf::encode(current( + Some(svm_msrs), + Some(osvw) + ))) + .unwrap(), + earlier() + ); + + // Each element is a message in a field of the saved state, 22 and 23, + // which these bytes pin: every build that holds the element reads + // them, and no later change may alter them. + for (state, bytes) in [ + ( + VpSavedState { + svm_msrs: Some(svm_msrs), + ..Default::default() + }, + &[0xb2, 0x01, 0x06, 0x08, 0x05, 0x10, 0x80, 0xa0, 0x01][..], + ), + ( + VpSavedState { + osvw: Some(osvw), + ..Default::default() + }, + &[0xba, 0x01, 0x04, 0x08, 0x05, 0x10, 0x02], + ), + ] { + assert_eq!(mesh_protobuf::encode(state), bytes); + assert_eq!( + mesh_protobuf::encode(mesh_protobuf::decode::(bytes).unwrap()), + bytes + ); + } + } + /// The components beyond x87 and SSE of the XSAVE layout of the tests of /// `write_standard`, each `(index, size, standard offset, aligned)`. In the /// compact format, with all of them present, they sit at the bytes 576 @@ -3907,4 +4186,127 @@ mod tests { assert!(read.interrupt_shadow); assert_eq!(read.interrupt_shadow_kind, None); } + + /// The activity registers' values with `interruption` in the pending + /// interruption register and `event` in the pending event register. + fn activity_values( + interruption: HvX64PendingInterruptionRegister, + event: u128, + ) -> [HvRegisterValue; 4] { + [ + 0u64.into(), + u64::from(interruption).into(), + 0u64.into(), + event.into(), + ] + } + + #[test] + fn activity_registers_read_back_as_written() { + for (pending_interruption, pending_event) in [ + (None, None), + (Some(PendingInterruption::Interrupt { vector: 0x30 }), None), + (Some(PendingInterruption::Nmi), None), + ( + Some(PendingInterruption::Exception { + vector: 14, + error_code: Some(2), + }), + Some(PendingEvent::Exception { + vector: 14, + error_code: Some(2), + parameter: 0x1000, + }), + ), + (None, Some(PendingEvent::ExtInt { vector: 0x20 })), + ] { + let written = Activity { + mp_state: MpState::Halted, + nmi_masked: true, + pending_event, + pending_interruption, + ..Default::default() + }; + let mut values = [HvRegisterValue::from(0u64); 4]; + written.get_values(values.iter_mut()); + assert_eq!(Activity::from_hv_values(values), Ok(written)); + } + } + + #[test] + fn activity_registers_refuse_what_the_activity_cannot_hold() { + let pending = |ty: u8| { + HvX64PendingInterruptionRegister::new() + .with_interruption_pending(true) + .with_interruption_type(ty) + .with_instruction_length(2) + .with_interruption_vector(0x80) + }; + // A software interrupt, privileged software exception, or software + // exception, which the hypervisor delivers with an instruction length, + // and the types that Hyper-V does not define. + for (ty, refused) in [ + (4, "a pending software interrupt (INT n)"), + (5, "a pending privileged software exception (INT1)"), + (6, "a pending software exception (INT3 or INTO)"), + ( + 1, + "a pending interruption of a type that Hyper-V does not define", + ), + ( + 7, + "a pending interruption of a type that Hyper-V does not define", + ), + ] { + assert_eq!( + Activity::from_hv_values(activity_values(pending(ty), 0)), + Err(refused), + "type {ty}" + ); + } + // An interruption that is not pending has no type. + assert_eq!( + Activity::from_hv_values(activity_values( + pending(4).with_interruption_pending(false), + 0 + )), + Ok(Activity::default()) + ); + // A vector beyond 8 bits. + assert!( + Activity::from_hv_values(activity_values( + pending(0).with_interruption_vector(0x100), + 0 + )) + .is_err() + ); + + // A pending event other than an exception or an extint, such as a + // memory intercept or hypercall output, and an exception whose vector + // is beyond 8 bits. + for ty in [1, 2, 3, 4, 6, 7] { + let event = HvX64PendingEventReg0::new() + .with_event_pending(true) + .with_event_type(ty); + assert_eq!( + Activity::from_hv_values(activity_values( + HvX64PendingInterruptionRegister::new(), + event.into() + )), + Err("a pending event other than an exception or an extint"), + "type {ty}" + ); + } + let exception = HvX64PendingExceptionEvent::new() + .with_event_pending(true) + .with_event_type(hvdef::HV_X64_PENDING_EVENT_EXCEPTION) + .with_vector(0x100); + assert!( + Activity::from_hv_values(activity_values( + HvX64PendingInterruptionRegister::new(), + exception.into() + )) + .is_err() + ); + } } diff --git a/vmm_core/virt_kvm/src/arch/x86_64/mod.rs b/vmm_core/virt_kvm/src/arch/x86_64/mod.rs index 7ce226b3d0..67c56a0e9b 100644 --- a/vmm_core/virt_kvm/src/arch/x86_64/mod.rs +++ b/vmm_core/virt_kvm/src/arch/x86_64/mod.rs @@ -563,6 +563,12 @@ impl ProtoPartition for KvmProtoPartition<'_> { return Err(SnpError::InvalidVmsaGpa(config.bsp.gpa).into()); } + // What the host's KVM keeps beyond the MSRs that it lists, as its + // supported CPUID says: whether it runs on SVM and supports OSVW. + let kvm_support = msr_state::KvmSupport::from_supported_cpuid(|leaf, subleaf| { + self.cpuid.result(leaf, subleaf, &[0; 4]) + }); + let cpuid = match &self.config.time_abi { Some(time_abi) => time_abi::partition_cpuid(&time_abi.cpuid), None => { @@ -650,7 +656,12 @@ impl ProtoPartition for KvmProtoPartition<'_> { // SNP guest's from its encrypted VMSA. caps.spec_ctrl &= self.sev.is_none() && self.msr_index_list.contains(&x86defs::X86X_MSR_SPEC_CTRL); - msr_state::set_capabilities(&mut caps, &self.msr_index_list, self.sev.is_some()); + msr_state::set_capabilities( + &mut caps, + &self.msr_index_list, + kvm_support, + self.sev.is_some(), + ); let mut gsi_routing = GsiRouting::new(); @@ -1217,6 +1228,15 @@ impl virt::BindProcessor for KvmProcessorBinder { time_abi.set_arch_capabilities(&kvm)?; } + // KVM gives the vCPU OSVW values that it derives from the host's, and + // lets them be read only once the vCPU's CPUID enumerates OSVW. + let osvw_at_bind = self + .partition + .caps + .osvw + .then(|| msr_state::osvw(&kvm)) + .transpose()?; + let mut vp = KvmProcessor { partition: &self.partition, inner, @@ -1232,6 +1252,7 @@ impl virt::BindProcessor for KvmProcessorBinder { vmtime: &mut self.vmtime, interrupt_window_stale: true, timer_at_stop: None, + osvw_at_bind, }; // 1. Reset the APIC state to clear the directed EOI bit, which is @@ -1303,9 +1324,33 @@ pub struct KvmProcessor<'a> { /// its LAPIC state is set. #[inspect(skip)] timer_at_stop: Option, + /// AMD's OSVW MSRs as KVM gave them to the vCPU, deriving them from the + /// host's, where the partition holds them, which a reset puts back. + #[inspect(skip)] + osvw_at_bind: Option, } impl KvmProcessor<'_> { + /// Resets the VP's state to what a new VP holds. The state's reset value + /// of the OSVW MSRs is 0, a CPU's before its firmware runs, but a new vCPU + /// holds the values that KVM derives from the host's, which a reboot's + /// firmware would write again, so the reset puts those back. + fn reset_state(&mut self) -> Result<(), virt::state::StateError> { + let vp_info = self.inner.vp_info; + let osvw_at_bind = self.osvw_at_bind; + let mut state = self.access_state(Vtl::Vtl0); + state.reset_all(&vp_info)?; + if let Some(osvw) = osvw_at_bind { + state + .set_osvw(&osvw) + .map_err(|err| virt::state::StateError::Access { + phase: "reset osvw", + err, + })?; + } + Ok(()) + } + /// Delivers any pending PIC interrupt. /// /// The VP must be known to be stopped and must have an open interrupt @@ -2208,8 +2253,7 @@ impl<'p> Processor for KvmProcessor<'p> { } fn reset(&mut self) -> Result<(), impl std::error::Error + Send + Sync + 'static> { - let vp_info = self.inner.vp_info; - self.access_state(Vtl::Vtl0).reset_all(&vp_info) + self.reset_state() } fn access_state(&mut self, vtl: Vtl) -> Self::StateAccess<'_> { diff --git a/vmm_core/virt_kvm/src/arch/x86_64/msr_state.rs b/vmm_core/virt_kvm/src/arch/x86_64/msr_state.rs index 457dca308c..72aec430f1 100644 --- a/vmm_core/virt_kvm/src/arch/x86_64/msr_state.rs +++ b/vmm_core/virt_kvm/src/arch/x86_64/msr_state.rs @@ -3,8 +3,8 @@ //! The guest-writable MSRs that the VP state holds as optional elements beyond //! the register-backed ones: which of them this backend can save for a -//! partition, and how it reads and writes the three feature and power MSRs -//! that no register name reaches. +//! partition, and how it reads and writes the three feature and power MSRs and +//! the AMD MSRs that no register name reaches. //! //! KVM keeps each of these in the vCPU, where a guest's write lands and a fresh //! vCPU starts with KVM's own value, so a restore that does not set them @@ -13,6 +13,8 @@ use crate::KvmError; use virt::x86::X86PartitionCapabilities; use virt::x86::vp::MiscMsrs; +use virt::x86::vp::Osvw; +use virt::x86::vp::SvmMsrs; /// The MSRs of [`MiscMsrs`], in the order of its fields. const MISC_MSRS: [u32; 3] = [ @@ -21,6 +23,18 @@ const MISC_MSRS: [u32; 3] = [ x86defs::X86X_MSR_POWER_CTL, ]; +/// The MSRs of [`SvmMsrs`], in the order of its fields. +const SVM_MSRS: [u32; 2] = [ + x86defs::X86X_AMD_MSR_VM_CR, + x86defs::X86X_AMD_MSR_VM_HSAVE_PA, +]; + +/// The MSRs of [`Osvw`], in the order of its fields. +const OSVW_MSRS: [u32; 2] = [ + x86defs::X86X_AMD_MSR_OSVW_ID_LENGTH, + x86defs::X86X_AMD_MSR_OSVW_ID_STATUS, +]; + /// Returns whether this backend can save and restore every MSR in `msrs` for a /// partition: the host has each, as `KVM_GET_MSR_INDEX_LIST` /// (`msr_index_list`) lists them, and the guest is not an SNP guest, whose @@ -29,17 +43,47 @@ fn saveable(msr_index_list: &[u32], snp: bool, msrs: &[u32]) -> bool { !snp && msrs.iter().all(|msr| msr_index_list.contains(msr)) } +/// What the host's KVM keeps for a vCPU beyond the MSRs that its +/// `KVM_GET_MSR_INDEX_LIST` names, as its supported CPUID tells. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub(super) struct KvmSupport { + /// KVM runs on AMD's SVM, as on an AMD host, whose vendor code keeps a + /// vCPU's `VM_CR` and `VM_HSAVE_PA`. + pub svm: bool, + /// KVM supports OSVW, as on a host that has it, which a vCPU needs for KVM + /// to keep its OSVW MSRs. + pub osvw: bool, +} + +impl KvmSupport { + /// Returns what KVM supports, from the result of each leaf and subleaf of + /// its supported CPUID, `cpuid`, which is zero for a leaf that it lacks. + pub(super) fn from_supported_cpuid(cpuid: impl Fn(u32, u32) -> [u32; 4]) -> Self { + let [_, ebx, ecx, edx] = cpuid(0, 0); + Self { + svm: x86defs::cpuid::Vendor::from_ebx_ecx_edx(ebx, ecx, edx).is_amd_compatible(), + osvw: virt::x86::enumerates_osvw(cpuid(0x8000_0001, 0)), + } + } +} + /// Sets the capabilities of the MSR state that this backend holds beyond -/// `IA32_SPEC_CTRL`: it clears `umwait_control`, which `from_cpuid` derived -/// from the guest's CPUID, for a host that lacks the MSR, and sets those that -/// no CPUID bit decides, where the host has the MSRs. +/// `IA32_SPEC_CTRL`: it clears `umwait_control`, `bndcfgs`, and `osvw`, which +/// `from_cpuid` derived from the guest's CPUID, for a host that lacks the MSR +/// or, for OSVW, whose KVM does not support OSVW, and sets those that no CPUID +/// bit decides, where the host's KVM keeps the MSRs, as `msr_index_list` and +/// `support` say. pub(super) fn set_capabilities( caps: &mut X86PartitionCapabilities, msr_index_list: &[u32], + support: KvmSupport, snp: bool, ) { caps.umwait_control &= saveable(msr_index_list, snp, &[x86defs::X86X_MSR_UMWAIT_CONTROL]); - // `IA32_DEBUGCTL` is the exception to the host's list: KVM virtualizes it + // KVM lists `IA32_BNDCFGS` only where it exposes MPX, which needs the + // host to enable MPX's state components, as Linux does on a CPU with MPX. + caps.bndcfgs &= saveable(msr_index_list, snp, &[x86defs::X86X_MSR_BNDCFGS]); + // `IA32_DEBUGCTL` is an exception to the host's list: KVM virtualizes it // for every vCPU, and `KVM_GET_MSRS` and `KVM_SET_MSRS` reach it on every // kernel that the time ABI supports, Linux 5.16 and later. A write fails // only for a bit that the KVM does not support (on VMX, which drops BTF and @@ -56,6 +100,18 @@ pub(super) fn set_capabilities( // the registers of the banks, whose number `IA32_MCG_CAP` gives for each // vCPU, so the capability follows `IA32_MCG_STATUS`. caps.machine_check = saveable(msr_index_list, snp, &[x86defs::X86X_MSR_MCG_STATUS]); + // KVM keeps AMD's OSVW MSRs for a vCPU that has OSVW, and lists neither of + // them. A vCPU has it where its CPUID enumerates it, which `from_cpuid` + // records, and KVM supports it: a kernel that keeps a vCPU's features + // within KVM's own, as recent ones do, refuses the MSRs of any other vCPU. + caps.osvw &= support.osvw && !snp; + // KVM on SVM keeps a guest's VM_CR and VM_HSAVE_PA whatever the guest's + // CPUID says. It lists `VM_HSAVE_PA`, and not `VM_CR`, which it emulates + // beside it, but it lists `VM_HSAVE_PA` wherever the host can read it, as + // an Intel host can under a hypervisor that reads it as 0, where KVM has + // no `VM_CR`, so the capability needs KVM to run on SVM too. + caps.svm_msrs = + support.svm && saveable(msr_index_list, snp, &[x86defs::X86X_AMD_MSR_VM_HSAVE_PA]); } /// Reads the feature and power MSRs of `vp`. @@ -89,6 +145,88 @@ pub(super) fn set_misc_msrs(vp: &kvm::Processor<'_>, value: &MiscMsrs) -> Result Ok(()) } +/// Reads AMD's SVM control and host save area MSRs of `vp`. +pub(super) fn svm_msrs(vp: &kvm::Processor<'_>) -> Result { + let mut values = [0; 2]; + vp.get_msrs(&SVM_MSRS, &mut values)?; + let [vm_cr, vm_hsave_pa] = values; + Ok(SvmMsrs { vm_cr, vm_hsave_pa }) +} + +/// Refuses a `VM_HSAVE_PA` that a guest could not have written: one that is +/// not page-aligned, or that lies beyond the guest's physical address width, +/// `physical_address_width`. KVM checks a guest's write so, but takes any value +/// from the host, and keeps only its page address. +fn check_restorable_vm_hsave_pa( + vm_hsave_pa: u64, + physical_address_width: u8, +) -> Result<(), KvmError> { + let beyond_width = vm_hsave_pa + .checked_shr(physical_address_width.into()) + .is_some_and(|high| high != 0); + if vm_hsave_pa & 0xfff != 0 || beyond_width { + return Err(KvmError::InvalidState( + "a VM_HSAVE_PA that is not a page address that the guest has", + )); + } + Ok(()) +} + +/// `VM_CR.SVMDIS`, which makes `EFER.SVME` reserved. +const VM_CR_SVMDIS: u64 = 1 << 4; + +/// Refuses a `VM_CR` that sets `SVMDIS` for a vCPU whose `EFER`, `efer`, which +/// the restore sets before `VM_CR`, has `SVME`. No guest holds both, since the +/// CPU refuses either write while the other is set, and KVM refuses the restore +/// only after it has set `SVMDIS` and `LOCK`, which it then keeps for the life +/// of the vCPU. +fn check_restorable_vm_cr(vm_cr: u64, efer: u64) -> Result<(), KvmError> { + if vm_cr & VM_CR_SVMDIS != 0 && efer & x86defs::X64_EFER_SVME != 0 { + return Err(KvmError::InvalidState( + "a VM_CR that sets SVMDIS while EFER.SVME is set", + )); + } + Ok(()) +} + +/// Writes AMD's SVM control and host save area MSRs of `vp`, whose guest's +/// physical address width is `physical_address_width`. KVM refuses a bit of +/// `VM_CR` outside the five that it emulates, so that MSR goes first, since KVM +/// takes the other from the host, so that a state that fails writes nothing, +/// and a `VM_HSAVE_PA` that the guest could not have written, or a `VM_CR` that +/// KVM would refuse only after it changed it, is refused before either is +/// written. +pub(super) fn set_svm_msrs( + vp: &kvm::Processor<'_>, + value: &SvmMsrs, + physical_address_width: u8, +) -> Result<(), KvmError> { + let SvmMsrs { vm_cr, vm_hsave_pa } = *value; + check_restorable_vm_hsave_pa(vm_hsave_pa, physical_address_width)?; + if vm_cr & VM_CR_SVMDIS != 0 { + check_restorable_vm_cr(vm_cr, vp.get_sregs()?.efer)?; + } + let [vm_cr_msr, vm_hsave_pa_msr] = SVM_MSRS; + vp.set_msrs(&[(vm_cr_msr, vm_cr), (vm_hsave_pa_msr, vm_hsave_pa)])?; + Ok(()) +} + +/// Reads AMD's OSVW MSRs of `vp`. +pub(super) fn osvw(vp: &kvm::Processor<'_>) -> Result { + let mut values = [0; 2]; + vp.get_msrs(&OSVW_MSRS, &mut values)?; + let [id_length, status] = values; + Ok(Osvw { id_length, status }) +} + +/// Writes AMD's OSVW MSRs of `vp`, which KVM takes with any value. +pub(super) fn set_osvw(vp: &kvm::Processor<'_>, value: &Osvw) -> Result<(), KvmError> { + let Osvw { id_length, status } = *value; + let [id_length_msr, status_msr] = OSVW_MSRS; + vp.set_msrs(&[(id_length_msr, id_length), (status_msr, status)])?; + Ok(()) +} + #[cfg(test)] mod tests { use super::*; @@ -97,6 +235,11 @@ mod tests { /// Returns the capabilities of a CPU whose CPUID has `waitpkg`, with every /// state that no CPUID bit decides left to the backend. pub(super) fn caps(waitpkg: bool) -> X86PartitionCapabilities { + caps_with(waitpkg, false) + } + + /// Returns the capabilities of a CPU whose CPUID has `waitpkg` and `mpx`. + fn caps_with(waitpkg: bool, mpx: bool) -> X86PartitionCapabilities { let topology = vm_topology::processor::TopologyBuilder::new_x86() .build(1) .unwrap(); @@ -106,7 +249,7 @@ mod tests { &mut |leaf, subleaf| match (leaf, subleaf) { (0, _) => [7, 0x756e_6547, 0x6c65_746e, 0x4965_6e69], (1, _) => [0, 0, u32::from(x2apic) << 21, 0], - (7, 0) => [0, 0, u32::from(waitpkg) << 5, 0], + (7, 0) => [0, u32::from(mpx) << 14, u32::from(waitpkg) << 5, 0], _ => [0; 4], }, ) @@ -129,6 +272,18 @@ mod tests { FULL_LIST.into_iter().filter(|&m| m != msr).collect() } + /// What KVM supports on an Intel host: neither SVM nor OSVW. + const INTEL: KvmSupport = KvmSupport { + svm: false, + osvw: false, + }; + + /// What KVM supports on an AMD host: SVM and OSVW. + const AMD: KvmSupport = KvmSupport { + svm: true, + osvw: true, + }; + /// The capabilities as `(umwait_control, debug_ctl, misc_msrs, /// machine_check)`. fn gates(caps: &X86PartitionCapabilities) -> (bool, bool, bool, bool) { @@ -143,13 +298,13 @@ mod tests { #[test] fn a_host_that_has_the_msrs_saves_what_the_guest_can_write() { let mut with = caps(true); - set_capabilities(&mut with, &FULL_LIST, false); + set_capabilities(&mut with, &FULL_LIST, INTEL, false); assert_eq!(gates(&with), (true, true, true, true)); // Without WAITPKG in the guest's CPUID, the guest cannot write // IA32_UMWAIT_CONTROL, whatever the host has. let mut without_waitpkg = caps(false); - set_capabilities(&mut without_waitpkg, &FULL_LIST, false); + set_capabilities(&mut without_waitpkg, &FULL_LIST, INTEL, false); assert_eq!(gates(&without_waitpkg), (false, true, true, true)); } @@ -176,21 +331,153 @@ mod tests { (x86defs::X86X_MSR_DEBUGCTL, (true, true, true, true)), ] { let mut caps = caps(true); - set_capabilities(&mut caps, &without(lacked), false); + set_capabilities(&mut caps, &without(lacked), INTEL, false); assert_eq!(gates(&caps), expected, "a host without MSR {lacked:#x}"); } // A host with none of the other MSRs saves none of the other states. let mut caps = caps(true); - set_capabilities(&mut caps, &[], false); + set_capabilities(&mut caps, &[], INTEL, false); assert_eq!(gates(&caps), (false, true, false, false)); } #[test] fn an_snp_guest_saves_none_of_them() { let mut caps = caps(true); - set_capabilities(&mut caps, &FULL_LIST, true); + set_capabilities(&mut caps, &FULL_LIST, INTEL, true); assert_eq!(gates(&caps), (false, false, false, false)); } + + #[test] + fn bndcfgs_needs_mpx_in_the_guest_and_the_msr_on_the_host() { + let with_bndcfgs = [&FULL_LIST[..], &[x86defs::X86X_MSR_BNDCFGS]].concat(); + let bndcfgs = |mpx, list: &[u32], snp| { + let mut caps = caps_with(false, mpx); + set_capabilities(&mut caps, list, INTEL, snp); + caps.bndcfgs + }; + // A guest with MPX, on a host whose KVM exposes it and so lists the + // MSR. + assert!(bndcfgs(true, &with_bndcfgs, false)); + // A host whose KVM does not list it, as one without MPX, holds none. + assert!(!bndcfgs(true, &FULL_LIST, false)); + // Without MPX in the guest's CPUID, the guest cannot write the MSR. + assert!(!bndcfgs(false, &with_bndcfgs, false)); + // Nor can KVM read an SNP guest's. + assert!(!bndcfgs(true, &with_bndcfgs, true)); + } + + /// Returns the capabilities of an AMD CPU whose CPUID has `osvw`. + fn amd_caps(osvw: bool) -> X86PartitionCapabilities { + let topology = vm_topology::processor::TopologyBuilder::new_x86() + .build(1) + .unwrap(); + let x2apic = topology.apic_mode() != vm_topology::processor::x86::ApicMode::XApic; + X86PartitionCapabilities::from_cpuid( + &topology, + &mut |leaf, subleaf| match (leaf, subleaf) { + (0, _) => [0xd, 0x6874_7541, 0x444d_4163, 0x6974_6e65], + (1, _) => [0, 0, u32::from(x2apic) << 21, 0], + (0x8000_0000, _) => [0x8000_0008, 0, 0, 0], + (0x8000_0001, _) => [0, 0, u32::from(osvw) << 9, 0], + _ => [0; 4], + }, + ) + .unwrap() + } + + #[test] + fn osvw_needs_osvw_in_the_guest_and_in_kvm() { + let osvw = |guest_osvw, kvm_osvw, snp| { + let mut caps = amd_caps(guest_osvw); + let support = KvmSupport { + osvw: kvm_osvw, + ..AMD + }; + set_capabilities(&mut caps, &FULL_LIST, support, snp); + caps.osvw + }; + // KVM does not list the MSRs, and keeps them for every vCPU whose + // CPUID enumerates OSVW where it supports OSVW, as on an AMD host. + assert!(osvw(true, true, false)); + assert!(!osvw(false, true, false)); + // A KVM that does not support OSVW, as on an Intel host, refuses them. + assert!(!osvw(true, false, false)); + // Nor can KVM read an SNP guest's. + assert!(!osvw(true, true, true)); + } + + #[test] + fn the_svm_msrs_need_kvm_on_svm_whatever_the_guest_has() { + let with_hsave = [&FULL_LIST[..], &[x86defs::X86X_AMD_MSR_VM_HSAVE_PA]].concat(); + let svm_msrs = |caps: X86PartitionCapabilities, list: &[u32], support, snp| { + let mut caps = caps; + set_capabilities(&mut caps, list, support, snp); + caps.svm_msrs + }; + // KVM on an AMD host, which lists VM_HSAVE_PA, keeps both MSRs, for a + // guest of either vendor's CPUID, though none of them has SVM. + assert!(svm_msrs(amd_caps(false), &with_hsave, AMD, false)); + assert!(svm_msrs(caps(false), &with_hsave, AMD, false)); + // KVM on an Intel host keeps neither, even where it lists VM_HSAVE_PA, + // as it does under a hypervisor that reads the MSR as 0. + assert!(!svm_msrs(amd_caps(true), &with_hsave, INTEL, false)); + assert!(!svm_msrs(amd_caps(true), &FULL_LIST, INTEL, false)); + // A host that does not list VM_HSAVE_PA has no SVM to keep them. + assert!(!svm_msrs(amd_caps(true), &FULL_LIST, AMD, false)); + // KVM cannot read an SNP guest's. + assert!(!svm_msrs(amd_caps(true), &with_hsave, AMD, true)); + } + + #[test] + fn kvm_support_follows_its_supported_cpuid() { + let support = |vendor: [u32; 3], ext1_ecx| { + KvmSupport::from_supported_cpuid(|leaf, _| match leaf { + 0 => [0xd, vendor[0], vendor[1], vendor[2]], + 0x8000_0001 => [0, 0, ext1_ecx, 0], + _ => [0; 4], + }) + }; + // "AuthenticAMD" and "GenuineIntel", as EBX, ECX, and EDX order them. + let amd = [0x6874_7541, 0x444d_4163, 0x6974_6e65]; + let intel = [0x756e_6547, 0x6c65_746e, 0x4965_6e69]; + assert_eq!(support(amd, 1 << 9), AMD); + assert_eq!( + support(amd, !(1 << 9)), + KvmSupport { + osvw: false, + ..AMD + } + ); + assert_eq!(support(intel, 0), INTEL); + } + + #[test] + fn a_restore_refuses_a_vm_hsave_pa_that_a_guest_could_not_write() { + // A page address within the guest's physical address width. + check_restorable_vm_hsave_pa(0, 48).unwrap(); + check_restorable_vm_hsave_pa(0x5000, 48).unwrap(); + check_restorable_vm_hsave_pa((1 << 48) - 0x1000, 48).unwrap(); + // Not page-aligned, or beyond the width. + for value in [0x5001, 0x5800, 1 << 48, !0xfff] { + assert!( + check_restorable_vm_hsave_pa(value, 48).is_err(), + "{value:#x}" + ); + } + } + + #[test] + fn a_restore_refuses_svmdis_while_efer_enables_svm() { + let svme = x86defs::X64_EFER_SVME; + // SVMDIS, with or without LOCK, while EFER.SVME is clear, and any + // other bits while it is set. + check_restorable_vm_cr(0x10, 0).unwrap(); + check_restorable_vm_cr(0x18, x86defs::X64_EFER_LME).unwrap(); + check_restorable_vm_cr(0x0f, svme).unwrap(); + // SVMDIS while EFER.SVME is set, which no guest can hold. + assert!(check_restorable_vm_cr(0x10, svme).is_err()); + assert!(check_restorable_vm_cr(0x18, svme | x86defs::X64_EFER_LME).is_err()); + } } /// Tests against the host's KVM. @@ -248,15 +535,27 @@ mod kvm_tests { assert_eq!(misc_msrs(&vp).unwrap(), value); } + /// What the host's KVM supports, from its supported CPUID. + fn host_support(kvm: &Kvm) -> KvmSupport { + let supported = kvm.supported_cpuid().unwrap(); + KvmSupport::from_supported_cpuid(|leaf, _| { + supported + .iter() + .find(|entry| entry.function == leaf) + .map_or([0; 4], |entry| [entry.eax, entry.ebx, entry.ecx, entry.edx]) + }) + } + /// The host's list has every MSR whose state a partition on it holds, so /// the partition's capabilities hold it. #[test] #[ignore = "requires /dev/kvm"] fn the_host_lists_the_msrs_that_its_partitions_hold() { - let list = Kvm::new().unwrap().msr_index_list().unwrap(); + let kvm = Kvm::new().unwrap(); + let list = kvm.msr_index_list().unwrap(); // The guest has no WAITPKG, so no state for IA32_UMWAIT_CONTROL. let mut caps = tests::caps(false); - set_capabilities(&mut caps, &list, false); + set_capabilities(&mut caps, &list, host_support(&kvm), false); assert!(caps.debug_ctl && caps.misc_msrs && caps.machine_check); assert!(!caps.umwait_control); } @@ -272,12 +571,15 @@ mod kvm_tests { vm.add_vp(0).unwrap(); let vp = vm.vp(0); - // KVM has IA32_DEBUGCTL on every host, and IA32_UMWAIT_CONTROL on a - // host with WAITPKG. + // KVM has IA32_DEBUGCTL on every host, IA32_UMWAIT_CONTROL on a host + // with WAITPKG, and IA32_BNDCFGS on a host with MPX. let mut registers = vec![(HvX64RegisterName::DebugCtl, 0x1d9)]; if list.contains(&0xe1) { registers.push((HvX64RegisterName::UmwaitControl, 0xe1)); } + if list.contains(&0xd90) { + registers.push((HvX64RegisterName::Bndcfgs, 0xd90)); + } for (name, msr) in registers { assert_eq!(super::super::regs::register_to_msr(name).unwrap(), msr); let mut value = [!0]; @@ -291,5 +593,97 @@ mod kvm_tests { vp.get_msrs(&[0xe1], &mut value).unwrap(); assert_eq!(value, [0x1005]); } + if list.contains(&0xd90) { + // MPX enabled at CPL 0, with BNDPRESERVE, and a bound directory + // at 0x1000. + vp.set_msrs(&[(0xd90, 0x1003)]).unwrap(); + let mut value = [0]; + vp.get_msrs(&[0xd90], &mut value).unwrap(); + assert_eq!(value, [0x1003]); + } + } + + /// KVM keeps VM_CR and VM_HSAVE_PA exactly where its partitions hold them: + /// where it runs on SVM and lists VM_HSAVE_PA, whatever the vCPU's CPUID + /// says. An Intel host whose own read of VM_HSAVE_PA succeeds, as under a + /// hypervisor that reads it as 0, lists it, but its KVM has no VM_CR. On an + /// AMD host, a new vCPU holds 0 in each, and the state reads back what it + /// set; a state that KVM refuses, or whose VM_HSAVE_PA a guest could not + /// write, writes nothing. + #[test] + #[ignore = "requires /dev/kvm"] + fn the_svm_msrs_are_kept_exactly_where_a_partition_holds_them() { + let kvm = Kvm::new().unwrap(); + let mut caps = tests::caps(false); + set_capabilities( + &mut caps, + &kvm.msr_index_list().unwrap(), + host_support(&kvm), + false, + ); + let mut vm = kvm.new_vm(VmType::Default).unwrap(); + vm.add_vp(0).unwrap(); + let vp = vm.vp(0); + assert_eq!(svm_msrs(&vp).is_ok(), caps.svm_msrs); + if !caps.svm_msrs { + println!("skipped: KVM does not run on SVM"); + return; + } + + assert_eq!(svm_msrs(&vp).unwrap(), SvmMsrs::default()); + // DPD and DIS_A20M in VM_CR, and a host save area at 0x5000. + let value = SvmMsrs { + vm_cr: 0x5, + vm_hsave_pa: 0x5000, + }; + set_svm_msrs(&vp, &value, 48).unwrap(); + assert_eq!(svm_msrs(&vp).unwrap(), value); + + for refused in [ + // A VM_CR bit beyond the five that KVM emulates. + SvmMsrs { + vm_cr: 0x20, + vm_hsave_pa: 0x6000, + }, + // A VM_HSAVE_PA that is not page-aligned. + SvmMsrs { + vm_cr: 0, + vm_hsave_pa: 0x6001, + }, + ] { + assert!(set_svm_msrs(&vp, &refused, 48).is_err(), "{refused:x?}"); + assert_eq!(svm_msrs(&vp).unwrap(), value); + } + } + + /// KVM keeps the OSVW MSRs for a vCPU whose CPUID enumerates OSVW, where it + /// supports OSVW, and the state reads back what it set. + #[test] + #[ignore = "requires /dev/kvm"] + fn the_osvw_msrs_are_kept_for_a_vcpu_with_osvw() { + let kvm = Kvm::new().unwrap(); + if !host_support(&kvm).osvw { + println!("skipped: KVM does not support OSVW"); + return; + } + let leaf = *kvm + .supported_cpuid() + .unwrap() + .iter() + .find(|entry| entry.function == 0x8000_0001) + .unwrap(); + let mut vm = kvm.new_vm(VmType::Default).unwrap(); + vm.add_vp(0).unwrap(); + let vp = vm.vp(0); + + // Without OSVW in its CPUID, the vCPU has neither MSR. + assert!(osvw(&vp).is_err()); + vp.set_cpuid(&[leaf]).unwrap(); + let value = Osvw { + id_length: 7, + status: 5, + }; + set_osvw(&vp, &value).unwrap(); + assert_eq!(osvw(&vp).unwrap(), value); } } diff --git a/vmm_core/virt_kvm/src/arch/x86_64/regs.rs b/vmm_core/virt_kvm/src/arch/x86_64/regs.rs index 106e2f107f..ba7269ff09 100644 --- a/vmm_core/virt_kvm/src/arch/x86_64/regs.rs +++ b/vmm_core/virt_kvm/src/arch/x86_64/regs.rs @@ -57,6 +57,7 @@ pub const fn register_to_msr(name: HvX64RegisterName) -> Result x86defs::X86X_MSR_SPEC_CTRL, HvX64RegisterName::UmwaitControl => x86defs::X86X_MSR_UMWAIT_CONTROL, HvX64RegisterName::DebugCtl => x86defs::X86X_MSR_DEBUGCTL, + HvX64RegisterName::Bndcfgs => x86defs::X86X_MSR_BNDCFGS, HvX64RegisterName::Xss => x86defs::X86X_MSR_XSS, HvX64RegisterName::UCet => x86defs::X86X_MSR_U_CET, diff --git a/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs b/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs index af90f1d245..fc241812b7 100644 --- a/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs +++ b/vmm_core/virt_kvm/src/arch/x86_64/vp_state.rs @@ -628,6 +628,34 @@ impl AccessVpState for KvmVpStateAccess<'_, '_> { super::machine_check::restore(&self.kvm(), value, self.vp.partition.mce_cmci_supported) } + fn bndcfgs(&mut self) -> Result { + self.get_register_state() + } + + fn set_bndcfgs(&mut self, value: &vp::Bndcfgs) -> Result<(), Self::Error> { + self.set_register_state(value) + } + + fn svm_msrs(&mut self) -> Result { + super::msr_state::svm_msrs(&self.kvm()) + } + + fn set_svm_msrs(&mut self, value: &vp::SvmMsrs) -> Result<(), Self::Error> { + super::msr_state::set_svm_msrs( + &self.kvm(), + value, + self.vp.partition.caps.physical_address_width, + ) + } + + fn osvw(&mut self) -> Result { + super::msr_state::osvw(&self.kvm()) + } + + fn set_osvw(&mut self, value: &vp::Osvw) -> Result<(), Self::Error> { + super::msr_state::set_osvw(&self.kvm(), value) + } + fn synic_msrs(&mut self) -> Result { self.get_register_state() } diff --git a/vmm_core/virt_mshv/src/x86_64/finalize.rs b/vmm_core/virt_mshv/src/x86_64/finalize.rs index 10a794b4f4..9c76f503c0 100644 --- a/vmm_core/virt_mshv/src/x86_64/finalize.rs +++ b/vmm_core/virt_mshv/src/x86_64/finalize.rs @@ -233,10 +233,13 @@ impl MshvPartitionInner { ); caps.nested_virt = self.config.nested_virt; caps.tsc_deadline = false; - // The VMSA holds an SNP guest's SPEC_CTRL, which the root cannot read. + // The VMSA holds an SNP guest's SPEC_CTRL, UMWAIT_CONTROL, and + // BNDCFGS, which the root cannot read. caps.spec_ctrl &= self.isolation_type() == virt::IsolationType::None; - // The VP state access does not hold UMWAIT_CONTROL. - caps.umwait_control = false; + caps.umwait_control &= self.isolation_type() == virt::IsolationType::None; + caps.bndcfgs &= self.isolation_type() == virt::IsolationType::None; + // The VP state access does not hold AMD's OSVW MSRs. + caps.osvw = false; caps.xsaves_state_bv_broken = true; // Ordinary state access does not freeze the partition clock. caps.can_freeze_time = false; diff --git a/vmm_core/virt_mshv/src/x86_64/mod.rs b/vmm_core/virt_mshv/src/x86_64/mod.rs index bf2e5b52ce..eb08531b9c 100644 --- a/vmm_core/virt_mshv/src/x86_64/mod.rs +++ b/vmm_core/virt_mshv/src/x86_64/mod.rs @@ -528,6 +528,9 @@ impl MshvPartitionInner { tsc_aux: false, spec_ctrl: false, umwait_control: false, + bndcfgs: false, + osvw: false, + svm_msrs: false, debug_ctl: false, misc_msrs: false, machine_check: false, diff --git a/vmm_core/virt_mshv/src/x86_64/vp_state.rs b/vmm_core/virt_mshv/src/x86_64/vp_state.rs index d349d5dbc8..e293540f9f 100644 --- a/vmm_core/virt_mshv/src/x86_64/vp_state.rs +++ b/vmm_core/virt_mshv/src/x86_64/vp_state.rs @@ -5,6 +5,7 @@ use crate::Error; use crate::ErrorInner; use crate::MshvProcessor; use crate::VcpuFdExt; +use hvdef::HvRegisterValue; use hvdef::HvX64RegisterName; use hvdef::hypercall::HvRegisterAssoc; use mshv_bindings::MSHV_VP_STATE_SIEFP; @@ -47,7 +48,17 @@ impl MshvProcessor<'_> { T: HvRegisterState, { let mut regs = T::default(); - let mut assoc = regs.names().map(|name| HvRegisterAssoc { + let values = self.get_register_values(regs.names())?; + regs.set_values(values.into_iter()); + Ok(regs) + } + + /// Returns the values of the registers that `names` lists, in its order. + pub(crate) fn get_register_values( + &self, + names: &[HvX64RegisterName; N], + ) -> Result<[HvRegisterValue; N], Error> { + let mut assoc = names.map(|name| HvRegisterAssoc { name: name.into(), pad: [0; 3], value: FromZeros::new_zeroed(), @@ -58,8 +69,7 @@ impl MshvProcessor<'_> { .get_hvdef_regs(&mut assoc[..]) .map_err(ErrorInner::Register)?; - regs.set_values(assoc.iter().map(|assoc| assoc.value)); - Ok(regs) + Ok(assoc.map(|assoc| assoc.value)) } fn set_state(&self, ty: u32, data: &[u8]) -> Result<(), Error> { @@ -253,7 +263,12 @@ impl AccessVpState for &'_ mut MshvProcessor<'_> { } fn activity(&mut self) -> Result { - let mut activity: vp::Activity = self.get_register_state()?; + // The hypervisor can hold a pending interruption that the activity + // cannot, such as a software interrupt that it is delivering, so the + // save refuses it rather than drop it. + let values = self.get_register_values(vp::Activity::default().names())?; + let mut activity = + vp::Activity::from_hv_values(values).map_err(ErrorInner::UnsupportedState)?; // The NMI pending bit is not part of the register state; it lives // in the APIC page. activity.nmi_pending = self.get_lapic()?.hv_apic_nmi_pending(); @@ -444,11 +459,11 @@ impl AccessVpState for &'_ mut MshvProcessor<'_> { } fn umwait_control(&mut self) -> Result { - Err(ErrorInner::UnsupportedState("umwait_control").into()) + self.get_register_state() } - fn set_umwait_control(&mut self, _value: &vp::UmwaitControl) -> Result<(), Self::Error> { - Err(ErrorInner::UnsupportedState("umwait_control").into()) + fn set_umwait_control(&mut self, value: &vp::UmwaitControl) -> Result<(), Self::Error> { + self.set_register_state(value) } fn debug_ctl(&mut self) -> Result { @@ -475,6 +490,30 @@ impl AccessVpState for &'_ mut MshvProcessor<'_> { Err(ErrorInner::UnsupportedState("machine_check").into()) } + fn bndcfgs(&mut self) -> Result { + self.get_register_state() + } + + fn set_bndcfgs(&mut self, value: &vp::Bndcfgs) -> Result<(), Self::Error> { + self.set_register_state(value) + } + + fn svm_msrs(&mut self) -> Result { + Err(ErrorInner::UnsupportedState("svm_msrs").into()) + } + + fn set_svm_msrs(&mut self, _value: &vp::SvmMsrs) -> Result<(), Self::Error> { + Err(ErrorInner::UnsupportedState("svm_msrs").into()) + } + + fn osvw(&mut self) -> Result { + Err(ErrorInner::UnsupportedState("osvw").into()) + } + + fn set_osvw(&mut self, _value: &vp::Osvw) -> Result<(), Self::Error> { + Err(ErrorInner::UnsupportedState("osvw").into()) + } + fn synic_msrs(&mut self) -> Result { self.get_register_state() } diff --git a/vmm_core/virt_whp/src/apic.rs b/vmm_core/virt_whp/src/apic.rs index 4a5989c28e..f692df527f 100644 --- a/vmm_core/virt_whp/src/apic.rs +++ b/vmm_core/virt_whp/src/apic.rs @@ -23,6 +23,7 @@ use std::sync::atomic::Ordering; use virt::VpIndex; use virt::io::CpuIo; use virt::irqcon::MsiRequest; +use virt::state::HvRegisterState; use virt::x86::MsrError; use virt::x86::vp; use virt_support_apic::ApicClient; @@ -180,9 +181,16 @@ impl ApicClient for WhpApicClient<'_, T> { impl WhpProcessor<'_> { pub(crate) fn save_activity(&mut self, vtl: Vtl) -> Result { + // The hypervisor can hold a pending interruption that the activity + // cannot, such as a software interrupt that it is delivering, so the + // save refuses it rather than drop it. + let values = self + .vp + .get_register_values(vtl, vp::Activity::default().names())?; + let registers = vp::Activity::from_hv_values(values).map_err(Error::UnsupportedState)?; let activity = match self.vp.partition.vtlp(vtl).lapic { LocalApicKind::Emulated(_) => { - let activity: vp::Activity = self.vp.get_register_state(vtl)?; + let activity = registers; let lapic = self.state.vtls[vtl].lapic.as_ref().unwrap(); vp::Activity { mp_state: if lapic.startup_suspend { @@ -203,7 +211,7 @@ impl WhpProcessor<'_> { } LocalApicKind::Offloaded => { // Get the NMI pending bit from the APIC. - let mut activity: vp::Activity = self.vp.get_register_state(vtl)?; + let mut activity = registers; let apic = vp::ApicRegisters::from_page( &self.vp.whp(vtl).get_apic().for_op("get apic state")?, ); diff --git a/vmm_core/virt_whp/src/lib.rs b/vmm_core/virt_whp/src/lib.rs index 75c4040830..093b77b921 100644 --- a/vmm_core/virt_whp/src/lib.rs +++ b/vmm_core/virt_whp/src/lib.rs @@ -1417,8 +1417,10 @@ impl WhpPartitionInner { let mut caps = caps.map_err(Error::Capabilities)?; caps.can_freeze_time = true; caps.tsc_deadline &= matches!(vtl0.lapic, LocalApicKind::Offloaded); - // The VP state access does not hold UMWAIT_CONTROL. - caps.umwait_control = false; + // The hypervisor leaves a guest's accesses of AMD's OSVW MSRs to + // the VMM, which reads them as 0 and refuses writes, so the VP + // holds no OSVW state to save. + caps.osvw = false; caps.xsaves_state_bv_broken = true; caps.dr6_tsx_broken = true; caps.nested_virt = nested_virt; diff --git a/vmm_core/virt_whp/src/regs.rs b/vmm_core/virt_whp/src/regs.rs index 4df59c00c3..ea478d3374 100644 --- a/vmm_core/virt_whp/src/regs.rs +++ b/vmm_core/virt_whp/src/regs.rs @@ -89,6 +89,8 @@ mod x86_64 { HvX64RegisterName::Sfmask => whp::abi::WHvX64RegisterSfmask, HvX64RegisterName::TscAux => whp::abi::WHvX64RegisterTscAux, HvX64RegisterName::SpecCtrl => whp::abi::WHvX64RegisterSpecCtrl, + HvX64RegisterName::UmwaitControl => whp::abi::WHvX64RegisterUnwaitControl, + HvX64RegisterName::Bndcfgs => whp::abi::WHvX64RegisterBndcfgs, HvX64RegisterName::Pat => whp::abi::WHvX64RegisterPat, HvX64RegisterName::MsrMtrrDefType => whp::abi::WHvX64RegisterMsrMtrrDefType, diff --git a/vmm_core/virt_whp/src/vp_state.rs b/vmm_core/virt_whp/src/vp_state.rs index 6b971125c9..cdd3eebfc3 100644 --- a/vmm_core/virt_whp/src/vp_state.rs +++ b/vmm_core/virt_whp/src/vp_state.rs @@ -62,7 +62,18 @@ impl WhpVpRef<'_> { T: HvRegisterState, { let mut regs = T::default(); - let names = regs.names().map(|name| hv_register_to_whp(name).unwrap()); + let values = self.get_register_values(vtl, regs.names())?; + regs.set_values(values.into_iter()); + Ok(regs) + } + + /// Returns the values of the registers that `names` lists, in its order. + pub(crate) fn get_register_values( + &self, + vtl: Vtl, + names: &[R; N], + ) -> Result<[HvRegisterValue; N], Error> { + let names = names.map(|name| hv_register_to_whp(name).unwrap()); let mut values = [HvRegisterValue::new_zeroed(); N]; self.whp(vtl) .get_registers(&names, unsafe { @@ -71,8 +82,7 @@ impl WhpVpRef<'_> { ) }) .for_op("get registers")?; - regs.set_values(values.into_iter()); - Ok(regs) + Ok(values) } } @@ -239,11 +249,11 @@ mod x86 { } fn umwait_control(&mut self) -> Result { - Err(Error::UnsupportedState("umwait_control")) + self.run.vp.get_register_state(self.vtl) } - fn set_umwait_control(&mut self, _value: &vp::UmwaitControl) -> Result<(), Self::Error> { - Err(Error::UnsupportedState("umwait_control")) + fn set_umwait_control(&mut self, value: &vp::UmwaitControl) -> Result<(), Self::Error> { + self.run.vp.set_register_state(self.vtl, value) } fn debug_ctl(&mut self) -> Result { @@ -270,6 +280,30 @@ mod x86 { Err(Error::UnsupportedState("machine_check")) } + fn bndcfgs(&mut self) -> Result { + self.run.vp.get_register_state(self.vtl) + } + + fn set_bndcfgs(&mut self, value: &vp::Bndcfgs) -> Result<(), Self::Error> { + self.run.vp.set_register_state(self.vtl, value) + } + + fn svm_msrs(&mut self) -> Result { + Err(Error::UnsupportedState("svm_msrs")) + } + + fn set_svm_msrs(&mut self, _value: &vp::SvmMsrs) -> Result<(), Self::Error> { + Err(Error::UnsupportedState("svm_msrs")) + } + + fn osvw(&mut self) -> Result { + Err(Error::UnsupportedState("osvw")) + } + + fn set_osvw(&mut self, _value: &vp::Osvw) -> Result<(), Self::Error> { + Err(Error::UnsupportedState("osvw")) + } + fn synic_msrs(&mut self) -> Result { match self.run.state.vtls[self.vtl].hv { Some(_) => { diff --git a/vmm_tests/vmm_tests/tests/virt_backend_tests/msr_state.rs b/vmm_tests/vmm_tests/tests/virt_backend_tests/msr_state.rs index d0057b074b..01eff2e98a 100644 --- a/vmm_tests/vmm_tests/tests/virt_backend_tests/msr_state.rs +++ b/vmm_tests/vmm_tests/tests/virt_backend_tests/msr_state.rs @@ -3,7 +3,8 @@ //! Guest-free contract tests of the guest-writable MSRs that a save holds beyond //! the registers: the feature and power controls, `IA32_UMWAIT_CONTROL`, -//! `IA32_DEBUGCTL`, and the machine-check state. +//! `IA32_BNDCFGS`, `IA32_DEBUGCTL`, the machine-check state, and AMD's SVM and +//! OSVW MSRs. //! //! A guest writes these MSRs at boot, a new VP starts without what it wrote, //! and a restore into a new VP must give the guest back what it saved. @@ -147,6 +148,161 @@ fn umwait_control_survives_a_save_and_restore( /// BUS_LOCK_DETECT, FREEZE_LBRS_ON_PMI, and RTM_DEBUG. const DEBUG_CONTROLS: [u64; 5] = [1 << 0, 1 << 1, 1 << 2, 1 << 11, 1 << 15]; +// A guest whose CPUID enumerates MPX can enable it at CPL 0 through +// IA32_BNDCFGS, which a new VP holds as 0. +backend_test!(bndcfgs_survives_a_save_and_restore, requires: [bndcfgs]); +fn bndcfgs_survives_a_save_and_restore( + _partition: &impl Partition, + processor: &mut impl Processor, +) -> anyhow::Result<()> { + let mut state = processor.access_state(Vtl::Vtl0); + anyhow::ensure!( + state.bndcfgs()? == vp::Bndcfgs::default(), + "a new VP does not hold the reset value of IA32_BNDCFGS" + ); + + // MPX enabled, with BNDPRESERVE, and a bound directory at 0x1000. + let written = vp::Bndcfgs { value: 0x1003 }; + state.set_bndcfgs(&written)?; + state.commit()?; + anyhow::ensure!( + state.bndcfgs()? == written, + "the VP does not hold the value that was just set" + ); + + let saved = save(&mut state)?; + state.set_bndcfgs(&vp::Bndcfgs::default())?; + state.commit()?; + + restore(&mut state, &saved)?; + let restored = state.bndcfgs()?; + anyhow::ensure!( + restored == written, + "the restore did not give the guest its configuration: saved {:#x}, restored {:#x}", + written.value, + restored.value + ); + Ok(()) +} + +// KVM on an AMD host keeps a guest's VM_CR and VM_HSAVE_PA, though the guest's +// CPUID has no SVM, and a new VP holds 0 in each. +backend_test!(svm_msrs_survive_a_save_and_restore, requires: [svm_msrs]); +fn svm_msrs_survive_a_save_and_restore( + _partition: &impl Partition, + processor: &mut impl Processor, +) -> anyhow::Result<()> { + let mut state = processor.access_state(Vtl::Vtl0); + anyhow::ensure!( + state.svm_msrs()? == vp::SvmMsrs::default(), + "a new VP does not hold the reset values of VM_CR and VM_HSAVE_PA: {:#x?}", + state.svm_msrs()? + ); + + // DPD and DIS_A20M in VM_CR, and a host save area at 0x5000. SVMDIS would + // stay set for the life of the vCPU, as KVM keeps it once set. + let written = vp::SvmMsrs { + vm_cr: 0x5, + vm_hsave_pa: 0x5000, + }; + state.set_svm_msrs(&written)?; + state.commit()?; + anyhow::ensure!( + state.svm_msrs()? == written, + "the VP does not hold the values that were just set" + ); + + let saved = save(&mut state)?; + state.set_svm_msrs(&vp::SvmMsrs::default())?; + state.commit()?; + + // A host save area that is not page-aligned, which a guest cannot write, + // is refused, and writes nothing. + anyhow::ensure!( + state + .set_svm_msrs(&vp::SvmMsrs { + vm_cr: 0x1, + vm_hsave_pa: 0x5008, + }) + .is_err(), + "a VM_HSAVE_PA that is not page-aligned is restored" + ); + anyhow::ensure!( + state.svm_msrs()? == vp::SvmMsrs::default(), + "a refused state changed the VP: {:#x?}", + state.svm_msrs()? + ); + + restore(&mut state, &saved)?; + let restored = state.svm_msrs()?; + anyhow::ensure!( + restored == written, + "the restore did not give the guest its values: saved {written:#x?}, restored {restored:#x?}" + ); + Ok(()) +} + +// A guest whose CPUID enumerates OSVW can write AMD's OSVW MSRs, whose values a +// new VP takes from the backend: KVM derives them from the host's. +backend_test!(osvw_survives_a_save_and_restore, requires: [osvw]); +fn osvw_survives_a_save_and_restore( + _partition: &impl Partition, + processor: &mut impl Processor, +) -> anyhow::Result<()> { + let mut state = processor.access_state(Vtl::Vtl0); + let new_vp = state.osvw()?; + + let written = vp::Osvw { + id_length: new_vp.id_length.wrapping_add(4), + status: new_vp.status ^ 0x5, + }; + state.set_osvw(&written)?; + state.commit()?; + anyhow::ensure!( + state.osvw()? == written, + "the VP does not hold the values that were just set" + ); + + let saved = save(&mut state)?; + state.set_osvw(&new_vp)?; + state.commit()?; + + restore(&mut state, &saved)?; + let restored = state.osvw()?; + anyhow::ensure!( + restored == written, + "the restore did not give the guest its values: saved {written:#x?}, restored {restored:#x?}" + ); + Ok(()) +} + +// A VM reset puts back the OSVW values that a new VP holds, as a reboot's +// firmware would write them again, not the reset value of the MSRs, 0. +backend_test!(a_reset_returns_osvw_to_what_a_new_vp_holds, requires: [osvw]); +fn a_reset_returns_osvw_to_what_a_new_vp_holds( + _partition: &impl Partition, + processor: &mut impl Processor, +) -> anyhow::Result<()> { + let new_vp = processor.access_state(Vtl::Vtl0).osvw()?; + { + let mut state = processor.access_state(Vtl::Vtl0); + state.set_osvw(&vp::Osvw { + id_length: new_vp.id_length.wrapping_add(4), + status: new_vp.status ^ 0x5, + })?; + state.commit()?; + } + + processor.reset()?; + + let reset = processor.access_state(Vtl::Vtl0).osvw()?; + anyhow::ensure!( + reset == new_vp, + "the reset did not give the VP the values of a new VP: new {new_vp:#x?}, reset {reset:#x?}" + ); + Ok(()) +} + // A backend drops the controls that its host or its guest's CPUID does not // enumerate, so the test saves the controls that the VP keeps. Linux sets // BUS_LOCK_DETECT when the guest's CPUID has it. diff --git a/vmm_tests/vmm_tests/tests/virt_backend_tests/native.rs b/vmm_tests/vmm_tests/tests/virt_backend_tests/native.rs index b6f80c6ac3..cee1d2fbf4 100644 --- a/vmm_tests/vmm_tests/tests/virt_backend_tests/native.rs +++ b/vmm_tests/vmm_tests/tests/virt_backend_tests/native.rs @@ -20,6 +20,9 @@ pub(crate) struct Capabilities { pub(crate) hv1: bool, pub(crate) spec_ctrl: bool, pub(crate) umwait_control: bool, + pub(crate) bndcfgs: bool, + pub(crate) osvw: bool, + pub(crate) svm_msrs: bool, pub(crate) debug_ctl: bool, pub(crate) misc_msrs: bool, pub(crate) machine_check: bool, @@ -43,6 +46,9 @@ where hv1: partition.caps().hv1, spec_ctrl: partition.caps().spec_ctrl, umwait_control: partition.caps().umwait_control, + bndcfgs: partition.caps().bndcfgs, + osvw: partition.caps().osvw, + svm_msrs: partition.caps().svm_msrs, debug_ctl: partition.caps().debug_ctl, misc_msrs: partition.caps().misc_msrs, machine_check: partition.caps().machine_check, diff --git a/vmm_tests/vmm_tests/tests/virt_backend_tests/unsupported_state.rs b/vmm_tests/vmm_tests/tests/virt_backend_tests/unsupported_state.rs index 8e5a5bbe4e..21528f984b 100644 --- a/vmm_tests/vmm_tests/tests/virt_backend_tests/unsupported_state.rs +++ b/vmm_tests/vmm_tests/tests/virt_backend_tests/unsupported_state.rs @@ -39,6 +39,12 @@ struct CpuState { misc_msrs: Option, #[mesh(20)] machine_check: Option, + #[mesh(21)] + bndcfgs: Option, + #[mesh(22)] + svm_msrs: Option, + #[mesh(23)] + osvw: Option, #[mesh(100)] synic: Option, } @@ -75,7 +81,7 @@ fn refuses_unsupported_cpu_state( let mut state = processor.access_state(Vtl::Vtl0); let before = state.registers()?; - let elements: [(&str, bool, fn(&mut CpuState)); 6] = [ + let elements: [(&str, bool, fn(&mut CpuState)); 9] = [ ("synic", true, |s| s.synic = Some(Default::default())), ("spec_ctrl", !caps.spec_ctrl, |s| { s.spec_ctrl = Some(Default::default()) @@ -92,6 +98,13 @@ fn refuses_unsupported_cpu_state( ("machine_check", !caps.machine_check, |s| { s.machine_check = Some(Default::default()) }), + ("bndcfgs", !caps.bndcfgs, |s| { + s.bndcfgs = Some(Default::default()) + }), + ("svm_msrs", !caps.svm_msrs, |s| { + s.svm_msrs = Some(Default::default()) + }), + ("osvw", !caps.osvw, |s| s.osvw = Some(Default::default())), ]; for (element, lacked, add) in elements { if !lacked {