Skip to content

[DO NOT REVIEW] virt: hold each SYSENTER MSR in the field that names it - #5100

Merged
Pedro Henrique Penna (ppenna) merged 1 commit into
feature/microvm/devfrom
copilot/nvx-issue-1367-sysenter-msr-order
Nov 14, 2026
Merged

Pedro Henrique Penna (ppenna) merged 1 commit into
feature/microvm/devfrom
copilot/nvx-issue-1367-sysenter-msr-order

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Holds IA32_SYSENTER_ESP and IA32_SYSENTER_EIP in the VirtualMsrs fields that name them, for microsoft/nvx#1367.

One commit on 5d2a71027, the head of feature/microvm/dev. microsoft/nvx#1588 pins it for microsoft/nvx#1367; this pull request lands first, by fast-forwarding feature/microvm/dev to c5509cde2.

Hold each SYSENTER MSR in the field that names it (c5509cde2)

  • Cause. VirtualMsrs::names lists SysenterEsp before SysenterEip, but get_values and set_values paired the registers with the fields in the other order. On KVM, MSHV, and WHP, which save and restore the type through those methods, sysenter_eip therefore held IA32_SYSENTER_ESP and sysenter_esp held IA32_SYSENTER_EIP. A save and a restore round-tripped, since the two swaps cancel, but anything that read the fields by name saw the wrong register: inspect output and tools that decode saved state. OpenHCL's TDX and SNP processors fill the same fields by name, so a field held different registers on different backends.
  • Change. get_values, set_values, and at_reset pair the registers with the fields in the order that names() lists them, and the fields are numbered in the order of the MSRs and declared in that order: field 3 is sysenter_esp and field 4 is sysenter_eip. mesh writes a message's fields in declaration order and reads them by number, so the bytes that the type encodes do not change. HvRegisterState documents the contract that VirtualMsrs broke: names, get_values, and set_values list the same registers in the same order.
  • Saved states. The saved states of KVM, MSHV, and WHP have always held IA32_SYSENTER_ESP in field 3 and IA32_SYSENTER_EIP in field 4, so each of them restores to the right registers, and a state saved after this change restores to the right registers in a build from before it. No OpenHCL saved state holds VirtualMsrs: the TDX, SNP, and CCA processors refuse to save, and the hypervisor-backed processor saves fields of its own. Of the 19 implementors of HvRegisterState, no other pairs its registers with its fields in an order that its names does not list.
  • Tests. virtual_msrs_write_each_register_from_its_own_field and virtual_msrs_read_each_register_into_its_own_field check that the registers that VirtualMsrs writes and reads carry the values of their own fields; the second is the issue's reproduction, registers SysenterEip and SysenterEsp holding 0x1111 and 0x2222, which read as sysenter_eip 0x2222 and sysenter_esp 0x1111 before the change. virtual_msrs_keep_the_layout_of_earlier_saved_states checks that a VirtualMsrs decodes a state that holds IA32_SYSENTER_ESP in field 3 and IA32_SYSENTER_EIP in field 4, and encodes one that does.

No version is created or bumped, and no snapshot, saved-state, or wire format changes.

Validation

With Rust 1.97.1 and cargo-nextest 0.9.133, on the rebased commit:

  • Reproduction. cargo nextest run --profile agent -p virt fails the three new tests at 5d2a71027 with only the tests applied, and passes them with this change.
  • Windows x64: cargo nextest run --profile agent -p virt passed 108 tests (1 skipped), cargo clippy --all-targets and cargo doc --no-deps on virt have no warnings, and cargo xtask fmt --fix changes nothing.
  • Linux x64, on nvx-mshv-test (Azure Linux 3.0): the same virt tests, clippy, and rustdoc pass, and cargo check passes for virt_mshv_vtl, virt_mshv, virt_kvm, and vmm_core, which use VirtualMsrs.
  • KVM, on nvx-kvm-test (a KVM virtual machine, Intel Xeon Platinum 8573C, Linux 6.18.57.1-1.azl3): cargo nextest run --profile agent --run-ignored all -p virt_kvm -p kvm passed 77 tests, the ignored /dev/kvm ones included.
  • CI. Run 43594226323 at c5509cde2 passed the six jobs that NVX's openvmm-ci requires: unit tests [x64-linux] (6582 tests) and unit tests [x64-windows] (5985 tests), which ran the three new virt tests, and run vmm-tests on x64-linux-amd-kvm (an AMD EPYC 9V45, 110 tests), x64-linux-intel-mshv, x64-windows-intel, and x64-windows-amd. The binary-size jobs passed, and the SEV-SNP job was not waited for: x64-windows-amd-snp failed. run vmm-tests [aarch64-windows] failed one test, multiarch::hyperv_openhcl_uefi_aarch64_ubuntu_2404_server_aarch64_boot_heavy, which timed out after 663 s in an aarch64 OpenHCL Ubuntu boot while the other 97 passed. That job passed in the four earlier runs of the branch, and this change touches only x86 code and a doc comment, so I did not rerun it.
  • Real guests. On nvx-kvm-test (KVM, Intel Xeon Platinum 8573C, Linux 6.18.57.1-1.azl3), nvx-mshv-test (MSHV, Intel Xeon Platinum 8573C, Linux 6.18.34.mshv3-1.azl3), and nvx-whp-test (WHP, Intel Xeon Platinum 8370C, Windows Server 2025), all virtual machines, a Linux guest of one or two CPUs read the IA32_SYSENTER_ESP and IA32_SYSENTER_EIP of each CPU through /dev/cpu/N/msr, with EIP the address of entry_SYSENTER_compat, and requested a snapshot. OpenVMM built from 157866605, which lacks only the EFER.SVME restore check of 5d2a71027, saved the ESP in field 3 and the EIP in field 4 of each VP's VirtualMsrs; OpenVMM built from this commit saved the same bytes. A snapshot that either build captured, restored with the other build, kept each CPU's two MSRs, on all three backends and for one and two CPUs.

On KVM, MSHV, and WHP, VirtualMsrs::sysenter_eip held IA32_SYSENTER_ESP
and sysenter_esp held IA32_SYSENTER_EIP. names() lists SysenterEsp
before SysenterEip, but get_values and set_values paired the registers
with the fields in the other order. A save and a restore round-tripped,
since the two swaps cancel, but anything that read the fields by name
saw the wrong register: inspect output and tools that decode saved
state. OpenHCL's TDX and SNP processors fill the same fields by name, so
a field held different registers on different backends.

Pair the registers with the fields in the order that names() lists them,
and number the fields in the order of the MSRs, field 3 for
IA32_SYSENTER_ESP and field 4 for IA32_SYSENTER_EIP, declaring them in
that order. The saved layout does not change, so no version or marker is
needed: the saved states of KVM, MSHV, and WHP have always held ESP in
field 3 and EIP in field 4, so each of them restores to the right
registers, and a state saved after this change restores to the right
registers in a build from before it. No OpenHCL saved state holds
VirtualMsrs: the TDX, SNP, and CCA processors refuse to save, and the
hypervisor-backed processor saves fields of its own. HvRegisterState now
documents the contract that VirtualMsrs broke: names, get_values, and
set_values list the same registers in the same order.

Tests: the three new tests in virt fail at 5d2a710 and pass here. Two
check that the registers that VirtualMsrs writes and reads carry the
values of their own fields; the second is the issue's reproduction,
registers SysenterEip and SysenterEsp holding 0x1111 and 0x2222, which
reads as sysenter_eip 0x2222 and sysenter_esp 0x1111 at 5d2a710. The
third checks that a VirtualMsrs decodes a state that holds
IA32_SYSENTER_ESP in field 3 and IA32_SYSENTER_EIP in field 4, and
encodes one that does. The 108 tests of virt pass on Windows and on
Azure Linux 3.0, with clippy and rustdoc.

On nvx-kvm-test, nvx-mshv-test, and nvx-whp-test, a guest read the
IA32_SYSENTER_ESP and IA32_SYSENTER_EIP of each of its one or two CPUs
through /dev/cpu/N/msr, with EIP the address of entry_SYSENTER_compat,
and requested a snapshot. OpenVMM built from 1578666, which lacks only
the EFER.SVME restore check of 5d2a710, saved the ESP in field 3 and
the EIP in field 4 of each VP's VirtualMsrs; OpenVMM built from this
commit saved the same bytes. A snapshot that either build captured,
restored with the other build, kept each CPU's two MSRs.

Reported in microsoft/nvx#1367.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added the release-nvx Targets a branch under feature/microvm/. label Nov 14, 2026
@ppenna
Pedro Henrique Penna (ppenna) merged commit c5509cd into feature/microvm/dev Nov 14, 2026
71 of 74 checks passed
@github-actions

Copy link
Copy Markdown

syeedali pushed a commit to syeedali/nvx that referenced this pull request Nov 15, 2026
On KVM, MSHV, and WHP, a VP's saved VirtualMsrs held IA32_SYSENTER_ESP
in sysenter_eip and IA32_SYSENTER_EIP in sysenter_esp, since get_values
and set_values paired the registers with the fields in the opposite
order of names(). A save and a restore round-tripped, because the two
swaps cancel, but anything that read the fields by name saw the wrong
register, and OpenHCL's TDX and SNP processors, which fill the fields by
name, disagreed with these backends (microsoft#1367).

Pin c5509cde20d624fa49b90722c34ad292ab7672cb, the head of
microsoft/openvmm#5100, one commit on 5d2a71027, the head of
feature/microvm/dev:

- c5509cde2 virt: hold each SYSENTER MSR in the field that names it

get_values, set_values, and at_reset now pair the registers with the
fields in the order that names() lists them, and the two fields swap
their declaration order and their numbers, so that field 3 holds
IA32_SYSENTER_ESP and field 4 holds IA32_SYSENTER_EIP, as KVM, MSHV, and
WHP have always saved them. The bytes that a VirtualMsrs encodes do not
change, so every snapshot of the current format restores with the right
SYSENTER_EIP and SYSENTER_ESP, and a snapshot that the pin saves
restores the same way in a build from before it. No OpenHCL saved state
holds VirtualMsrs, so none is misread. No version is created or bumped,
and no snapshot or wire format changes.

Tests: the revision's three tests of virt fail at 5d2a71027 and pass
with it, and its 108 tests of virt pass on Windows and on Azure Linux
3.0. On nvx-kvm-test its 77 tests of virt_kvm and kvm, the ignored
/dev/kvm ones included, pass as well. A guest on nvx-kvm-test,
nvx-mshv-test, and nvx-whp-test read the IA32_SYSENTER_ESP and
IA32_SYSENTER_EIP of each of its one or two CPUs and requested a
snapshot: OpenVMM built from 157866605 and OpenVMM built from the pin's
tree saved the same bytes, and a snapshot that either build captured,
restored with the other build, kept each CPU's two MSRs. With OpenVMM
built from the pin's tree and the artifacts of v0.1.0-dev.0b1029a5c372,
the microVM scenarios smp, smp-snapshot, snapshot-core,
snapshot-rollback, restore-downtime, restore-processors, snapshot-tiers,
and time-abi-conformance pass at 1 and 2 processors on all three hosts.

NVX needs no coordinated change: the control-session vectors, the source
excerpts, the contract versions, the activation bound, and the CPU
profiles that nvx-rs copies from OpenVMM match the revision's, and
nothing of NVX names the fields of VirtualMsrs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
syeedali pushed a commit to syeedali/nvx that referenced this pull request Nov 15, 2026
The NVX pin moves to c5509cde2, position 280, 156 commits past the
mapped tip. Position 280, the one commit of microsoft/openvmm#5100 on
5d2a71027, the head of feature/microvm/dev, is not yet assigned to a PR.

Position 280 makes VirtualMsrs, the virtual MSRs of a VP, hold
IA32_SYSENTER_ESP in sysenter_esp and IA32_SYSENTER_EIP in sysenter_eip,
which KVM, MSHV, and WHP held in the other's field, since get_values and
set_values paired the registers with the fields in the opposite order of
names. It swaps the fields and their numbers, so that the type encodes
the same bytes as before, and documents on HvRegisterState that its
three methods list the same registers in the same order. It changes
code that upstream main has and no commit of the chain changes, adds
three tests to the tests module that position 132 adds and position 251
extends, and changes nothing that a mapped PR adds. Replaying it on
upstream main, 7d90c4104, conflicts only where its tests extend that
module, which upstream main lacks, so they follow positions 132 and 251
upstream, and the rest of the commit applies there without conflicts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-nvx Targets a branch under feature/microvm/.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant