Skip to content

openvmm: pin the IOAPIC's zero-extended wide reads - #1570

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/issue-1140-ioapic-wide-read
Nov 13, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/issue-1140-ioapic-wide-read

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1140

What and why

This pins microsoft/openvmm#5084 at e2458a6cef958c3ac2d710e13a6ba490a9911b88. With the pin, a guest read of the emulated IOAPIC's index or data register that is wider than 4 bytes, such as a 64-bit mov from 0xfec00000, returns the 32-bit register zero-extended. Before, such a read panicked OpenVMM's vCPU thread and ended the VM.

At NVX's previous pin, 09cd771c0:

  • Cause. IoApicDevice::mmio_read computed n = data.len().min(4) and then called data.copy_from_slice(&v.to_ne_bytes()[..n]), which requires both slices to have the same length. Reads of 1 to 4 bytes worked. A 5- to 8-byte read panicked with "copy_from_slice: source slice length (4) does not match destination slice length (8)" at vm/devices/chipset/src/ioapic/mod.rs:724:14.
  • Reach. The IOAPIC is OpenVMM's own on every backend (KVM runs with a split irqchip), so the guest's MMIO reaches IoApicDevice on KVM, MSHV, and WHP alike. Any guest that can issue the access, such as a process with /dev/mem or a kernel driver, could end its own VM.

What the OpenVMM pull request changes

microsoft/openvmm#5084 adds one signed commit on 09cd771c0, the head of feature/microvm/dev. It has a good signature (openvmm_pin.py source).

Position Commit Change
274 e2458a6ce mmio_read copies the register's n low bytes into data[..n] and zeroes data[n..]. A wider read returns the 32-bit register zero-extended, as Rtc::io_read does for a wide read of its ports since microsoft/openvmm#5072, and a narrower one still returns the register's low bytes. A read of another offset still fails with InvalidRegister and leaves the buffer as it was. The test mmio_reads_of_any_size_are_zero_extended sets the IOAPIC's ID and redirection entry 5, then reads the index register and the data register (with the ID, the version, and both halves of entry 5 selected) at every length from 1 to 8 bytes, into a buffer of 0xa5 bytes, and requires the register's low bytes followed by zeros.

The patch that Ruize Tang (@tangruize) attached to #1140 makes the same change.

Pin details

  • Roadmap. The pin is 150 commits past the mapped tip.
    • Position 274 is not yet assigned to a PR.
    • Its code is upstream main's. Its test goes into the device's top-level mod tests, in a file whose saved-state tests (save_restore::tests) position 183 also changes. It changes none of position 183's code and nothing that a mapped PR adds.
    • Replaying 274 on upstream main, 7d90c4104, applies without conflicts (merged tree 49dce963c). There, cargo check -p chipset passes, and cargo nextest run -p chipset passes all 69 tests, including the new one, so the commit can go upstream on its own.
  • Prerequisites. None. The pull request's base, 09cd771c0, is NVX's previous pin and the head of feature/microvm/dev.
  • Equivalence. None claimed; this pin replaces no reviewed revision.
  • Coordinated changes. None. The control-session vectors, the source excerpts, the contract versions, the activation bound, and the CPU profiles that nvx-rs copies from OpenVMM all match the revision's.

Scope and limits

Landing order

microsoft/openvmm#5084 lands first, by fast-forwarding feature/microvm/dev from 09cd771c0 to e2458a6ce (python3 scripts/openvmm_pin.py land e2458a6cef958c3ac2d710e13a6ba490a9911b88 --remote origin --push). This pull request merges after that. OpenVMM's merge button would squash the commit and leave the pinned revision off the branch, so don't use it.

Validation

NVX

Run on Windows:

  • python scripts\openvmm_pin.py source e2458a6cef958c3ac2d710e13a6ba490a9911b88 --pr 5084: passed. The revision is the pull request's head, with a good signature.
  • python scripts\openvmm_pin.py check-roadmap: passed, with 274 positions and 285 references.
  • python scripts\openvmm_pin.py check-branch --target origin/dev and check-branch --topic: passed, with two signed commits and no coordinated files.
  • python scripts\check_markdown_links.py and git diff --check: passed.
  • python scripts\nvx.py verify: passed.
  • python scripts\local_gates.py --base origin/dev --only test-changes markdown-links test-layout openvmm-vectors openvmm-cpu-profiles nvx-cli --run --keep-going, with NVX_RS set to a release nvx-rs:
    • test-changes, markdown-links, test-layout, openvmm-vectors, and openvmm-cpu-profiles: passed. test-changes found no changes to existing tests.
    • nvx-cli: 2198 tests ran, 88 skipped. Two steps that the gate's script never reached because of one failure, scripts\test_adversarial.py -v (84 tests OK, 11 skipped) and the eight nvx.py … --help commands, then passed when run on their own.
    • The failure, in IssueWorkspaceTests.test_find_groups_exact_issue_matches_from_every_source, is a flaky test, unrelated to this change. The test's temporary repository gets commits with fresh timestamps, so their SHAs are random. find names a commit candidate by its 12-digit abbreviated SHA and subject. The test asserts that no candidate's name contains 570, which such a prefix does about once in 200 runs. It failed 3 times in about 490 runs: once in the gate, once in 8 standalone runs, and twice in loops. The last loop captured the offending candidate, commits on origin/dev | 0534a5706fbf Fix #57 on dev. This branch changes no scripts.

OpenVMM

For chipset, with this change, with Rust 1.97.1 and cargo-nextest 0.9.133:

  • Windows x64: cargo check, cargo clippy --all-targets, cargo doc --no-deps, and cargo nextest run --profile agent (132 passed) passed, and cargo xtask fmt --fix changed nothing.
  • Linux x64, on nvx-kvm-test: cargo check, cargo clippy --all-targets, cargo doc --no-deps, and cargo nextest run --profile agent (132 passed) passed.
  • Regression: at 09cd771c0, mmio_reads_of_any_size_are_zero_extended fails at its first 5-byte read with the panic above, at mod.rs:724:14.

Hosts

The checkouts were at #1569's head, f02bdcd53. On each host, nvx-rs download installed v0.1.0-dev.131aa44c4207, the nearest first-parent ancestor with a release. Then nvx.py build-openvmm built OpenVMM from the pinned revision. With the gitlink staged at e2458a6ce, the hosts' index tree was d1062d7d0, the tree of this branch's pin commit. record-openvmm-provenance reports source_revision e2458a6ce and a clean source.

A static guest program in the Alpine microVM mapped /dev/mem at 0xfec00000. It read the index (+0x00) and data (+0x10) registers with 32-bit loads, then with 64-bit loads. It got 0x34 and 0x21 with 1 processor and 0x34 and 0x20 with 2. nvx.py test-microvm --backend BACKEND --scenario interrupt-map --scenario smp --processors 1 2 ran smp with 1 and 2 processors and interrupt-map once.

Host Platform Host type OpenVMM 09cd771c0, 1 processor OpenVMM e2458a6ce (SHA-256), 1 and 2 processors Scenarios at e2458a6ce
nvx-kvm-test linux-kvm virtual-machine 32-bit reads correct, first 64-bit read panics at mod.rs:724:14 7a9db54a…: 64-bit reads zero-extended passed
nvx-mshv-test linux-mshv virtual-machine 32-bit reads correct, first 64-bit read panics at mod.rs:724:14 ca02013e…: 64-bit reads zero-extended passed
nvx-whp-test windows-whp virtual-machine 32-bit reads correct, first 64-bit read panics at mod.rs:724:14 41c642c6…: 64-bit reads zero-extended passed

CI

  • NVX CI: run 43516816365, of 799ad739c, passed: the microVM matrix on KVM, MSHV, and WHP, NVX CLI on Linux and Windows, the performance gate, OpenVMM CI of the pinned revision, and Required status check.
  • OpenVMM's CI of e2458a6ce: run 43511358061. The six jobs that NVX requires passed: unit tests [x64-linux] and [x64-windows], and run vmm-tests for x64-linux-amd-kvm, x64-linux-intel-mshv, x64-windows-intel, and x64-windows-amd. Two jobs that NVX doesn't require failed in the first attempt:
    • run vmm-tests [x64-windows-intel-tdx] failed one test of 119. memstat::…_memory_validation_debug_very_heavy, a Windows OpenHCL guest's memory test, timed out after 770 s. This change can't affect it: reads of up to 4 bytes behave as before, and wider ones panicked before. TDX passed at the base, so this was a transient timeout. Its one rerun passed, and so did openvmm checkin gates.
    • run vmm-tests [x64-windows-amd-snp] fails at the base too (run 43489085265). 19 of 86 tests failed there and 16 here, 11 of them in both runs, mostly with "VM did not start after reset in the required time". Neither run logs an IOAPIC warning or error. It wasn't rerun.

A guest read of the emulated IOAPIC's index or data register wider
than 4 bytes, such as a 64-bit mov from 0xfec00000, panicked OpenVMM's
VP thread with "copy_from_slice: source slice length (4) does not
match destination slice length (8)" in IoApicDevice::mmio_read, which
ended the VM (#1140).

Pin e2458a6cef958c3ac2d710e13a6ba490a9911b88, the head of
microsoft/openvmm#5084, one commit on 09cd771c0, the head of
feature/microvm/dev:

- e2458a6ce chipset: zero-extend wide IOAPIC MMIO reads

mmio_read copied the 32-bit register's min(len, 4) low bytes into the
whole buffer, so only reads of at most 4 bytes matched the slice
lengths. It now copies those bytes into the start of the buffer and
zeroes the rest, so a wider read returns the register zero-extended,
as the CMOS RTC does for a wide read of its ports, and a narrower one
still returns the register's low bytes.

Evidence: with OpenVMM built from 09cd771c0, the previous pin, an
Alpine guest on 1 processor that maps /dev/mem at 0xfec00000 and reads
the index and data registers with 32-bit and then 64-bit loads
panicked OpenVMM at vm/devices/chipset/src/ioapic/mod.rs:724:14 on its
first 64-bit load, on nvx-kvm-test, nvx-mshv-test, and nvx-whp-test.
At the pin, its 64-bit loads read the values of its 32-bit loads,
zero-extended, on all three hosts, on 1 and 2 processors.

Tests: with OpenVMM built from the pin, the microVM scenarios smp, on
1 and 2 processors, and interrupt-map pass on nvx-kvm-test,
nvx-mshv-test, and nvx-whp-test. The revision's new chipset test,
mmio_reads_of_any_size_are_zero_extended, fails at 09cd771c0 with the
panic above, and its 132 chipset tests pass on Windows and on Linux,
on nvx-kvm-test.

NVX needs no coordinated change: the control-session vectors, the
source excerpts, the contract versions, the activation bound, and the
CPU profiles that nvx-rs copies from OpenVMM match the revision's.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The NVX pin moves to e2458a6ce, position 274, 150 commits past the
mapped tip. Position 274, the one commit of microsoft/openvmm#5084 on
09cd771c0, the head of feature/microvm/dev, is not yet assigned to a
PR.

Position 274 makes the IOAPIC's mmio_read zero-extend a read of its
index or data register wider than 4 bytes, which panicked the vCPU
thread, and so OpenVMM, because it copied the register's low bytes
into the whole buffer. The code is upstream main's, and the commit
adds a test to the device's tests, in a file whose saved-state tests
position 183 also changes; it changes none of position 183's code and
nothing that a mapped PR adds. Replaying it on upstream main,
7d90c4104, applies without conflicts, and chipset type-checks there
with its tests, which pass, so it can go upstream on its own.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the roadmap pin and bookkeeping update.

No open issues 🎉

What changed in this PR

This PR pins the OpenVMM fix for #1140, preventing wide IOAPIC register reads from panicking the VM by zero-extending the 32-bit value.

Changes:

  • Updates the roadmap’s pinned revision to e2458a6ce at position 274.
  • Adjusts chain counts and records the new commit as unassigned to an upstream PR.
  • Documents the fix’s scope and its ability to go upstream independently.
File Description
doc/​openvmm-upstream-roadmap.md Records the new pin, updates chain bookkeeping, and explains the IOAPIC fix’s upstream applicability.

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Nov 13, 2026
Merged via the queue into dev with commit 6f08411 Nov 13, 2026
48 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-1140-ioapic-wide-read branch November 13, 2026 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openvmm: 64-bit IOAPIC MMIO reads panic in IoApicDevice::mmio_read

2 participants