Repository navigation
openvmm: skip a periodic timer's missed periods at once - #1565
Merged
Pedro Henrique Penna (ppenna) merged 2 commits intoNov 13, 2026
Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
Pedro Henrique Penna (ppenna) merged 2 commits into
Conversation
OpenVMM's VmTimerPeriodic moved its deadline one period per iteration until it passed the current VM time, so after a restore whose downtime moved VM time far past the deadline, the poll iterated once for every missed period (#1164). The RTC uses it for its periodic interrupt, at up to 8192 Hz, and for its update-ended interrupt, at 1 Hz, and the RTC's state unit polls the device as the restored VM starts, so a guest that had enabled the RTC's periodic interrupt at 8192 Hz ran again only some 15 minutes after a restore from a 30-day downtime. Pin 5ca20ee081c255bed107e58b6dc5b5b769b6101d, the head of microsoft/openvmm#5073, one commit on 60e160369, the head of feature/microvm/dev: - 5ca20ee08 vmcore: skip a periodic timer's missed periods at once A running timer's poll now moves its timeout to the first of its deadlines after now in one step, and registers its waker for that deadline under the same lock and clock reading as the check that found the timeout due, so that it neither loops nor wakes its own task. Evidence: in NVX's Alpine microVM, a guest enabled the RTC's periodic interrupt at 8192 Hz through /dev/port and requested a capture, and each restore timed the guest's first line after the capture: after the real downtime, and, with OpenVMM's downtime-add-s hook, after a simulated day and after 30 simulated days. With OpenVMM 60e160369, they took 0.09 s, 30.8 s, and 921.7 s on nvx-kvm-test, 0.04 s, 30.7 s, and 917.3 s on nvx-mshv-test, and 0.15 s, 27.2 s, and 809.8 s on nvx-whp-test. With OpenVMM built from the pin, they took 0.07 s, 0.09 s, and 0.11 s on nvx-kvm-test, 0.04 s, 0.04 s, and 0.06 s on nvx-mshv-test, and 0.59 s, 0.18 s, and 0.20 s on nvx-whp-test, and after each restore the guest found the RTC's periodic interrupt flag set, and set again 10 ms after its read had cleared it. Tests: with OpenVMM built from the pin, the microVM scenarios guest-boot, lifecycle, interrupt-map, snapshot-core, restore-processors, smp-snapshot, restore-downtime, time-abi-conformance, and time-host-correction pass with 1 and 2 processors on nvx-kvm-test, nvx-mshv-test, and nvx-whp-test. The revision's vmcore, chipset, and chipset_legacy tests, 155 in all, pass on Windows and on Linux, on nvx-kvm-test. NVX needs no coordinated change: the control-session vectors, the source excerpts, the contract versions, the activation bound, and the CPU profiles that nvx-rs copies from OpenVMM match the revision's. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The NVX pin moves to 5ca20ee08, position 268, 137 commits past the mapped tip. Position 268, which microsoft/openvmm#5073 proposes for feature/microvm/dev on its head, 60e160369, is unassigned. The Scope also says that microsoft/openvmm#5072 merged position 267. Position 268 makes VmTimerPeriodic skip at once the deadlines that VM time passed, and register its next deadline under the timer state's lock, in code that upstream main has, and adds tests, in a file that PR10's position 019 and positions 156, 157, and 255 also change, none of them in the code that it changes. It changes nothing that a mapped PR adds. Replaying it on upstream main, 7d90c4104, applies without conflicts, and vmcore type-checks there with all its targets and its 35 tests pass, so it can go upstream on its own. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The OpenVMM pin and roadmap updates have no active review issues.
No open issues 🎉
What changed in this PR
This PR addresses #1164 by pinning an OpenVMM fix that skips missed periodic timer deadlines in one step, avoiding long restore stalls when RTC periodic interrupts are enabled.
Changes:
- Advances the OpenVMM pin to
5ca20ee08. - Maps the fix to roadmap position 268 and updates the chain counts.
- Documents the fix’s scope and independent upstream applicability.
| File | Description |
|---|---|
doc/openvmm-upstream-roadmap.md |
Records the new pin, position 268, updated commit counts, and upstream applicability. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-1164-periodic-timer-catch-up
branch
November 13, 2026 06:31
This was referenced Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1164
What and why
A restore after a long downtime stalled when the guest had enabled the CMOS RTC's periodic interrupt. At 8192 Hz, status A's rate selector 3, the restored guest ran again only some 15 minutes after a restore from a 30-day downtime, on KVM, MSHV, and WHP alike, and some 30 s after a 1-day one.
Root cause, in OpenVMM:
VmTimerPeriodic::poll_timeoutinvm/vmcore/src/vmtime.rsmoved its deadline one period per loop iteration until it passed the current VM time, and returned ready once for all the deadlines that it passed. A restore's downtime advance moves VM time past the deadline by the whole downtime, some 21 billion periods of 122 µs in 30 days, and the RTC's state unit polls the device, under its lock, as the restored VM starts. The RTC's update-ended interrupt, at 1 Hz, uses the same timer, and a period below VM time's 100 ns resolution counted as zero, so the loop would have spun forever once such a timer fired; the RTC never asks for one.NVX's own guests leave the RTC's periodic interrupt off: their kernel has no RTC driver (
CONFIG_RTC_CLASSis not set), which is why therestore-downtimescenario, which restores after a simulated year, never met the stall. A guest that programs the RTC through/dev/portoriopermdoes.This builds on #1563, which has merged; its pin,
60e160369, is the head offeature/microvm/dev.The pin, microsoft/openvmm#5073 at
5ca20ee081c255bed107e58b6dc5b5b769b6101d, adds one commit on60e160369:5ca20ee08vmcore: skip a periodic timer's missed periods at once. A running timer's poll moves its timeout to the first of its deadlines after now in one step, from the number of whole periods since the timeout, and registers its waker for that deadline in a newTimerState::poll_periodic_timeout, under the timer state's lock and with the same clock reading as the check that found the timeout due. A ready poll so accounts for every deadline up to now, and the deadlines stay a whole number of periods after the timer's start, as before.startcounts a period shorter than 100 ns as 100 ns, and caps one ati64::MAXunits, the longest interval that VM time's circular order puts after a time. Its tests, in a newperiodicmodule ofvmtime's tests, catch an 8192 Hz timer up across 30 days of VM time, run a 50 ns timer with VM time stopped and running, and catch up across VM time's wrap onto a deadline.The NVX commits:
c13e733cf7fopenvmm: pin the periodic timer's one-step catch-up. It changes only the gitlink, from60e160369to5ca20ee08.7287f794836doc: map the periodic timer's one-step catch-up in the OpenVMM roadmap. Position 268, unassigned.Decisions:
ArcMutexChipsetDeviceUnit::runpolls a device again at once when its event is signaled, so the unit's task would never yield. Registering the deadline under the timer state's lock leaves no such window, so the poll neither loops nor wakes its own task.VmTimeKeeper::restorerather than the chain'sadvance, which reset VM time the same way, so that the commit applies to upstream main on its own.Not in this change, found while working on it:
pal_async's thread-pool timer.Timer::set_deadlineinsupport/pal/pal_async/src/windows/tp.rspanics for a deadline that has passed, which VM time's timers can hand it, asTimerState::poll_timeoutalready does with a timeout at exactly now. OpenVMM runs VM time onVmTaskDrivers overDefaultDriver, whose IOCP and epoll timers accept such deadlines, so this is not reachable there.scripts/nvx_hosts.py runfrom a Windows controller.SubprocessRunnerwrites the script through a text-mode pipe, which turns its line feeds into CRLF, so a bash script fails on a Linux host (set: pipefail\r: invalid option name). The Linux runs below therefore passed their bash scripts throughnvx_hosts.py exec, as base64 decoded into a temporary file.Validation
Local checks at this head,
7287f794836(Windows):python scripts/openvmm_pin.py check-branch --topic --target origin/agents/issue-1151-rtc-wide-reads: passed (2 signed commits, no coordinated files).python scripts/openvmm_pin.py check-roadmap --worktree: passed (268 positions, 277 references).python scripts/openvmm_pin.py source 5ca20ee081c255bed107e58b6dc5b5b769b6101d --pr 5073: good signature, and the revision is the pull request's head.python scripts/nvx.py verify: passed.python scripts/local_gates.py --base origin/dev --only test-changes markdown-links test-layout openvmm-vectors openvmm-cpu-profiles --run: passed. That ischeck_test_changes.py(0 changes to existing tests),check_markdown_links.py,check_test_layout.py, and the ignoredopenvmm_vectors(4) andopenvmm_cpu_profiles(1) tests ofnvx-rs.nvx-cligate, its commands afterpip install, whose one requirement, PyYAML, is installed, withNVX_RSset to a releasenvx-rsbuilt here and Git'sshonPATH: passed. The unit tests ran 2198 tests (88 skipped) andtest_adversarial.py84 (11 skipped).openvmm-cigate,python scripts/nvx.py check-openvmm-ci --revision 5ca20ee081c255bed107e58b6dc5b5b769b6101d: OpenVMM's CI of the pin, run 43368520199, passed every job that NVX requires; see the CI paragraph below.OpenVMM package checks at
5ca20ee08, with Rust 1.97.1: on Windows, forvmcore,cargo check,cargo clippy --all-targets(clean apart from the environment's LNK4199 linker notices),cargo doc --no-deps, andcargo nextest run --profile agent(40 passed), andcargo nextest run --profile agent -p chipset -p chipset_legacy(115 passed), thencargo xtask fmt --fix(no changes); on Linux, on nvx-kvm-test,cargo nextest run --profile agent -p vmcore -p chipset -p chipset_legacy(155 passed) andcargo test -p vmcore --lib(40 passed). The three new tests passed 200 stress iterations on each. With60e160369's code,test_skips_missed_periods_at_onceandtest_rounds_short_periods_updo not complete within their 10 s watchdog, andtest_catch_up_across_the_wrapleaves the timeout at now,VmTime(170)whereVmTime(180)is expected. Replayed on upstream main,7d90c4104, the commit applies without conflicts, andvmcoretype-checks there with all its targets and its 35 tests pass.Reproduction, in NVX's Alpine microVM: the guest enabled the RTC's periodic interrupt at 8192 Hz through
/dev/port, writing status A0x23and setting status B's periodic interrupt enable, and requested a capture. Each restore of that snapshot timed the guest's first line after the capture: after the real downtime, then with--x-time-abi-test-hook downtime-add-s=86400anddowntime-add-s=2592000. Every restore then found status C's periodic and IRQ flags set,0xc0, and set again 10 ms after the read that cleared them:60e16036960e16036960e1603695ca20ee085ca20ee085ca20ee08The "before" runs used a temporary worktree of
0e159a5808e, #1563's head, on each host, with the guest artifacts of the releasev0.1.0-dev.8e9e9eb74cf2, whichnvx-rs downloadinstalled, and OpenVMM built from60e160369(clean) bynvx.py build-openvmm --backend BACKEND:35af18e3on KVM,1d54e162on MSHV (musl), and89ffd0afon WHP. The "after" runs used a temporary worktree of52ca738f654, whose tree is this head's, with the same release's guest artifacts and OpenVMM built from5ca20ee08(clean) the same way:f260557don KVM,1237bbd3on MSHV (musl), and1c68b25eon WHP.The guest script, which the driver typed into the guest's console with the harness's
OpenvmmProcessRemote runs at the head's tree, in the temporary worktree of
52ca738f654, with OpenVMM built from5ca20ee08. The scenarios areguest-boot,lifecycle, which reads the RTC's status B,interrupt-map,snapshot-core,restore-processors,smp-snapshot,restore-downtime,time-abi-conformance, andtime-host-correction, with--processors 1 2:f260557d1237bbd3(musl)1c68b25eNo bare-metal profile is configured; the change is algorithmic and was reproduced and fixed on all three backends, so bare-metal hosts and the remaining scenarios are left to CI.
CI at this head,
7287f794836: passed, run 43371731765, withOpenVMM CI of the pinned revision, which found that OpenVMM's CI of5ca20ee08, run 43368520199, passed every job that NVX requires. That run'srun vmm-tests [x64-windows-amd-snp], which NVX does not require, failed in four of its 114 tests.multiarch::vmgs::hyperv_openhcl_uefi_x64_windows_datacenter_core_2025_x64_prepped_snp_vmgstool_copy_igvmfilefailed as it did at60e160369, in microsoft/openvmm#5072's run 43352068114: the runner's Hyper-V failed to load the CVM firmwarevmfirmwarecvm.dll(0x80070002).multiarch::tpm::…_snp_ak_cert_retry,multiarch::tpm::…_snp_hw_ak_stable, andmultiarch::vmbus_relay::…_snp_vmbus_relaytimed out after about 12 minutes each. The same job fails with timeouts in the same TPM and VMBus relay tests in the same hours on OpenVMM pull requests that lack this change, such as run 43330262358 (vmbus_relay::…_snp_vmbus_relay, 757 s), run 43334289464 (tpm::…_snp_ak_cert_cache, 719 s), and run 43342660705 (tpm::…_snp_cvm_guest, 792 s), so it predates this change.No existing test changes, so no
tests-modifiedlabel.5ca20ee081c255bed107e58b6dc5b5b769b6101d.60e160369, the head offeature/microvm/devsince [DO NOT REVIEW] chipset: zero-extend wide reads of the RTC's ports openvmm#5072 (openvmm: zero-extend wide reads of the RTC's ports #1563) landed. There are no other prerequisites.b41e992bb.feature/microvm/devfrom60e160369to exactly5ca20ee081c255bed107e58b6dc5b5b769b6101d. OpenVMM's merge button would squash the commit and leave the pin off the branch. This pull request then merges through the queue.