diff --git a/doc/design/3.2-clock-and-time.md b/doc/design/3.2-clock-and-time.md index 036d48207..9c39121d8 100644 --- a/doc/design/3.2-clock-and-time.md +++ b/doc/design/3.2-clock-and-time.md @@ -241,15 +241,34 @@ so the saved TSC dates the saved count. The restore runs no timer until it advances the count, by `ticks` and by the ticks for which the TSC has run from the target that the restore wrote to it. -A one-shot timer that expired before the capture keeps its counts. KVM's saved -LAPIC state does not show whether KVM had delivered the timer's interrupt, so -on KVM the restored timer fires once more: the interrupt is due if KVM had not -delivered it, and spurious if the guest had already taken it. KVM fires it at -once from Linux 6.3, and a full timer period later before. On KVM, an expired -timer whose unmasked LVT entry delivers a fixed interrupt to a valid vector -that is pending in the interrupt request register (IRR), which already holds -the interrupt, as after a downtime that expires the timer, reads an initial -count of zero after the restore, so that KVM does not fire it again. +A one-shot timer that expired before the capture keeps its counts, except +that on KVM, capture saves one whose interrupt the processor took with an +initial count of zero, which the restored guest reads instead of the count +that it wrote; Linux never reads it. KVM's LAPIC state shows such a timer like +one whose interrupt is still due, since KVM holds the expiry of a timer that +expires while its processor is outside the guest until the processor next +runs, and KVM's LAPIC state write fires a timer whose current count is zero +again, at once from Linux 6.3 and a full timer period later before. So when +the KVM backend stops a processor, it reads the timer's count and then has KVM +deliver the interrupts that it holds, without running the processor. KVM +records an expiry only once its host timer fires, which a loaded host can +delay, so if the timer had already expired at that read, the backend has KVM +deliver again until the interrupt appears, for up to 20 ms. Of the expired +timers whose unmasked LVT entry delivers a fixed interrupt to a valid vector, +capture then saves one that expired after the read with its vector pending in +the interrupt request register (IRR), and takes one whose vector is still +clear then for one whose interrupt the processor took. On KVM, such a timer +whose vector is pending in IRR, which already holds the interrupt, as after a +downtime that expires the timer, reads an initial count of zero after the +restore, so that KVM does not fire it again. + +Two limits remain on KVM. If KVM records an expiry more than 20 ms late, +capture saves its interrupt as taken, and the restore does not deliver it. A +KVM that runs the timer on the VMX preemption timer, as on bare-metal Intel +hosts, records no expiry while the processor is outside guest mode, so on such +a host capture saves a timer that had expired when its processor stopped as +KVM reports it, and the restored timer fires once more: the interrupt is due +if KVM had not delivered it, and spurious if the guest had already taken it. ## 3.2.6 Backend Obligations diff --git a/doc/openvmm-upstream-roadmap.md b/doc/openvmm-upstream-roadmap.md index dccf0a18e..39fda910a 100644 --- a/doc/openvmm-upstream-roadmap.md +++ b/doc/openvmm-upstream-roadmap.md @@ -10,9 +10,10 @@ summary, and commit mapping below cover positions 001-131, the PR09 fixups at positions 241-243, the PR13 fixup at position 247, the PR18 fixup at position 248, the PR20 fixup at position 249, and the PR19 fixup at position 250. -- NVX's pinned OpenVMM revision: `024ecee62d7c9b70456b63a5b4f94a6488f835cd`, position 258, the - head of microsoft/openvmm#5053, whose three commits follow position 255, `90f87319d`, the head - of `feature/microvm/dev`, which microsoft/openvmm#5052 merged with position 254 after position +- NVX's pinned OpenVMM revision: `c72fc56a3716fafaf1a9b5473c2de646c2a74815`, position 261, the + head of microsoft/openvmm#5055, whose three commits follow position 258, `024ecee62`, the head of + `feature/microvm/dev`, which microsoft/openvmm#5053 merged with positions 256 and 257 after + position 255, `90f87319d`, which microsoft/openvmm#5052 merged with position 254 after position 253, `3cd9f149b`, which microsoft/openvmm#5046 merged after position 252, `9f10ce7a8`, which microsoft/openvmm#5044 merged after position 251, `174da3a0d`, which microsoft/openvmm#5042 merged after position 250, `23136b647`, @@ -45,8 +46,8 @@ position 221 into positions 035 and 036, and combined v5.1 positions 222 and 223, the two commits of microsoft/openvmm#4984, into position 220. Positions 001-147 keep their positions, v5.1 positions 149-220 are positions 148-219, and the tree of position 220 is the tree of - `3f2ab4c2f`; see [The v6.0 reorganization](#the-v60-reorganization). The pin is 127 commits past - the mapped tip on the same first-parent chain. Positions 132-240, 244-246, and 251-258 are not + `3f2ab4c2f`; see [The v6.0 reorganization](#the-v60-reorganization). The pin is 130 commits past + the mapped tip on the same first-parent chain. Positions 132-240, 244-246, and 251-261 are not yet assigned to PRs, positions 241-243 are PR09 fixups, position 247 is a PR13 fixup, position 248 is a PR18 fixup, position 249 is a PR20 fixup, and position 250 is a PR19 fixup; see [Commits after the mapped tip](#commits-after-the-mapped-tip). @@ -1029,7 +1030,8 @@ ba105dd7f -> 365-day maximum snapshot downtime [253] -> restored VM time run from the TSCs' restart [254-255] -> LAPIC timer restores without drift or a downtime refire [256-258] - -> 024ecee62 (NVX pin) + -> LAPIC timer captures that keep a taken timer from refiring [259-261] + -> c72fc56a3 (NVX pin) ``` For upstream review, PR01-PR06 and PR14 can be rebased independently onto current upstream main, @@ -1066,7 +1068,7 @@ helper that the case uses. ## Commits after the mapped tip -The NVX pin extends the chain past position 131 without merges. Of the 127 later commits, +The NVX pin extends the chain past position 131 without merges. Of the 130 later commits, positions 241-243 are PR09 fixups, position 247 is a PR13 fixup, position 248 is a PR18 fixup, position 249 is a PR20 fixup, position 250 is a PR19 fixup, and the others are not yet classified or assigned to PRs. @@ -1074,7 +1076,7 @@ Each of positions 132-186 combines the v2.4 commits that the table lists, as v3. and 162-200 did, which developed one concept; "(part)" marks a v2.4 fixup that split across commits. Positions 187-191 followed the v3.0 reorganization, positions 192-205 are the ABI restart of the v4.0 reorganization, positions 206-211 followed it, positions 212-218 followed the v5.0 -reorganization, positions 219 and 220 followed the v5.1 rebase, and positions 221-258 followed the +reorganization, positions 219 and 220 followed the v5.1 rebase, and positions 221-261 followed the v6.0 reorganization, so they combine none, apart from position 220, which combines v5.1 positions 222 and 223. Each commit has the position that it had in v5.1 through position 147, and one less from v5.1 position 149 on; see @@ -1210,6 +1212,9 @@ at `e1ca353eb`, whose first 210 commits are those of `feature/microvm/release/v4 | 256 | `2a0401c43` | - | - | - | - | virt_kvm: share the guest RAM of the real-mode KVM tests (microsoft/openvmm#5053) | | 257 | `21a76f878` | - | - | - | - | virt_kvm: do not fire an expired LAPIC timer whose interrupt is pending (microsoft/openvmm#5053) | | 258 | `024ecee62` | - | - | - | - | vmm_core, virt: run a restored LAPIC timer from the time ABI anchors (microsoft/openvmm#5053) | +| 259 | `727cad63f` | - | - | - | - | kvm: deliver the interrupts that KVM holds without running the VP (microsoft/openvmm#5055) | +| 260 | `9f14b54fd` | - | - | - | - | virt_kvm: keep a restore from firing a taken one-shot LAPIC timer again (microsoft/openvmm#5055) | +| 261 | `c72fc56a3` | - | - | - | - | virt_kvm: build the LAPIC timer KVM tests for musl (microsoft/openvmm#5055); fold into 260 | The remarks that say whether commits apply to upstream main record what the v3.0 reorganization found on its base, `b2676222b`, at the positions that the commits have now. @@ -1847,6 +1852,24 @@ files, seven of which upstream main lacks, those that PR10's positions 024 and 0 position 036, and positions 132, 135, 139, and 256 add, so it follows the time ABI series upstream. +Positions 259-261 have the KVM backend tell a one-shot LAPIC timer whose interrupt the guest took +apart from one whose expiry KVM holds when it saves a VP, and change nothing that a mapped PR +adds. Position 259 adds a call to the `kvm` crate's VP runner that has KVM deliver the interrupts +that it holds without entering the guest, in a file that upstream main has and that PR10's +position 026 and positions 133, 138, 147, and 148 also change, none of them in the runner's +`KVM_RUN` and exit decoding, which it changes and upstream main has. Replaying it on upstream main +applies without conflicts, and the `kvm` crate type-checks there, so it can go upstream on its own. + +Position 260 adds a LAPIC timer module to `virt_kvm`'s x86-64 backend, which the VP run loop, the +partition, and the VP state's `apic` and `set_apic` use, in files that upstream main has and that +PR10's positions 023 and 026, PR17's position 077, and positions 133, 138, 144, 147, 149, 150, +255, 256, 257, and 258 also change, and rewrites the paragraph of the Guide's timekeeping reference +that position 257 adds. Replaying it on upstream main conflicts in that paragraph, in the module +list of the x86-64 module, and in the VP state module, whose KVM LAPIC page from position 257 its +KVM tests use, with position 256's guest RAM and position 259's delivery, so it follows positions +256, 257, and 259 upstream. Position 261 changes only position 260's KVM tests, which it builds for +musl, for which the `libc` crate defines no `SIGEV_THREAD_ID`, so it folds into position 260. + ## PR summary Each count is of v6.0 commits, which include the fixups that the v3.0, v4.0, v5.0, and v6.0 diff --git a/openvmm b/openvmm index 024ecee62..c72fc56a3 160000 --- a/openvmm +++ b/openvmm @@ -1 +1 @@ -Subproject commit 024ecee62d7c9b70456b63a5b4f94a6488f835cd +Subproject commit c72fc56a3716fafaf1a9b5473c2de646c2a74815