From 819b6b61c378d4a8230aaed20859f5cfbf040e5d Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Mon, 9 Nov 2026 05:27:01 -0800 Subject: [PATCH 1/3] openvmm: pin the refusal of earlier snapshot formats by their version Restoring a snapshot of an earlier OpenVMM directly, with `openvmm --restore-snapshot` or the management RPC, fails with a decoding error rather than with E_SNAPSHOT_VERSION, which run.md promises. Manifest version 6, which OpenVMM wrote until its v4 microVM chain restarted the manifest at version 2, gives fields 11 and 13 other types than version 2 does, and the first manifests of version 2 give fields 9 and 10 other types, but restore decoded the whole manifest before it checked the version and magic: failed to decode snapshot manifest: decoding failed: expected a varint nvx-rs, which reads a template's manifest itself, already checks the version and magic first, so only a direct restore was affected. Pin microsoft/openvmm#4994, 98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7, two commits on feature/microvm/dev ed8fc0713: - openvmm_helpers: refuse earlier snapshot formats before decoding them. Restore decodes the version alone, then the magic alone, and refuses any other with E_SNAPSHOT_VERSION before it decodes the rest. - openvmm_helpers: reserve the manifest fields that upstream assigns. Upstream's SaveVM and RestoreVM (microsoft/openvmm#4553) moved their snapshot ID from field 8, which this manifest gives to the length of state.bin, to field 18, and reserve fields 19 and 20, which no field of this manifest takes. No field moves, so the format stays manifest version 2, and every existing snapshot stays restorable. The protocol vectors and CPU profiles that nvx-rs copies are unchanged. On KVM and MSHV, on Azure Linux 3 virtual machines, and on WHP, on a Windows virtual machine, a microVM snapshot that the OpenVMM of NVX release v0.1.0-dev.304f8e1686b4 captured, of manifest version 6, which that OpenVMM restores, fails to restore with the error above at the previous pin, and with [E_SNAPSHOT_VERSION] snapshot manifest version 6 is not supported (expected 2); recapture the snapshot at this one. With this pin, the snapshot-core, run-console-restore, snapshot-tiers, scratch-snapshot, console-snapshot, filesystem-snapshot, network-snapshot, smp-snapshot, restore-processors, restore-memory, snapshot-rollback, restore-downtime, platform-restore-processors, platform-zero-fill, platform-network-policy, untiered-network-policy, and sandbox-blocks microVM scenarios pass on all three backends. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- openvmm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openvmm b/openvmm index ed8fc0713..98e5b3ca5 160000 --- a/openvmm +++ b/openvmm @@ -1 +1 @@ -Subproject commit ed8fc0713b4c6301d3b82ffe88817e70084d84b8 +Subproject commit 98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7 From 08d13987bc8347e277d7b941b7a16a463d9cd16f Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Mon, 9 Nov 2026 05:27:01 -0800 Subject: [PATCH 2/3] doc: say that OpenVMM refuses earlier manifests before decoding them Snapshots of an earlier OpenVMM in run.md said that OpenVMM refuses their manifest by its version with E_SNAPSHOT_VERSION. Until the previous commit's pin, it did so only once it had decoded the whole manifest, which fails first for manifest version 6 and the first manifests of version 2, whose fields have other types than version 2 gives them. With the pin, OpenVMM decodes the version and then the format magic before the rest, so it refuses every earlier manifest with E_SNAPSHOT_VERSION: by its version, or, for the earliest manifests of version 2, which have no magic, by the magic that they lack. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- doc/run.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/run.md b/doc/run.md index 3b62433fc..8c6cc3f51 100644 --- a/doc/run.md +++ b/doc/run.md @@ -680,8 +680,10 @@ platform snapshot. OpenVMM's v4 microVM restarted the versions of its contracts: microVM ABI 1, boot layout 1, time ABI 1, restore packet 1, and snapshot manifest 2, whose version 1 is upstream OpenVMM's. It reads no snapshot of an earlier OpenVMM, -whose manifest it refuses by its version with `E_SNAPSHOT_VERSION`, and -`nvx-rs` refuses a template of an earlier OpenVMM by its snapshot format. +whose manifest it refuses with `E_SNAPSHOT_VERSION` by its version, or by its +missing format magic for the earliest manifests of version 2, before it +decodes the rest, and `nvx-rs` refuses a template of an earlier OpenVMM by its +snapshot format. Capture them again; to restore them as they are, use NVX commit `304f8e1686b4fba188ad2d9c8d932498a6ca2980` with the OpenVMM that it pins. From 57efcf7c52399c7f39ab4dd194e937cdc201518a Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Mon, 9 Nov 2026 05:25:46 -0800 Subject: [PATCH 3/3] doc: map the refusal of earlier snapshot formats in the OpenVMM roadmap Map positions 230 and 231, the two commits of microsoft/openvmm#4994, which NVX now pins at 98e5b3ca5, 100 commits past the mapped tip: - 230, openvmm_helpers: refuse earlier snapshot formats before decoding them. Restore decodes the manifest's version and then its magic, each alone, and refuses any other with E_SNAPSHOT_VERSION before it decodes the rest, since positions 195 and 205 restarted the manifest at version 2 and numbered its fields without gaps. - 231, openvmm_helpers: reserve the manifest fields that upstream assigns. A comment and the snapshot format guide keep fields 18-20 for upstream's SaveVM and RestoreVM. Both are unassigned, and neither is a fixup of a mapped PR: they change the manifest decoding and header check of PR11's position 031, which PR12's position 035 moves to the opened snapshot directory, and their tests use the machine contract of PR17's position 074. They conflict with upstream main in all four of their files, so they follow PR11, PR12, and PR17 upstream. Under the open upstream pull requests, note microsoft/openvmm#4553, which proposes no chain commit: it records its snapshot ID in field 18, moved from field 8 after microsoft/nvx#1210, and reserves fields 19 and 20, so a rebase onto it adds the field to the chain's manifest, and nvx-rs's template-manifest reader must accept it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- doc/openvmm-upstream-roadmap.md | 55 +++++++++++++++++++++++++++------ 1 file changed, 45 insertions(+), 10 deletions(-) diff --git a/doc/openvmm-upstream-roadmap.md b/doc/openvmm-upstream-roadmap.md index bfe37f5fc..13328e94a 100644 --- a/doc/openvmm-upstream-roadmap.md +++ b/doc/openvmm-upstream-roadmap.md @@ -8,10 +8,11 @@ reorganization kept it there; see [Upstream status](#upstream-status). - Mapped fork tip: `7677ad8d56a59864475012b63b6f01a22575c30d`, position 131. The graph, PR summary, and commit mapping below cover positions 001-131. -- NVX's pinned OpenVMM revision: `ed8fc0713b4c6301d3b82ffe88817e70084d84b8`, position 229, the - head of microsoft/openvmm#4993, whose commit follows position 228, `d6addf8c1`, the head of - `feature/microvm/dev`, which microsoft/openvmm#4992 merged after position 227, `154548a0e`, - which microsoft/openvmm#4991 merged after position 226, `881619911`, which +- NVX's pinned OpenVMM revision: `98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7`, position 231, the + head of microsoft/openvmm#4994, whose two commits follow position 229, `ed8fc0713`, the head of + `feature/microvm/dev`, which microsoft/openvmm#4993 merged after position 228, `d6addf8c1`, + which microsoft/openvmm#4992 merged after position 227, `154548a0e`, which + microsoft/openvmm#4991 merged after position 226, `881619911`, which microsoft/openvmm#4989 merged after position 225, `99aac19b3`, which microsoft/openvmm#4988 merged after position 224, `adb7c8bc8`, which microsoft/openvmm#4987 merged after position 223, `90f9e6180`. Positions 221-223, the @@ -23,8 +24,8 @@ position 221 into positions 035 and 036, and combined v5.1 positions 222 and 223, the two commits of microsoft/openvmm#4984, into position 220. Positions 001-147 keep their positions, v5.1 positions 149-220 are positions 148-219, and the tree of position 220 is the tree of - `3f2ab4c2f`; see [The v6.0 reorganization](#the-v60-reorganization). The pin is 98 commits past - the mapped tip on the same first-parent chain. Positions 132-229 are not yet assigned to PRs; see + `3f2ab4c2f`; see [The v6.0 reorganization](#the-v60-reorganization). The pin is 100 commits past + the mapped tip on the same first-parent chain. Positions 132-231 are not yet assigned to PRs; see [Commits after the mapped tip](#commits-after-the-mapped-tip). - Upstream main: `ba105dd7f00499acc1862456c179c0930cb2c8f7` (2026-11-06), the fork base itself; see [Upstream main past the fork base](#upstream-main-past-the-fork-base). @@ -263,6 +264,16 @@ successful, and a `Retain` console on a named pipe needs that write handling for Position 014 stays whole in PR06. If the review of microsoft/openvmm#4609 narrows it as that of microsoft/openvmm#4605 did, its write handling joins position 041 in PR13. +microsoft/openvmm#4553, upstream's SaveVM and RestoreVM for the management RPC, proposes no chain +commit either. It records a snapshot ID in the manifest's field 18, to which it moved the ID from +field 8, the chain's `state_size_bytes`, after microsoft/nvx#1210 raised the clash, and it reserves +fields 19 and 20 for `compatibility_bundle_digest` and `memory_artifacts`; position 231 keeps those +three fields free. It also changes `write_snapshot` in the snapshot publication of +`openvmm_helpers`, which PR11's positions 031-033 and later chain commits change. A rebase onto it +therefore adds `snapshot_id` to the chain's `SnapshotManifest` and reconciles the publication, and +NVX's pin of that rebase needs `nvx-rs`'s reader of a template's manifest, which refuses fields +that it does not know, to accept field 18. + | Roadmap PR | Positions | Upstream pull requests, in position order | | --- | ---: | --- | | PR01 | 002-004 | microsoft/openvmm#4595, microsoft/openvmm#4618, microsoft/openvmm#4615 | @@ -971,7 +982,9 @@ ba105dd7f -> restore listener of a disconnected boot console [227] -> management RPC with the command line's microVM settings [228] -> staged manifest removed before a failed capture resumes [229] - -> ed8fc0713 (NVX pin) + -> earlier snapshot formats refused before decoding [230] + -> manifest fields reserved for upstream [231] + -> 98e5b3ca5 (NVX pin) ``` For upstream review, PR01-PR06 and PR14 can be rebased independently onto current upstream main, @@ -987,13 +1000,13 @@ reorganization folded the PR20 and PR12 fixups into the commits that they fix; s ## Commits after the mapped tip -The NVX pin extends the chain past position 131 without merges. None of the 98 later commits is a +The NVX pin extends the chain past position 131 without merges. None of the 100 later commits is a fixup of a mapped PR, and none is yet classified or assigned to PRs. Each of positions 132-186 combines the v2.4 commits that the table lists, as v3.0 positions 145-160 and 162-200 did, which developed one concept; "(part)" marks a v2.4 fixup that split across commits. Positions 187-191 followed the v3.0 reorganization, positions 192-205 are the ABI restart of the v4.0 reorganization, positions 206-211 followed it, positions 212-218 followed the v5.0 -reorganization, positions 219 and 220 followed the v5.1 rebase, and positions 221-229 followed the +reorganization, positions 219 and 220 followed the v5.1 rebase, and positions 221-231 followed the v6.0 reorganization, so they combine none, apart from position 220, which combines v5.1 positions 222 and 223. Each commit has the position that it had in v5.1 through position 147, and one less from v5.1 position 149 on; see @@ -1100,6 +1113,8 @@ at `e1ca353eb`, whose first 210 commits are those of `feature/microvm/release/v4 | 227 | `154548a0e` | - | - | - | - | microvm: let a restore attach a listener to a disconnected boot console (microsoft/openvmm#4991) | | 228 | `d6addf8c1` | - | - | - | - | vmservice: carry microVM blocks, network, RAM file, and share policy (microsoft/openvmm#4992) | | 229 | `ed8fc0713` | - | - | - | - | openvmm_helpers: don't resume beside a restorable staged snapshot (microsoft/openvmm#4993) | +| 230 | `8a994544a` | - | - | - | - | openvmm_helpers: refuse earlier snapshot formats before decoding them (microsoft/openvmm#4994) | +| 231 | `98e5b3ca5` | - | - | - | - | openvmm_helpers: reserve the manifest fields that upstream assigns (microsoft/openvmm#4994) | The remarks that say whether commits apply to upstream main record what the v3.0 reorganization found on its base, `b2676222b`, at the positions that the commits have now. @@ -1191,7 +1206,16 @@ now removes its manifest first, and a cleanup that cannot prove the manifest gon 033's uncertain outcome, on which the controller of PR17's position 076 terminates the source, where position 032 let it resume beside a snapshot that restore could open. It also rewords that outcome's message, the controller's log, and the sentences on a failed publication that position -076 adds to the snapshot and snapshot format guides. +076 adds to the snapshot and snapshot format guides. Position 230 changes the manifest's decoding, +which PR11's position 031 bounds and checks and PR12's position 035 moves to the snapshot directory +that it opens: positions 195 and 205 restarted the manifest at version 2 and numbered its fields +without gaps, which gives some fields of earlier formats other types, so restore now decodes the +version and then the magic, each alone, and refuses any other with `E_SNAPSHOT_VERSION` before it +decodes the rest, and the version and magic checks of position 031's header check become helpers +that both share. It also says why in the manifest section of the snapshot format guide, which +position 195 rewrote. Position 231 adds a comment to the manifest's fields, which position 205 +numbers, and a sentence to the snapshot format guide's rule for extending the manifest, that keep +fields 18-20 for upstream's SaveVM and RestoreVM, microsoft/openvmm#4553. Assigning the later commits therefore means revisiting those PRs, not only appending new ones. Position 177, by contrast, changes only `disk_blob`, which upstream main and the fork base @@ -1378,6 +1402,17 @@ add, and their tests. Replaying it on upstream main conflicts in all four of its main lacks the controller, the staging of positions 032 and 033, and those sentences, so it follows PR11 and PR17 upstream. +Positions 230 and 231 change only the snapshot manifest of `openvmm_helpers`, its format checks, +and its restore, which PR11's positions 031 and 032, PR12's position 035, PR17's positions 074 and +076, PR19's positions 092 and 093, and positions 136, 147, 166, 184, 195, 204, 205, and 215 also +change, and the snapshot format guide's sections on the manifest. In those files they change only +the manifest's decoding and header check, which positions 031 and 035 add, and comment on the +fields that position 205 numbers, and they add tests that use the machine contract of PR17's +position 074. Replaying them on upstream main conflicts in all four of their files: upstream main +lacks the format magic and header check of position 031, the restore of position 035, the machine +contract of position 074, and the fields that position 205 numbers, so they follow PR11, PR12, and +PR17 upstream. + ## PR summary Each count is of v6.0 commits, which include the fixups that the v3.0, v4.0, v5.0, and v6.0