Skip to content

[DO NOT REVIEW] openvmm_helpers: refuse earlier snapshot formats before decoding them - #4994

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
feature/microvm/devfrom
copilot/nvx-issue-1210-manifest-alignment
Nov 9, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
feature/microvm/devfrom
copilot/nvx-issue-1210-manifest-alignment

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Refuses the snapshots of an earlier OpenVMM by their manifest's version and magic, with E_SNAPSHOT_VERSION, before it decodes the rest of their manifest, and reserves the manifest fields that upstream's SaveVM and RestoreVM assign, for microsoft/nvx#1210. A microVM snapshot that NVX release v0.1.0-dev.304f8e1686b4 captured, of manifest version 6, failed to restore with failed to decode snapshot manifest: decoding failed: expected a varint, rather than with the [E_SNAPSHOT_VERSION] ... recapture the snapshot that the Guide's snapshot format and NVX's guide to snapshots of an earlier OpenVMM promise.

Two commits on ed8fc0713, the head of feature/microvm/dev. microsoft/nvx#1417 pins this head.

Refuse earlier snapshot formats before decoding them (8a994544a)

  • Cause. decode_snapshot_manifest in openvmm_helpers::snapshot::restore decoded the whole manifest with mesh::payload::decode, and only then checked its version and magic in validate_manifest_header. The v4.0 microVM chain restarted the manifest at version 2 and numbered its fields without gaps, so earlier formats give some field numbers other types, which mesh refuses before any check runs:
    • Version 6 keeps its machine contract, a message, in field 11, where version 2 keeps the saved-state schema version, a varint, and its schema version, a varint, in field 13, where version 2 keeps the capture tier, a string: expected a varint.
    • The first version 2, which predates the format magic, keeps the SHA-256 digests of state.bin and memory.bin in fields 9 and 10, where version 2 now keeps its machine contract and its magic: unknown wire type 3.
  • Change. format::decode_manifest decodes the version alone, which every format, upstream's included, keeps in field 1, and refuses any other version with E_SNAPSHOT_VERSION; then the magic alone, field 10, whose type only the formats of version 2 agree on, and refuses any other magic the same way; and only then decodes the whole manifest and runs the rest of validate_manifest_header. decode_snapshot_manifest uses it, so OpenedSnapshot::open, read_snapshot, read_snapshot_manifest, and read_snapshot_with_memory, and with them the CLI's --restore-snapshot and the management RPC's restore, refuse earlier formats this way. The version and magic checks move into helpers that validate_manifest_header shares, with the same messages. NVX's own reader of a template's manifest already checks the version and the magic first.
  • Compatibility. The snapshot format is unchanged, and so is every check of a manifest of version 2. Bytes that are not a manifest still fail to decode.
  • Tests. read_snapshot_refuses_a_version_6_manifest_by_its_version and read_snapshot_refuses_an_early_version_2_manifest_by_its_magic write a manifest of each layout beside a snapshot's other artifacts and require [E_SNAPSHOT_VERSION], the version's or the magic's message, and recapture the snapshot from both read_snapshot and read_snapshot_manifest. Each fails at 99aac19b3 with the error above.
  • Guide. The snapshot format's manifest section says that restore decodes the version and then the magic before the rest of the manifest, and why.

Reserve the manifest fields that upstream assigns (98e5b3ca5)

  • Cause. Upstream's SaveVM and RestoreVM, ttrpc: add SaveVM and RestoreVM APIs #4553, record a snapshot ID in the manifest. They first took field 8, which this manifest gives to state_size_bytes, a varint that a GUID cannot decode; they have since moved snapshot_id to field 18, so that neither side renumbers a field, and reserved fields 19 and 20 for compatibility_bundle_digest and memory_artifacts. This manifest's fields end at 15, so a field that it adds after 16 and 17 would take 18.
  • Change. A comment where SnapshotManifest declares its fields, and the Guide's rule for extending the format, keep fields 18, 19, and 20 for upstream's fields.
  • Compatibility. No field changes, so the manifest stays version 2, and every snapshot stays restorable.

Validation

With Rust 1.97.1, at this head:

  • Windows: cargo clippy --all-targets, cargo doc --no-deps, and cargo nextest run --profile agent for openvmm_helpers: 122 passed; cargo xtask fmt --fix, clean.
  • Linux (Azure Linux 3.0): the same: 128 passed.
  • On KVM and MSHV, on Azure Linux 3.0 virtual machines, and on WHP, on a Windows virtual machine: the OpenVMM of NVX release v0.1.0-dev.304f8e1686b4 captured a microVM snapshot of manifest version 6, which it restores. ed8fc0713 refuses it with failed to decode snapshot manifest (decoding failed: expected a varint), and this head with [E_SNAPSHOT_VERSION] snapshot manifest version 6 is not supported (expected 2); recapture the snapshot.
  • NVX's snapshot scenarios snapshot-core, run-console-restore, snapshot-tiers, scratch-snapshot, console-snapshot, filesystem-snapshot, network-snapshot, smp-snapshot, restore-processors, restore-memory, snapshot-rollback, restore-downtime, platform-restore-processors, platform-zero-fill, platform-network-policy, untiered-network-policy, and sandbox-blocks pass on all three backends with OpenVMM built at this head; openvmm: pin the refusal of earlier snapshot formats by their version nvx#1417 reports them.

At a851d870b, the same two commits on 99aac19b3 (an identical range-diff), the same end-to-end check and scenarios passed on all three backends, and 99aac19b3 failed the same way. The two new tests fail at 99aac19b3 on Windows with expected a varint and unknown wire type 3.

Notes

  • Land it by fast-forwarding feature/microvm/dev to this head, 98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7, since this repository's merge button squashes and would leave NVX's pin off the branch.

Restoring a snapshot of an earlier OpenVMM fails with a decoding error,
not with E_SNAPSHOT_VERSION and its request to recapture the snapshot.
Restore decodes the whole manifest, and only then checks its version
and magic, but the earlier formats give some field numbers other types
than manifest version 2 does:

- Version 6, which OpenVMM wrote until the v4.0 microVM chain restarted
  the manifest at version 2, keeps its machine contract, a message, in
  field 11 and its saved-state schema version, a varint, in field 13,
  where version 2 keeps the schema version and the capture tier, a
  string:

    failed to decode snapshot manifest: decoding failed: expected a
    varint

- The first version 2, which predates the format magic, keeps the
  SHA-256 digests of state.bin and memory.bin in fields 9 and 10, where
  version 2 now keeps its machine contract and its magic:

    failed to decode snapshot manifest: decoding failed: unknown wire
    type 3

The Guide says that restore rejects any other version or magic with
E_SNAPSHOT_VERSION, and so does NVX's guide to snapshots of an earlier
OpenVMM (microsoft/nvx#1210). On KVM and MSHV, on Azure Linux 3
virtual machines, and on WHP, on a Windows virtual machine, the OpenVMM
of NVX release v0.1.0-dev.304f8e1686b4 captured a microVM snapshot of
manifest version 6, which it restores, and which feature/microvm/dev
99aac19 refuses with the first error.

Decode the version alone, which every manifest format, upstream's
included, keeps in field 1, and then the magic alone, which version 2
keeps in field 10, and refuse any other version or magic with
E_SNAPSHOT_VERSION before decoding the rest of the manifest. With this
change, those restores fail with

    [E_SNAPSHOT_VERSION] snapshot manifest version 6 is not supported
    (expected 2); recapture the snapshot

A malformed manifest still fails to decode, and a manifest of version 2
still gets every check that it got before.

Tests: read_snapshot_refuses_a_version_6_manifest_by_its_version and
read_snapshot_refuses_an_early_version_2_manifest_by_its_magic, in
openvmm_helpers' restore tests, each of which fails at
feature/microvm/dev 99aac19 with the error above.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Upstream's SaveVM and RestoreVM (#4553) record a
snapshot ID in the manifest. It first took field 8, which this
manifest gives to the length of state.bin, a varint that a GUID cannot
decode. Upstream has since moved `snapshot_id` to field 18 instead, so
that neither side renumbers a field, and reserves fields 19 and 20 for
`compatibility_bundle_digest` and `memory_artifacts`, as its pull
request records (microsoft/nvx#1210).

This manifest's fields end at 15, so the field that it adds next after
16 and 17 would take 18. Record the reservation where the fields are
declared, and in the Guide's rule for extending the format, so that no
field of this manifest takes 18, 19, or 20. No field changes, so the
format stays version 2, and every snapshot stays restorable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added Guide release-nvx Targets a branch under feature/microvm/. labels Nov 9, 2026
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Nov 9, 2026
Restoring a snapshot of an earlier OpenVMM directly, with
`openvmm --restore-snapshot` or the management RPC, fails with a
decoding error rather than with E_SNAPSHOT_VERSION, which run.md
promises. Manifest version 6, which OpenVMM wrote until its v4 microVM
chain restarted the manifest at version 2, gives fields 11 and 13 other
types than version 2 does, and the first manifests of version 2 give
fields 9 and 10 other types, but restore decoded the whole manifest
before it checked the version and magic:

    failed to decode snapshot manifest: decoding failed: expected a
    varint

nvx-rs, which reads a template's manifest itself, already checks the
version and magic first, so only a direct restore was affected.

Pin microsoft/openvmm#4994, 98e5b3ca504ea18ffdcc7181e27516f7d7e5a6d7,
two commits on feature/microvm/dev ed8fc0713:

- openvmm_helpers: refuse earlier snapshot formats before decoding
  them. Restore decodes the version alone, then the magic alone, and
  refuses any other with E_SNAPSHOT_VERSION before it decodes the rest.
- openvmm_helpers: reserve the manifest fields that upstream assigns.
  Upstream's SaveVM and RestoreVM (microsoft/openvmm#4553) moved their
  snapshot ID from field 8, which this manifest gives to the length of
  state.bin, to field 18, and reserve fields 19 and 20, which no field
  of this manifest takes.

No field moves, so the format stays manifest version 2, and every
existing snapshot stays restorable. The protocol vectors and CPU
profiles that nvx-rs copies are unchanged.

On KVM and MSHV, on Azure Linux 3 virtual machines, and on WHP, on a
Windows virtual machine, a microVM snapshot that the OpenVMM of NVX
release v0.1.0-dev.304f8e1686b4 captured, of manifest version 6, which
that OpenVMM restores, fails to restore with the error above at the
previous pin, and with

    [E_SNAPSHOT_VERSION] snapshot manifest version 6 is not supported
    (expected 2); recapture the snapshot

at this one. With this pin, the snapshot-core, run-console-restore,
snapshot-tiers, scratch-snapshot, console-snapshot, filesystem-snapshot,
network-snapshot, smp-snapshot, restore-processors, restore-memory,
snapshot-rollback, restore-downtime, platform-restore-processors,
platform-zero-fill, platform-network-policy, untiered-network-policy,
and sandbox-blocks microVM scenarios pass on all three backends.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Nov 9, 2026
Map positions 230 and 231, the two commits of microsoft/openvmm#4994,
which NVX now pins at 98e5b3ca5, 100 commits past the mapped tip:

- 230, openvmm_helpers: refuse earlier snapshot formats before decoding
  them. Restore decodes the manifest's version and then its magic,
  each alone, and refuses any other with E_SNAPSHOT_VERSION before it
  decodes the rest, since positions 195 and 205 restarted the manifest
  at version 2 and numbered its fields without gaps.
- 231, openvmm_helpers: reserve the manifest fields that upstream
  assigns. A comment and the snapshot format guide keep fields 18-20
  for upstream's SaveVM and RestoreVM.

Both are unassigned, and neither is a fixup of a mapped PR: they change
the manifest decoding and header check of PR11's position 031, which
PR12's position 035 moves to the opened snapshot directory, and their
tests use the machine contract of PR17's position 074. They conflict
with upstream main in all four of their files, so they follow PR11,
PR12, and PR17 upstream.

Under the open upstream pull requests, note microsoft/openvmm#4553,
which proposes no chain commit: it records its snapshot ID in field 18,
moved from field 8 after #1210, and reserves fields 19 and
20, so a rebase onto it adds the field to the chain's manifest, and
nvx-rs's template-manifest reader must accept it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) merged commit 98e5b3c into feature/microvm/dev Nov 9, 2026
73 of 74 checks passed
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Nov 9, 2026
Maps NVX's new pin, 6cb6ba87c, the head of microsoft/openvmm#4995, at
position 232, after position 231, 98e5b3ca5, the head of
feature/microvm/dev, which microsoft/openvmm#4994 merged. The pin is
101 commits past the mapped tip, and positions 132-232 stay unassigned.

Position 232 refuses a paired-scratch restore mode that OpenVMM does
not know with the machine contract shape check's error, where the
capture-tier check panicked. It changes the restore-mode match that
position 166 adds to the capture-tier check of PR19's position 093, and
adds a restore test that uses the paired-scratch manifest of PR19's
position 092 and the resume claim of position 093. Replaying it on
upstream main conflicts in both of its files, since upstream main lacks
the microVM snapshot module of PR17's position 074 and the restore's
tiers and paired scratch, so it follows PR17 and PR19 upstream.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Guide release-nvx Targets a branch under feature/microvm/.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant