Skip to content

feat: grow hotplug-ready snapshots with virtio-mem - #88

Draft
mkagenius wants to merge 5 commits into
mainfrom
codex/snapshot-memory-growth
Draft

mkagenius wants to merge 5 commits into
mainfrom
codex/snapshot-memory-growth

Conversation

@mkagenius

@mkagenius mkagenius commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Tarit snapshots currently restore with their original guest-visible memory. This adds opt-in virtio-mem templates: boot with 2048 MiB, reserve a 4096 MiB maximum, then restore with target_memory_mib: 4096 while preserving running process and RAM state.

Draft: serial Linux/KVM acceptance passed with corrected test fixtures. Concurrent clone isolation, jailed hardware mode, overcommit and full integration/soak qualification remain untested.

The change carries the memory layout, device bitmap and requested target through snapshot/restore, CLI, orchestrator, peer RPC and generated SDKs. Restore publishes only after the device and guest agent confirm that the requested RAM is online. Shrinking and exceeding the saved maximum fail. Jailed preparation preserves the memory configuration, and pending memory IRQs are signalled after all restored vCPU/LAPIC state is installed.

The maximum remains reserved/backed from creation and determines admission, cgroup budgeting and snapshot extent size. A 2 GiB boot / 4 GiB maximum template has a 4 GiB RAM snapshot extent. This does not implement host-memory overcommit. Existing snapshots are not converted. New templates require the updated VMM, orchestrator, guest kernel and agent; older VMMs reject the new snapshot state. The pinned kernel candidate requires normal promotion/release before the download helper can fetch it.

Hardware evidence:

  • Unchanged production code from 66f9333f0c2a8fca3eeb429b2e136c047792bd35 passed the serial 2→4 GiB gate in 238.33 seconds with corrected fixtures and a shell driver adapted only to fixture/source paths. The unmodified old harness did not pass.
  • Boot MemTotal was 2,043,016 KiB; restored MemTotal was 4,140,168 KiB. The running witness retained its PID, nonce and page markers, touched 3 GiB with zero bad pages, and survived a second snapshot/restore without a target override.
  • Oversized/shrinking requests were rejected. Capacity reuse was verified after explicit test cleanup, not as proof of automatic rollback alone. Serial mode did not test simultaneous source/clone isolation.
  • Fixture-only commit 18e142fbbcc625868a7fbaf0e70cb260034948a5 adds MAP_NORESERVE to the witness's virtual mapping, checks the redacted public API error plus the internal reason in the private daemon log, and adds witness startup diagnostics. It changes no production code and does not enable host overcommit.
  • The committed tree is exactly the recorded fixture-corrected test tree 6ad60bb244f526d955b2527c5917a25638a2484f. C fixture SHA256: dc8752e024805d2224c4d2bc3ba85f0aa615cc7a0fa564a66cd3ddc38c9bed27; Python fixture SHA256: 426960c49dbe52e241efcbdfa96435c8337ef2e188c9e9c8c7d1a9d99e892943. This commit was not rerun on hardware after publication.

Validation:

  • The fixture patch applies cleanly, matches the recorded hashes/tree, parses as Python, passes shell syntax checks and cross-compiles the C witness with strict warnings.
  • On production head 66f9333, protocol, VMM (including Linux/KVM compile/lint), SDK, musl and security CI passed. Kernel reproducibility CI passed config/hash verification, a clean byte-for-byte rebuild and candidate upload; production trees and kernel inputs are unchanged by the fixture commit.
  • 385 local daemon tests and 143 Linux device tests passed, including jailed memory preservation and deferred/saved IRQ regressions. Three existing seccomp worker tests were excluded locally because x86 emulation does not support them; VMM CI passed on Linux.
  • New fixture-head CI is pending. The preceding orchestrator CI failed only on three existing atomic API deprecations tracked separately in fix(orch): preserve saturating atomic updates on Rust 1.99 #87.

Before readiness claims: qualify concurrent clone isolation, production jail/cgroup/seccomp, authenticated lazy restore, SMP, network/volume load, forced agent/driver timeout and full integration/soak behavior. See docs/memory-growth.md for resource semantics, rollout constraints and the test command. No merge, deployment or release publication is included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant