microvm durabledir support#533
Merged
Dmitry Berkovich (dberkov) merged 3 commits intoJul 25, 2026
Merged
Conversation
Benjamin Elder (BenTheElder)
force-pushed
the
microvm-durabledir
branch
2 times, most recently
from
July 25, 2026 00:21
a3a971e to
e2f023d
Compare
DurableDir volumes shipped for gVisor only; the micro-VM runtime dropped
Container.durable_dir_volumes on the floor and the API rejected the
combination outright. Give the guest the volume:
- cmd/ateom-microvm/internal/tarutil: archive/restore a directory tree
preserving modes, ownership, times, symlinks and hardlinks, with
os.Root-confined extraction. Snapshot support (a later commit) ships the
volume as a tar; no reusable helper existed, and atelet's image untar does
not preserve ownership.
- A SECOND virtiofsd serves the actor's durable-dir directory read-write
with cache=auto. The kataShared share stays read-only at cache=always,
which is what its TODO asked for: writable volumes get their own share
rather than weakening the overlay lower's caching.
- The guest mounts the new tag at /run/ateom-durable and each WORKLOAD
container binds <tag>/<volume> at its declared paths. The carrier is
deliberately left alone: its rootfs is the read-only lower, where the
agent could not create a missing mount point.
- Lift the CEL rule banning durableDir on sandboxClass microvm.
ateom learns the volume name from the directory layout atelet prepares: the
wire protocol carries mount paths only, and the API allows at most one
durable-dir volume, so anything but exactly one directory is an error.
Snapshot scopes are unchanged so far, so the contents live only as long as
the actor. Checkpoint and restore follow.
The micro-VM runtime ignored the snapshot scope: every checkpoint took a
whole-guest CH snapshot, and durable-dir volumes — now host-backed rather
than in guest RAM — were left out of it entirely, so their contents did not
survive a suspend.
Checkpoint, with the guest paused either way:
- Full keeps the CH snapshot (moved to snapshotVMState) and adds a tar of
the volumes.
- Data writes only that tar: no memory image, and no base-id, since nothing
will reattach to the frozen virtio-fs lower.
- Any other scope is rejected rather than silently treated as Full.
Restore mirrors it. The volumes are extracted before anything can observe
them — for Full before the share's virtiofsd starts, for Data before the
workload cold-starts. Full then resumes the guest as before; Data cold-boots
the actor, which is a Run with the volumes already populated, so RunWorkload's
body becomes coldBootActor and both paths share it. Cold boot gates on readyz,
so a Data-scope resume returns with the actor serving.
Pausing first is what makes the tar coherent: the durable share is served
write-through, so every completed guest write is already on the host and no
further ones can arrive. The tar's duration is logged next to the snapshot's —
it is on the suspend path and scales with the volume's contents.
With two virtio-fs devices, the snapshot's fs sockets are now repointed by
device tag instead of all being set to the overlay lower's socket, which would
have handed the guest the wrong filesystem.
Give the micro-VM counter demo the same durableDir volume the gVisor one has, and drop TestDurableDirLifecycle's micro-VM skip so the scope matrix (Full/Full, Data/Full, Data/Data) runs against both runtimes. TestActorLifecycle no longer special-cases micro-VM to expect a file counter of -1. That value was the demo failing to write: it counts in /home/counter/a.txt, which nothing created in the micro-VM rootfs. The volume mounts there now, so both runtimes assert the same counters. Also correct the glossary: an ActorTemplate may declare one DurableDir volume, not several — CEL has enforced that since the feature landed — and Full scope captures the volumes whether or not the runtime keeps them inside rootfs (the micro-VM does not).
Benjamin Elder (BenTheElder)
force-pushed
the
microvm-durabledir
branch
from
July 25, 2026 01:48
e2f023d to
448400a
Compare
Dmitry Berkovich (dberkov)
merged commit Jul 25, 2026
aa1d14a
into
agent-substrate:main
11 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #530
NOTE: This temporarily shares the "only one durabledir" limit, but there's no reason we couldn't lift that for uVM, and TODOs are added for this.
cc Dmitry Berkovich (@dberkov)