Skip to content

feat(daemon): probe the VM image's models in the background and carry them across upgrades - #2508

Merged
zfy0701 merged 4 commits into
mainfrom
claude/vm-model-probe
Sep 25, 2026
Merged

zfy0701 merged 4 commits into
mainfrom
claude/vm-model-probe

Conversation

@zfy0701

@zfy0701 zfy0701 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The VM strategy learned its runtime models only from a real session's turn (S2c), recorded against one image identity. Every upgrade changes the identity, so the record was discarded, and each runtime's VM entry in facts/daemon-runtimes advertised no models until a session of that runtime ran a turn in a VM. The console showed "0 models" for all of them. It was also a chicken-and-egg problem: an agent could not be configured from a VM catalog that only a session could fill.

Change

Probe the VM the way the host is probed, whenever the strategy is available.

  • The cheap availability probe is unchanged. sandbox.microsandbox: false, no usable /dev/kvm or no image reference means nothing is installed, pulled or booted.
  • Once it passes, the ready daemon runs a background task, once a run and never blocking startup. It installs msb and prepares the image when needed, as a first session would. Then it boots one probe VM (probe/models) through the manager's serialized start.
  • Inside that VM it runs the host's runtime probe (probeRuntime: spawn, initialize, throwaway session/new, per-runtime budget) for each image runtime, one at a time, through the shim, via the in-process executor entry.
  • Each runtime launches as its own local VM session would (prepareMicrosandboxLaunch, in a scope of its own under run/vm-probe), with this machine's credential step. The VM's mounts and credential bindings are the union of those launches.
  • A runtime that disagrees with the others on a binding name, a mount, or another runtime's protected credential file is left out with a warning.
  • Results go into strategies.microsandbox.models/modelsSource: probed. A failed probe keeps the previous list.
  • The VM is drained and removed when the sweep ends. Shutdown aborts the sweep without waiting on it, so an in-flight pull cannot hold the drain. The next probe removes a probe VM that an earlier run left behind.
  • Once shutdown has begun, nothing resumes. Readiness stops after msb's install, the manager's recovery and the image preparation, so a manager installed during shutdown is never adopted. A closed manager refuses to pull or to start or restart the preparation VM.
  • The real-session capture stays as a refresh.

Carry the record across an image change. microsandbox/image-models.json is read whichever image wrote it. It is loaded at startup even before msb is installed, so an upgrade that also moves the msb pin starts from it too. Each runtime keeps its last list as cached until the new image's probe replaces it. There is no per-version matching.

The upgrade pre-pull does not probe. It runs as a separate short-lived process of the next release, with no runtime catalog, store or VM driver. The carried-over lists cover the new daemon until its own background probe replaces them a few minutes after it starts.

Docs: §5 of session-executors.md and daemon-sandbox-backends.md. "Boots no VM" now applies only to the availability probe; they also describe the background probe and the carry-over.

The console change (the model picker and the Decision rule editor reading the entry for the agent's strategy) follows in a separate PR.

Verification

  • New test/microsandbox-model-probe.test.ts covers:
    • one VM with every runtime's own workspace and HOME, launched serially;
    • discard before and after, and boot before any runtime's budget;
    • placeholders bound once, and a conflicting binding left out;
    • a failing runtime reported while the rest still run;
    • a shutdown that stops the sweep.
  • test/daemon-sandbox-capability.test.ts covers:
    • the previous image's list reported as cached at startup, before msb is installed;
    • the probe replacing it with probed and recording it;
    • a failure keeping it;
    • no probe when microsandbox is unavailable or microsandbox: false;
    • shutdown during the msb install or the manager's recovery adopting and preparing nothing.
  • test/microsandbox-driver.test.ts covers:
    • the record carrying across a moved tag and a new image, and being replaced one runtime at a time;
    • shutdown during the image pull starting no preparation VM.
  • The shutdown tests fail with the fence removed.
  • Ran locally with --maxWorkers=2: those three files, plus daemon-model-selection, daemon-birth-strategy, executor-placement, model-catalog-daemon, daemon-k8s-mode, cluster-probe and runtime-prober (293 tests, all passing). Daemon typecheck (which includes the smoke script) and eslint on the changed files also pass.
  • Not run locally: anything that boots a real VM needs Linux with KVM. scripts/smoke-microsandbox-runtime.mts gains a step that boots the probe VM. There it probes the daemon's fixture ACP agent, run inline by the image's Node, checks the exact model list it advertises, and checks that the VM is removed. CI's microsandbox smoke job runs it for this change. The first CI run showed the image's signed-out Claude adapter refusing session/new with an internal error; that is why the step now uses the fixture.

🤖 Generated with Claude Code . Claude Opus 5.5

… them across upgrades

The VM strategy's model list was only learned from a real session's turn,
and an image change discarded the recorded list, so after every upgrade each
runtime's VM entry advertised no models until a session ran in a VM.

- Whenever the microsandbox strategy is available, the ready daemon runs the
  host's runtime probe for the image's runtimes in one probe VM, in the
  background: it installs msb and prepares the image when needed, boots
  `probe/models` through the manager's serialized start, and probes each
  runtime serially through the shim, launched as its own local VM session
  would be, with this machine's credential step. The VM is removed when the
  sweep ends; shutdown aborts the sweep without waiting on it.
- The cheap availability probe is unchanged: no KVM or `microsandbox: false`
  means nothing is installed, pulled or booted.
- `microsandbox/image-models.json` is no longer discarded when the image
  changes: each runtime's last list is carried over as `cached` until the new
  image's probe replaces it, and it is read at startup before msb is installed.
- The real-session capture stays as a refresh.
- The microsandbox smoke script probes the image's runtime in a real VM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zfy0701
zfy0701 enabled auto-merge (squash) September 25, 2026 12:54

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The background VM probe introduces a shutdown race. When shutdown starts while msb installation or image preparation is pending, aborting the sweep does not stop microsandboxReady(). That readiness task can finish after stop() has passed its VM teardown and then continue preparing the image or creating its preflight VM. Please fence the readiness path at shutdown (including the post-install and post-pull continuations) so a probe started at readiness cannot create work after teardown. I did not find other blocking issues in the VM model probe. Focused tests were reviewed but not run here because this checkout has no installed dependencies.

sent by review-bot (Codex · gpt-6-sol) · open in session

Comment thread packages/daemon/src/daemon.ts
A shutdown during msb's install, the manager's recovery or the image pull
could let the background probe's readiness resume after teardown: adopt a
freshly installed manager, then pull and boot the preparation VM. Readiness
now stops at each step once shutdown has begun, and a closed manager refuses
to pull or to start or restart the preparation VM.

The microsandbox smoke step now probes the fixture ACP agent through the
image's Node and checks the model list it advertises, instead of relying on
how a signed-out image adapter answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The update guards shutdown during msb installation, recovery, and the image pull. One preparation-VM race remains: probe() checks closed before awaiting VM creation, but stopAll() only drains environments and does not track that preparation VM. If shutdown starts while creation is pending, teardown can finish; creation then resolves and the probe runs its VM preflight after shutdown. Please fence that in-flight creation/preflight path before treating the shutdown issue as resolved. I found no other blocking issue in this revision. I reviewed the new tests but did not run them because this checkout has no installed dependencies.

sent by review-bot (Codex · gpt-6-sol) · open in session

Comment thread packages/daemon/src/microsandbox/driver.ts
The post-pull check ran before the preparation VM's creation was awaited, and
no environment tracks that VM, so a shutdown could finish while the create
was pending and the preflight then ran in the new VM. The VM now checks the
manager is still open once it exists, before running anything, and the
manager's shutdown waits until the preparation VM is gone. It still does not
wait for the pull before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The preparation VM is now tracked through teardown, so shutdown waits for an in-flight creation to be cleaned up, and the closed-manager check prevents its preflight from running afterward. This resolves my prior blocking finding. I reviewed the added regression test and the new diff; I did not run tests locally because dependencies are not installed in this checkout.

sent by review-bot (Codex · gpt-6-sol) · open in session

The daemon project shares one module registry per worker, so the vi.mock of
the install module did not apply once another file had loaded the daemon, and
the file was missing from the list of mocking tests. The daemon takes the
install as an option instead, as it already takes the host check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. This revision replaces the test-wide msb install mock with a daemon constructor test seam; production still calls the same installMicrosandbox function, and the previously approved shutdown fencing is unchanged. I found no blocking issue. I reviewed the focused diff but did not run tests because dependencies are not installed in this checkout.

sent by review-bot (Codex · gpt-6-sol) · open in session

@zfy0701
zfy0701 merged commit d74ff39 into main Sep 25, 2026
13 checks passed
@zfy0701
zfy0701 deleted the claude/vm-model-probe branch September 25, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant