Skip to content

nvx-rs: let an adapter read the boot console of a template's clones #996

Closed

Description

Follow-up to the snapshot/restore workstream (#833).

Goal. An adapter can read a clone's boot console, as it can a cold sandbox's. After a restore, that console holds what the guest's kernel and init write to hvc0, such as a panic, an oops, or an out-of-memory kill. Today it is discarded, so a clone that fails after its restore, before or after Ready, leaves no record of why.

Today (dev c3f15cc8; OpenVMM 832d5bb4, NVX's pin):

  • A template's capture and each clone's launch pass --virtio-console none, a boot console that discards its output (nvx-rs/src/host/template/store/capture.rs:317-318, nvx-rs/src/host/template/clone/launch.rs:482-483). NVX's own launches do the same (nvx-rs/src/host/launch.rs:260-261, tests/runtime-adapter/src/layered.rs:320-321). An adapter's cold launch chooses its own boot console, and may listen at an endpoint, read it, and keep a console log. ClonePlan::launch renders all of a clone's arguments, and takes no console.
  • OpenVMM records the boot console's attachment in the snapshot, and a restore must request that attachment: a template captured with none restores only with none (openvmm/openvmm_entry/src/microvm/console.rs:575, :639). Only a listener may be replaced at a restore, by a listener of the same kind at another endpoint (microvm_console_listener_replacement_matches, console.rs:44). On Linux, the listener's socket must be in the snapshot's parent directory (validate_microvm_console_attachment_namespace, console.rs:421, checked at config.rs:186), as the control socket must be, which is why a clone's launch reserves its control socket beside the snapshot.
  • A listener holds the guest's console output while no reader is connected (VirtioConsoleDisconnectPolicy::Retain, vm/devices/virtio/virtio_resources/src/console/attachment.rs:13-18), and the guest's console writes then wait for a reader. A template captured with a listener would need a reader during its capture, and each of its clones would need one for as long as it runs.

Proposal. One of:

  1. OpenVMM, then nvx-rs (preferred): let a restore attach a listener boot console to a snapshot captured with none. The guest's device is the same either way; only the host's attachment changes. The listener discards output while no reader is connected, as none does, so a clone never waits for a reader that has gone away. Templates keep capturing with none. ClonePlan::launch takes an optional boot console, reserves its endpoint beside the snapshot as it does the control socket (on Windows, a named pipe), and returns the endpoint, from which the adapter reads.
  2. nvx-rs only: capture each template with a listener in the capture's transaction directory, which the capture reads, and keeps as the capture's console log for diagnosing a failed capture. Each clone's launch replaces the listener with its own, beside the snapshot, and its adapter must read it for as long as the clone runs, as for a cold launch with a listener.

Acceptance:

  • A clone launched with a boot console delivers to its reader what the guest writes to hvc0 after the restore, on KVM, MSHV, and WHP. A clone launched without one restores as today.
  • After a crash of the caller, a reattach can read the clone's console again.
  • With the first option, a clone whose reader disconnects keeps running, and OpenVMM still refuses any other change of attachment at a restore.

Not in scope: the control console, and the boot consoles of cold launches.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions