Skip to content

openvmm: after a microVM restore, the TSC leads VM time by the restore's length #1371

Closed

Description

@tangruize

After a time-ABI microVM restore, the guest's TSC is ahead of VM time, which drives the emulated devices' timers, by the window from the restore anchor to the start of VM time. In the real worker that window is milliseconds, and it grows with any delay in the restore. The capture side compensates for its own window (the recorded cut), but nothing compensates on the restore side, so every restore adds this skew between the guest's TSC and its device clocks. No guest or host action is needed. The TSC set on KVM, MSHV and WHP resumes the guest TSC at the anchor, so all three are affected; it was measured on KVM.

Reproduction

A KVM test (not attached) ran time_abi_restore's steps in order, with the real backend's capture anchor and synchronized TSC set and vmcore's VmTimeKeeper. TSC/F minus VM time should match the captured relation; it exceeded it by 32.1 µs for a 32.4 µs window, and by 20.116 ms for a 20.114 ms window with 20 ms inserted.

Cause

LoadedVm::time_abi_restore sets the TSC from the downtime selected at the restore anchor, after which the TSC runs with host time, and derives VM time's advance from the same downtime:

// openvmm/openvmm_core/src/worker/dispatch/time_abi.rs
let set =
    backend(partition.as_ref(), &hypervisor)?.set_synchronized_tsc(&mut |sample| {
        // ...
        let target =
            virt::time_abi::downtime::tsc_target(capture.tsc, selected.nanos, tsc_hz)?;
// ...
let vm_time_downtime_ns = input.contract.vm_time_downtime_ns(downtime.nanos)?;

VM time stays stopped until VmTimeKeeper::start, which anchors it at a later Instant::now():

// vm/vmcore/src/vmtime.rs
pub async fn start(&mut self) {
    let vmtime = self.time.stop_time().expect("should be stopped");
    self.stop_host_time = None;
    let timestamp = Timestamp::new(vmtime, Instant::now());

The window between the two covers the LAPIC advance, the VM-time RPCs, the rest of load, and the start of the units.

Fix

No patch is attached; the options are:

  • Start VM time against the restore anchor's host time instead of Instant::now().
  • Advance VM time once by the measured anchor-to-start interval before it starts.
  • Document the skew, and its bound, as accepted.

Related: #1366.

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions