From d501e50660136000ec93412aea78fdafb9cf391c Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Mon, 9 Nov 2026 05:24:48 -0800 Subject: [PATCH] microvm: carry a clone's activation record in the restore packet A platform clone takes the settings of its own launch, its activation record, from its host after the restore: NVX's guest waits, once it has acknowledged the restore, for the record in an Activate on its control session, which the host sends once it has connected. The record is small, bounded, and carries no secret, so the restore packet, which the guest reads before it acknowledges the restore, can carry it, and the clone can then activate before the host connects, without a request of the host's (microsoft/nvx#544). The packet's flag bit 5, ACTIVATION, which was reserved, now says that the packet ends, after the DNS server, with the record's length, a little-endian u32 from 1 to 1024, which keeps the record aligned to the guest's four-byte reads, and the record's bytes, which OpenVMM passes to the guest without interpreting them. The packet stays version 1: every guest that decodes version 1 refuses a packet that sets a reserved bit, rather than ignore its record, and a packet without the flag keeps version 1's layout and meaning, so snapshots whose guests predate the flag stay restorable. --restore-activation-record PATH, and the management RPC's MicrovmSnapshotConfig.restore_activation_record_path (15), which takes the option, read the record of a restore before any vCPU runs, and at most one byte past the bound. Only a platform snapshot admits it: a snapshot of a later tier continues a tenant's workload with the settings that it was captured with, and one without a tier has no activation. Such a restore gates host input until the guest acknowledges it, so the guest has read the record before host input reaches it. Tests: the packet's encoding with a record, its bounds, and malformed lengths; the portb device serving a record whose last read is partial; the option's rules on the command line and in the RPC; the tier check; the file's bounds; and the packet's fields. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Guide/src/reference/openvmm/management/cli.md | 16 +- .../src/reference/openvmm/management/grpc.md | 1 + Guide/src/user_guide/openvmm/snapshots.md | 23 ++- openvmm/openvmm_entry/src/cli_args/microvm.rs | 79 +++++++- openvmm/openvmm_entry/src/microvm/restore.rs | 124 ++++++++++++- openvmm/openvmm_entry/src/ttrpc/microvm.rs | 18 ++ .../src/vmservice.proto | 4 + vm/devices/chipset/src/microvm.rs | 53 ++++++ .../chipset_resources/src/microvm_time.rs | 172 +++++++++++++++++- 9 files changed, 467 insertions(+), 23 deletions(-) diff --git a/Guide/src/reference/openvmm/management/cli.md b/Guide/src/reference/openvmm/management/cli.md index a314642337b..b41f2cc449f 100644 --- a/Guide/src/reference/openvmm/management/cli.md +++ b/Guide/src/reference/openvmm/management/cli.md @@ -521,9 +521,10 @@ describes the source definitions. Every microVM restore makes the restore packet available on the private portb restore channel. The packet carries 64 bytes of fresh entropy, the online-VP target, the memory-expansion ranges, the downtime, the TSC rate - deviation, and, at a restore that replaces the network policy of a - snapshot without a tier, the network settings of the restore's policy; the - guest must consume it and explicitly reseed its RNG. + deviation, at a restore that replaces the network policy of a snapshot + without a tier, the network settings of the restore's policy, and, at a + platform restore with `--restore-activation-record`, the clone's activation + record; the guest must consume it and explicitly reseed its RNG. [Snapshots](../../../user_guide/openvmm/snapshots.md) documents its format, the portb status bits, and the time sample. Every microVM portb device also reports generation-ID support in status bit @@ -597,6 +598,15 @@ describes the source definitions. clone from its host, and the guest of a snapshot without a tier from the restore packet, which carries them, and whose restore gates host input until the guest acknowledges that it applied them. +* `--restore-activation-record `: End the restore packet of a platform + snapshot's clone with the bytes of this file, the clone's activation record, + which gives the clone the settings of its own launch. The file holds 1 to + 1024 bytes, which OpenVMM reads before any vCPU runs and passes to the guest + without interpreting them, so they must carry no secret. The restore gates + host input until the guest acknowledges it, which the guest does once it has + read the packet. OpenVMM rejects the option before any vCPU starts for a + cold boot, for a snapshot of another tier or without a tier, and for a file + that is empty or longer than 1024 bytes. * `--restore-generation-id <32-HEX-DIGITS>`: State the immutable storage generation of this restore's sandbox blocks. A `workload-start` or `instance-checkpoint` snapshot binds its read-only layers and paired scratch diff --git a/Guide/src/reference/openvmm/management/grpc.md b/Guide/src/reference/openvmm/management/grpc.md index d27c8979537..aad0916a94a 100644 --- a/Guide/src/reference/openvmm/management/grpc.md +++ b/Guide/src/reference/openvmm/management/grpc.md @@ -289,6 +289,7 @@ option. | `MicrovmSnapshotConfig.restore_generation_id` | `--restore-generation-id` | | `MicrovmSnapshotConfig.restore_absent_scratch` | `--restore-absent-scratch` | | `MicrovmSnapshotConfig.restore_network_policy` | `--restore-network-policy` | +| `MicrovmSnapshotConfig.restore_activation_record_path` | `--restore-activation-record` | A string field takes its option's syntax: `MicrovmNetwork.address` is `IPv4/PREFIX`, such as `10.0.0.2/24`, each egress rule is diff --git a/Guide/src/user_guide/openvmm/snapshots.md b/Guide/src/user_guide/openvmm/snapshots.md index 6196eb4f9ec..19bef73de7e 100644 --- a/Guide/src/user_guide/openvmm/snapshots.md +++ b/Guide/src/user_guide/openvmm/snapshots.md @@ -256,7 +256,8 @@ from the data port. Its 24-byte little-endian header holds: - the magic `OVR` and the version; - flags: bit 0, the downtime came from host UTC; bit 1, a memory target; bit 2, the guest must acknowledge the restore through port `0x605`; bit 3, - a test hook is active; bit 4, the packet carries network settings; + a test hook is active; bit 4, the packet carries network settings; bit 5, + the packet ends with an activation record; - the online-VP target (0 for none), the memory-range count, and the network byte; - the generation counter (`u32`), which is 0 at cold boot and one more than @@ -265,10 +266,11 @@ from the data port. Its 24-byte little-endian header holds: - the downtime in nanoseconds (`u64`). The `(u64 GPA start, u64 length)` memory ranges, 64 bytes of fresh entropy, and -a 16-byte DNS server follow the header. Status bits 2, 3, and 4 report an -online-VP target, a memory target, and one or more memory ranges, so a -zero-range target can skip post-restore repair. The packet carries no host UTC: -the guest sets its clock from a time sample. +a 16-byte DNS server follow the header, and then, with flag bit 5, the +activation record: its length (`u32`), then its bytes. Status bits 2, 3, and 4 +report an online-VP target, a memory target, and one or more memory ranges, so +a zero-range target can skip post-restore repair. The packet carries no host +UTC: the guest sets its clock from a time sample. A restore whose `--restore-network-policy` replaces the network policy of a snapshot without a tier sets flag bit 4: the packet carries the network @@ -283,6 +285,17 @@ acknowledgement, which the guest gives once it has applied the settings. Every other restore, a platform clone's included, leaves flag bit 4 clear and the network byte and DNS server zero. +A platform restore with `--restore-activation-record ` sets flag bit 5: +the packet ends with the bytes of that file, 1 to 1024 of them, the clone's +activation record, which gives the clone the settings of its own launch. +OpenVMM reads the file before any vCPU runs, and passes its bytes to the guest +without interpreting them, so they must carry no secret. Such a restore +requires the guest's acknowledgement, which the guest gives once it has read +the packet. Every other restore leaves flag bit 5 clear, and its packet ends +with the DNS server, as before the flag. A guest that predates the flag +refuses a packet that sets it, whose bit 5 it reserves, rather than ignore its +record. + Status bit 6 advertises the time sample. Writing `0xa7` to the status port latches a 48-byte little-endian sample, version 1: diff --git a/openvmm/openvmm_entry/src/cli_args/microvm.rs b/openvmm/openvmm_entry/src/cli_args/microvm.rs index ce1969715ef..74f64c5d4a3 100644 --- a/openvmm/openvmm_entry/src/cli_args/microvm.rs +++ b/openvmm/openvmm_entry/src/cli_args/microvm.rs @@ -313,6 +313,16 @@ pub struct MicrovmCli { #[clap(long, requires = "restore_snapshot")] pub restore_network_policy: bool, + /// Hand the guest of a platform snapshot's clone the bytes of this file, + /// its activation record, at the end of its restore packet. + /// + /// The record holds 1 to 1024 bytes, which OpenVMM reads before any vCPU + /// runs and passes to the guest without interpreting them, so they must + /// carry no secret. The restore gates host input until the guest has + /// read the packet and acknowledged the restore. + #[clap(long, value_name = "PATH", requires = "restore_snapshot")] + pub restore_activation_record: Option, + /// Immutable storage generation of the sandbox blocks of this restore. /// /// Required to restore a `workload-start` or `instance-checkpoint` @@ -822,13 +832,14 @@ impl Options { && self.microvm.restore_memory.is_none() && !self.microvm.restore_absent_scratch && !self.microvm.restore_network_policy + && self.microvm.restore_activation_record.is_none() && self.microvm.restore_generation_id.is_none() && self.microvm.memory_capacity.is_none() && self.microvm.snapshot_generation_id.is_none() && self.microvm.snapshot_scratch_restore_mode.is_none() && self.microvm.microvm_control_console.is_none() && !self.microvm.microvm_control_auth_stdin, - "--network-profile, --net-tap, --mount, --mount-aggregate, --microvm-sandbox-block, --microvm-workload-identity, --microvm-lifecycle, --microvm-control-console, --microvm-control-auth-stdin, --restore-processors, --restore-memory, --restore-absent-scratch, --restore-network-policy, --restore-generation-id, --memory-capacity, snapshot block policy, and microVM network policy require a microVM machine" + "--network-profile, --net-tap, --mount, --mount-aggregate, --microvm-sandbox-block, --microvm-workload-identity, --microvm-lifecycle, --microvm-control-console, --microvm-control-auth-stdin, --restore-processors, --restore-memory, --restore-absent-scratch, --restore-network-policy, --restore-activation-record, --restore-generation-id, --memory-capacity, snapshot block policy, and microVM network policy require a microVM machine" ); return Ok(()); } @@ -952,6 +963,10 @@ impl Options { self.restore_snapshot.is_some() || !self.microvm.restore_network_policy, "--restore-network-policy requires --restore-snapshot" ); + anyhow::ensure!( + self.restore_snapshot.is_some() || self.microvm.restore_activation_record.is_none(), + "--restore-activation-record requires --restore-snapshot" + ); anyhow::ensure!( self.restore_snapshot.is_some() || self.microvm.restore_generation_id.is_none(), "--restore-generation-id requires --restore-snapshot" @@ -1841,6 +1856,68 @@ mod tests { assert!(standard.validate_microvm_options().is_err()); } + #[cfg(guest_arch = "x86_64")] + #[test] + fn test_microvm_restore_activation_record_requires_a_microvm_restore() { + let restore = Options::try_parse_from([ + "openvmm", + "--machine", + "microvm", + "--restore-snapshot", + "snapshot", + "--restore-activation-record", + "record", + ]) + .unwrap(); + restore.validate_microvm_options().unwrap(); + assert_eq!( + restore.microvm.restore_activation_record.as_deref(), + Some(std::path::Path::new("record")) + ); + + // A cold boot has no restore packet. + assert!( + Options::try_parse_from([ + "openvmm", + "--machine", + "microvm", + "--restore-activation-record", + "record", + ]) + .is_err() + ); + // Clap accepts it when --snapshot-destination, which conflicts with + // --restore-snapshot, lifts the requirement, so the validation refuses + // it. + let cold_boot = Options::try_parse_from([ + "openvmm", + "--machine", + "microvm", + "--snapshot-destination", + "snapshot", + "--restore-activation-record", + "record", + ]) + .unwrap(); + assert_eq!( + cold_boot + .validate_microvm_options() + .unwrap_err() + .to_string(), + "--restore-activation-record requires --restore-snapshot" + ); + // Nor does a standard machine. + let standard = Options::try_parse_from([ + "openvmm", + "--restore-snapshot", + "snapshot", + "--restore-activation-record", + "record", + ]) + .unwrap(); + assert!(standard.validate_microvm_options().is_err()); + } + #[cfg(guest_arch = "x86_64")] #[test] fn test_microvm_sandbox_block_parser_and_validation() { diff --git a/openvmm/openvmm_entry/src/microvm/restore.rs b/openvmm/openvmm_entry/src/microvm/restore.rs index d1609e98c26..c8c17144313 100644 --- a/openvmm/openvmm_entry/src/microvm/restore.rs +++ b/openvmm/openvmm_entry/src/microvm/restore.rs @@ -14,6 +14,7 @@ use crate::cli_args::DiskCliKind; use crate::cli_args::microvm::MachineProfileCli; use crate::storage_builder::microvm::MicrovmSandboxBlockGeometry; use anyhow::Context; +use chipset_resources::microvm_time::MAX_RESTORE_ACTIVATION_LEN; use chipset_resources::microvm_time::RESTORE_ENTROPY_LEN; use chipset_resources::microvm_time::RestoreMemoryRange; use chipset_resources::microvm_time::RestoreNetwork; @@ -33,6 +34,7 @@ use openvmm_helpers::snapshot::microvm::SnapshotMachineContract; use openvmm_helpers::snapshot::microvm::SnapshotMemoryExpansionRange; use openvmm_helpers::snapshot::microvm::SnapshotMicrovmSandboxBlock; use openvmm_helpers::snapshot::restore::OpenedSnapshot; +use std::io::Read; use std::path::Path; use virt::time_abi::HostIdentity; use virt::time_abi::TimeAbiTestHooks; @@ -107,6 +109,10 @@ pub(crate) struct MicrovmRestore { /// restore's policy, which the guest applies before it acknowledges the /// gated restore. pub(crate) network_settings: bool, + /// The activation record that `--restore-activation-record` hands a + /// platform snapshot's clone in its restore packet, which the guest reads + /// before it acknowledges the gated restore. + pub(crate) activation_record: Option>, /// Whether `--restore-memory` selected an explicit RAM target. pub(crate) memory_target_requested: bool, /// Fresh private GPA ranges selected for this restore. @@ -132,7 +138,8 @@ impl MicrovmRestore { /// Returns the controller's fields of the restore packet for a time ABI /// restore, with `network`, the network settings of the restore's policy /// when [`network_settings`](Self::network_settings) asks for them, and - /// the generation ID they imply. The generation counter is the snapshot's + /// the [`activation_record`](Self::activation_record), if any, and the + /// generation ID they imply. The generation counter is the snapshot's /// plus one. pub(crate) fn time_abi_restore_packet( &self, @@ -149,6 +156,7 @@ impl MicrovmRestore { &self.memory_ranges, self.gate_required, network, + self.activation_record.clone(), ) } } @@ -157,9 +165,9 @@ impl MicrovmRestore { /// restore of a snapshot with the time contract `time`, and the generation ID /// they imply. The generation counter is the snapshot's plus one, /// `ack_required` makes the guest acknowledge the restore through port -/// `0x605`, and `network`, the network settings of a restore that replaces -/// the network policy of a snapshot without a tier, makes the packet version -/// 6. +/// `0x605`, `network` carries the network settings of a restore that replaces +/// the network policy of a snapshot without a tier, and `activation` the +/// activation record of a platform snapshot's clone. pub(crate) fn restore_packet_base( time: &SnapshotTimeContract, restore_online_vp_count: Option, @@ -167,6 +175,7 @@ pub(crate) fn restore_packet_base( memory_ranges: &[SnapshotMemoryExpansionRange], ack_required: bool, network: Option, + activation: Option>, ) -> anyhow::Result<([u8; 16], RestorePacketBase)> { let generation = time.capture_generation.checked_add(1).ok_or_else(|| { virt::time_abi::TimeAbiError::new( @@ -199,6 +208,7 @@ pub(crate) fn restore_packet_base( .collect(), entropy, network, + activation, }; base.validate().context("restore packet is invalid")?; generation_id_create.complete("restore", "generation_id_create", Default::default()); @@ -215,10 +225,12 @@ pub(crate) fn restore_packet_base( /// `--restore-absent-scratch` attaches its fresh scratch with no file. Only a /// platform snapshot, or one without a tier, admits /// `--restore-network-policy`, and only a platform snapshot admits -/// `--mount-aggregate` on the filesystem slot that it left dormant. A restore -/// that replaces the network policy of a snapshot without a tier hands the -/// guest the new policy's network settings in its restore packet, and gates -/// host input until the guest has applied them. +/// `--mount-aggregate` on the filesystem slot that it left dormant, and +/// `--restore-activation-record`. A restore that replaces the network policy +/// of a snapshot without a tier hands the guest the new policy's network +/// settings in its restore packet, and gates host input until the guest has +/// applied them, and so does a restore that hands a platform snapshot's clone +/// its activation record, until the guest has read it. pub(crate) fn prepare_restore( opt: &mut Options, restore_snapshot: Option<&OpenedSnapshot>, @@ -228,6 +240,7 @@ pub(crate) fn prepare_restore( let mut absent_scratch = None; let mut restore_gate_required = false; let mut network_settings = false; + let mut activation_record = None; let restore_memory_target_requested = opt.microvm.restore_memory.is_some(); let mut restore_memory_ranges = Vec::new(); if let Some(contract) = @@ -270,6 +283,11 @@ pub(crate) fn prepare_restore( network_settings = true; restore_gate_required = true; } + if let Some(path) = opt.microvm.restore_activation_record.as_deref() { + validate_activation_record_restore(&manifest.snapshot_tier)?; + activation_record = Some(read_activation_record(path)?); + restore_gate_required = true; + } validate_filesystem_aggregate_restore( &manifest.snapshot_tier, contract.microvm_filesystem.is_some(), @@ -427,6 +445,7 @@ pub(crate) fn prepare_restore( machine_contract: restore_machine_contract, gate_required: restore_gate_required, network_settings, + activation_record, memory_target_requested: restore_memory_target_requested, memory_ranges: restore_memory_ranges, private_scratch_dir, @@ -478,6 +497,40 @@ fn restore_packet_carries_network_settings( restore_network_policy && snapshot_tier.is_empty() } +/// Rejects `--restore-activation-record` for a snapshot of a tier other than +/// the platform tier, `snapshot_tier`. +/// +/// A platform snapshot is captured before any tenant input, so that each of +/// its clones takes the settings of its own launch, which its activation +/// record carries. A snapshot of a later tier continues a tenant's workload +/// with the settings that it was captured with, and one without a tier has no +/// activation. +fn validate_activation_record_restore(snapshot_tier: &str) -> anyhow::Result<()> { + anyhow::ensure!( + snapshot_tier == openvmm_helpers::snapshot::format::SNAPSHOT_TIER_PLATFORM, + "--restore-activation-record requires a platform snapshot" + ); + Ok(()) +} + +/// Reads the activation record at `path`, which the restore packet hands the +/// guest without interpreting it: 1 to [`MAX_RESTORE_ACTIVATION_LEN`] bytes, +/// of which it reads at most one more, so that a longer file fails without +/// being read whole. +fn read_activation_record(path: &Path) -> anyhow::Result> { + let file = fs_err::File::open(path).context("failed to open the activation record")?; + let mut record = Vec::new(); + file.take(MAX_RESTORE_ACTIVATION_LEN as u64 + 1) + .read_to_end(&mut record) + .with_context(|| format!("failed to read the activation record {}", path.display()))?; + anyhow::ensure!( + (1..=MAX_RESTORE_ACTIVATION_LEN).contains(&record.len()), + "activation record {} must hold 1 to {MAX_RESTORE_ACTIVATION_LEN} bytes", + path.display() + ); + Ok(record) +} + /// Rejects `--mount-aggregate`, which `aggregate` reports, on the filesystem /// slot that a snapshot of a tier other than the platform tier left dormant, /// which `captured_filesystem` reports when false. @@ -1135,6 +1188,50 @@ pub(crate) mod tests { } } + #[test] + fn only_a_platform_restore_hands_over_an_activation_record() { + use openvmm_helpers::snapshot::format::SNAPSHOT_TIER_INSTANCE_CHECKPOINT; + use openvmm_helpers::snapshot::format::SNAPSHOT_TIER_PLATFORM; + use openvmm_helpers::snapshot::format::SNAPSHOT_TIER_WORKLOAD_START; + + validate_activation_record_restore(SNAPSHOT_TIER_PLATFORM).unwrap(); + for tier in [ + "", + SNAPSHOT_TIER_WORKLOAD_START, + SNAPSHOT_TIER_INSTANCE_CHECKPOINT, + ] { + assert_eq!( + validate_activation_record_restore(tier) + .unwrap_err() + .to_string(), + "--restore-activation-record requires a platform snapshot", + "{tier}" + ); + } + } + + #[test] + fn activation_records_hold_one_to_the_most_bytes() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("record"); + for length in [1, 602, MAX_RESTORE_ACTIVATION_LEN] { + let record: Vec = (0..length).map(|i| i as u8).collect(); + fs_err::write(&path, &record).unwrap(); + assert_eq!(read_activation_record(&path).unwrap(), record); + } + for length in [0, MAX_RESTORE_ACTIVATION_LEN + 1, 1 << 20] { + fs_err::write(&path, vec![0x5a; length]).unwrap(); + let err = read_activation_record(&path).unwrap_err(); + assert!( + err.to_string().ends_with(&format!( + "must hold 1 to {MAX_RESTORE_ACTIVATION_LEN} bytes" + )), + "{err:#}" + ); + } + assert!(read_activation_record(&dir.path().join("missing")).is_err()); + } + #[test] fn only_a_platform_restore_attaches_an_aggregate_to_a_dormant_slot() { use openvmm_helpers::snapshot::format::SNAPSHOT_TIER_INSTANCE_CHECKPOINT; @@ -1193,6 +1290,7 @@ pub(crate) mod tests { machine_contract: Some(contract), gate_required: true, network_settings: false, + activation_record: None, memory_target_requested: true, memory_ranges: vec![SnapshotMemoryExpansionRange { gpa_start: 0x1_0000_0000, @@ -1242,6 +1340,16 @@ pub(crate) mod tests { assert_eq!(base.network, Some(network)); assert!(base.ack_required); + // The packet carries a platform clone's activation record. + let restore = MicrovmRestore { + activation_record: Some(vec![0xa1, 0xa2]), + ..time_abi_restore(6) + }; + let (_, base) = restore.time_abi_restore_packet(None, None).unwrap(); + assert_eq!(base.activation.as_deref(), Some(&[0xa1, 0xa2][..])); + assert_eq!(base.network, None); + assert!(base.ack_required); + assert!(restore.time_abi_restore_packet(Some(3), None).is_err()); let err = time_abi_restore(u32::MAX) .time_abi_restore_packet(None, None) diff --git a/openvmm/openvmm_entry/src/ttrpc/microvm.rs b/openvmm/openvmm_entry/src/ttrpc/microvm.rs index 9652d8dfb5f..db9176fc637 100644 --- a/openvmm/openvmm_entry/src/ttrpc/microvm.rs +++ b/openvmm/openvmm_entry/src/ttrpc/microvm.rs @@ -153,6 +153,7 @@ impl MicrovmRequest { restore_generation_id, restore_absent_scratch, restore_network_policy, + restore_activation_record_path, } = microvm_snapshot.unwrap_or_default(); let snapshot_destination = absolute_path(destination_path)?; let restore_snapshot = absolute_path(restore_path)?; @@ -169,6 +170,10 @@ impl MicrovmRequest { (!restore_generation_id.is_empty(), "restore_generation_id"), (restore_absent_scratch, "restore_absent_scratch"), (restore_network_policy, "restore_network_policy"), + ( + !restore_activation_record_path.is_empty(), + "restore_activation_record_path", + ), ] { anyhow::ensure!(restore || !present, "{field} requires restore_path"); } @@ -367,6 +372,7 @@ impl MicrovmRequest { generation_id_option(&restore_generation_id, "restore_generation_id")?; opt.microvm.restore_absent_scratch = restore_absent_scratch; opt.microvm.restore_network_policy = restore_network_policy; + opt.microvm.restore_activation_record = absolute_path(restore_activation_record_path)?; opt.microvm.cpu_profile = (!cpu_profile.is_empty()).then_some(cpu_profile); Ok(Self { @@ -1198,6 +1204,7 @@ mod tests { assert_eq!(rpc.restore_memory, cli.restore_memory); assert_eq!(rpc.restore_absent_scratch, cli.restore_absent_scratch); assert_eq!(rpc.restore_network_policy, cli.restore_network_policy); + assert_eq!(rpc.restore_activation_record, cli.restore_activation_record); assert_eq!(rpc.restore_generation_id, cli.restore_generation_id); assert_eq!(rpc.restore_gate_timeout_ms, cli.restore_gate_timeout_ms); assert_eq!(rpc.snapshot_destination, cli.snapshot_destination); @@ -1439,6 +1446,8 @@ mod tests { let snapshot = snapshot.to_str().unwrap(); let ready = std::env::temp_dir().join("ready.sock"); let ready = ready.to_str().unwrap(); + let record = std::env::temp_dir().join("activation.bin"); + let record = record.to_str().unwrap(); let request = vmservice::CreateVmRequest { config: Some(vmservice::VmConfig { machine_profile: vmservice::vm_config::MachineProfile::Microvm as i32, @@ -1490,6 +1499,7 @@ mod tests { restore_generation_id: vec![0x22; 16], restore_absent_scratch: true, restore_network_policy: true, + restore_activation_record_path: record.to_owned(), ..Default::default() }), }; @@ -1500,6 +1510,8 @@ mod tests { snapshot, "--restore-ready-path", ready, + "--restore-activation-record", + record, "--restore-processors", "2", "--restore-gate-timeout-ms", @@ -1891,6 +1903,12 @@ mod tests { capture(&|request| snapshot(request).restore_network_policy = true), "restore_network_policy requires restore_path", ), + ( + capture(&|request| { + snapshot(request).restore_activation_record_path = "record".to_owned(); + }), + "restore_activation_record_path requires restore_path", + ), ( capture(&|request| snapshot(request).restore_path = "snapshot".to_owned()), "snapshot capture and restore paths are mutually exclusive", diff --git a/openvmm/openvmm_ttrpc_vmservice/src/vmservice.proto b/openvmm/openvmm_ttrpc_vmservice/src/vmservice.proto index 819a01c87a5..fe8f69d76c6 100644 --- a/openvmm/openvmm_ttrpc_vmservice/src/vmservice.proto +++ b/openvmm/openvmm_ttrpc_vmservice/src/vmservice.proto @@ -1067,6 +1067,10 @@ message MicrovmSnapshotConfig { // policy that the NIC of a platform snapshot, or of a snapshot without a // tier, was captured with (--restore-network-policy). bool restore_network_policy = 14; + // File whose 1 to 1024 bytes, the activation record of a platform + // snapshot's clone, end the clone's restore packet + // (--restore-activation-record). They carry no secret. + string restore_activation_record_path = 15; } message CreateVMRequest { diff --git a/vm/devices/chipset/src/microvm.rs b/vm/devices/chipset/src/microvm.rs index 04eef555d48..e657c888c73 100644 --- a/vm/devices/chipset/src/microvm.rs +++ b/vm/devices/chipset/src/microvm.rs @@ -1579,6 +1579,7 @@ mod tests { ranges: Vec::new(), entropy: [0x7e; 64], network: None, + activation: None, }; let (send, recv) = mesh::oneshot(); let mut restored = MicrovmPortb::new( @@ -1673,6 +1674,7 @@ mod tests { ranges, entropy: [0x5a; 64], network: None, + activation: None, }; let (send, recv) = mesh::oneshot(); let mut portb = time_abi_portb(Some( @@ -1724,6 +1726,7 @@ mod tests { )), announce_listeners: true, }), + activation: None, }; let (send, recv) = mesh::oneshot(); let mut portb = time_abi_portb(Some( @@ -1764,6 +1767,54 @@ mod tests { ); } + #[test] + fn a_restore_packet_carries_an_activation_record() { + let record: Vec = (0..=u8::MAX).cycle().take(1021).collect(); + let base = RestorePacketBase { + online_vp_count: 0, + memory_target: false, + ack_required: true, + generation: 7, + ranges: Vec::new(), + entropy: [0x5a; 64], + network: None, + activation: Some(record.clone()), + }; + let (send, recv) = mesh::oneshot(); + let mut portb = time_abi_portb(Some( + chipset_resources::microvm::MicrovmRestorePacketSource { + base: base.clone(), + time: recv, + selected: None, + }, + )); + send.send(RestoreTimeRecord { + downtime_ns: 0, + downtime_utc: false, + rate_deviation: 0, + test_hooks: false, + }); + assert!(matches!( + portb.io_write(STATUS_PORT, &[RESTORE_PACKET_SELECT]), + IoResult::Ok + )); + // The guest reads the packet four bytes at a time, and its last read + // takes the one byte that remains. + let bytes = read_record(&mut portb, DATA_PORT, base.encoded_len()); + assert_eq!(bytes.len(), 104 + 4 + record.len()); + assert_ne!( + bytes[4] & chipset_resources::microvm_time::FLAG_ACTIVATION, + 0 + ); + assert_eq!(bytes[104..], [&[0xfd, 0x03, 0, 0], &record[..]].concat()); + assert_eq!(RestorePacket::decode(&bytes).unwrap().base, base); + // Reading the last byte consumed the packet. + assert_eq!( + read_status(&mut portb), + STATUS_GENERATION_ID_AVAILABLE | STATUS_TIME_SAMPLE_AVAILABLE + ); + } + fn time_abi_portb( restore: Option, ) -> MicrovmPortb { @@ -1909,6 +1960,7 @@ mod tests { ranges: Vec::new(), entropy: [0x3c; 64], network: None, + activation: None, }; let time = RestoreTimeRecord { downtime_ns: 1_000_000, @@ -1979,6 +2031,7 @@ mod tests { ranges: Vec::new(), entropy: [0; 64], network: None, + activation: None, }, time: recv, selected: None, diff --git a/vm/devices/chipset_resources/src/microvm_time.rs b/vm/devices/chipset_resources/src/microvm_time.rs index 2d314b8ce88..9938245935a 100644 --- a/vm/devices/chipset_resources/src/microvm_time.rs +++ b/vm/devices/chipset_resources/src/microvm_time.rs @@ -31,6 +31,12 @@ pub const RESTORE_ENTROPY_LEN: usize = 64; /// address, or an IPv4 address and twelve zero bytes, or zeros without a DNS /// server. pub const RESTORE_DNS_SERVER_LEN: usize = 16; +/// The length of the activation record's length field, which follows the +/// DNS server record in a packet with [`FLAG_ACTIVATION`], and keeps the +/// record's bytes aligned to the guest's four-byte reads. +pub const RESTORE_ACTIVATION_LENGTH_LEN: usize = 4; +/// The most bytes of an activation record. +pub const MAX_RESTORE_ACTIVATION_LEN: usize = 1024; /// The most memory ranges a packet can carry. pub const MAX_RESTORE_RANGES: usize = u8::MAX as usize; /// The online VP targets a packet can carry; 0 means no processor target. @@ -50,8 +56,18 @@ pub const FLAG_TEST_HOOKS: u8 = 1 << 3; /// restore that replaces the network policy of a snapshot without a tier. /// Without it, both are zero. pub const FLAG_NETWORK: u8 = 1 << 4; -const RESTORE_PACKET_FLAGS: u8 = - FLAG_DOWNTIME_UTC | FLAG_MEMORY_TARGET | FLAG_ACK_REQUIRED | FLAG_TEST_HOOKS | FLAG_NETWORK; +/// The packet ends with the activation record of a platform clone, which +/// the launcher supplied: its length, a little-endian `u32` from 1 to +/// [`MAX_RESTORE_ACTIVATION_LEN`], then its bytes, which OpenVMM passes to +/// the guest without reading them. A guest that predates the flag refuses +/// the packet by its reserved bit, rather than ignore the record. +pub const FLAG_ACTIVATION: u8 = 1 << 5; +const RESTORE_PACKET_FLAGS: u8 = FLAG_DOWNTIME_UTC + | FLAG_MEMORY_TARGET + | FLAG_ACK_REQUIRED + | FLAG_TEST_HOOKS + | FLAG_NETWORK + | FLAG_ACTIVATION; /// The bits of the network byte that give the family of the DNS server: 0 /// for none, [`NETWORK_DNS_IPV4`], or [`NETWORK_DNS_IPV6`]. @@ -118,6 +134,10 @@ pub enum RestorePacketError { /// not zero, or is not zero although the network byte names no DNS server. #[error("invalid DNS server")] DnsServer, + /// The activation record is empty, or longer than + /// [`MAX_RESTORE_ACTIVATION_LEN`] bytes. + #[error("activation record is {0} bytes, not 1 to {MAX_RESTORE_ACTIVATION_LEN}")] + Activation(usize), /// The record length does not match its counts. #[error("record is {actual} bytes, expected {expected}")] Length { @@ -259,6 +279,11 @@ pub struct RestorePacketBase { /// The network settings of a restore that replaces the network policy of /// a snapshot without a tier, which set [`FLAG_NETWORK`]. pub network: Option, + /// The activation record of a platform clone, which sets + /// [`FLAG_ACTIVATION`]: 1 to [`MAX_RESTORE_ACTIVATION_LEN`] bytes that + /// the launcher supplied, which the guest decodes. The record carries no + /// secret, since the packet is no confidential channel. + pub activation: Option>, } impl RestorePacketBase { @@ -278,6 +303,11 @@ impl RestorePacketBase { if let Some(network) = &self.network { network.validate()?; } + if let Some(record) = &self.activation + && !(1..=MAX_RESTORE_ACTIVATION_LEN).contains(&record.len()) + { + return Err(RestorePacketError::Activation(record.len())); + } Ok(()) } @@ -287,6 +317,10 @@ impl RestorePacketBase { + RESTORE_RANGE_LEN * self.ranges.len() + RESTORE_ENTROPY_LEN + RESTORE_DNS_SERVER_LEN + + self + .activation + .as_ref() + .map_or(0, |record| RESTORE_ACTIVATION_LENGTH_LEN + record.len()) } } @@ -330,6 +364,7 @@ impl RestorePacket { (self.base.ack_required, FLAG_ACK_REQUIRED), (self.time.test_hooks, FLAG_TEST_HOOKS), (self.base.network.is_some(), FLAG_NETWORK), + (self.base.activation.is_some(), FLAG_ACTIVATION), ] { if set { flags |= flag; @@ -360,6 +395,12 @@ impl RestorePacket { network.dns_server_record() }), ); + if let Some(record) = &self.base.activation { + // The validation bounds the record by MAX_RESTORE_ACTIVATION_LEN, + // which fits the length field. + bytes.extend_from_slice(&(record.len() as u32).to_le_bytes()); + bytes.extend_from_slice(record); + } debug_assert_eq!(bytes.len(), self.base.encoded_len()); Ok(bytes) } @@ -383,7 +424,27 @@ impl RestorePacket { let range_count = usize::from(header[6]); let entropy_offset = RESTORE_PACKET_HEADER_LEN + RESTORE_RANGE_LEN * range_count; let dns_server_offset = entropy_offset + RESTORE_ENTROPY_LEN; - let expected = dns_server_offset + RESTORE_DNS_SERVER_LEN; + let activation_offset = dns_server_offset + RESTORE_DNS_SERVER_LEN; + let record_offset = activation_offset + RESTORE_ACTIVATION_LENGTH_LEN; + let expected = if flags & FLAG_ACTIVATION != 0 { + let length: &[u8; RESTORE_ACTIVATION_LENGTH_LEN] = bytes + .get(activation_offset..record_offset) + .and_then(|length| length.try_into().ok()) + .ok_or(RestorePacketError::Length { + expected: record_offset, + actual: bytes.len(), + })?; + // A length beyond the bound fails before the packet's length + // is checked against it. + let length = u32::from_le_bytes(*length); + let length = usize::try_from(length) + .ok() + .filter(|length| *length <= MAX_RESTORE_ACTIVATION_LEN) + .ok_or(RestorePacketError::Activation(length as usize))?; + record_offset + length + } else { + activation_offset + }; if bytes.len() != expected { return Err(RestorePacketError::Length { expected, @@ -403,8 +464,10 @@ impl RestorePacket { } }) .collect(); - let dns_server_record: &[u8; RESTORE_DNS_SERVER_LEN] = - bytes[dns_server_offset..].try_into().unwrap(); + let dns_server_record: &[u8; RESTORE_DNS_SERVER_LEN] = bytes + [dns_server_offset..activation_offset] + .try_into() + .unwrap(); let network = if flags & FLAG_NETWORK != 0 { Some(RestoreNetwork::decode(header[7], dns_server_record)?) } else if header[7] != 0 || *dns_server_record != [0; RESTORE_DNS_SERVER_LEN] { @@ -412,6 +475,7 @@ impl RestorePacket { } else { None }; + let activation = (flags & FLAG_ACTIVATION != 0).then(|| bytes[record_offset..].to_vec()); let packet = Self { base: RestorePacketBase { online_vp_count: header[5], @@ -421,6 +485,7 @@ impl RestorePacket { ranges, entropy: bytes[entropy_offset..dns_server_offset].try_into().unwrap(), network, + activation, }, time: RestoreTimeRecord { downtime_ns: u64_at(16), @@ -626,6 +691,7 @@ mod tests { ranges: Vec::new(), entropy: entropy(), network: None, + activation: None, }, time: RestoreTimeRecord { downtime_ns: 0x0102_0304_0506_0708, @@ -655,6 +721,7 @@ mod tests { ], entropy: entropy(), network: None, + activation: None, }, time: RestoreTimeRecord { downtime_ns: 30_000_000_000, @@ -713,7 +780,8 @@ mod tests { }; assert_eq!(mutate(0, b'X'), RestorePacketError::Magic); assert_eq!(mutate(3, 4), RestorePacketError::Version(4)); - assert_eq!(mutate(4, 0x2a), RestorePacketError::Reserved); + assert_eq!(mutate(4, 0x4a), RestorePacketError::Reserved); + assert_eq!(mutate(4, 0x8a), RestorePacketError::Reserved); assert_eq!(mutate(7, 1), RestorePacketError::Reserved); assert_eq!(mutate(5, 3), RestorePacketError::OnlineVpCount(3)); assert_eq!(mutate(4, 0x08), RestorePacketError::MemoryRanges(2)); @@ -905,6 +973,98 @@ mod tests { )); } + fn activation_packet(record: Vec) -> RestorePacket { + let mut packet = packet_with_ranges(); + packet.base.activation = Some(record); + packet + } + + #[test] + fn golden_packet_with_an_activation_record() { + let packet = activation_packet(vec![0xa1, 0xa2, 0xa3]); + let bytes = packet.encode().unwrap(); + let plain = packet_with_ranges().encode().unwrap(); + // The activation flag, then the record after the DNS server: its + // length, a little-endian u32, and its bytes. + let mut expected = plain.clone(); + expected[4] |= FLAG_ACTIVATION; + expected.extend([3, 0, 0, 0, 0xa1, 0xa2, 0xa3]); + assert_eq!(bytes, expected); + assert_eq!(bytes.len(), packet.base.encoded_len()); + assert_eq!(RestorePacket::decode(&bytes).unwrap(), packet); + + // The longest record. + let packet = activation_packet(vec![0x5a; MAX_RESTORE_ACTIVATION_LEN]); + let bytes = packet.encode().unwrap(); + assert_eq!( + bytes.len(), + plain.len() + RESTORE_ACTIVATION_LENGTH_LEN + MAX_RESTORE_ACTIVATION_LEN + ); + assert_eq!( + bytes[plain.len()..plain.len() + 4], + [0x00, 0x04, 0x00, 0x00] + ); + assert_eq!(RestorePacket::decode(&bytes).unwrap(), packet); + } + + #[test] + fn activation_records_are_bounded() { + for length in [0, MAX_RESTORE_ACTIVATION_LEN + 1] { + assert_eq!( + activation_packet(vec![0; length]).encode().unwrap_err(), + RestorePacketError::Activation(length) + ); + } + + let good = activation_packet(vec![0xa1, 0xa2, 0xa3]).encode().unwrap(); + let length = good.len() - 7; + // An empty record, and one longer than the bound, whose bytes are + // all there, or whose length is too large to read. + let mut empty = good[..length + 4].to_vec(); + empty[length..].copy_from_slice(&[0; 4]); + assert_eq!( + RestorePacket::decode(&empty).unwrap_err(), + RestorePacketError::Activation(0) + ); + let mut long = good[..length].to_vec(); + long.extend((MAX_RESTORE_ACTIVATION_LEN as u32 + 1).to_le_bytes()); + long.resize(long.len() + MAX_RESTORE_ACTIVATION_LEN + 1, 0); + assert_eq!( + RestorePacket::decode(&long).unwrap_err(), + RestorePacketError::Activation(MAX_RESTORE_ACTIVATION_LEN + 1) + ); + let mut huge = good[..length].to_vec(); + huge.extend(u32::MAX.to_le_bytes()); + assert_eq!( + RestorePacket::decode(&huge).unwrap_err(), + RestorePacketError::Activation(u32::MAX as usize) + ); + // A packet that ends within the length, or before or after the + // record's bytes. + for end in [length, length + 3, good.len() - 1] { + assert!(matches!( + RestorePacket::decode(&good[..end]).unwrap_err(), + RestorePacketError::Length { .. } + )); + } + let mut trailing = good.clone(); + trailing.push(0); + assert!(matches!( + RestorePacket::decode(&trailing).unwrap_err(), + RestorePacketError::Length { .. } + )); + // Without the flag, the packet ends at the DNS server. + let mut unflagged = good.clone(); + unflagged[4] &= !FLAG_ACTIVATION; + assert_eq!( + RestorePacket::decode(&unflagged).unwrap_err(), + RestorePacketError::Length { + expected: length, + actual: good.len() + } + ); + } + fn sample() -> TimeSample { TimeSample { test_hooks: true,