Repository navigation
Description
Summary
The nightly full run of dev at 4f9a3214f failed in Package nvx crate: the build of the packed nvx crate with every feature panicked in its build script, because the release that nvx-rs/artifacts.json pins, v0.1.0-dev.9ceb89fcc31f, declares neither a time ABI version nor a restore-packet version in its SOURCE-MANIFEST.json. #1018 raised the crate's time ABI to 2 and its restore packet to 5, and made the build script require a release to declare both, but artifacts.json still pins the release of October 4, which predates them.
Every bundled build of dev that downloads the pinned release fails the same way, so crates.io gets no new version of nvx, and the nightly stays red, until a pull request repins a compatible release, as The nvx crate says. The OpenVMM pin plays no part: the job checks out no submodule, and #1133, which pins microVM v3.0, changes no contract.
Incident
-
Workflow: CI run 41276515539, attempt 1,
scheduleondev, at4f9a3214f09b1474311ad1bdc957014828f7f03b, the merge of openvmm: pin microVM v2.4, rebased onto upstream main #1075. Nightly failure report: ci: the nightly full run is failing #1134. -
Failed job:
Package nvx crate, stepPackage, check, and build the nvx crate, onubuntu-latestwith Rust 1.95.0. Sincenvx-rs/changed after23ad267292b4, from which0.1.0-alpha.8was packed, the job packed0.1.0-alpha.9, and then built it with every feature, bundling the KVM release:warning: nvx@0.1.0-alpha.9: nvx: downloading https://github.com/microsoft/nvx/releases/download/v0.1.0-dev.9ceb89fcc31f/nvx-0.1.0-linux-kvm.tar.gz thread 'main' (3117) panicked at build.rs:354:9: nvx bundled artifacts: .../nvx-0.1.0-linux-kvm/SOURCE-MANIFEST.json does not declare the compatible OpenVMM contracts (time_abi_version: expected 2, found none; restore_packet_version: expected 5, found none) -
Downstream:
Publish nvx cratewas skipped. The run's other 35 jobs passed, and the rest of its skipped jobs run only for pushes, pull requests or merge groups. -
Not superseded: this is the newest full run, and
devhas since taken only two[skip ci]commits, which leavenvx-rs/unchanged, so the next nightly fails the same way.
History
- Add aci_edge_sandboxes with an OpenVMM sandbox backend #307 pinned
v0.1.0-dev.89c192e79f0dfor the crate's predecessor,aci_edge_sandboxes. Track aci_edge_sandboxes bugs: uncancellable async exec, stale bundled pin, lock-file leak, and signed CIDR prefixes #364 found that pin stale after the move to thenvx-microvm-v2-control-v2control contract, the same failure as this one, and aci_edge_sandboxes: fix four bugs from the crate's bug hunt #367 repinnedv0.1.0-dev.9ceb89fcc31f. nvx-rs: extract the NVX mechanics from aci_edge_sandboxes #519 moved the pin tonvx-rs/artifacts.jsonunchanged, and nothing has repinned it since. - The nightly of October 31, run 41172170265 at
23ad267292b4, published0.1.0-alpha.8with that pin, when the build script checked only the microVM ABI, the control session and the control contract. - time ABI: publish the host's clock correction with each time sample #1018, merged on October 31 as
10b405b29, raisedTIME_ABI_VERSIONfrom 1 to 2 andRESTORE_PACKET_VERSIONfrom 4 to 5, and replaced the three fields ofmanifest_is_compatiblewith the five ofcontract::manifest_mismatches. Its review asked to "require a compatibleartifacts.jsonpin before publishing", which the build script's check now does. time ABI: publish the host's clock correction with each time sample #1018 could not repin, since no release declared time ABI 2 before it merged;v0.1.0-dev.10b405b29f0a, from its merge, is the first that does. - nvx-rs: final acceptance for the image-backed runtime #624 owned the re-pin of
bundleduntil its rescoping on November 1 dropped it. nvx-rs slimming 4.2.3: the old configs, the old launch code andbundledare gone #1122 deletesbundled, its build script andartifacts.json, which ends this kind of failure, but it waits for nvx-rs slimming 4.2.2: the old clone path, template store and sandbox store are gone #1121.
Root cause
The pinned release's SOURCE-MANIFEST.json declares these contract versions, and no others:
{"microvm_abi_version": 2, "control_session_protocol_version": 1, "control_contract_revision": "nvx-microvm-v2-control-v2"}At 4f9a3214f, contract::manifest_mismatches also requires time_abi_version 2 and restore_packet_version 5, so stage() in nvx-rs/build.rs refuses the release. No pull request job builds bundled against the pin: the --all-features builds of check-nvx-rs set NVX_BUNDLE=skip, and its bundling test stages a synthetic release from the commit's own SOURCE-MANIFEST.json. Only the nightly Package nvx crate job downloads the pinned release, so a contract change that merges without a repin fails there first.
Reproduction
On Windows, with Rust 1.95.0:
- The
SOURCE-MANIFEST.jsonof each package, read from the release assets themselves: the pinned release'slinux-kvmandwindows-whppackages lack both fields, whilev0.1.0-dev.10b405b29f0aandv0.1.0-dev.4f9a3214f09bdeclare all five values that the crate expects. - The build script of
4f9a3214f, run bycargo check --locked --features bundledwithNVX_BUNDLE_DIRnaming the pinned release'sSOURCE-MANIFEST.jsonandSHA256SUMS, panics atbuild.rs:354:9with the same two mismatches. With those ofv0.1.0-dev.4f9a3214f09b, it passes the contract check.
Fix
Repin nvx-rs/artifacts.json to v0.1.0-dev.4f9a3214f09b, the release of the nightly's own commit. Its OpenVMM is dev's pin, and its SOURCE-MANIFEST.json declares every contract version of nvx::contract. The GitHub release API gives these digests:
| Package | SHA-256 |
|---|---|
linux-kvm |
928aaf6f46c9b90346f45953268260d868fd52b49c0d58d9ce16ac897aba280c |
linux-mshv |
a66fc9b6ba1579a5e9935af2d27e3b47b25a5f0f200315f0a4f2ff34bbfa327d |
windows-whp |
2690b089f07ceab05b0ed221db65be3132262eef34349d9630f68388fc5c9a53 |
Then dispatch CI on dev, or wait for the next nightly, so that Package nvx crate and Publish nvx crate publish 0.1.0-alpha.9, and a passing nightly closes #1134.
Activity
nvx-rs: bundle a release that declares time ABI 2