From def7d1adf82fa9488befb0627b1bffa1e4796908 Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Fri, 30 Oct 2026 13:06:49 -0700 Subject: [PATCH] doc: describe the roadmap commit of an OpenVMM pin promotion The nvx-openvmm-promote skill described a promotion as one commit that changes only the openvmm gitlink, and required exactly one commit on the target. Each of the twelve promotions from #801 to #1042 instead followed its gitlink commit with a doc: commit that maps the pinned commits in doc/openvmm-upstream-roadmap.md, as 6d3442d14 followed 70adf8918 in #1042. Nothing said which of the roadmap's sections a promotion updates; #918, for one, left PR18's node in the dependency graph at 11 commits while the PR summary gave 13. A new section of the skill makes the roadmap commit part of the procedure. It takes the previous pin from the target's gitlink, and stops unless the pinned revision descends from it with no merge since the fork base. It tells an OpenVMM pull request that branched from an older head of feature/microvm/dev, which needs a rebase but leaves the roadmap valid, from a rebase of the whole chain, which shifts every position and needs the roadmap remapped. It derives the pin's position, its distance from the mapped tip, and the new positions from Git rather than by counting, after unshallowing a submodule that the Copilot setup's --depth 1 left shallow, whose boundary would cut the counts short without an error. It lists what a promotion updates: the Scope's pin and chain of OpenVMM pull requests, whose previous proposal has usually merged since; the branch DAG, whose final step grows only when it is unassigned; and the table of the commits after the mapped tip, whose remarks it backs with commands that find the earlier commits that change the same files and, separately, check with git merge-tree whether the new commits apply to upstream main. Only a type-check of their packages on that merged tree lets the remarks say that they can go upstream on their own. A fixup of mapped work also updates its PR's row, graph node, and commit mapping. The section ends by checking that the branch holds exactly the two commits and that the second changes only the roadmap. The pull-request section rebuilds both commits when the target moves, and the report covers both. Vectors: since the skill described the openvmm_vectors comparison, the test has come to check three excerpts of OpenVMM's sources and four versions of nvx::contract against the submodule too, while the skill still had a failure copy "both files". It now gives each kind of failure its fix, reruns the comparison until it passes, since each test stops at its first mismatch and only the NVX CLI jobs run these ignored tests, and runs the check-nvx-rs checks, whose runtime adapter compiles the excerpts. Landing: .gitmodules tracks feature/microvm/dev, and microsoft/openvmm allows only squash merges, which would land a commit other than the one that NVX pins. OpenVMM pull requests land by fast-forwarding that branch to the pinned revision, as microsoft/openvmm#4818 and microsoft/openvmm#4820 did. The skill now says so, pushes only when the user asks, and has the NVX pull request state that landing order. doc/contribute.md and .github/copilot-instructions.md described a one-commit, gitlink-only promotion too; they now describe both commits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/copilot-instructions.md | 8 +- .github/skills/nvx-openvmm-promote/SKILL.md | 214 +++++++++++++++++--- doc/contribute.md | 15 +- 3 files changed, 201 insertions(+), 36 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 2c4b681e3..94c75504c 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -8,9 +8,11 @@ - `openvmm/` is a Git submodule that tracks the `feature/microvm/dev` branch of `microsoft/openvmm`. For OpenVMM source changes, work in the nested repository and follow [its instructions](../openvmm/.github/copilot-instructions.md). For - an NVX pin promotion, use the `nvx-openvmm-promote` skill and keep the - superproject change gitlink-only unless the task explicitly requires - coordinated NVX changes. + an NVX pin promotion, use the `nvx-openvmm-promote` skill: keep the gitlink + commit gitlink-only unless the task explicitly requires coordinated NVX + changes, and map the pinned commits in + [the OpenVMM upstream roadmap](../doc/openvmm-upstream-roadmap.md) in a + separate commit. - Preserve dirty state in both repositories. Do not reset, clean, switch, update, or synchronize the submodule over uncommitted work. diff --git a/.github/skills/nvx-openvmm-promote/SKILL.md b/.github/skills/nvx-openvmm-promote/SKILL.md index 01a29ea54..9c7d6b667 100644 --- a/.github/skills/nvx-openvmm-promote/SKILL.md +++ b/.github/skills/nvx-openvmm-promote/SKILL.md @@ -1,14 +1,15 @@ --- name: nvx-openvmm-promote -description: "Promote an OpenVMM pull request or revision into NVX as a clean gitlink update. Use for OpenVMM pin updates, clean replacement PRs, source-equivalence checks, signed integration commits, dependency validation, and exact-head NVX CI validation." +description: "Promote an OpenVMM pull request or revision into NVX as a clean gitlink update and map it in the OpenVMM upstream roadmap. Use for OpenVMM pin updates, clean replacement PRs, source-equivalence checks, signed integration commits, dependency validation, OpenVMM roadmap updates, and exact-head NVX CI validation." argument-hint: " [target=origin/dev] [Draft|Ready]" --- # Promote An OpenVMM Revision Move one reviewed OpenVMM revision into NVX without mixing the source repository's -history into the superproject. Preserve exact provenance and validate only claims -supported by the selected source and current NVX head. +history into the superproject, and map its commits in the OpenVMM upstream roadmap. +Preserve exact provenance and validate only claims supported by the selected source and +current NVX head. ## 1. Establish Both Repository States @@ -71,8 +72,10 @@ reviewed revision: ## 3. Prepare A Clean NVX Integration Branch Fetch the target branch narrowly and record its exact commit. A normal promotion is -one NVX commit directly on the current target and changes only the `openvmm` gitlink. -Do not merge OpenVMM commits or copy their history into NVX. +two NVX commits directly on the current target: one that changes only the `openvmm` +gitlink, then one that maps the pinned commits in the OpenVMM upstream roadmap, as +[the next section](#4-map-the-pin-in-the-openvmm-roadmap) describes. Do not merge +OpenVMM commits or copy their history into NVX. If replacing a stale integration branch, create a fresh branch or isolated worktree from the refreshed target rather than importing merge or rebase history. With both @@ -86,17 +89,31 @@ git diff --name-only git diff --check ``` -`nvx-rs`'s protocol tests read copies of OpenVMM's control-session protocol vectors. -Compare them with the revision's files, as the NVX CLI jobs do: +`nvx-rs` reads copies of what it needs from OpenVMM, because the `nvx-rs` CI job cannot +check out the submodule: the control-session protocol vectors, excerpts of OpenVMM's +sources, and, in `nvx::contract`, the versions of OpenVMM's snapshot format, saved-state +schema, time ABI, and restore packet. Compare them with the revision's, as the NVX CLI +jobs do: ```console cargo test --locked --manifest-path nvx-rs/Cargo.toml --no-default-features --test openvmm_vectors -- --ignored ``` -A failure means that the revision changes the `NVXS` vectors that NVX's host and guest -must follow. Copy both files unchanged into `nvx-rs/tests/vectors/openvmm/`, record the -revision in [the vectors' README](../../../nvx-rs/tests/vectors/README.md#openvmms-vectors), -and run the `nvx-rs` tests, which may need coordinated changes too. +Each test stops at its first failure, which names a copy that the revision changes: + +- A vector file differs: the revision changes the `NVXS` vectors that NVX's host and + guest must follow. Copy the file unchanged into `nvx-rs/tests/vectors/openvmm/`. +- An excerpt is no longer a run of whole lines of its OpenVMM file: copy the new lines + unchanged into it. +- A contract version differs: change the version in `nvx::contract`, and the code that + reads what it versions. + +After copying a vector file or an excerpt, record the revision in +[the vectors' README](../../../nvx-rs/tests/vectors/README.md#openvmms-vectors). Rerun the +comparison after each fix until it passes, since only the NVX CLI jobs run these ignored +tests. Then run the checks of [check-nvx-rs](../../actions/check-nvx-rs/action.yml), which +also build the runtime adapter that compiles the excerpts; they may need coordinated +changes too. `nvx-rs run` also reads a copy of the identities of OpenVMM's built-in CPU profiles, from which it guides a failed cold boot on a CPU that none serves. Compare it with the revision's @@ -108,19 +125,20 @@ cargo test --locked --manifest-path nvx-rs/Cargo.toml --no-default-features --te A failure means that the revision adds, revises, or removes a built-in profile. Update `nvx-rs/src/host/cli/cpu_generations.json` and the copy of `scripts/nvx_tools/time_abi.py`, -which `scripts/test_time_abi.py` compares with the same profiles, as coordinated changes. +which `scripts/test_time_abi.py` compares with the same profiles, as coordinated changes, +and rerun the comparison until it passes. -Before committing, require `git diff --name-only` to contain exactly `openvmm` -unless coordinated NVX changes were explicitly requested, or the revision needs them, -as a change to OpenVMM's protocol vectors or CPU profiles does. Stage only the gitlink and -those changes, and name the coordinated changes in the pull request. +Before committing the gitlink, require `git diff --name-only` to contain exactly +`openvmm` unless coordinated NVX changes were explicitly requested, or the revision needs +them, as a change to what `nvx-rs` copies from OpenVMM or to its CPU profiles does. Stage +only the gitlink and those changes, and name the coordinated changes in the pull request. Commit only when the user requested it. Follow the repository's subject convention, -sign the commit with the user's existing configuration, and stop rather than falling +sign each commit with the user's existing configuration, and stop rather than falling back to an unsigned commit if signing fails. Never amend, force-push, or rewrite an already published integration commit without explicit authorization. -After a requested commit, verify that it is the only commit directly on the target -and that the target is its sole parent, then verify its signature: +After a requested commit, verify that it is the only commit on the target so far and +that the target is its sole parent, then verify its signature: ```console git diff --name-only ..HEAD @@ -132,7 +150,130 @@ git log --show-signature --format=fuller -1 Require the count to be exactly `1` and the parent line to contain exactly the `HEAD` and `` object IDs. Reject a different parent or any additional parent. -## 4. Validate The Exact Pin +## 4. Map The Pin In The OpenVMM Roadmap + +The [OpenVMM upstream roadmap](../../../doc/openvmm-upstream-roadmap.md) numbers the +commits of the microVM chain by position, from 001 on the fork base, and records NVX's +pin. Record the promotion there in a second commit, directly on the gitlink commit, that +changes only the roadmap, with a subject such as +`doc: map the in the OpenVMM roadmap`. Its message names the new positions and +the new pin, says what the commits change, and gives the pin's new distance from the +mapped tip. + +Take the fork base and the mapped tip from the roadmap's Scope, and the previous pin from +`git rev-parse :openvmm`, which must match the Scope's pin. Derive every +number from Git rather than by counting, and from complete history: a shallow boundary, +such as the `--depth 1` initialization of the +[Copilot setup](../../workflows/copilot-setup-steps.yml) leaves, cuts counts and logs +short without an error. When `git -C openvmm rev-parse --is-shallow-repository` prints +`true`, run `git -C openvmm fetch --unshallow `, from a remote that resolves to +`microsoft/openvmm`, and require the check to print `false` before going on: + +```console +git -C openvmm merge-base --is-ancestor +git -C openvmm rev-list --count --merges .. +git -C openvmm rev-list --count .. +git -C openvmm rev-list --count .. +git -C openvmm log --reverse --abbrev=9 --format='%h %s' .. +``` + +Require the first command to succeed and the second to print `0`; otherwise stop and +ask. When the first fails, list the pinned commits that the revision lacks: + +```console +git -C openvmm log --oneline .. +``` + +If they are only the latest commits of the chain, the revision's pull request branched +from an older head of `feature/microvm/dev`: the roadmap stays valid, and the pull request +needs a rebase onto the pin and new OpenVMM CI before its promotion. If they are every +commit of the chain, the chain itself was rebased, as onto a new upstream base: positions +shift, and the whole roadmap needs remapping, as the `feature/microvm/release/v2.3` +rebase did. + +The third command prints the pin's position, the fourth its distance from the mapped tip, +and the fifth lists the new positions in order. Any commit's position is +`git -C openvmm rev-list --count ..`. + +Record the new commits as unassigned. An unassigned range grows only when the new +positions directly follow it; after a fixup step, start a new range. Update these parts, +following their wording: + +- **Scope**: the pin's full SHA and position; the chain back from it, naming the OpenVMM + pull request that proposes or merged each group of positions and the current head of + `feature/microvm/dev`; the pin's distance from the mapped tip; and the unassigned + ranges. The previous pin's pull request has usually merged since, so check each state + with `gh pr view --repo microsoft/openvmm --json state,mergeCommit`. +- **Actual branch DAG**: the final step, extended when it is unassigned and otherwise + followed by a new `unassigned` step, and the 9-character SHA of the `(NVX pin)` line. +- **Commits after the mapped tip**: the count and the ranges of each kind in its opening + paragraph, and one row per topic, in chain order. A row gives inclusive positions, the + 9-character SHAs of the first and last commit, the count, and what the commits change, + ending with the OpenVMM pull request. +- **The remarks after that table**: which earlier chain commits change the same files, + and, separately, whether the new commits apply to upstream main as they are, so they + can go upstream on their own. Sharing a file decides neither: only a commit that changes + what an earlier one added revisits that commit's PR. + +For the remarks, list the files that the new commits change and the earlier chain commits +that change them too, then check whether the new commits apply to upstream main, which the +Scope records: + +```console +git -C openvmm diff --name-only +git -C openvmm log --reverse --abbrev=9 --format='%h %s' .. -- ... +git -C openvmm merge-tree --write-tree --name-only --merge-base +``` + +The last command replays the new commits on upstream main without touching a working tree. +It exits with `1` when they conflict, listing the conflicting files, as for positions 366 +and 367, which change fork-only CPU checks, and with `0` when they apply cleanly, as +positions 368 and 369 do although positions 043 and 301 change other parts of their file. +A clean merge shows only that the commits apply, not that upstream has the code that they +use. Before the remarks say that they can go upstream on their own, type-check their +packages on the merged tree, whose ID is the first line of that output, in a temporary +worktree. Give it an absolute `` outside both checkouts, since `git -C` +resolves a relative one from another directory, and return to the NVX root before +removing it: + +```console +git -C openvmm worktree add --detach +git -C read-tree -u --reset +cd +cargo check -p +cd +git -C openvmm worktree remove --force +``` + +Without that check, say only that the commits apply to upstream main without conflicts. + +A new commit that fixes a commit that a roadmap PR maps is instead a fixup of that PR, to +fold into the commit that it fixes. End its row with `fold into `, give it its +own `PRnn fixup` step in the DAG, and also update the fixup positions that the Scope and +the commit mapping's introduction list, the fold sentences after the DAG, the PR's row in +the PR summary, its node's count in the dependency graph, and its list and count in the +commit mapping. + +Before committing, confirm that the Scope's pin is `git rev-parse HEAD:openvmm`, then run +`git diff --check` and the link check with the platform launcher, +`python3 scripts/check_markdown_links.py` on Linux or +`python scripts\check_markdown_links.py` on Windows. After a requested commit, +verify that the branch holds exactly the two commits, that the gitlink commit is the +roadmap commit's sole parent, that the roadmap commit changes only the roadmap, and its +signature: + +```console +git rev-list --count ..HEAD +git rev-list --parents --max-count=1 HEAD +git diff --name-only HEAD~1 HEAD +git log --show-signature --format=fuller -1 +``` + +Require the count to be exactly `2`, the parent line to contain exactly the `HEAD` and +gitlink-commit object IDs, and the file list to be exactly +`doc/openvmm-upstream-roadmap.md`. + +## 5. Validate The Exact Pin Run NVX verification with the platform launcher: @@ -155,11 +296,12 @@ Use `nvx-run` for a smoke test, `nvx-debug` for a failure, and `nvx-benchmark` o when performance evidence is required. Report unavailable KVM, MSHV, or WHP coverage as pending CI rather than substituting another backend. -## 5. Create And Validate The Pull Request +## 6. Create And Validate The Pull Request -Immediately before pushing, refresh the target and confirm the integration commit is -still directly based on it. If the target moved, rebuild the clean integration commit -on the new target and rerun validation; do not present stale evidence as current. +Immediately before pushing, refresh the target and confirm the gitlink commit is still +directly based on it. If the target moved, rebuild both commits on the new target, +deriving the roadmap's numbers again, and rerun validation; do not present stale +evidence as current. The pull-request body should identify: @@ -167,8 +309,11 @@ The pull-request body should identify: - Scope and explicit exclusions. - Required dependencies and their states. - Tree, patch, or behavioral equivalence evidence using the precise verified term. +- The roadmap positions of the pinned commits. - Local validation and unavailable platform gates. - The exact NVX head SHA submitted to CI. +- The landing order: the OpenVMM pull request lands, as described below, before the NVX + pull request merges. Ask for Draft or Ready state before creation. Ready pull requests start the complete [NVX matrix](../../workflows/ci.yml); changing an already active pull request to Draft @@ -176,9 +321,24 @@ does not cancel its current run. After each push, follow only the workflow run w head SHA matches the current pull-request head. Treat older runs as superseded and use `nvx-ci-investigation` for a failed or unexpectedly cancelled current-head run. +`.gitmodules` tracks `feature/microvm/dev`, so the pinned revision must be on that branch +before the NVX pull request merges. `microsoft/openvmm` allows only squash merges, which +would land a different commit, so land the OpenVMM pull request by fast-forwarding the +branch to the pinned revision instead, from a remote that resolves to `microsoft/openvmm`, +and only when the user asks: + +```console +git -C openvmm push :refs/heads/feature/microvm/dev +``` + +Never force it. If the branch has moved past the pull request's base, the pull request +needs a rebase, new OpenVMM CI, and a new promotion. + ## Report Report the source PR and revision, signature result, dependency state, equivalence -method and result, NVX target commit, final changed-file set, local validation, the -comparisons of OpenVMM's protocol vectors and CPU profiles, pull-request state and URL, -exact-head workflow URL, and every blocked or pending gate. +method and result, NVX target commit, both commits and their changed-file sets, the +roadmap positions and the pin's distance from the mapped tip, local validation, the +comparisons of what `nvx-rs` copies from OpenVMM and of its CPU profiles, pull-request +state and URL, exact-head workflow URL, the OpenVMM pull request's landing state, and +every blocked or pending gate. diff --git a/doc/contribute.md b/doc/contribute.md index 0976f3b4f..e85298f5b 100644 --- a/doc/contribute.md +++ b/doc/contribute.md @@ -191,21 +191,24 @@ into planned pull requests to `main` and marks those that stay fork-only. A pin promotion moves the `openvmm` gitlink to a reviewed OpenVMM revision: -- Make the promotion a one-commit topic branch based directly on the current - `dev`; that commit should change only the gitlink unless the revision needs - coordinated NVX changes, with a subject such as `openvmm: pin auto's fallback - to host CPU profiles`. Never merge OpenVMM history into NVX. +- Make the promotion a two-commit topic branch based directly on the current + `dev`. The first commit should change only the gitlink unless the revision + needs coordinated NVX changes, with a subject such as `openvmm: pin auto's + fallback to host CPU profiles`. The second maps the pinned commits in the + [OpenVMM upstream roadmap](openvmm-upstream-roadmap.md) and changes nothing + else, with a subject such as `doc: map the VM time fixes in the OpenVMM + roadmap`. Never merge OpenVMM history into NVX. - After committing, run `scripts/nvx.py verify`, which checks that the submodule checkout matches the recorded pin. - `nvx-rs` checks copies of OpenVMM's control-session protocol vectors, which the NVX CLI jobs compare with the submodule's files. When a revision changes them, copy them into `nvx-rs/tests/vectors/openvmm/` unchanged and record the - revision, in the same commit; see + revision, in the gitlink commit; see [OpenVMM's vectors](../nvx-rs/tests/vectors/README.md#openvmms-vectors). `nvx-rs run` also reads a copy of the identities of OpenVMM's built-in CPU profiles, `nvx-rs/src/host/cli/cpu_generations.json`, which the NVX CLI jobs compare with the pinned profiles; a revision that adds, revises, or removes - a profile updates it in the same commit. + a profile updates it in the gitlink commit. - In the pull request, name the OpenVMM pull request and the exact pinned SHA, the scope and any prerequisite revisions, and the validation you ran. - A gitlink change is an input of every CI suite, so its pull request runs