Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/actions/validate-nvx/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ runs:
set -euo pipefail
python3 -m pip install --requirement scripts/setup/azure/requirements.txt
python3 -m compileall -q scripts
python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_kconfig.py scripts/test_kernel_config.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_mutants.py scripts/test_nvx_coverage.py scripts/test_nvx_scenario_coverage.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py scripts/test_test_layout.py scripts/test_test_changes.py scripts/test_public_api.py scripts/test_scripts_coverage.py scripts/test_openvmm_git.py scripts/test_openvmm_pin.py scripts/test_openvmm_release.py scripts/test_local_gates.py scripts/test_pr_threads.py scripts/test_ci_triage.py scripts/test_issue_workspace.py scripts/test_host_worktree.py scripts/test_rebase_check.py scripts/test_nvx_hosts.py scripts/test_host_probe.py scripts/test_benchmark_series.py scripts/test_label_issues.py -v
python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_kconfig.py scripts/test_kernel_config.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_mutants.py scripts/test_nvx_coverage.py scripts/test_nvx_scenario_coverage.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py scripts/test_test_layout.py scripts/test_test_changes.py scripts/test_public_api.py scripts/test_scripts_coverage.py scripts/test_openvmm_git.py scripts/test_openvmm_pin.py scripts/test_openvmm_release.py scripts/test_local_gates.py scripts/test_pr_threads.py scripts/test_ci_triage.py scripts/test_issue_workspace.py scripts/test_host_worktree.py scripts/test_rebase_check.py scripts/test_nvx_hosts.py scripts/test_host_probe.py scripts/test_benchmark_series.py scripts/test_label_issues.py scripts/test_issue_link.py -v
python3 scripts/test_adversarial.py -v
python3 scripts/nvx.py --help
python3 scripts/nvx.py test-openvmm-unit --help
Expand All @@ -30,7 +30,7 @@ runs:
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
python -m compileall -q scripts
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_kconfig.py scripts/test_kernel_config.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_mutants.py scripts/test_nvx_coverage.py scripts/test_nvx_scenario_coverage.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py scripts/test_test_layout.py scripts/test_test_changes.py scripts/test_public_api.py scripts/test_scripts_coverage.py scripts/test_openvmm_git.py scripts/test_openvmm_pin.py scripts/test_openvmm_release.py scripts/test_local_gates.py scripts/test_pr_threads.py scripts/test_ci_triage.py scripts/test_issue_workspace.py scripts/test_host_worktree.py scripts/test_rebase_check.py scripts/test_nvx_hosts.py scripts/test_host_probe.py scripts/test_benchmark_series.py scripts/test_label_issues.py -v
python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_kconfig.py scripts/test_kernel_config.py scripts/test_guest_binaries.py scripts/test_microvm_tests.py scripts/test_nightly_ci.py scripts/test_development_release.py scripts/test_crate_release.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py scripts/test_markdown_links.py scripts/test_module_diagrams.py scripts/test_nvx_mutants.py scripts/test_nvx_coverage.py scripts/test_nvx_scenario_coverage.py scripts/test_setup_azure.py scripts/test_setup_azure_linux.py scripts/test_setup_azure_windows.py scripts/test_powershell_testing.py scripts/test_test_layout.py scripts/test_test_changes.py scripts/test_public_api.py scripts/test_scripts_coverage.py scripts/test_openvmm_git.py scripts/test_openvmm_pin.py scripts/test_openvmm_release.py scripts/test_local_gates.py scripts/test_pr_threads.py scripts/test_ci_triage.py scripts/test_issue_workspace.py scripts/test_host_worktree.py scripts/test_rebase_check.py scripts/test_nvx_hosts.py scripts/test_host_probe.py scripts/test_benchmark_series.py scripts/test_label_issues.py scripts/test_issue_link.py -v
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
python scripts\test_adversarial.py -v
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
Expand Down
7 changes: 7 additions & 0 deletions .github/skills/nvx-rebase-openvmm-chain/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,13 @@ absence of an OpenVMM pull request, and the signatures), what changed in
OpenVMM and in NVX, the equivalence evidence, and the landing order of
section 7. Ask whether it is Draft or Ready.

A release completes no issue, so the
[Issue link](../../../doc/ci.md#issue-link) check flags its pull request and
creates the `no-issue` label when the repository lacks it. Then label the pull
request `no-issue`, which exempts it, with
`gh pr edit P --repo microsoft/nvx --add-label no-issue`. A pin that promotes
an OpenVMM fix is not a release: it references the fix's issue instead.

## 7. Land The Release

Once OpenVMM CI passed NVX's required jobs on the new head, and only when the
Expand Down
64 changes: 64 additions & 0 deletions .github/workflows/issue-link.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Issue link

# Checks that a pull request references an issue of the repository: GitHub
# links it to one, or its description holds Fixes, Closes, Part of, or Refs
# and the issue (doc/ci.md#issue-link). An unlinked pull request gets the
# needs-issue label and one comment that says how to fix it, and the check
# fails; a linked or exempt one loses them. pull_request_target runs the
# workflow as the default branch holds it, with a token that may label and
# comment, for pull requests from forks too. So the job runs the default
# branch's script, which reads the pull request through the API and never
# runs its code.
on:
pull_request_target:
types:
- opened
- reopened
- synchronize
# An edit of the description, or of the base, changes the verdict
# without a push.
- edited
- labeled
- unlabeled

permissions: {}

jobs:
check:
name: Check that the pull request references an issue
# No condition: a run whose job a condition skips would replace the result
# of the run before it, so every event checks the pull request again.
runs-on: ubuntu-latest
timeout-minutes: 5
# Runs of one pull request go one at a time, so that each reads what the
# run before it wrote: canceling a run would not cancel a write that
# GitHub had already received. A newer waiting run replaces an older one,
# so the newest event's run comes last.
concurrency:
group: issue-link-${{ github.event.pull_request.number }}
cancel-in-progress: false
# Labelling and commenting on a pull request need pull-requests: write.
# Reading an issue of a public repository needs no issues permission.
permissions:
contents: read
pull-requests: write
steps:
# The default branch's script runs, whatever the pull request's base,
# so that no other branch's script runs with the token.
- name: Checkout
uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
with:
ref: ${{ github.event.repository.default_branch }}
persist-credentials: false

# Only the pull request's number reaches the job; the script reads the
# rest through the API and never prints the description.
- name: Check that the pull request references an issue
env:
GH_TOKEN: ${{ github.token }}
NUMBER: ${{ github.event.pull_request.number }}
run: >-
python3 scripts/check_issue_link.py
--repository "${GITHUB_REPOSITORY}"
--summary "${GITHUB_STEP_SUMMARY}"
"${NUMBER}"
76 changes: 74 additions & 2 deletions doc/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -2055,8 +2055,9 @@ started it.
The job checks out the default branch, never the run's code. Its token can
read the repository, the run's jobs and logs, and their check annotations,
and write issues, and no other workflow can write issues; the
[Test changes](#test-changes) workflow labels and comments on pull requests
with a token that writes pull requests instead. The job runs one
[Test changes](#test-changes) and [Issue link](#issue-link) workflows label
and comment on pull requests with tokens that write pull requests instead. The
job runs one
report at a time; a newer reporter can replace one that waits for its turn,
but it reports the latest nightly result anyway. Its job summary repeats
the report.
Expand Down Expand Up @@ -2215,6 +2216,77 @@ The `quality` job checks the [test layout](contribute.md#test-layout) with
where tests live, which test hooks a production file may hold, and which the
flagging script shares to read Rust.

## Issue link

The [`issue-link.yml`](../.github/workflows/issue-link.yml) workflow checks
that a pull request references an issue of this repository, as
[Commits and pull requests](contribute.md#commits-and-pull-requests) asks. A
pull request is linked when GitHub links it to an issue of the repository, as
a closing keyword in the description of a pull request to the default branch,
or the Development sidebar, does, or when its description holds a keyword and
an issue: `close`, `closes`, `closed`, `fix`, `fixes`, `fixed`, `resolve`,
`resolves`, `resolved`, `Part of`, or `Refs`, in any case and with an
optional colon, then `#N`, `microsoft/nvx#N`, or the issue's URL, where `N` is
an issue, open or closed. The description counts whatever the base, so a
stacked pull request, to which a closing keyword links no issue, counts too,
and so do `Part of` and `Refs`, which never link one. A pull request's number,
a missing number, a bare `#N`, and another repository's issue do not count.
The script reads the description as GitHub renders it on the pull request's
page, through GitHub's
[Markdown API](https://docs.github.com/en/rest/markdown/markdown#render-a-markdown-document),
and only the text that GitHub shows counts: not a reference in an HTML
comment, such as the template's, in an attribute, such as a link's title, or
in code, inline, fenced, indented, or in `<pre>`, nor a keyword that code or
the end of a block, such as a paragraph, a list item, a heading, or a table
cell, separates from its number. A link counts by where it goes, and of its
text only a keyword that starts it: a link to an issue or a pull request on
GitHub, such as the link that GitHub makes of `#N`, as that keyword and
`OWNER/REPO#N`, so `[Part of #N]` linked to the issue counts and `[#N]`
linked to another repository's issue does not, and any other link as code
does. Such a link is `https://github.com/OWNER/REPO/issues/N` or
`.../pull/N`, whose scheme and host may take any case or be left out; a link
relative to the page counts as code does. In such a link, as in an issue's URL
that GitHub shows as text, nothing may follow `N` but a slash, a query, or a
fragment, and in text the punctuation that ends a sentence, so that no `..`
takes it elsewhere, and the text after a link does not extend `N`. It looks up
only the first 10 numbers that the description references, so a description
cannot make it call the API without bound, which is one more reason to
reference the issue on the first line.

It runs when a pull request to any branch, drafts included, is opened,
reopened, pushed to, edited, labelled, or unlabelled. The runs of one pull
request go one at a time, so that each reads what the run before it wrote,
since canceling a run would not cancel a write that GitHub had already
received; a newer waiting run replaces an older one, so the newest event's run
comes last. Its job has no condition, since a run whose job
a condition skipped would replace the result of the run before it. An
unlinked pull request gets the `needs-issue` label and one comment, which a
hidden marker at its start identifies, that says how to reference an issue,
and the check fails. Editing the description runs it again: a linked pull
request loses the label, and the comment, if there is one, says that it
passes. Pull requests by `dependabot[bot]` or `github-actions[bot]`, and those
labelled `no-issue`, which a maintainer applies to a pull request that needs
no issue, such as a release pin of the OpenVMM chain, are exempt; closed and
merged ones are left alone. The workflow creates either label when the
repository lacks it. `Required status check` does not wait for the workflow. A
GitHub API error fails the check. The script reads everything before its first
change, so an error while reading changes nothing, and the next run completes
the changes of a run that fails while writing. The job summary gives the
verdict and the numbers that the description references, never the
description.

As with [Test changes](#test-changes), `pull_request_target` gives the job a
token that can label and comment, for pull requests from forks too, so the
job runs the workflow and [`check_issue_link.py`](../scripts/check_issue_link.py)
of the default branch, whose checkout keeps no credentials, and never a pull
request's code. Only the pull request's number reaches the job; the script
reads the pull request, has GitHub render its description, and reads the
issues that GitHub links to it, the issues that its description names, and
its comments through the API. The token can read the
repository's contents and write pull requests, but not issues: reading an
issue of a public repository needs no permission. A change to the workflow or
to the script takes effect once it merges into `dev`.

## crates.io releases

A scheduled or manually dispatched full run of `dev` publishes the
Expand Down
2 changes: 1 addition & 1 deletion doc/contribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ with the deploy key and on the hypervisor runners before it merges; see
| Commit body | What changed and why, when the subject does not say enough. |
| Pull request title | The commit subject of a single-commit pull request; otherwise a summary in the same form. |
| Pull request body | The sections of the [pull request template](../.github/pull_request_template.md): **What does this pull request change?**, **Why is this change needed?**, **How was this change validated?**, with the checks that you ran and the gates that you leave to CI, such as backends that you could not test, and **Checklist**. Add **Changes to existing tests** when `check_test_changes.py` lists any, and **OpenVMM pin** when the `openvmm` gitlink moves. |
| Issue references | `Closes #N` or `Fixes #N` first when the pull request completes the issue; `Part of #N` or `Refs #N` otherwise. |
| Issue references | Every pull request references an issue of this repository on the first line of its description: `Closes #N` or `Fixes #N` when it completes the issue, and `Part of #N` or `Refs #N` otherwise. When no issue covers the change, open one first. The [Issue link](ci.md#issue-link) workflow labels a pull request that references none `needs-issue`; a maintainer labels one that needs none, such as a release pin of the OpenVMM chain, `no-issue`. |
| Review threads | Reply with the commit that addresses the comment, or with the evidence that no change is needed, then resolve the thread once the pull request's head holds that outcome. `scripts/pr_threads.py` lists the threads, and checks each reply and resolution. |

Leave the performance histories in `data/*.csv` to CI: successful `dev`
Expand Down
6 changes: 5 additions & 1 deletion doc/project-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ nvx/
| | |-- ci.yml Main build, test, benchmark, and release workflow
| | |-- copilot-code-review.yml Copilot code review environment
| | |-- copilot-setup-steps.yml Copilot cloud agent environment
| | |-- issue-link.yml Check that every pull request references an issue
| | |-- module-diagrams.yml Module diagrams of the nvx crate, drawn every night
| | |-- nightly-ci-report.yml Report of failed nightly CI runs
| | |-- nvx-coverage.yml Coverage of nvx's own tests, for dev and pull requests
Expand Down Expand Up @@ -182,6 +183,7 @@ nvx/
| | |-- setup-windows-whp.ps1 Windows/WHP development-host and Actions runner bootstrap
| | `-- tool-versions.conf Shared tool release and checksum pins
| |-- benchmark_series.py Benchmark series with CI's validation, for local and remote hosts
| |-- check_issue_link.py Checks that a pull request references an issue
| |-- check_markdown_links.py Markdown link, anchor, and doc/ path check run by CI
| |-- check_test_changes.py Flags the changes of a pull request to existing tests
| |-- check_test_layout.py Check that tests live apart from the code that they test
Expand Down Expand Up @@ -242,7 +244,7 @@ reusable workflows: `build-openvmm-binary.yml` builds OpenVMM for one target,
`run-platform.yml` benchmarks one platform. The full runs of `ci.yml` on `dev`
also publish the `nvx` crate to
crates.io. The other workflows report failed nightly CI runs, flag the pull
requests that change existing tests, draw the module
requests that change existing tests or reference no issue, draw the module
diagrams of the `nvx` crate every night, run adversarial campaigns, measure
the coverage of the `nvx` crate's tests and of the Python tooling's unit tests
as `dev` changes, measure every week which host code of the `nvx` crate the
Expand Down Expand Up @@ -393,6 +395,8 @@ hooks that `test-layout-allowlist.json` allows them, and
`check_test_changes.py` flags the pull requests that change existing tests; see
[Test layout](contribute.md#test-layout) and
[Changes to existing tests](contribute.md#changes-to-existing-tests).
`check_issue_link.py` flags the pull requests that reference no issue; see
[Issue link](ci.md#issue-link).
`public_api.py` checks the public API of `nvx` against its snapshots in
`nvx/public-api/`, and rewrites them; see [Rust crates](ci.md#rust-crates).

Expand Down
Loading
Loading