Skip to content

ci: check that every pull request references an issue - #1679

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/issue-1676-issue-link-check
Nov 17, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/issue-1676-issue-link-check

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Part of #1676

What does this pull request change?

Before this change, nothing checks that a pull request references an issue.
With it, the Issue link workflow checks each pull request when it is opened,
reopened, pushed, edited, or labelled, and a pull request that references no
issue gets the needs-issue label, a comment that says how to reference one,
and a failed check. Start with .github/workflows/issue-link.yml, then
scripts/check_issue_link.py: main, evaluate, and publish decide and
write, and GitHub.render and referenced_numbers read the description.

ci: check that every pull request references an issue adds the workflow
and its stdlib-only script, which reads the pull request through gh api, in
UTF-8 whatever the locale:

  • Linked. GitHub links the pull request to an issue of this repository
    (GraphQL closingIssuesReferences, which includes the Development sidebar),
    or its description holds close, fix, or resolve (in any form), Part of, or
    Refs, in any case and with an optional colon, then #N,
    microsoft/nvx#N, or the issue's URL, where N is an issue, open or
    closed. A pull request's number, a missing number, a number of more than 10
    digits, a bare #N, and another repository's issue do not count. The
    script reads the description as GitHub renders it, through the Markdown API
    (POST /markdown, mode: gfm, in the repository's context), and takes the
    text of the HTML with Python's html.parser: only the text that GitHub
    shows counts, not an HTML comment, an attribute such as a link's title or
    an image's alt text, or code, inline, fenced, indented, or in <pre>, nor a
    keyword that code or the end of a block, such as a paragraph, a list item, a
    heading, or a table cell, separates from its number. A link counts by where
    it goes, and of its text only a keyword that starts it: a link to an issue
    or a pull request on GitHub, such as the link that GitHub makes of #N, as
    that keyword and OWNER/REPO#N, so
    [Part of #12](https://github.com/microsoft/nvx/issues/12) counts and
    [#12](https://github.com/microsoft/openvmm/issues/12) does not, and any
    other link as code does. Such a link's scheme and host may take any case or
    be left out, as in /microsoft/nvx/issues/12, but nothing may follow the
    number other than a slash, a query, or a fragment, so no .. takes it
    elsewhere, and the text after the link does not extend the number. An
    issue's URL that GitHub shows as text follows the same rule, but for the
    punctuation that ends a sentence. Elements that GitHub
    renders within a line, such as emphasis, links, images, pictures, and ruby,
    separate nothing; any other element does, so an element that GitHub adds
    later can only make the check flag a pull request, never pass one.
  • Bounded lookups. The script looks up at most the first 10 numbers of a
    description, so a description cannot make it call the API without bound.
    Past that bound it flags the pull request, and its comment says why, rather
    than failing with an API error, which would change nothing.
  • Exempt. Pull requests by dependabot[bot] or github-actions[bot], or
    labelled no-issue. Closed pull requests are left alone.
  • Outcomes. A linked or exempt pull request loses needs-issue, has the
    marker comment rewritten if it has one, and passes. An unlinked one gets
    needs-issue, creating needs-issue and no-issue when either is missing,
    and one marker comment that says how to reference an issue, and fails, so
    editing the description reruns the check and clears it without a push. An
    API error, or a label that GitHub refuses to create, fails it with exit
    status 2. Only a comment of github-actions[bot] that starts with
    <!-- nvx-issue-link --> is the script's, so a comment that quotes the
    marker is never edited, and the script skips other authors' comments,
    including a deleted account's, which have no author.
  • Writes. The script does every read before its first write, so an error
    while reading changes nothing; its writes are idempotent, so the next run
    completes a partial write. Its job summary names the verdict and the numbers,
    never the description.
  • Security. pull_request_target runs the default branch's workflow and
    script, and the job checks out the default branch without persisted
    credentials, so no code of the pull request runs. Rendering the description
    through the Markdown API needs no permission beyond contents: read. The workflow sets
    permissions: {}, and the job gets contents: read and
    pull-requests: write, so it cannot write issues; only the pull request's
    number reaches the shell, through env. The job has no condition, since a
    skipped run would replace the previous verdict, and its concurrency group
    runs the checks of one pull request one at a time, since canceling a run
    would not cancel a write that GitHub had already received; a newer waiting
    run replaces an older one.
  • validate-nvx runs scripts/test_issue_link.py on Linux and Windows.
  • doc/ci.md gets an
    Issue link
    section after "Test changes", and its nightly report paragraph names it. The
    "Issue references" row of doc/contribute.md says that every pull request
    needs a reference and names the workflow, needs-issue, and no-issue, and
    doc/project-structure.md lists the workflow and the script.

skills: label pull requests of OpenVMM releases no-issue tells section 6
of nvx-rebase-openvmm-chain to label a release's NVX pull request
no-issue, since a release rebase completes no issue; the pin of an OpenVMM
fix keeps referencing the fix's issue.

Left out. As the issue says, this does not close unlinked pull requests,
make the check required, or change ci.yml, Required status check, or the
merge queue. The check counts what GitHub shows, so a reference that GitHub
shows but does not link, such as one in raw HTML that GitHub's autolinker
skips, still counts, and the lookup decides whether it names an issue. A link
elsewhere counts as code does, so in [Fixes](https://example.com) #12 the
link separates the keyword from its number, and so does a link relative to the
page, such as ../issues/12, since where it goes depends on the page. The
script makes one more API call for each run, to render the description.

Remaining from #1676.

  • Task 7, after merge. pull_request_target runs the default branch's
    workflow, so this pull request is not checked by it. Verify with throwaway
    pull requests, closed afterwards: (a) no reference, then an edit to
    Part of #N; (b) Fixes #N against a base other than dev; (c) one from a
    fork, if one is at hand; and a draft. That run also confirms that
    GITHUB_TOKEN reads closingIssuesReferences and the issues API and
    renders Markdown, and that gh api follows a transferred issue's redirect,
    which the dry runs below could check only with a user token.
  • The CodeQL Actions criterion waits for ci: enable CodeQL code scanning for Actions, Python, and Rust #1655.

Why is this change needed?

doc/contribute.md asks every pull request to reference an issue on the first
line of its description, but nothing checks it, so only a reviewer can notice
a pull request that does not. #1675 references issues only inside the
template's HTML comment, and #1516's Refs #1509 names a pull request, so
neither references an issue.

Alternatives rejected:

  • A regular expression over the raw description, which would count the
    references in the template's comment and in code, as in ci: add basic codeql workflow file #1675.
  • A parser of the description written here, which the first five
    revisions of this pull request used: it read CommonMark's blocks, GFM, and
    raw HTML line by line, and each of Copilot's five reviews found cases where
    it read the description otherwise than GitHub does, such as attributes that
    hold <!--, link titles, and GFM tables, and CodeQL flagged its HTML
    regular expressions (py/bad-tag-filter). GitHub's own renderer agrees with
    the pull request's page by construction, costs one API call of about a
    second for a 64 KiB description, and leaves only HTML to read, which
    Python's standard library parses.
  • GitHub's links alone (closingIssuesReferences): Part of and Refs,
    which the template asks for when a pull request completes only part of an
    issue, link nothing.
  • A pull_request trigger, whose token cannot label or comment on a pull
    request from a fork. pull_request_target can, and the job runs only the
    default branch's code.
  • Filtering the marker comments by author in the gh api jq query, as
    Copilot suggested: filtering in Python lets the tests' fake gh api return
    the API's author: null as it is.

How was this change validated?

Checks and results. On Windows (Python 3.14.10), on this branch, rebased
onto e5330ad60af, with Git's usr\bin on PATH and XDG_CACHE_HOME
unset:

Command Result
python scripts/local_gates.py --base origin/dev --run, at ee871eb7f5f, before the review fixes all 5 gates passed; nvx-cli ran 2283 tests, OK (skipped=91), and 84, OK (skipped=11)
python scripts/local_gates.py --base origin/dev --only test-changes markdown-links test-layout quality-code --run --keep-going, at the head all 4 passed
python -m unittest scripts/test_issue_link.py, at the head; nvx-cli's only module that the change since that run affects 40 tests OK; against the parser-based script of c6f3211129f, the renderer's tests give 87 failures and 1 error, the cases added for the local review fail without its fixes, six cases of test_a_link_counts_by_where_it_goes fail with the script of 953c98f11c4, ten cases of the two link tests added for the next review fail with the script of 56dfa154e00, and eleven cases of the two tests added since fail with the script of d7ac2b20dc3
ruff format --check, ruff check, and python -m pyright on scripts/check_issue_link.py and scripts/test_issue_link.py, at the head passed; 0 errors
git -c sequence.editor=: rebase -i --autosquash --exec "python -m unittest scripts/test_issue_link.py" e5330ad60af, after each round of fixes passed at each commit
python scripts/rebase_check.py verify, after the rebase onto origin/dev passed
python scripts/check_test_changes.py --base origin/dev 0 change(s) to existing tests; relocated unchanged: 0; added: 62

The gates are test-changes, markdown-links, test-layout, quality-code (Ruff,
Pyright on Linux and Windows, ShellCheck, and shfmt), and nvx-cli (the
validate-nvx unittest suites). The issue expected the validate-nvx wiring
to be a change to existing tests, but both of its changed lines only gain
scripts/test_issue_link.py, which check_test_changes.py counts as an
addition.

Dry runs of the final script against the live repository, with
python scripts/check_issue_link.py --repository microsoft/nvx --dry-run P:

Pull request Verdict Exit status Detail
#1682 linked 0 GitHub links it to #1474
#1679 linked 0 its description references #1676
#1675 unlinked 1 references only inside the template's comment
#1516 unlinked 1 Refs #1509 names a pull request
#1510 unlinked 1 Refs #1509 names a pull request
#1509 unlinked 1
#1687, #1686 closed 0 left alone; open at earlier dry runs, which found GitHub linking them to #1684 and #1683
#1681, #1678, #1672 closed 0 left alone

Reading each description confirms the verdicts. The tests' HTML follows what
gh api markdown returned for each case, in the repository's context: issue
links, <p dir="auto">, code, tables wrapped in
<markdown-accessiblity-table>, task lists, and alerts. On the 243 cases that
the earlier parser was checked on, rendered by GitHub, the final reading
agrees on every number with an independent harness that reads the HTML
without the link rules, except for the two links elsewhere,
[Fixes #12](https://example.com) and a raw <a href="x"> around
Refs #12, which count as code; the fixes for the review of d7ac2b20dc3
change no number on those cases. A 64 KiB description renders in
0.64 to 1.08 s, and the script reads the text of the largest rendering,
321 KB of table HTML, in at most 57 ms.

One verdict changes on purpose: Fixes <!-- #8 --> #12 now references #12,
since GitHub hides the comment and shows Fixes #12.

Copilot's review of ee871eb7f5f found that indented code counted, that a
fence's container did not bound it, that hidden text between a keyword and its
number joined them, and that lookups were unbounded. Its review of
9cc107c329e found that a canceled run's write could land after the next run
read, that a number too long for int() raised, and that any 422 from creating
a label passed. Its review of dfd2a9d76a3 found that a sibling list item did
not end the paragraph before it, that a comment's state did not outlast its
block, that a Setext underline did not end its paragraph, that <!-->,
<!--->, and --!> did not end a comment, and that a deleted account's
comment, which has no author, failed the check. Its review of 4324f4844b7
found that the script read HTML blocks other than comments as Markdown, so a
comment in a <div> across a blank line hid nothing and their backticks made
code, that a blank line did not end an empty list item, and that gh's input
and output used the locale's encoding; its fourth thread, that a lazy line
starting a numbered item from 2 continues a quote, does not match GitHub's
rendering, and its reply shows why. Its review of c6f3211129f found that an attribute holding <!--
opened a comment and that backticks in attributes made code, that a link's
title counted as text, and that GFM tables were read as paragraphs, and
CodeQL flagged the regular expression of HTML blocks (py/bad-tag-filter).
Rather than patch the parser a fifth time, the script now has GitHub render
the description, which answers all four; a local review of that change
found that ruby and picture, which GitHub keeps within a line, ended a
line, and that the render test also passed on the raw description, both
fixed. Its review of 953c98f11c4 found that a link whose text is #N
counted as this repository's issue wherever it went, now fixed: a link counts
by its destination. Its other thread asked that a comment between a keyword
and its number keep separating them; GitHub shows Refs <!-- #8 --> #1680 as
Refs #1680, with #1680 linked, so the check counts it, and the reply shows
the rendering. Its review of 56dfa154e00 found that a link counted only with
a lowercase, absolute href, that anything after the number, .. included,
still counted, and that a link to an issue dropped the keyword that starts its
text, all fixed and checked against live renders. Its review of
d7ac2b20dc3 found that an issue's URL shown as text still counted with ..
after its number, now held to a link's path rule, though GitHub links such a
URL as #N and the check counts it by that link, and that the text after a
link to an issue extended its number, so [#1676](.../pull/1679)6 read as
16796 and #1676-related as nothing, now fixed with a boundary after the link.
The fixes are folded into the first commit, and all twenty-seven threads are
resolved.

Backend, host OS, host type, and guest. None applies: the change is CI
tooling and touches no runtime code.

Left to CI. The Ubuntu runs of check-quality and of the nvx-cli suites.
This is a draft, so CI skips its lint, build, and test jobs until it is ready.
The Issue link workflow itself runs only once merged, as task 7 above says.

Checklist

  • Tests cover the new or fixed behavior, in the layout that
    python3 scripts/check_test_layout.py checks.
  • python3 scripts/check_test_changes.py --base origin/dev lists no change
    to an existing test, or a Changes to existing tests section below
    explains each one.
  • The documentation that the change affects is updated, including
    .github/copilot-instructions.md and the skills when a convention or
    workflow changes.
  • A change to the public API of nvx updates nvx-rs/public-api/ with
    python3 scripts/public_api.py --update.
  • Snapshots, protocols, and on-disk formats stay compatible, or the
    description says how the change versions or migrates them.

Copilot AI balanced review requested due to automatic review settings November 17, 2026 02:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Markdown parsing can misidentify issue references around fenced, indented, and inline code, while unbounded API requests can exhaust timeouts or quotas.

4 open issues
What changed in this PR

Adds a standalone CI check that flags pull requests without an NVX issue reference, supporting the contribution policy without changing required checks or the merge queue.

Changes:

  • Adds a trusted default-branch workflow and script to validate references and manage labels and marker comments.
  • Adds parser, API, and workflow tests to Linux and Windows validation.
  • Documents the check and the exemption for OpenVMM release pull requests.
File Description
scripts/​test_issue_link.py Tests parsing, verdicts, API handling, and workflow safeguards.
scripts/​check_issue_link.py Validates issue links and updates labels, comments, and summaries.
doc/​project-structure.md Lists the new workflow and script.
doc/​contribute.md Explains issue-reference requirements and exemptions.
doc/​ci.md Documents triggers, outcomes, permissions, and trusted execution.
.github/​workflows/​issue-link.yml Runs the check on pull-request changes with limited permissions.
.github/​skills/​nvx-rebase-openvmm-chain/​SKILL.md Adds no-issue guidance for release pull requests.
.github/​actions/​validate-nvx/​action.yml Runs the new tests on Linux and Windows.

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from ee871eb to 9cc107c Compare November 17, 2026 02:49
Copilot AI balanced review requested due to automatic review settings November 17, 2026 02:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Canceled runs can leave stale labels or duplicate comments, while suppressed 422 errors can hide API failures and oversized issue numbers can crash the check.

3 open issues
4 resolved since last review

Review effort: Balanced

Comment thread .github/workflows/issue-link.yml Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from 9cc107c to dfd2a9d Compare November 17, 2026 03:06
Copilot AI balanced review requested due to automatic review settings November 17, 2026 03:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The issue-link check can count hidden references as valid and fail on unrelated comments from deleted accounts, allowing unlinked PRs through or blocking legitimate ones.

5 open issues
3 resolved since last review

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from dfd2a9d to 4fdd771 Compare November 17, 2026 03:34
Copilot AI balanced review requested due to automatic review settings November 17, 2026 03:34
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from 4fdd771 to 4324f48 Compare November 17, 2026 03:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Copilot AI balanced review requested due to automatic review settings November 17, 2026 03:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Incorrect Markdown container handling can count references inside code or comments, allowing PRs without valid issue links to pass.

4 open issues

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from 4324f48 to c6f3211 Compare November 17, 2026 04:12
Copilot AI balanced review requested due to automatic review settings November 17, 2026 04:12
Comment thread scripts/check_issue_link.py Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved parsing bugs in tables, HTML attributes, and link titles can misclassify issue references, allowing unlinked PRs through or incorrectly flagging them.

3 open issues
4 resolved since last review

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from c6f3211 to 953c98f Compare November 17, 2026 05:01
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py, reads
the pull request through gh api, in UTF-8 whatever the locale, and
passes it when GitHub links it to an issue of the repository
(closingIssuesReferences) or when its description holds close, fix,
resolve (in any form), Part of, or Refs, with an optional colon,
then #N, microsoft/nvx#N, or the issue's URL, where N is an issue. A
pull request's number, a missing number, a bare #N, and another
repository's issue do not count. The script reads the description as
GitHub renders it, through the Markdown API in the repository's context,
and only the text that GitHub shows counts: not an HTML comment, an
attribute such as a link's title, or code, inline, fenced, indented, or
in <pre>, nor a keyword that code or the end of a block, such as a list
item or a table cell, separates from its number. GitHub's own renderer
keeps the check in step with what the pull request's page shows, which a
parser of CommonMark, GFM tables, and raw HTML written here kept
missing. It looks up at most the first 10 numbers, so a description
cannot make it call the API without bound. Pull requests by
dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1687, #1686,
and #1682, which GitHub links to issues, and #1679, whose description
references #1676, and flag #1675, #1516, #1510, and #1509, as reading
their descriptions confirms. A 64 KiB description renders in about a
second, and its text reads in at most 60 ms.

test_issue_link.py covers the reading of GitHub's HTML, as gh api
markdown renders it: keywords and targets, other repositories, code and
<pre>, attributes, the ends of blocks and table cells, and inline
elements; the verdicts and the writes through a fake gh api, the render
request, gh's input and output in UTF-8, the bound on lookups, an API
error at each read, a comment that quotes the marker or has no author,
and the workflow's triggers, permissions, checkout, concurrency, and
inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 17, 2026 05:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Comment boundaries can disappear before scanning, and external issue links can be mistaken for local references, allowing invalid references to pass.

2 open issues
3 resolved since last review

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from 953c98f to 56dfa15 Compare November 17, 2026 05:18
Copilot AI balanced review requested due to automatic review settings November 17, 2026 05:18
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py, reads
the pull request through gh api, in UTF-8 whatever the locale, and
passes it when GitHub links it to an issue of the repository
(closingIssuesReferences) or when its description holds close, fix,
resolve (in any form), Part of, or Refs, with an optional colon,
then #N, microsoft/nvx#N, or the issue's URL, where N is an issue. A
pull request's number, a missing number, a bare #N, and another
repository's issue do not count. The script reads the description as
GitHub renders it, through the Markdown API in the repository's context,
and only the text that GitHub shows counts: not an HTML comment, an
attribute such as a link's title, or code, inline, fenced, indented, or
in <pre>, nor a keyword that code or the end of a block, such as a list
item or a table cell, separates from its number. A link counts by where
it goes, whatever its text: a link to an issue or a pull request as
OWNER/REPO#N, and any other link as code does. GitHub's own renderer
keeps the check in step with what the pull request's page shows, which a
parser of CommonMark, GFM tables, and raw HTML written here kept
missing. It looks up at most the first 10 numbers, so a description
cannot make it call the API without bound. Pull requests by
dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1687, #1686,
and #1682, which GitHub links to issues, and #1679, whose description
references #1676, and flag #1675, #1516, #1510, and #1509, as reading
their descriptions confirms. A 64 KiB description renders in about a
second, and its text reads in at most 60 ms.

test_issue_link.py covers the reading of GitHub's HTML, as gh api
markdown renders it: keywords and targets, other repositories, code and
<pre>, attributes, links, the ends of blocks and table cells, and inline
elements; the verdicts and the writes through a fake gh api, the render
request, gh's input and output in UTF-8, the bound on lookups, an API
error at each read, a comment that quotes the marker or has no author,
and the workflow's triggers, permissions, checkout, concurrency, and
inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Issue-link validation can accept links that resolve to a different issue and reject valid references in link text or differently formatted URLs.

3 open issues
2 resolved since last review

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from 56dfa15 to d7ac2b2 Compare November 17, 2026 05:39
Copilot AI balanced review requested due to automatic review settings November 17, 2026 05:39
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py, reads
the pull request through gh api, in UTF-8 whatever the locale, and
passes it when GitHub links it to an issue of the repository
(closingIssuesReferences) or when its description holds close, fix,
resolve (in any form), Part of, or Refs, with an optional colon,
then #N, microsoft/nvx#N, or the issue's URL, where N is an issue. A
pull request's number, a missing number, a bare #N, and another
repository's issue do not count. The script reads the description as
GitHub renders it, through the Markdown API in the repository's context,
and only the text that GitHub shows counts: not an HTML comment, an
attribute such as a link's title, or code, inline, fenced, indented, or
in <pre>, nor a keyword that code or the end of a block, such as a list
item or a table cell, separates from its number. A link counts by where
it goes, and of its text only a keyword that starts it: a link to an
issue or a pull request, absolute or root-relative, with nothing after
the number other than a slash, a query, or a fragment, as that keyword
and OWNER/REPO#N, and any other link as code does. GitHub's own renderer
keeps the check in step with what the pull request's page shows, which a
parser of CommonMark, GFM tables, and raw HTML written here kept
missing. It looks up at most the first 10 numbers, so a description
cannot make it call the API without bound. Pull requests by
dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1687, #1686,
and #1682, which GitHub links to issues, and #1679, whose description
references #1676, and flag #1675, #1516, #1510, and #1509, as reading
their descriptions confirms. A 64 KiB description renders in about a
second, and its text reads in at most 60 ms.

test_issue_link.py covers the reading of GitHub's HTML, as gh api
markdown renders it: keywords and targets, other repositories, code and
<pre>, attributes, links, the ends of blocks and table cells, and inline
elements; the verdicts and the writes through a fake gh api, the render
request, gh's input and output in UTF-8, the bound on lookups, an API
error at each read, a comment that quotes the marker or has no author,
and the workflow's triggers, permissions, checkout, concurrency, and
inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The link checker can accept raw URLs pointing outside NVX and merge linked issue numbers with following text, producing incorrect validation results.

2 open issues
3 resolved since last review

Review effort: Balanced

Comment thread scripts/check_issue_link.py Resolved
Comment thread scripts/check_issue_link.py Resolved
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-1676-issue-link-check branch from d7ac2b2 to be68c6d Compare November 17, 2026 06:14
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py, reads
the pull request through gh api, in UTF-8 whatever the locale, and
passes it when GitHub links it to an issue of the repository
(closingIssuesReferences) or when its description holds close, fix,
resolve (in any form), Part of, or Refs, with an optional colon,
then #N, microsoft/nvx#N, or the issue's URL, where N is an issue. A
pull request's number, a missing number, a bare #N, and another
repository's issue do not count. The script reads the description as
GitHub renders it, through the Markdown API in the repository's context,
and only the text that GitHub shows counts: not an HTML comment, an
attribute such as a link's title, or code, inline, fenced, indented, or
in <pre>, nor a keyword that code or the end of a block, such as a list
item or a table cell, separates from its number. A link counts by where
it goes, and of its text only a keyword that starts it: a link to an
issue or a pull request, absolute or root-relative, with nothing after
the number other than a slash, a query, or a fragment, as that keyword
and OWNER/REPO#N, which no text after the link extends, and any other
link as code does. An issue's URL that GitHub shows as text follows the
same rule, but for the punctuation that ends a sentence. GitHub's own
renderer keeps the check in step with what the pull request's page
shows, which a parser of CommonMark, GFM tables, and raw HTML written
here kept missing. It looks up at most the first 10 numbers, so a
description cannot make it call the API without bound. Pull requests by
dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1687, #1686,
and #1682, which GitHub links to issues, and #1679, whose description
references #1676, and flag #1675, #1516, #1510, and #1509, as reading
their descriptions confirms. A 64 KiB description renders in about a
second, and its text reads in at most 60 ms.

test_issue_link.py covers the reading of GitHub's HTML, as gh api
markdown renders it: keywords and targets, other repositories, code and
<pre>, attributes, links, the ends of blocks and table cells, and inline
elements; the verdicts and the writes through a fake gh api, the render
request, gh's input and output in UTF-8, the bound on lookups, an API
error at each read, a comment that quotes the marker or has no author,
and the workflow's triggers, permissions, checkout, concurrency, and
inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 17, 2026 06:14
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py, reads
the pull request through gh api, in UTF-8 whatever the locale, and
passes it when GitHub links it to an issue of the repository
(closingIssuesReferences) or when its description holds close, fix,
resolve (in any form), Part of, or Refs, with an optional colon,
then #N, microsoft/nvx#N, or the issue's URL, where N is an issue. A
pull request's number, a missing number, a bare #N, and another
repository's issue do not count. The script reads the description as
GitHub renders it, through the Markdown API in the repository's context,
and only the text that GitHub shows counts: not an HTML comment, an
attribute such as a link's title, or code, inline, fenced, indented, or
in <pre>, nor a keyword that code or the end of a block, such as a list
item or a table cell, separates from its number. A link counts by where
it goes, and of its text only a keyword that starts it: a link to an
issue or a pull request, absolute or root-relative, with nothing after
the number other than a slash, a query, or a fragment, as that keyword
and OWNER/REPO#N, which no text after the link extends, and any other
link as code does. An issue's URL that GitHub shows as text follows the
same rule, but for the punctuation that ends a sentence. GitHub's own
renderer keeps the check in step with what the pull request's page
shows, which a parser of CommonMark, GFM tables, and raw HTML written
here kept missing. It looks up at most the first 10 numbers, so a
description cannot make it call the API without bound. Pull requests by
dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1687, #1686,
and #1682, which GitHub links to issues, and #1679, whose description
references #1676, and flag #1675, #1516, #1510, and #1509, as reading
their descriptions confirms. A 64 KiB description renders in about a
second, and its text reads in at most 60 ms.

test_issue_link.py covers the reading of GitHub's HTML, as gh api
markdown renders it: keywords and targets, other repositories, code and
<pre>, attributes, links, the ends of blocks and table cells, and inline
elements; the verdicts and the writes through a fake gh api, the render
request, gh's input and output in UTF-8, the bound on lookups, an API
error at each read, a comment that quotes the marker or has no author,
and the workflow's triggers, permissions, checkout, concurrency, and
inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A microVM release rebase or reorganization completes no issue, so its
NVX pull request references none, and the Issue link check flags it.
Section 6 of nvx-rebase-openvmm-chain now tells the agent to label the
pull request no-issue, which exempts it, once the check has created the
label, and says that a pin that promotes an OpenVMM fix references the
fix's issue instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The privileged pull_request_target workflow cannot run from this PR, leaving token permissions, label and comment writes, and fork behavior unverified until after merge.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) marked this pull request as ready for review November 17, 2026 06:35
@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Nov 17, 2026
Merged via the queue into dev with commit 28f42dd Nov 17, 2026
89 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-1676-issue-link-check branch November 17, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants