Report validation results on pull requests - #7
Merged
Conversation
The IG Publisher reports validation results in qa.xml rather than through its exit code, so a build with errors finished green and nobody noticed unless they opened qa.html. Pull requests only ever got a link to the published preview. A validation step now renders qa.xml into inline annotations and a summary table, and fails the job when an error survives. Warnings are annotated but do not fail; the ones meant to stay belong in input/ignoreWarnings.txt, which the publisher applies first. That summary is posted to the pull request together with the deployment link, from the build job when the PR is already open. Repo-internal pull requests no longer build a second time. The branch's push event has already built that commit, so the pull_request run was pure duplication - forks and Dependabot keep building through their PR, since they have no push event to rely on. That leaves a gap: push the branch, open the PR a moment later, and the push run is done looking for a PR while the pull_request run is skipped. The pr-comment job closes it by replaying the finished run's result - it finds the push run for the head commit, downloads its summary artifact and posts the same comment. It deliberately stands back while a push build is still in flight, since that run will find the PR by itself. Both paths share .github/scripts/pr-comment.js so the marker they use to find and update their comment cannot drift apart and start posting duplicates. Ported from Gefyra/KDL-IG-Publisher, where this has been in use. All five action versions verified to exist; the workflow parses and refers only to files that are present.
Contributor
FHIR Validation Results (filter: warnings)❌ 0 errors (0 filtered),
Deployment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The IG Publisher reports validation results in
qa.xmlrather than through its exit code. A build with errors therefore finished green, and nobody noticed unless they openedqa.htmlthemselves. Pull requests only ever got a link to the published preview.This ports the setup already in use in KDL-IG-Publisher, in all three stages.
1. The build fails on validation errors
patrick-werner/validation-outcome-markdown-renderer@v1turnsqa.xmlinto inline annotations on the diff and a summary table, and fails the job when an error survives. Warnings are annotated but do not fail — the ones meant to stay belong ininput/ignoreWarnings.txt, which the publisher applies first.2. The summary is posted to the pull request
Together with the deployment link, from the build job. It runs even when validation failed — that is when the report matters most — but not when the job was cancelled.
3. …and also when the PR is opened after the build
Repo-internal pull requests no longer build a second time: the branch's push event already built that commit. Forks and Dependabot keep building through their PR, since they have no push event to rely on.
That creates a gap. Push the branch, open the PR a moment later, and nothing reports back — the
pull_requestrun is skipped, and the push run finished looking for a PR before one existed. Thepr-commentjob closes it: it finds the push run for the head commit, downloads itspr-summaryartifact and posts the same comment. It builds nothing.Details that matter in that job:
actions: readto read another run's artifact, which the workflow's default permissions do not include..github/scriptsonly, not the IG sources.Both paths share
.github/scripts/pr-comment.js, so the marker they use to find and update their comment cannot drift apart and start posting duplicates. It also truncates at 60000 characters, below GitHub's 65536 comment limit, and paginates when searching — the marker comment is the oldest one on the PR and would otherwise be missed on a busy one.Verification
build,pr-commentcheckout@v6,upload-artifact@v6,download-artifact@v6,github-script@v8,validation-outcome-markdown-renderer@v1.github/scripts/pr-comment.js,scripts/gh-pages.sh,input/ignoreWarnings.txtbuildBodyhelpernullwith neither input, composes marker + findings + deployment otherwiseNote
The behaviour of the three stages together can only really be seen on a live pull request. This one is it: if the comment below carries a findings table and a preview link, all three worked.