docs: capture how to test the AzDO release pipeline (previewRun + DryRun read-only)#18135
Draft
radical wants to merge 1 commit into
Draft
docs: capture how to test the AzDO release pipeline (previewRun + DryRun read-only)#18135radical wants to merge 1 commit into
radical wants to merge 1 commit into
Conversation
…ease-pipeline testing guidance The AzDO release pipeline (release-publish-nuget.yml) never compiles on a GitHub PR, so template errors and unsafe dry-run side effects pass every PR check and only surface at release time. Document how to test it and add a review rule for keeping DryRun read-only. - New .github/instructions/release-pipeline.instructions.md (auto-applied to release-publish-nuget.yml, publish-*.yml, eng/pipelines/scripts/**, and release-github-tasks.yml): every side-effecting step must be a no-op under DryRun=true, including the GitHub Tasks dispatch; and never write a literal template expression inside an inlined script block (the template engine evaluates it even in comments/strings -> "Unable to convert from Object to String"). - pr-testing/ci-infra-testing.md (Track B): new "Testing a release-pipeline change" section -- previewRun compile-check first, then a DryRun run pinned to a real def-1602 source build (branch pushed to the mirror); DryRun read-only audit table + the GitHubTasks-not-gated-by-DryRun caveat; corrected the stale "can't run the release pipeline from a feature branch" claim. - azdo-internal/SKILL.md: release/unofficial pipelines added to Related Pipelines with their (stable) definition IDs; previewRun compile-check recipe; pin-the-source-build REST runs API recipe; api-version 7.1 quirk; refined the publish/release dry-run validation guidance. - AGENTS.md: register the new instructions file. Refs microsoft#18129 (release pipeline DryRun is not fully read-only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 18135Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 18135" |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds documentation and agent-skill/instructions content to capture how to validate changes to the AzDO release pipeline (release-publish-nuget.yml), which is never compiled during a GitHub PR — meaning template errors and DryRun gaps only surface at release time. The changes are purely documentation/tooling, with no product or pipeline code modifications.
Changes:
- New
.github/instructions/release-pipeline.instructions.mdwith review rules for DryRun read-only enforcement and template-expression traps in inlined scripts. - Extended
pr-testingskill (ci-infra-testing.md) with a new "Testing a release-pipeline change" section coveringpreviewRun, DryRun audit, and the knownGitHubTasksgap (#18129). - Extended
azdo-internalskill (SKILL.md) withpreviewRuncompile-check recipe, RESTrunsAPI for pinning a source build, hardcoded definition IDs (1600/1601/1602), and updated "Validating publish/release-only changes safely" section.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
AGENTS.md |
Registers new release-pipeline.instructions.md in the pattern-based instructions table |
.github/instructions/release-pipeline.instructions.md |
New instructions file with DryRun read-only rules and template-expression trap warnings |
.agents/skills/pr-testing/ci-infra-testing.md |
Adds release-pipeline testing section (previewRun, DryRun audit table, known gap caveat) and corrects stale "can't run from feature branch" claim |
.agents/skills/azdo-internal/SKILL.md |
Adds release/unofficial pipelines to Related Pipelines table, previewRun recipe, REST runs API recipe for pinning resources, updates validation guidance |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was missing
The AzDO release pipeline (
eng/pipelines/release-publish-nuget.yml, defmicrosoft-aspire-Release-To-NuGet) is never compiled on a GitHub PR — AzDO only compiles it at run time, and no AzDO pipeline runs on a GitHub PR. So two whole classes of defect pass every PR check (and the offlineInfrastructure.Tests) and only surface at release time:Both were hit while testing a recent release-pipeline change. A literal template expression inside an inlined PowerShell script block comment compiled the parameters object to a string and broke the pipeline:
And a
DryRun=truerun still dispatched a realrelease-github-tasks.ymlActions run, because that stage is gated onSkipGitHubTasks, notDryRun(tracked separately by #18129).None of our skills or review instructions captured how to catch either before merge.
The change (docs / agent-tooling only)
New
.github/instructions/release-pipeline.instructions.md(auto-applied to the release pipeline YAML,publish-*.yml,eng/pipelines/scripts/**, andrelease-github-tasks.yml): a review rule that every side-effecting step must be a no-op underDryRun=true— including the GitHub Tasks dispatch — plus the literal-template-expression-in-inlined-script trap.pr-testingskillci-infra-testing.md(Track B): new "Testing a release-pipeline change" section —previewRuncompile-check first, then aDryRun=truerun pinned to a real def-1602 source build; a DryRun read-only audit table; theGitHubTasks-not-gated-by-DryRuncaveat; and a fix for a stale claim that the release pipeline "can't be run from a feature branch" (it can, under DryRun).azdo-internalskillSKILL.md: release + unofficial pipelines added to Related Pipelines with their (stable) definition IDs; apreviewRuncompile-without-queuing recipe; the RESTrunsAPI recipe for pinning a consumed source build (az pipelines runcan't pin a resource version); and the--api-version 7.1quirk.AGENTS.md: register the new instructions file.Surprises / call-outs
.agents/skills/onmain(they were relocated from.github/skills/); these edits target themainlayout.Refs #18129