Use RegenPreview parallel regeneration in publish pipeline for manual runs - #11713
Conversation
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 1 pipeline(s). 1 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
commit: |
There was a problem hiding this comment.
Pull request overview
Shares SDK regeneration helpers and enables faster per-library regeneration for manual publish runs.
Changes:
- Extracts discovery, parallel regeneration, and reporting into
RegenPreview.psm1. - Adds manual-run parallel regeneration to the PR submission pipeline.
- Documents the shared workflow and preserves the existing automated path.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
Submit-AzureSdkForNetPr.ps1 |
Adds optional parallel regeneration and failure reporting. |
RegenPreview.psm1 |
Provides shared regeneration helpers. |
RegenPreview.ps1 |
Delegates existing behavior to shared helpers. |
docs/RegenPreview.md |
Documents helper APIs and pipeline usage. |
publish.yml |
Enables parallel regeneration for manual runs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (3)
packages/http-client-csharp/eng/scripts/Submit-AzureSdkForNetPr.ps1:624
- This derives the parallel scope from the requested switches instead of
$emitterPatterns, which only contains emitters that were prepared successfully. If the Azure or management generator setup fails in the caught blocks above, its switch remains true and this branch still regenerates those libraries with stale artifacts, contradicting the existing “continuing without ... regeneration” recovery. Restrict discovery directly to the successfully prepared emitter patterns.
$allLibraries = @(Get-SdkLibrariesToRegenerate -SdkRepoPath $tempDir)
$librariesToRegenerate = @(Filter-LibrariesByGenerator `
-Libraries $allLibraries `
-Azure:$RegenerateAzureLibraries `
packages/http-client-csharp/eng/scripts/Submit-AzureSdkForNetPr.ps1:68
- The PR description promises a
-RegenerationThrottleLimitparameter, but only the parallel switch is exposed here and the invocation always uses the helper's automatic 4–12 job limit. Manual pipeline callers therefore cannot apply the documented throttle control. Add the integer parameter and forward it toInvoke-SdkLibraryRegeneration -ThrottleLimit.
This issue also appears on line 621 of the same file.
[Parameter(Mandatory = $false)]
[switch]$UseParallelRegeneration
packages/http-client-csharp/eng/scripts/RegenPreview.psm1:1032
- The shared discovery API is missing the
-AzureLibrariesOnlyoption described by this PR. The removed local helper explicitly excludedMicrosoft.*and every name not starting withAzure., but all three replacement call sites inRegenPreview.ps1now use this unrestricted helper. Any matching non-Azure library is therefore newly included in local selection/regeneration, despite the stated behavior-preservation goal. Add the switch here and pass it from the local script's discovery calls to retain the previous contract.
[Parameter(Mandatory = $false)]
[string[]]$EmitterPackageJsonPaths
The publish pipeline's regen step regenerates SDK libraries one service directory at a time via
dotnet msbuild eng/service.proj, whileRegenPreview.ps1already regenerates libraries concurrently. This shares that mechanism withSubmit-AzureSdkForNetPr.ps1, gated to manual pipeline runs. Pipeline parameters and the set of files committed to the azure-sdk-for-net PR are unchanged.RegenPreview.psm1— new shared helpers (moved out ofRegenPreview.ps1)Get-SdkLibrariesToRegenerate— discovers libraries whosetsp-location.yamlreferences a TypeSpec C# emitter package json.-EmitterPackageJsonPathsrestricts to specific emitters;-AzureLibrariesOnlypreserves the local script'sAzure.*filter.Invoke-SdkLibraryRegeneration— installs tsp-client and builds the codegen plugin once, then runsdotnet build /t:GenerateCode /p:SkipTspClientInstall=true /p:SkipBuildPlugin=trueper library in parallel. Options:-ThrottleLimit,-NpmRegistry(temporary.envoverride),-AdditionalBuildArgs,-SerialServiceDirectories.Write-RegenerationReport— pass/fail summary, optional JSON report path.-SerialServiceDirectoriesruns those libraries (e.g.ai, whose libraries share a generator plugin output folder) in a serial batch after the parallel one, replacing the old/m:1 /p:BuildInParallel=falsehandling.RegenPreview.ps1Submit-AzureSdkForNetPr.ps1-UseParallelRegenerationswitch and-RegenerationThrottleLimit. When set, the discovered libraries are regenerated in parallel with/p:Trace=trueinstead of looping over service directories.Register-StepFailurepath, so manual runs still fail before opening a PR.publish.yml${{ replace(replace('True', ne(variables['Build.Reason'], 'Manual'), ''), 'True', '-UseParallelRegeneration') }}docs/RegenPreview.mdNote for reviewers
The parallel path regenerates only the libraries matching the emitter patterns, whereas the msbuild path regenerates every generatable project in a matched service directory — so the resulting PR could contain slightly fewer changes if a matched service directory also holds libraries generated by other emitters.