simplify ama-logs release process by zanejohnson-azure · Pull Request #1744 · microsoft/Docker-Provider

zanejohnson-azure · 2026-07-17T06:25:34Z

Summary

Mirrors the ama-metrics (Azure/prometheus-collector) pipeline model in ama-logs so the build pipeline packages + pushes the Helm chart to cidev (ungated), and the AKS prod release consumes that build by version = the build run's runName and promotes image + chart cidev→ciprod.

Goal: decouple chart packaging from release. The build pipeline produces a versioned, packaged chart in cidev; the release pipeline consumes that exact packaged chart for promotion — instead of re-templating/re-packaging at release time.

Scope: this PR covers the build pipeline and the AKS prod release (promotion). The Arc release's consumption of the promoted -arc chart is intentionally out of scope here and lands in a follow-up (the managed-Ev2 Arc rollout in #1745, with the Arc prod-release YAML changes staged on branch zane/arc-prod-release-changes). The Arc prod-release pipeline is therefore unchanged in this PR. The build still emits the -arc chart and the AKS release still promotes it, so the promoted copy is ready for that follow-up.

What changed

Build pipeline (`azure_pipeline_mergedbranches.yaml`)

Derive SEMVER from the new top-level VERSION file and stamp the run number via ##vso[build.updatebuildnumber] (so runName = SEMVER).
Add a helm package step; push both the AKS (:<tag>) and Arc (:<tag>-arc) chart variants to cidev ACR (mirrored to cidev MCR). Ungated — runs on merge.

AKS prod release pipeline (`ci-aks-prod-release.yaml`)

Source the release version solely from the consumed build's runName (AgentImageTagSuffix = $(resources.pipeline._ci-aks-prod-release.runName)) — no manual ADO version vars.
Collapse the CDPX tag vars (CDPXLinuxTag, CDPXWindowsTag) into the single AgentImageTagSuffix, matching ama-metrics (one tag per artifact; source/dest differ only by repo).
Add the pushChartToAcr.sh Ev2 shell extension to promote the chart (both :<ver> and :<ver>-arc) cidev→ciprod alongside the existing image promotion.

Chart templatization (`charts/azuremonitor-containerinsights/`)

Split Chart.yaml/values.yaml into Chart-template.yaml/values-template.yaml so the version is injected at build time.
Add .helmignore to exclude source files from the packaged chart.

Cleanup

Delete dormant ManagedSDPReleasePipeline.yml (unused SDP pipeline).

Version flow

build run  ──> VERSION file ──> ##vso[build.updatebuildnumber]  (runName = SEMVER)
                                        │
                                        ▼
                            AKS prod release (consumes build)
                              AgentImageTagSuffix = runName
                              promotes image + chart (<ver>, <ver>-arc) cidev→ciprod

Testing

see work item 38731269

…line Mirror the ama-metrics approach so a downstream release pipeline can later consume a versioned, packaged Helm chart: - Add repo-root VERSION file (3.6.0) as the SEMVER base. - Derive SEMVER = <VERSION>-<branch>-<date>-<commit> in the setup step and use it for the linux/windows image tags. - Templatize the azuremonitor-containerinsights chart (Chart-template.yaml / values-template.yaml); generate Chart.yaml/values.yaml at build via envsubst and gitignore the generated files. Add generate_helm_files.sh local-dev helper. - Add a helm_chart job that lints, packages, and (on non-PR ci_prod) pushes the chart to the cidev ACR OCI path and publishes a pipeline artifact. - Regenerate Chart.yaml/values.yaml from the templates in the shared Helm deploy template (ama-logs-helm-deploy.yaml) so the e2e FilePath deploys still have a complete chart after the committed files were removed. - trunc 63 the chart: labels across the templates so long dev-build SEMVERs stay within the 63-char Kubernetes label limit. - Neutralize stale ARG IMAGE_TAG defaults in the Dockerfiles. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

Build pipeline (azure_pipeline_mergedbranches.yaml): package the chart a second time as <SEMVER>-arc and push it to cidev alongside the plain <SEMVER> tag; record both refs in metadata.json; add pushChartToAcr.sh to the AKS Managed-SDP Ev2 artifacts tar. Prod release: add a PushChartToACR Ev2 shell extension that oras-copies both cidev chart tags to ciprod under the clean release tag (<AgentImageTagSuffix> and <AgentImageTagSuffix>-arc), mirroring the agent image promotion and reusing existing ScopeBindings/Configuration tokens. WaitForMCRImages now also polls the ciprod chart repo for both tags before deploying. The -arc tag serves the AKS Arc rollout and the plain tag the AKS extension rollout. Migrating the Arc release pipeline to consume the promoted -arc chart from ciprod MCR is a follow-up PR. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

The prod release hand-set AgentImageTagSuffix via VAR_AGENT_IMAGE_TAG_SUFFIX. Mirror ama-metrics instead: the build pipeline sets its run number to the SEMVER (##vso[build.updatebuildnumber]), and the release derives AgentImageTagSuffix from the picked build run's runName ($(resources.pipeline._ci-aks-prod-release.runName)). A release now just picks a build run and uses whatever version is inside it, keeping the ciprod image/chart tag and helm deploys in lockstep with the build. Also drop the now-always-true ne(AgentImageTagSuffix, '') clause from the Stage_3 gating condition. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

Build pipeline (azure_pipeline_mergedbranches.yaml): remove the IS_PR/IS_MAIN_BRANCH condition on the "Push Helm chart to ACR" step so the chart is pushed to cidev on every build, matching ama-metrics whose chart push is ungated. The ADO pipeline-artifact publish stays gated to non-PR ci_prod builds. This lets feature-branch build runs produce a cidev chart the prod release can promote. VERSION: 3.6.0 -> 3.5.0 so the SEMVER-derived image/chart tags use 3.5.0. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

The build generates Chart.yaml/values.yaml from the *-template.yaml files before 'helm package', but with no .helmignore the packaged .tgz also shipped the Chart-template.yaml, values-template.yaml, and generate_helm_files.sh source files. Exclude them so the published chart contains only the real chart contents. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

…DP pipeline ci-aks-prod-release.yaml: the cidev->ciprod promotion SOURCE tags now derive from the picked build run's runName, matching the DEST tag (AgentImageTagSuffix) and mirroring ama-metrics resources.pipeline.<alias>.runName. CDPXLinuxTag = runName; CDPXWindowsTag = win-runName. Removed the two "Set CDPX * Tag" Bash tasks that re-derived these at runtime from the build artifact's metadata.json, consolidating to a single runName mechanism (the build sets its run number to SEMVER via ##vso[build.updatebuildnumber], and tags the images/chart with the same SEMVER, so runName resolves to exactly what was published). Deleted deployment/mergebranch-multiarch-agent-deployment-Managed-SDP/ ServiceGroupRoot/ManagedSDPReleasePipeline.yml, a dormant standalone OneBranch pipeline that was never registered in ADO and is referenced nowhere; the live release runs entirely through the inline Ev2RARollout@2 ("Ev2 Managed SDP - Deploy") in ci-aks-prod-release.yaml. This file held the last VAR_AGENT_IMAGE_TAG_SUFFIX / VAR_CDPX_*_TAG version inputs. The rest of ServiceGroupRoot/ (RolloutSpec.json, RolloutParameter.json, Scripts, etc.) is kept - the live pipeline consumes it. Net: no hand-set ADO variable feeds the release version anymore; a release picks a build run and promotes exactly the image + chart tags that build produced. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

…arity) The prod release pipeline carried a legacy CDPX* tag/registry variable set (CDPXLinux/Windows ACR|Registry|RepoName + CDPXLinuxTag/CDPXWindowsTag) from an era when the cidev source tag differed from the clean release tag. Now the build publishes cidev artifacts under clean SEMVER (= the build run's runName), so source tag == dest tag and the CDPX distinction is redundant. This matches how ama-metrics promotes: one tag per artifact, symmetric across source/dest, only the repo differs (cidev -> ciprod). - ci-aks-prod-release.yaml: delete the CDPX var block (6 dead ACR/Registry/ RepoName vars never consumed anywhere + CDPXLinuxTag/CDPXWindowsTag that just duplicated runName / win-runName); drop cdpxLinuxTag/cdpxWindowsTag from configurationOverrides. - RolloutParameter.json: remove the 3 CDPX_TAG env blocks; repoint each source path to __AGENT_IMAGE_TAG_SUFFIX__ (cidev:<tag>, cidev:win-<tag>, cidev/azuremonitor-containers:<tag>) so source mirrors dest. - ScopeBindings.json / Configurations.Public.Prod.json: drop the cdpx bindings and settings (keep overrideTag). - pushAgentToAcr.sh / pushChartToAcr.sh: remove the now-unused CDPX_TAG empty guard (AGENT_IMAGE_TAG_SUFFIX is already guarded); refresh the chart-script header comment. Net effect: image + both chart tags all derive from one value, the picked build run's runName (= SEMVER). No version-carrying ADO variables remain. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

azure-pipelines · 2026-07-17T06:25:43Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

…e chart push Mirror the ama-metrics model for the Arc K8s extension prod-release pipeline. The build packages the Helm chart to cidev and the AKS prod release promotes it to the single ciprod MCR path (<ver> plain + <ver>-arc). The Arc release now ONLY performs the rollout/registration of that already-promoted chart instead of packaging and pushing its own copy. ci-arc-k8s-extension-prod-release.yaml: - CHART_VERSION now derives from the picked build run: $(resources.pipeline._ci-arc-k8s-extension-prod-release.runName)-arc (was the hand-set $(VAR_CHART_VERSION)). - Delete the two in-pipeline Chart Push stages (Stage_Canary_MCR, Stage_1). No stage had an explicit dependsOn on either, so implicit file-order gating rewires cleanly: Stage_Canary_Regions is now first and Stage_2 gates on Wait_After_Canary. All deploy/wait/JIT stages are unchanged. arcExtensionRelease.sh: - Repoint all chart paths (REGISTRY_PATH_CANARY_STABLE/PROD_STABLE and the three MCR_NAME_PATH helm-pull sources) to the single promoted ciprod path mcr.microsoft.com/azuremonitor/containerinsights/ciprod/azuremonitor-containers. - Add a header comment documenting the consolidation. Operational note: the Arc release must run after the AKS prod release has promoted the chart for the same build run (both key off runName), so ciprod:<runName>-arc exists when the helm-pull validation runs. Dormant after this change (tracked for a later cleanup PR): the deployment/arc-k8s-extension-Managed-SDP/ chart-push folder + its build packaging, and the now-dead ADO vars/overrides (VAR_CHART_VERSION, ACRName/RepoType, acrName/ repoType). Left in place to keep this a minimal repoint. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

…o preview Per microsoft/mcr #5138 reviewer decision, the public chart repo is restructured so charts live under a dedicated `helmchart/` segment on MCR, decoupled from the agent image repos. Dev/build charts reuse the already onboarded `preview` chart repo (no `helmchart/` prefix, "for now"); prod charts are promoted to `helmchart/containerinsights/ciprod/...`. Build (azure_pipeline_mergedbranches.yaml): - Add `chartRepoName` var = .../public/azuremonitor/containerinsights/preview, decoupled from the cidev image `repoImageName`. Repoint both helm pushes (<SEMVER> + <SEMVER>-arc), metadata.json refs, and the log message from repoImageName to chartRepoName. Update cidev-referencing comments. AKS prod release: - ci-aks-prod-release.yaml: WaitForMCRImages verify gate PROD_MCR_CHART_REPO -> /azuremonitor/helmchart/containerinsights/ciprod/azuremonitor-containers. - RolloutParameter.json: SOURCE_CHART_FULL_PATH cidev -> preview; DEST_CHART_REPO adds helmchart/ (keeps __AGENT_RELEASE__ parametrization). - pushChartToAcr.sh: overwrite-guard CHART_MCR_REPO adds helmchart/. Arc prod release (arcExtensionRelease.sh): - All 5 ciprod chart refs (2 https REGISTRY_PATH_* + 3 oci MCR_NAME_PATH) add helmchart/. Header comment updated (build pushes to preview, not cidev). Image paths are unchanged (agent image stays on cidev/ciprod). oras copy / helm pull take full paths, so the dev(preview)/prod(helmchart) asymmetry is mechanically fine. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

Gate every rollout stage's approval on the promoted <version>-arc chart being present on the ciprod MCR path, so the Arc release fails fast in the pipeline (before the approval request and the Ev2 rollout) instead of mid-rollout inside the Ev2 shell script. - ci-arc-k8s-extension-prod-release.yaml: add a PowerShell preflight step to the releaseGating job of all 6 rollout stages. It queries the anonymous MCR v2 tags/list API for azuremonitor/helmchart/containerinsights/ciprod/azuremonitor-containers and checks for tag $(CHART_VERSION) (= runName + "-arc"), with a short retry loop to absorb ACR->MCR mirror propagation. On miss it logs an actionable error and exits 1, which skips approval and the Ev2 rollout via the existing dependsOn chain (approval dependsOn releaseGating; rollout dependsOn approval). arcExtensionRelease.sh is unchanged (keeps its per-stage helm pull as an Ev2-time last-resort check). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

…ransfer) Test-prep changes to validate the Helm chart promotion path without producing a ciprod ama-logs image on MCR: - VERSION 3.5.0 -> 3.4.0: avoid colliding test SEMVER tags with the team's in-progress 3.5.0 GA rollout. - AKS Managed-SDP RolloutSpec.json: drop shell/PushAgentToACR and shell/PushAgentToACR-1 (Linux/Windows image transfer); keep only shell/PushChartToACR so the run promotes the chart (plain + -arc) but no image. - ci-aks-prod-release.yaml Stage_3: condition -> false. Its WaitForMCRImages gate requires the image tags (would hang 24h) and its cluster deploys need the image; disabled for the chart-only test. Original condition preserved in a REVERT comment. Temporary; revert after chart-promotion testing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

The script duplicated the pipeline's inline envsubst generation and was never invoked by any automation (build pipeline and helm-deploy templates inline the same substitution). Drop it and its .helmignore entry; local chart generation, if needed, can be done with the same envsubst commands. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

Restore PushAgentToACR + PushAgentToACR-1 alongside PushChartToACR so the Stage_2 EV2 rollout promotes the ama-logs images (preview/cidev -> ciprod) in addition to the chart. The chart pins pods to mcr.microsoft.com/azuremonitor/containerinsights/ciprod:<tag>, so the Arc canary rollout requires the ciprod image to exist or pods hit ImagePullBackOff. This is the production-correct action set for this PR. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

For the current release the Arc prod release pipeline stays unchanged from ci_prod (still packages/pushes the chart in-pipeline). The consume-promoted- ciprod-chart changes are preserved on branch zane/arc-prod-release-changes and will be re-applied next release when managed-EV2 (PR #1745) handles Arc rollout. Reverted to ci_prod: - .pipelines/ci-arc-k8s-extension-prod-release.yaml - deployment/arc-k8s-extension-release-v2-Managed-SDP/ServiceGroupRoot/Scripts/arcExtensionRelease.sh Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

The chart+image promotion test is done, so restore production values: - VERSION 3.4.0 -> 3.5.0 - ci-aks-prod-release.yaml Stage_3: restore condition: and(eq(variables.IS_PR, false), eq(variables.IS_MAIN_BRANCH, true)) (was temporarily condition: false for the chart-only test) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83c76146-9fbb-4014-998a-eeed6f589b81

zanejohnson-azure · 2026-07-21T16:57:30Z

/azp run

azure-pipelines · 2026-07-21T16:57:45Z

Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions.

zanejohnson-azure and others added 7 commits July 15, 2026 22:45

zanejohnson-azure requested a review from a team as a code owner July 17, 2026 06:25

zanejohnson-azure and others added 6 commits July 17, 2026 11:20

zanejohnson-azure changed the title ~~Package Helm chart in ama-logs build pipeline and consume by runName in prod release~~ simplify ama-logs release process Jul 20, 2026

zanejohnson-azure and others added 2 commits July 20, 2026 15:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

simplify ama-logs release process#1744

simplify ama-logs release process#1744
zanejohnson-azure wants to merge 15 commits into
ci_prodfrom
zanejohnson-azure-helm-package-ama-logs-build

zanejohnson-azure commented Jul 17, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Jul 17, 2026

Uh oh!

zanejohnson-azure commented Jul 21, 2026

Uh oh!

azure-pipelines Bot commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

zanejohnson-azure commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Build pipeline (azure_pipeline_mergedbranches.yaml)

AKS prod release pipeline (ci-aks-prod-release.yaml)

Chart templatization (charts/azuremonitor-containerinsights/)

Cleanup

Version flow

Testing

Uh oh!

azure-pipelines Bot commented Jul 17, 2026

Uh oh!

zanejohnson-azure commented Jul 21, 2026

Uh oh!

azure-pipelines Bot commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

zanejohnson-azure commented Jul 17, 2026 •

edited

Loading

Build pipeline (`azure_pipeline_mergedbranches.yaml`)

AKS prod release pipeline (`ci-aks-prod-release.yaml`)

Chart templatization (`charts/azuremonitor-containerinsights/`)