Skip to content

improvement: ephemeral environment deploy testing for Renovate PRs #1576

Description

@ixxeL2097

Context

Currently, Renovate PRs that update Helm chart versions or image tags are auto-merged (patch/minor) without any functional validation. If a chart upgrade contains a breaking change, it only surfaces after merge and ArgoCD sync — i.e. in production.

Goal

Implement ephemeral environment testing on Renovate PRs before merge to:

  • Deploy the proposed chart/image change to a short-lived namespace or cluster
  • Run smoke tests / health checks against the ephemeral deployment
  • Report pass/fail back to the PR as a required status check
  • Block platformAutomerge until the check passes
  • Tear down the ephemeral environment after the check

Proposed approach

  1. Trigger: GitHub Actions workflow on pull_request targeting main, filtered on paths gitops/manifests/**
  2. Ephemeral namespace: create renovate-test-<pr-number> namespace in the genmachine cluster (via kubectl or ArgoCD CLI)
  3. Deploy: helm install the changed chart with the PR values against the ephemeral namespace
  4. Validate: run a configurable health-check (e.g. kubectl rollout status, HTTP probe, or a dedicated test job)
  5. Cleanup: delete the namespace on workflow completion (success or failure)
  6. Status check: set as required in branch protection to block Renovate automerge

Considerations

  • Needs cluster access from GitHub Actions (kubeconfig secret or OIDC)
  • Renovate automerge should be gated on this check passing (requiredStatusChecks)
  • Some charts (storage, CRDs) may need special handling or exclusion
  • Cost: ephemeral namespaces are cheap but CI time increases

Tasks

  • Design the GHA workflow for ephemeral deploy
  • Provision cluster access for GHA (OIDC or kubeconfig via Vault)
  • Implement namespace lifecycle (create / deploy / health-check / delete)
  • Add required status check to branch protection
  • Configure Renovate automergeStrategy to respect the check
  • Document the flow and exclusion list for non-testable charts

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions