ci: component-scoped e2e testing on kind for Renovate PRs - #35
Draft
jdaln wants to merge 7 commits into
Draft
Conversation
Behavior-preserving: kind-argo-test.sh keeps its assertions and CLI contract (CLUSTER/KEEP); cluster creation, overlay install, rollout waits and AppProject apply move to the shared lib so the component e2e runner can reuse them.
Profiles (exareme2, mip-stack, eck, haproxy, datacatalog) deploy the changed component through Argo CD at the PR head SHA and assert service-level health (Synced/Healthy, rollouts, in-cluster curl with content assertions). Committed manifests keep targetRevision: main; the runner rewrites them at runtime with check_zero-style assertions. tests/e2e holds the CI-only shims: StorageClass aliases for the CephFS classes, a /32 MetalLB pool carrying the production VIP, a federation-A overlay (drops the sample exareme2-overlay app, downgrades RWX claims to RWO, disables keycloak/public ingress) and shrunk ECK values.
changes job maps the diff to profiles (scripts/e2e/changed-profiles.sh, kept in sync with the workflow paths and renovate managerFilePatterns); one matrix job per profile on kind; render-only job for submariner and the hybrid federation overrides. Renovate manages the cert-manager / MetalLB pins in the e2e scripts via an annotation manager.
teardown_cluster captured $? after the trap's rm -rf, turning any e2e failure into exit 0. It now takes the exit code explicitly.
- changed-profiles.sh fails closed when the base ref is unresolvable (was: silent all-false mapping -> green check testing nothing) - ClusterIssuer health neutralized via CI-only argocd-cm override so Let's Encrypt account registration can never gate the haproxy profile - projects/templates/** now triggers the mip-stack profile (the federation AppProject chart was fully untested) - kind-argo-smoke paths include scripts/e2e/lib.sh (the smoke test's own bring-up code was no longer covered by its trigger) - ensure_probe recreates a Completed probe pod from a prior KEEP=1 run - wait_app tolerates 3 min of ComparisonError (cold chart-repo clone) before failing fast - ci-secrets generates passwords outside the heredocs so a generator failure aborts instead of applying empty secrets - revision rewrite is now a structured yq edit conditioned on the mip-infra repoURL instead of an unconditional sed over rendered YAML - eck profile overlays the production eck-stack Application (values-only patch) instead of maintaining a forked copy - workflow: dynamic matrix from the changes job (no idle runners), kind-action gets the 4-node config directly (no create-then-recreate), dispatch concurrency keyed by profile, e2e timeout 50m
- mip-stack: the upstream chart's 'default true $ingress.enabled' swallows enabled:false (sprig treats false as empty), so the Ingress deployed anyway and, with no controller in the profile, pinned the app at Progressing for the full 900s. Gate via global.publicHost='' (the chart's other, reliable condition), like the datacatalog overlay. - haproxy: the production configmap enforces ssl-redirect on :80, so the content assertion got 30x with an empty body. Assert the redirect itself on :80, then assert the echo body over https (-k, self-signed default cert); the unknown-Host negative check moves to https too. - eck: a 1-node ES is legitimately yellow once Kibana creates a replica-carrying index (the CR briefly said green, the API said yellow). Accept green/yellow and add a document write + search as the actual service-level assertion. exareme2 and datacatalog passed unchanged on the first run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When Renovate bumps a cluster component (exareme2, mip-stack, ECK, haproxy ingress, datacatalog), the PR now deploys that component on an ephemeral kind cluster through Argo CD at the PR's revision and asserts service-level health before merge.
How
changesjob maps the diff to profiles (scripts/e2e/changed-profiles.sh); a dynamic matrix runs only the requested profiles, each on its own runner.scripts/e2e/lib.sh, reused by the existing smoke test), then applies the component's production Application manifests withtargetRevision: mainrewritten to the PR head SHA at runtime (structured yq edit, conditioned on the mip-infra repoURL, with check_zero assertions). Committed files keepmain, socheck-main-revisionsis untouched./healthcheck+ non-empty/algorithms, ES cluster green via CRD status and API, haproxy routing an echo backend's body through the production Service (which holds the real VIP via a MetalLB /32 shim), datacatalog actuatorUP.tests/e2e/: StorageClass aliases for the CephFS classes, MetalLBpool-no-auto, cert-manager (ACME never load-bearing via a CI-only health override), non-interactive secrets.Docs:
docs/testing.mdrewritten.Validation
This PR touches
scripts/e2e/**, which maps to all profiles — the checks on this PR are the pipeline's own end-to-end validation. Expect first-run findings (e.g. the exact mip-stack backend endpoint is asserted loosely until confirmed here).Local:
git push && PROFILE=exareme2 KEEP=1 bash scripts/e2e/run-e2e.sh