From a745456903427fc5f2879e6328c7409c939ce05f Mon Sep 17 00:00:00 2001 From: vrabbi Date: Sun, 16 Aug 2026 18:10:19 +0300 Subject: [PATCH 1/5] docs: state cross-cluster webhook failover is out of scope Each cluster runs an independent operator install. Fleet consistency is CI (convctl test/diff --live per kubecontext), not a shared webhook. Closes #85 --- docs/architecture.md | 22 ++++++++++++++++++++++ docs/faq.md | 10 ++++++++++ docs/limitations.md | 2 +- docs/roadmap.md | 2 +- 4 files changed, 34 insertions(+), 2 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 8235535..3638bd8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -53,6 +53,28 @@ Each `ConversionWebhookServer` replica is symmetric and self-sufficient — ther - **Readiness** gates on both informer cache sync *and* a completed first reconcile pass over every currently-existing config, closing the classic "added to Service endpoints before the registry is populated" gap. - A registry miss (a `ConversionReview` for an XRD this replica has no compiled plan for) fails closed with a clear `503`, rather than guessing. +## One cluster, one install + +Every operator replica and every `ConversionWebhookServer` replica is +self-sufficient **inside a single Kubernetes cluster**. There is no leader +election on the webhook-server path, no shared conversion state, and no +network hop to another cluster on a `ConversionReview`. That is a design +constraint, not a current gap. + +**Cross-cluster webhook failover is out of scope.** A webhook-server in +cluster A must not serve conversions for a resource whose apiserver lives +in cluster B. Each cluster runs its own fully independent install +(chart + `ConversionWebhookServer` + configs). Fleet consistency is a +**CI problem**, not an operator feature: run the same config through +[`convctl test --live`](cli.md#pre-upgrade-checks-testing-against-everything-that-already-exists) +and [`convctl diff --live`](cli.md#convctl-diff) against every kubecontext +before merge. + +Asking the operator to fail over conversion state across clusters would +need a redesign (shared registry, cross-cluster identity, a different +availability model). Until that exists, do not point `spec.conversion` +at a Service in another cluster. + ## Safety by construction: the hazards this design closes | Scenario | Mitigation | diff --git a/docs/faq.md b/docs/faq.md index 42c4718..9aec331 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -72,6 +72,16 @@ Yes — `convctl test --live` fetches every existing instance of the target type ## Operations and scaling +### Can one webhook-server serve conversions for resources in another cluster? + +No. Each cluster runs its own operator install. Cross-cluster webhook +failover (cluster A's pods converting objects for cluster B's apiserver) +is out of scope — the runtime has no shared state and no leader election +to make that safe. Use the same config YAML in each cluster and +`convctl test --live` / `diff --live` per kubecontext to keep them +consistent. See +[Architecture: One cluster, one install](architecture.md#one-cluster-one-install). + ### Can I run more than one `ConversionWebhookServer`? Yes — it's designed for this. Each instance is cluster-scoped, deployable, and independently scalable (Deployment, Service, cert-manager Certificate, optional HPA/PodDisruptionBudget). The chart installs exactly one, marked `default: true`; you can create more for tenancy or scale-out and assign specific configs to them via `webhookServerRef`. Every replica of every instance is symmetric and self-sufficient — no leader election, no shared state — so horizontal scaling is just adding pods. diff --git a/docs/limitations.md b/docs/limitations.md index fd304d8..3326a5a 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -23,7 +23,7 @@ This page is deliberately blunt about what the operator does *not* do today, so - **CRD schema changes require a manual step on Helm upgrade.** CRDs are installed once at `helm install` and never touched by `helm upgrade`/`helm uninstall` (Helm's own recommended convention for CRD-heavy charts) — see [Upgrading](installation.md#upgrading). - **Per-pod webhook-server state isn't surfaced back into `ConversionWebhookServer.status`.** `status.assignedConfigs` reflects *desired* assignment computed by the shared resolver, not confirmation that every replica has actually finished compiling and loading a given config — that's a deliberate trade-off to avoid the operator's reconcile loop depending on network calls to webhook-server pods. Check each pod's `/debug/registry` endpoint or metrics for real per-pod state. - **Spoke-to-spoke conversions always route through the hub** — two `Convert` calls, never a direct spoke-to-spoke path. This keeps compilation cost linear in the number of spoke versions. A 1000-element `forEach` spoke-to-spoke convert is ~2.3× a single hop and still under 1 ms ([Capacity planning](operations/capacity.md)); shortcut plans are not implemented. -- **No cross-cluster or multi-region coordination.** Every `ConversionWebhookServer` instance and every operator replica assumes a single Kubernetes cluster. +- **No cross-cluster or multi-region coordination.** Every `ConversionWebhookServer` instance and every operator replica assumes a single Kubernetes cluster. A webhook-server in cluster A serving conversions for a resource in cluster B is an explicit non-goal — see [Architecture: One cluster, one install](architecture.md#one-cluster-one-install). Keep configs consistent across a fleet with `convctl test --live` / `diff --live` per kubecontext, not by sharing webhook endpoints. ## Scale envelopes diff --git a/docs/roadmap.md b/docs/roadmap.md index 0f2cb26..08d8280 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -35,7 +35,7 @@ if one of them matters to you sooner. - **Strategies stay additive.** A new strategy is a `*Params` type, an `Op`, a compile resolver, webhook validation, a CLI fixture, and a docs page — see [Adding a strategy](contributing/adding-a-strategy.md). - **Observability is chart-optional.** ServiceMonitor / PrometheusRule / Grafana dashboards ship with the chart and stay off unless enabled. -- **No cross-cluster coordination.** Every operator and webhook-server replica assumes a single cluster ([Limitations](limitations.md)). +- **No cross-cluster coordination.** Every operator and webhook-server replica assumes a single cluster ([Architecture: One cluster, one install](architecture.md#one-cluster-one-install), [Limitations](limitations.md)). --- From d9672c4d713726739335f018cb0fdb12eebdd05f Mon Sep 17 00:00:00 2001 From: vrabbi Date: Sun, 16 Aug 2026 18:11:06 +0300 Subject: [PATCH 2/5] docs: add fleet CI pattern for convctl test and diff Copy-pasteable shell loop and GitHub Actions matrix that run diff --live plus test --live against every kubecontext and collect per-cluster JUnit. Closes #82 --- docs/architecture.md | 2 +- docs/cli.md | 3 + docs/faq.md | 2 +- docs/gitops/convctl-fleet.gha.yml | 62 +++++++++++++++++++++ docs/gitops/convctl-fleet.sh | 89 ++++++++++++++++++++++++++++++ docs/gitops/fleet-ci.md | 85 ++++++++++++++++++++++++++++ docs/limitations.md | 2 +- docs/operations/upgrade-runbook.md | 3 +- mkdocs.yml | 1 + 9 files changed, 245 insertions(+), 4 deletions(-) create mode 100644 docs/gitops/convctl-fleet.gha.yml create mode 100755 docs/gitops/convctl-fleet.sh create mode 100644 docs/gitops/fleet-ci.md diff --git a/docs/architecture.md b/docs/architecture.md index 3638bd8..0eadc31 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -68,7 +68,7 @@ in cluster B. Each cluster runs its own fully independent install **CI problem**, not an operator feature: run the same config through [`convctl test --live`](cli.md#pre-upgrade-checks-testing-against-everything-that-already-exists) and [`convctl diff --live`](cli.md#convctl-diff) against every kubecontext -before merge. +before merge — see [Fleet CI](gitops/fleet-ci.md). Asking the operator to fail over conversion state across clusters would need a redesign (shared registry, cross-cluster identity, a different diff --git a/docs/cli.md b/docs/cli.md index 27131b9..f794a7d 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -452,6 +452,9 @@ convctl test --crd crd.yaml --config new-crd-config.yaml --live \ This is the tool to run before applying a new or changed `XRDConversionConfig`/`CRDConversionConfig`: does it hold up against every object that already exists in the cluster, not just your fixtures? `--kubeconfig`/`--context` resolve exactly like `kubectl` does. The invoking identity only needs `get`/`list` on the target resource type — no write access, and nothing related to this operator's own CRDs or webhook server. +To run the same pair of checks (`diff --live` + `test --live`) against every +cluster in a fleet before merge, see [Fleet CI](gitops/fleet-ci.md). + ## Shell completion `convctl completion [bash|zsh|fish|powershell]` (built into Cobra) prints a completion script for your shell: diff --git a/docs/faq.md b/docs/faq.md index 9aec331..65c4f72 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -79,7 +79,7 @@ failover (cluster A's pods converting objects for cluster B's apiserver) is out of scope — the runtime has no shared state and no leader election to make that safe. Use the same config YAML in each cluster and `convctl test --live` / `diff --live` per kubecontext to keep them -consistent. See +consistent ([Fleet CI](gitops/fleet-ci.md)). See [Architecture: One cluster, one install](architecture.md#one-cluster-one-install). ### Can I run more than one `ConversionWebhookServer`? diff --git a/docs/gitops/convctl-fleet.gha.yml b/docs/gitops/convctl-fleet.gha.yml new file mode 100644 index 0000000..b14efa0 --- /dev/null +++ b/docs/gitops/convctl-fleet.gha.yml @@ -0,0 +1,62 @@ +# Reference GitHub Actions workflow — copy into a platform repo. +# This file is not wired into this operator's own CI. +# +# Each matrix.context is one cluster. fail-fast is false so one red +# cluster does not hide the rest. Upload the JUnit / diff JSON as +# artifacts for a test reporter or PR review. +name: convctl fleet + +on: + pull_request: + paths: + - "**/xrdconversionconfig.yaml" + - "**/crdconversionconfig.yaml" + - ".github/workflows/convctl-fleet.yml" + +permissions: + contents: read + +jobs: + fleet: + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + context: [prod-us, prod-eu] + steps: + - uses: actions/checkout@v4 + + - name: Install convctl + run: | + # Pin to a released convctl, or build from this repo: + # go install github.com/terasky-oss/declarative-conversion-operator/cmd/convctl@latest + echo "install convctl and place it on PATH" >&2 + exit 1 + + - name: Write kubeconfig + env: + KUBECONFIG_B64: ${{ secrets.FLEET_KUBECONFIG }} + run: | + mkdir -p "${HOME}/.kube" + printf '%s' "${KUBECONFIG_B64}" | base64 -d > "${HOME}/.kube/config" + chmod 600 "${HOME}/.kube/config" + + - name: convctl diff --live + run: | + mkdir -p fleet-out + convctl diff --config path/to/xrdconversionconfig.yaml --live \ + --context "${{ matrix.context }}" -o json \ + > "fleet-out/${{ matrix.context }}.diff.json" + + - name: convctl test --live + run: | + convctl test --config path/to/xrdconversionconfig.yaml \ + --xrd path/to/xrd.yaml --live \ + --context "${{ matrix.context }}" \ + --output junit --output-file "fleet-out/${{ matrix.context }}.junit.xml" + + - uses: actions/upload-artifact@v4 + if: always() + with: + name: convctl-${{ matrix.context }} + path: fleet-out/ diff --git a/docs/gitops/convctl-fleet.sh b/docs/gitops/convctl-fleet.sh new file mode 100755 index 0000000..ae5a5ff --- /dev/null +++ b/docs/gitops/convctl-fleet.sh @@ -0,0 +1,89 @@ +#!/usr/bin/env bash +# Fleet gate: convctl diff --live + test --live against every cluster. +# Copy into a platform repo or run from a checkout of this operator. +# +# Required: +# CONVCTL_CONFIG path to XRDConversionConfig / CRDConversionConfig +# CONVCTL_XRD or CONVCTL_CRD +# Cluster list (exactly one): +# CONTEXTS space-separated kubeconfig context names +# KUBECONFIGS space-separated kubeconfig file paths +# +# Optional: +# CONVCTL convctl binary (default: convctl on PATH) +# KUBECONFIG kubeconfig used with CONTEXTS (kubectl default if unset) +# OUT_DIR where per-cluster JUnit/JSON land (default: ./convctl-fleet-out) +# SKIP_DIFF set to 1 to run only test --live +set -euo pipefail + +CONVCTL="${CONVCTL:-convctl}" +OUT_DIR="${OUT_DIR:-./convctl-fleet-out}" +SKIP_DIFF="${SKIP_DIFF:-0}" + +die() { echo "error: $*" >&2; exit 2; } + +[[ -n "${CONVCTL_CONFIG:-}" ]] || die "CONVCTL_CONFIG is required" +if [[ -n "${CONVCTL_XRD:-}" && -n "${CONVCTL_CRD:-}" ]]; then + die "set CONVCTL_XRD or CONVCTL_CRD, not both" +fi +if [[ -z "${CONVCTL_XRD:-}" && -z "${CONVCTL_CRD:-}" ]]; then + die "CONVCTL_XRD or CONVCTL_CRD is required" +fi +if [[ -n "${CONTEXTS:-}" && -n "${KUBECONFIGS:-}" ]]; then + die "set CONTEXTS or KUBECONFIGS, not both" +fi +if [[ -z "${CONTEXTS:-}" && -z "${KUBECONFIGS:-}" ]]; then + die "set CONTEXTS (context names) or KUBECONFIGS (kubeconfig paths)" +fi +command -v "${CONVCTL}" >/dev/null 2>&1 || die "${CONVCTL} not found on PATH" + +schema_flags=() +if [[ -n "${CONVCTL_XRD:-}" ]]; then + schema_flags+=(--xrd "${CONVCTL_XRD}") +else + schema_flags+=(--crd "${CONVCTL_CRD}") +fi + +mkdir -p "${OUT_DIR}" +failed=0 + +run_one() { + local label="$1" kubeconfig="$2" context="$3" + local safe kube_flags=() + safe="$(printf '%s' "${label}" | tr -c 'A-Za-z0-9._-' '_')" + [[ -n "${kubeconfig}" ]] && kube_flags+=(--kubeconfig "${kubeconfig}") + [[ -n "${context}" ]] && kube_flags+=(--context "${context}") + + echo "=== ${label} ===" + if [[ "${SKIP_DIFF}" != 1 ]]; then + if ! "${CONVCTL}" diff --config "${CONVCTL_CONFIG}" --live \ + "${kube_flags[@]}" -o json \ + > "${OUT_DIR}/${safe}.diff.json"; then + echo "diff --live failed on ${label}" >&2 + failed=1 + fi + fi + if ! "${CONVCTL}" test --config "${CONVCTL_CONFIG}" --live \ + "${schema_flags[@]}" "${kube_flags[@]}" \ + --output junit --output-file "${OUT_DIR}/${safe}.junit.xml" \ + --quiet; then + echo "test --live failed on ${label}" >&2 + failed=1 + fi +} + +if [[ -n "${CONTEXTS:-}" ]]; then + for ctx in ${CONTEXTS}; do + run_one "${ctx}" "${KUBECONFIG:-}" "${ctx}" + done +else + for kc in ${KUBECONFIGS}; do + run_one "$(basename "${kc}")" "${kc}" "" + done +fi + +if [[ "${failed}" -ne 0 ]]; then + echo "fleet gate failed; reports in ${OUT_DIR}" >&2 + exit 1 +fi +echo "fleet gate passed; reports in ${OUT_DIR}" diff --git a/docs/gitops/fleet-ci.md b/docs/gitops/fleet-ci.md new file mode 100644 index 0000000..3bc3636 --- /dev/null +++ b/docs/gitops/fleet-ci.md @@ -0,0 +1,85 @@ +# Fleet CI: `convctl test` and `diff` across kubecontexts + +Each cluster runs its own operator install +([Architecture: One cluster, one install](../architecture.md#one-cluster-one-install)). +The operator does not sync conversion state between clusters. Before you +merge a config change, run the same two read-only checks against **every** +cluster that will apply that YAML: + +1. `convctl diff --live` — what would applying this file change on that + cluster (coverage, rule claims, lossiness). +2. `convctl test --live` — does the proposed mapping still hold up against + every live object of the target type. + +Neither command writes to the cluster. The invoking identity only needs +`get`/`list` on the target XRD/CRD and its instances. + +## Shell loop + +[`convctl-fleet.sh`](convctl-fleet.sh) is a copy-pasteable wrapper. It +walks `CONTEXTS` (space-separated kubeconfig context names), writes one +JUnit file per cluster, and exits non-zero if any cluster failed. + +```console +# Two kind clusters sharing ~/.kube/config: +export CONTEXTS="kind-fleet-a kind-fleet-b" +export CONVCTL_XRD=examples/field-rename/xrd.yaml +export CONVCTL_CONFIG=examples/field-rename/xrdconversionconfig.yaml +./docs/gitops/convctl-fleet.sh +``` + +`KUBECONFIG` / `--kubeconfig` resolve the same way `kubectl` does. To use +one kubeconfig file per cluster instead of contexts, set `KUBECONFIGS` to +a list of paths (the script uses each file's `current-context`). + +## GitHub Actions matrix + +[`convctl-fleet.gha.yml`](convctl-fleet.gha.yml) is a reference workflow, +not a job this repository runs. Copy it into your platform repo and +replace the `context` matrix with your fleet. Each matrix leg is one +cluster; `actions/upload-artifact` collects the JUnit files so a +test-reporter can show a per-cluster breakdown. + +```yaml +strategy: + fail-fast: false + matrix: + context: [prod-us, prod-eu, staging] +``` + +`fail-fast: false` is required: a red cluster must not hide the others. + +## What "pass" means + +| Command | Exit 0 | Exit 1 | Exit 2 | +|---|---|---|---| +| `convctl diff --live` | Cluster config and the file are equivalent | Any coverage/claim/lossy delta | Usage or cluster error | +| `convctl test --live` | Every path passed, or every loss was already `acknowledgeLossy` | Unacknowledged loss or conversion error | Usage or cluster error | + +A fleet gate should fail the PR if **any** cluster returns 1 or 2. A +delta on `diff` is not automatically a bug — it is the change you are +about to roll out. Typical pattern: require `test --live` green on every +cluster, and treat `diff --live` as a required review artifact (upload +the JSON) unless you are enforcing "no accidental coverage change." + +## Two-cluster check + +Verify the loop against two kind clusters that share one kubeconfig: + +```console +kind create cluster --name fleet-a +kind create cluster --name fleet-b +# install the operator + apply the field-rename XRD on both, then: +CONTEXTS="kind-fleet-a kind-fleet-b" \ + CONVCTL_XRD=examples/field-rename/xrd.yaml \ + CONVCTL_CONFIG=examples/field-rename/xrdconversionconfig.yaml \ + ./docs/gitops/convctl-fleet.sh +``` + +## Related + +- [CLI: pre-upgrade checks](../cli.md#pre-upgrade-checks-testing-against-everything-that-already-exists) +- [CLI: `convctl diff`](../cli.md#convctl-diff) +- [Upgrade runbook](../operations/upgrade-runbook.md) +- XRD lifecycle GitOps demo (Flux/Argo + in-cluster `convctl test --live`): + [`examples/crossplane-xr-multiversion/gitops/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/crossplane-xr-multiversion/gitops) diff --git a/docs/limitations.md b/docs/limitations.md index 3326a5a..a76d541 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -23,7 +23,7 @@ This page is deliberately blunt about what the operator does *not* do today, so - **CRD schema changes require a manual step on Helm upgrade.** CRDs are installed once at `helm install` and never touched by `helm upgrade`/`helm uninstall` (Helm's own recommended convention for CRD-heavy charts) — see [Upgrading](installation.md#upgrading). - **Per-pod webhook-server state isn't surfaced back into `ConversionWebhookServer.status`.** `status.assignedConfigs` reflects *desired* assignment computed by the shared resolver, not confirmation that every replica has actually finished compiling and loading a given config — that's a deliberate trade-off to avoid the operator's reconcile loop depending on network calls to webhook-server pods. Check each pod's `/debug/registry` endpoint or metrics for real per-pod state. - **Spoke-to-spoke conversions always route through the hub** — two `Convert` calls, never a direct spoke-to-spoke path. This keeps compilation cost linear in the number of spoke versions. A 1000-element `forEach` spoke-to-spoke convert is ~2.3× a single hop and still under 1 ms ([Capacity planning](operations/capacity.md)); shortcut plans are not implemented. -- **No cross-cluster or multi-region coordination.** Every `ConversionWebhookServer` instance and every operator replica assumes a single Kubernetes cluster. A webhook-server in cluster A serving conversions for a resource in cluster B is an explicit non-goal — see [Architecture: One cluster, one install](architecture.md#one-cluster-one-install). Keep configs consistent across a fleet with `convctl test --live` / `diff --live` per kubecontext, not by sharing webhook endpoints. +- **No cross-cluster or multi-region coordination.** Every `ConversionWebhookServer` instance and every operator replica assumes a single Kubernetes cluster. A webhook-server in cluster A serving conversions for a resource in cluster B is an explicit non-goal — see [Architecture: One cluster, one install](architecture.md#one-cluster-one-install). Keep configs consistent across a fleet with [CI](gitops/fleet-ci.md) (`convctl test --live` / `diff --live` per kubecontext), not by sharing webhook endpoints. ## Scale envelopes diff --git a/docs/operations/upgrade-runbook.md b/docs/operations/upgrade-runbook.md index 9a31bef..a82d054 100644 --- a/docs/operations/upgrade-runbook.md +++ b/docs/operations/upgrade-runbook.md @@ -31,7 +31,8 @@ addition. `convctl test --live` is the one that matters most before a change lands: your fixtures are what you thought of, and the cluster is what people actually -created. +created. For more than one cluster, use the [fleet CI](../gitops/fleet-ci.md) +loop or matrix — do not point one webhook-server at another cluster. Then apply and watch it through: diff --git a/mkdocs.yml b/mkdocs.yml index c6d428d..fd76e1c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -87,6 +87,7 @@ nav: - HA checklist: operations/ha-checklist.md - Capacity planning: operations/capacity.md - HPA on conversion QPS: operations/hpa-custom-metrics.md + - Fleet CI (many kubecontexts): gitops/fleet-ci.md - CLI Reference: cli.md - Strategy Reference: - strategies/index.md From c92ff3a14ff17d6960a289582ff2318afe14b820 Mon Sep 17 00:00:00 2001 From: vrabbi Date: Sun, 16 Aug 2026 18:12:51 +0300 Subject: [PATCH 3/5] feat: add convctl test --live --contexts and --kubeconfig-dir Run the same pre-upgrade test against every listed cluster and emit one JUnit document with a suite per cluster. One context or one kubeconfig keeps the existing single-cluster report. Closes #83 --- docs/cli.md | 11 +- docs/faq.md | 2 +- docs/gitops/fleet-ci.md | 19 ++++ internal/cli/fleet.go | 205 +++++++++++++++++++++++++++++++++++++ internal/cli/fleet_test.go | 131 ++++++++++++++++++++++++ internal/cli/report.go | 27 +++-- internal/cli/root.go | 121 +++++++++++++++++----- internal/cli/test.go | 6 ++ 8 files changed, 484 insertions(+), 38 deletions(-) create mode 100644 internal/cli/fleet.go create mode 100644 internal/cli/fleet_test.go diff --git a/docs/cli.md b/docs/cli.md index f794a7d..61b99ae 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -71,8 +71,10 @@ convctl test --crd crd.yaml --config crdconversionconfig.yaml --samples ./sample | `-c, --config` | Path to an `XRDConversionConfig` or `CRDConversionConfig` YAML file. **Required.** | | `-s, --samples` | Path to a directory of sample objects — one file per sample (or multi-doc YAML). Mutually exclusive with `--live`; exactly one of the two is required. | | `--live` | Fetch samples from a live cluster instead — see [Pre-upgrade checks](#pre-upgrade-checks-testing-against-everything-that-already-exists) below. | -| `--kubeconfig` | Path to a kubeconfig file. Only used with `--live`. Falls back to `$KUBECONFIG`, then `~/.kube/config`, exactly like `kubectl`. | -| `--context` | Kubeconfig context to use. Only used with `--live`. Falls back to the kubeconfig's `current-context`. | +| `--kubeconfig` | Path to a kubeconfig file. Only used with `--live`. Falls back to `$KUBECONFIG`, then `~/.kube/config`, exactly like `kubectl`. Mutually exclusive with `--kubeconfig-dir`. | +| `--context` | Kubeconfig context to use. Only used with `--live`. Falls back to the kubeconfig's `current-context`. Mutually exclusive with `--contexts`. | +| `--contexts` | Repeatable / comma-separated kubeconfig context names. Runs `--live` once per context and aggregates the report. A single name keeps the one-cluster report shape. | +| `--kubeconfig-dir` | Directory of kubeconfig files. `--live` runs against each file (that file's `current-context`, or each `--contexts` name). README / hidden files are skipped. | | `-o, --output` | `table` (default), `json`, or `junit` — see [Output formats](#output-formats) below. | | `--output-file` | Write the full report to this file instead of stdout. A short pass/loss/fail/error summary is still printed to stdout either way, so a CI log isn't empty on success. | | `--strict` | Escalate warnings (e.g. a rule that's never exercised by any sample) to failures. | @@ -454,6 +456,9 @@ This is the tool to run before applying a new or changed `XRDConversionConfig`/` To run the same pair of checks (`diff --live` + `test --live`) against every cluster in a fleet before merge, see [Fleet CI](gitops/fleet-ci.md). +`convctl test --live --contexts east,west -o junit` produces one JUnit +document with a `` per cluster (a cluster that cannot be +reached is an `` suite, not a silent skip). ## Shell completion @@ -469,7 +474,7 @@ Once installed, flags complete as follows: - `--xrd` / `--crd` / `--config` / `--sample` on offline commands complete YAML files; `--samples` completes directories. - `migrate-storage --xrd` / `--crd` complete **cluster resource names** (XRDs and CRDs listed from the current kubeconfig context), not files. `--namespace` completes live namespaces the same way. -- `--context` on `test`, `diff`, and `migrate-storage` completes kubeconfig context names. `--kubeconfig` already on the command line is honored, so `convctl migrate-storage --kubeconfig ./other --context ` lists contexts from that file. +- `--context` on `test`, `diff`, and `migrate-storage` completes kubeconfig context names. `--contexts` on `test` uses the same list. `--kubeconfig` already on the command line is honored, so `convctl migrate-storage --kubeconfig ./other --context ` lists contexts from that file. - `--output` and `test --fail-on` complete their allowed values. Cluster lookups during completion time out after two seconds and fall back to no suggestions if the apiserver is unreachable, so a hung cluster cannot freeze tab-complete. diff --git a/docs/faq.md b/docs/faq.md index 65c4f72..3182aae 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -68,7 +68,7 @@ Yes — `convctl test --live` fetches every existing instance of the target type ### Does this integrate with CI? -`convctl test` supports `--output json` for scripting and `--output junit` for CI systems with JUnit test-result reporting (GitHub Actions, GitLab, Jenkins), plus `--output-file` to write the report to a specific path. Exit codes are deliberately distinct for "a test failed" (`1`) versus "the tool was used wrong" (`2`), so CI can tell the two apart. +`convctl test` supports `--output json` for scripting and `--output junit` for CI systems with JUnit test-result reporting (GitHub Actions, GitLab, Jenkins), plus `--output-file` to write the report to a specific path. Exit codes are deliberately distinct for "a test failed" (`1`) versus "the tool was used wrong" (`2`), so CI can tell the two apart. For a fleet, `--live --contexts` / `--kubeconfig-dir` emit one aggregated JUnit document — see [Fleet CI](gitops/fleet-ci.md). ## Operations and scaling diff --git a/docs/gitops/fleet-ci.md b/docs/gitops/fleet-ci.md index 3bc3636..aac943d 100644 --- a/docs/gitops/fleet-ci.md +++ b/docs/gitops/fleet-ci.md @@ -14,6 +14,25 @@ cluster that will apply that YAML: Neither command writes to the cluster. The invoking identity only needs `get`/`list` on the target XRD/CRD and its instances. +## Built-in: `convctl test --live --contexts` + +Once you have more than one context in a single kubeconfig: + +```console +convctl test --xrd xrd.yaml --config proposed.yaml --live \ + --contexts kind-fleet-a,kind-fleet-b \ + --output junit --output-file fleet.junit.xml +``` + +`--kubeconfig-dir ./clusters/` is the same idea when each cluster has its +own kubeconfig file. One context or one file keeps the existing +single-cluster report. A connection error on one cluster is recorded as a +failed suite; the others still run. + +`convctl diff` stays one cluster per invocation (`--context`). The +[shell loop](#shell-loop) still wraps both commands when you want +`diff --live` in the same gate. + ## Shell loop [`convctl-fleet.sh`](convctl-fleet.sh) is a copy-pasteable wrapper. It diff --git a/internal/cli/fleet.go b/internal/cli/fleet.go new file mode 100644 index 0000000..db84eba --- /dev/null +++ b/internal/cli/fleet.go @@ -0,0 +1,205 @@ +/* +Copyright 2026 The declarative-conversion-operator Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + 10|Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package cli + +import ( + "fmt" + "io" + "os" + "path/filepath" + "strings" +) + +// liveTarget is one cluster --live should hit. +type liveTarget struct { + Label string + Kubeconfig string + Context string +} + +// ClusterResult is one cluster's contribution to a fleet test run. +type ClusterResult struct { + Label string `json:"label"` + Error string `json:"error,omitempty"` + Report *Report `json:"report,omitempty"` +} + +// FleetReport aggregates --live results across kubeconfigs/contexts. +type FleetReport struct { + Clusters []ClusterResult `json:"clusters"` +} + +func resolveLiveTargets(opts TestOptions) ([]liveTarget, error) { + if opts.KubeconfigDir != "" && opts.Kubeconfig != "" { + return nil, fmt.Errorf("--kubeconfig and --kubeconfig-dir are mutually exclusive") + } + if len(opts.Contexts) > 0 && opts.KubeContext != "" { + return nil, fmt.Errorf("--context and --contexts are mutually exclusive") + } + + if opts.KubeconfigDir == "" && len(opts.Contexts) == 0 { + label := opts.KubeContext + if label == "" { + label = "current-context" + } + return []liveTarget{{ + Label: label, + Kubeconfig: opts.Kubeconfig, + Context: opts.KubeContext, + }}, nil + } + + var files []string + if opts.KubeconfigDir != "" { + listed, err := listKubeconfigFiles(opts.KubeconfigDir) + if err != nil { + return nil, err + } + if len(listed) == 0 { + return nil, fmt.Errorf("no kubeconfig files under %s", opts.KubeconfigDir) + } + files = listed + } else { + files = []string{opts.Kubeconfig} + } + + var out []liveTarget + if len(opts.Contexts) == 0 { + for _, f := range files { + label := filepath.Base(f) + if label == "." || label == "" { + label = "current-context" + } + out = append(out, liveTarget{Label: label, Kubeconfig: f, Context: ""}) + } + return out, nil + } + for _, f := range files { + for _, ctx := range opts.Contexts { + if ctx == "" { + return nil, fmt.Errorf("--contexts contains an empty name") + } + label := ctx + if f != "" { + label = filepath.Base(f) + ":" + ctx + } + out = append(out, liveTarget{Label: label, Kubeconfig: f, Context: ctx}) + } + } + return out, nil +} + +func listKubeconfigFiles(dir string) ([]string, error) { + entries, err := os.ReadDir(dir) + if err != nil { + return nil, fmt.Errorf("reading --kubeconfig-dir %s: %w", dir, err) + } + var out []string + for _, e := range entries { + if e.IsDir() || strings.HasPrefix(e.Name(), ".") { + continue + } + name := e.Name() + lower := strings.ToLower(name) + if strings.HasSuffix(lower, ".md") || strings.HasPrefix(lower, "readme") { + continue + } + out = append(out, filepath.Join(dir, name)) + } + return out, nil +} + +func (f FleetReport) WriteJUnit(w io.Writer) error { + root := junitTestSuites{Name: "convctl fleet"} + var totalTime float64 + for _, c := range f.Clusters { + if c.Error != "" { + suite := junitTestSuite{ + Name: c.Label, + Tests: 1, + Errors: 1, + Cases: []junitTestCase{{ + Name: c.Label, + Classname: "convctl.fleet", + Error: &junitMessage{Message: "cluster error", Content: c.Error}, + }}, + } + root.Suites = append(root.Suites, suite) + root.Tests++ + root.Errors++ + continue + } + if c.Report == nil { + continue + } + suite := c.Report.junitSuite() + suite.Name = c.Label + "/" + suite.Name + for i := range suite.Cases { + suite.Cases[i].Classname = c.Label + "." + suite.Cases[i].Classname + } + root.Suites = append(root.Suites, suite) + root.Tests += suite.Tests + root.Failures += suite.Failures + root.Errors += suite.Errors + var seconds float64 + _, _ = fmt.Sscanf(suite.Time, "%f", &seconds) + totalTime += seconds + } + root.Time = fmt.Sprintf("%.6f", totalTime) + return writeJUnitSuites(w, root) +} + +func (f FleetReport) WriteTable(w io.Writer) { + for _, c := range f.Clusters { + _, _ = fmt.Fprintf(w, "=== %s ===\n", c.Label) + if c.Error != "" { + _, _ = fmt.Fprintf(w, "ERROR: %s\n\n", c.Error) + continue + } + if c.Report != nil { + c.Report.WriteTable(w) + _, _ = fmt.Fprintln(w) + } + } + f.WriteSummaryLine(w) +} + +func (f FleetReport) decideExitCode(failOn string, strict bool) int { + code := ExitOK + for _, c := range f.Clusters { + if c.Error != "" { + return ExitTestFailure + } + if c.Report == nil { + continue + } + if decideExitCode(c.Report, failOn, strict) != ExitOK { + code = ExitTestFailure + } + } + return code +} + +func (f FleetReport) WriteSummaryLine(w io.Writer) { + failed := 0 + for _, c := range f.Clusters { + if c.Error != "" || (c.Report != nil && decideExitCode(c.Report, failOnLoss, false) != ExitOK) { + failed++ + } + } + _, _ = fmt.Fprintf(w, "FLEET: %d clusters, %d failed\n", len(f.Clusters), failed) +} diff --git a/internal/cli/fleet_test.go b/internal/cli/fleet_test.go new file mode 100644 index 0000000..20a7aea --- /dev/null +++ b/internal/cli/fleet_test.go @@ -0,0 +1,131 @@ +/* +Copyright 2026 The declarative-conversion-operator Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + 10|Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package cli + +import ( + "bytes" + "encoding/xml" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestResolveLiveTargets_SingleDefault(t *testing.T) { + got, err := resolveLiveTargets(TestOptions{Live: true}) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Label != "current-context" { + t.Fatalf("single --live should stay one current-context target, got %+v", got) + } +} + +func TestResolveLiveTargets_OneContextMatchesSingleCluster(t *testing.T) { + got, err := resolveLiveTargets(TestOptions{Live: true, Contexts: []string{"prod"}}) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Context != "prod" || got[0].Label != "prod" { + t.Fatalf("one --contexts value must be a single-cluster target, got %+v", got) + } +} + +func TestResolveLiveTargets_Contexts(t *testing.T) { + got, err := resolveLiveTargets(TestOptions{Live: true, Contexts: []string{"east", "west"}}) + if err != nil { + t.Fatal(err) + } + if len(got) != 2 || got[0].Context != "east" || got[1].Context != "west" { + t.Fatalf("expected east then west, got %+v", got) + } +} + +func TestResolveLiveTargets_KubeconfigDir(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "east.kubeconfig"), []byte("apiVersion: v1\nkind: Config\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "west.kubeconfig"), []byte("apiVersion: v1\nkind: Config\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "README.md"), []byte("ignore"), 0o644); err != nil { + t.Fatal(err) + } + got, err := resolveLiveTargets(TestOptions{Live: true, KubeconfigDir: dir}) + if err != nil { + t.Fatal(err) + } + if len(got) != 2 { + t.Fatalf("expected 2 kubeconfigs (README skipped), got %+v", got) + } +} + +func TestResolveLiveTargets_RejectsMixedFlags(t *testing.T) { + if _, err := resolveLiveTargets(TestOptions{Kubeconfig: "a", KubeconfigDir: "b"}); err == nil { + t.Fatal("expected --kubeconfig and --kubeconfig-dir to conflict") + } + if _, err := resolveLiveTargets(TestOptions{KubeContext: "a", Contexts: []string{"b"}}); err == nil { + t.Fatal("expected --context and --contexts to conflict") + } +} + +func TestFleetReport_WriteJUnit_PerClusterSuites(t *testing.T) { + rep, err := RunTest(TestOptions{ + XRDPath: "testdata/xrd.yaml", + ConfigPath: "testdata/config.yaml", + SamplesDir: "testdata/samples", + }) + if err != nil { + t.Fatal(err) + } + fleet := FleetReport{Clusters: []ClusterResult{ + {Label: "east", Report: rep}, + {Label: "west", Error: "dial timeout"}, + }} + var buf bytes.Buffer + if err := fleet.WriteJUnit(&buf); err != nil { + t.Fatal(err) + } + var parsed junitTestSuites + if err := xml.Unmarshal(buf.Bytes(), &parsed); err != nil { + t.Fatalf("not JUnit XML: %v\n%s", err, buf.String()) + } + if len(parsed.Suites) != 2 { + t.Fatalf("expected 2 testsuites, got %d", len(parsed.Suites)) + } + if !strings.HasPrefix(parsed.Suites[0].Name, "east/") { + t.Fatalf("first suite should be prefixed with cluster label, got %q", parsed.Suites[0].Name) + } + if parsed.Suites[1].Errors != 1 { + t.Fatalf("unreachable cluster should be a suite error, got %+v", parsed.Suites[1]) + } + if parsed.Errors < 1 { + t.Fatal("fleet totals must count the cluster error") + } +} + +func TestFleetReport_DecideExitCode(t *testing.T) { + ok := &Report{} + fleet := FleetReport{Clusters: []ClusterResult{ + {Label: "a", Report: ok}, + {Label: "b", Error: "no such host"}, + }} + if fleet.decideExitCode(failOnLoss, false) != ExitTestFailure { + t.Fatal("a cluster connection error must fail the fleet run") + } +} diff --git a/internal/cli/report.go b/internal/cli/report.go index 3474292..91e9f67 100644 --- a/internal/cli/report.go +++ b/internal/cli/report.go @@ -196,7 +196,7 @@ type junitMessage struct { // (informational, not a failure); "fail" (unacknowledged loss) and "error" // map to / respectively, so CI systems that only surface // failed testcases still catch exactly what --fail-on would. -func (r *Report) WriteJUnit(w io.Writer) error { +func (r *Report) junitSuite() junitTestSuite { suite := junitTestSuite{Name: fmt.Sprintf("%s/%s", r.Meta.ResourceKind, r.Meta.Resource)} var totalTime float64 for _, s := range r.Samples { @@ -223,16 +223,10 @@ func (r *Report) WriteJUnit(w io.Writer) error { } suite.Tests = len(suite.Cases) suite.Time = fmt.Sprintf("%.6f", totalTime) + return suite +} - root := junitTestSuites{ - Name: fmt.Sprintf("%s conversion test: %s", r.Meta.ResourceKind, r.Meta.Resource), - Tests: suite.Tests, - Failures: suite.Failures, - Errors: suite.Errors, - Time: suite.Time, - Suites: []junitTestSuite{suite}, - } - +func writeJUnitSuites(w io.Writer, root junitTestSuites) error { if _, err := io.WriteString(w, xml.Header); err != nil { return err } @@ -245,6 +239,19 @@ func (r *Report) WriteJUnit(w io.Writer) error { return err } +func (r *Report) WriteJUnit(w io.Writer) error { + suite := r.junitSuite() + root := junitTestSuites{ + Name: fmt.Sprintf("%s conversion test: %s", r.Meta.ResourceKind, r.Meta.Resource), + Tests: suite.Tests, + Failures: suite.Failures, + Errors: suite.Errors, + Time: suite.Time, + Suites: []junitTestSuite{suite}, + } + return writeJUnitSuites(w, root) +} + func issuesText(issues []Issue) string { if len(issues) == 0 { return "" diff --git a/internal/cli/root.go b/internal/cli/root.go index adfc95a..4e873eb 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -169,7 +169,8 @@ func newTestCmd() *cobra.Command { xrdPath, crdPath, configPath, samplesDir, output, failOn, outputFile string skipIdentity, strict, live, quiet bool versionPairs []string - kubeconfig, kubeContext string + kubeconfig, kubeContext, kubeconfigDir string + contexts []string concurrency int ) cmd := &cobra.Command{ @@ -193,6 +194,11 @@ Samples come from exactly one of: the usual $KUBECONFIG and ~/.kube/config fallbacks). It only needs get/list on the target resource — no write access, and no access to the operator's own CRDs. +--contexts and --kubeconfig-dir run the same --live test against every listed +cluster and emit one aggregated JUnit report (one per cluster). +A single context or a single kubeconfig file keeps the existing one-cluster +report shape. + --output selects table (default), json, or junit (for CI test-result reporters). --output-file writes the full report to a path instead of stdout; a short pass/loss/fail/error summary still prints to stdout either way. @@ -211,41 +217,51 @@ results are collected by sample index, never by completion order.`, default: return fmt.Errorf("invalid --fail-on value %q (want %s, %s, or %s)", failOn, failOnNone, failOnWarn, failOnLoss) } - rep, err := RunTest(TestOptions{ + if !live && (len(contexts) > 0 || kubeconfigDir != "") { + return fmt.Errorf("--contexts and --kubeconfig-dir require --live") + } + opts := TestOptions{ XRDPath: xrdPath, CRDPath: crdPath, ConfigPath: configPath, SamplesDir: samplesDir, SkipIdentity: skipIdentity, RestrictVersionPairs: versionPairs, Live: live, Kubeconfig: kubeconfig, KubeContext: kubeContext, + Contexts: contexts, KubeconfigDir: kubeconfigDir, Concurrency: concurrency, Quiet: quiet, - }) - if err != nil { - return err - } - - var buf bytes.Buffer - switch output { - case "json": - err = writeJSONTo(&buf, rep) - case "junit": - err = rep.WriteJUnit(&buf) - default: - rep.WriteTable(&buf) } + targets, err := resolveLiveTargets(opts) if err != nil { return err } - if outputFile != "" { - if err := os.WriteFile(outputFile, buf.Bytes(), 0o644); err != nil { - return fmt.Errorf("writing report to %s: %w", outputFile, err) + if !live || len(targets) <= 1 { + if live && len(targets) == 1 { + opts.Kubeconfig = targets[0].Kubeconfig + opts.KubeContext = targets[0].Context + } + rep, err := RunTest(opts) + if err != nil { + return err } - _, _ = fmt.Fprintf(cmd.OutOrStdout(), "output written to file %s\n", outputFile) - rep.WriteSummaryLine(cmd.OutOrStdout()) - } else { - _, _ = cmd.OutOrStdout().Write(buf.Bytes()) + return writeTestOutput(cmd, output, outputFile, failOn, strict, rep, nil) } - exitCode = decideExitCode(rep, failOn, strict) - return nil + fleet := FleetReport{} + for _, t := range targets { + if !quiet { + _, _ = fmt.Fprintf(cmd.ErrOrStderr(), "=== %s ===\n", t.Label) + } + one := opts + one.Kubeconfig = t.Kubeconfig + one.KubeContext = t.Context + one.Contexts = nil + one.KubeconfigDir = "" + rep, runErr := RunTest(one) + if runErr != nil { + fleet.Clusters = append(fleet.Clusters, ClusterResult{Label: t.Label, Error: runErr.Error()}) + continue + } + fleet.Clusters = append(fleet.Clusters, ClusterResult{Label: t.Label, Report: rep}) + } + return writeTestOutput(cmd, output, outputFile, failOn, strict, nil, &fleet) }, } cmd.Flags().StringVarP(&xrdPath, "xrd", "x", "", "Path to an XRD YAML file (required for an XRDConversionConfig)") @@ -255,6 +271,8 @@ results are collected by sample index, never by completion order.`, cmd.Flags().BoolVar(&live, "live", false, "Fetch samples from a live cluster instead of --samples (a pre-upgrade check against real objects)") cmd.Flags().StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig file (default: $KUBECONFIG, then ~/.kube/config); only used with --live") cmd.Flags().StringVar(&kubeContext, "context", "", "Kubeconfig context to use (default: the kubeconfig's current-context); only used with --live") + cmd.Flags().StringSliceVar(&contexts, "contexts", nil, "Run --live against each of these kubeconfig contexts and aggregate the report; mutually exclusive with --context") + cmd.Flags().StringVar(&kubeconfigDir, "kubeconfig-dir", "", "Directory of kubeconfig files; --live runs against each file (current-context unless --contexts is also set)") cmd.Flags().StringVarP(&output, "output", "o", "table", "Output format: table|json|junit") cmd.Flags().StringVar(&outputFile, "output-file", "", "Write the full report to this file instead of stdout; a short summary still prints to stdout") cmd.Flags().BoolVar(&skipIdentity, "skip-identity", false, "Skip trivial same-version passthrough checks") @@ -268,10 +286,18 @@ results are collected by sample index, never by completion order.`, cmd.MarkFlagsMutuallyExclusive("xrd", "crd") cmd.MarkFlagsOneRequired("samples", "live") cmd.MarkFlagsMutuallyExclusive("samples", "live") + cmd.MarkFlagsMutuallyExclusive("context", "contexts") + cmd.MarkFlagsMutuallyExclusive("kubeconfig", "kubeconfig-dir") registerOfflineFlagCompletions(cmd) registerKubeFlagCompletions(cmd) registerOutputCompletions(cmd, "table", "json", "junit") _ = cmd.RegisterFlagCompletionFunc("fail-on", cobra.FixedCompletions([]string{failOnNone, failOnWarn, failOnLoss}, cobra.ShellCompDirectiveNoFileComp)) + if cmd.Flags().Lookup("contexts") != nil { + _ = cmd.RegisterFlagCompletionFunc("contexts", completeKubeContexts) + } + if cmd.Flags().Lookup("kubeconfig-dir") != nil { + _ = cmd.MarkFlagDirname("kubeconfig-dir") + } return cmd } @@ -349,6 +375,53 @@ const ( failOnLoss = "loss" ) +func writeTestOutput(cmd *cobra.Command, output, outputFile, failOn string, strict bool, rep *Report, fleet *FleetReport) error { + var buf bytes.Buffer + var err error + switch { + case fleet != nil: + switch output { + case "json": + err = writeJSONTo(&buf, fleet) + case "junit": + err = fleet.WriteJUnit(&buf) + default: + fleet.WriteTable(&buf) + } + default: + switch output { + case "json": + err = writeJSONTo(&buf, rep) + case "junit": + err = rep.WriteJUnit(&buf) + default: + rep.WriteTable(&buf) + } + } + if err != nil { + return err + } + if outputFile != "" { + if err := os.WriteFile(outputFile, buf.Bytes(), 0o644); err != nil { + return fmt.Errorf("writing report to %s: %w", outputFile, err) + } + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "output written to file %s\n", outputFile) + if fleet != nil { + fleet.WriteSummaryLine(cmd.OutOrStdout()) + } else { + rep.WriteSummaryLine(cmd.OutOrStdout()) + } + } else { + _, _ = cmd.OutOrStdout().Write(buf.Bytes()) + } + if fleet != nil { + exitCode = fleet.decideExitCode(failOn, strict) + } else { + exitCode = decideExitCode(rep, failOn, strict) + } + return nil +} + func decideExitCode(rep *Report, failOn string, strict bool) int { if failOn == failOnNone { return ExitOK diff --git a/internal/cli/test.go b/internal/cli/test.go index 197f9d5..2914944 100644 --- a/internal/cli/test.go +++ b/internal/cli/test.go @@ -47,6 +47,12 @@ type TestOptions struct { Live bool Kubeconfig string KubeContext string + // Contexts, when set, runs --live once per kubeconfig context and + // aggregates the reports. Mutually exclusive with KubeContext. + Contexts []string + // KubeconfigDir, when set, treats each regular file in the directory + // as a kubeconfig. Mutually exclusive with Kubeconfig. + KubeconfigDir string // Concurrency is how many samples to test at once. Zero or negative // means runtime.GOMAXPROCS(0). Testing a cluster's entire population From 922425191fec14c769dea5f21d5d34454c8587a5 Mon Sep 17 00:00:00 2001 From: vrabbi Date: Sun, 16 Aug 2026 18:13:56 +0300 Subject: [PATCH 4/5] docs: add Flux and Argo examples for operator apply order HelmRelease / Application trees install CRDs and the default ConversionWebhookServer before XRDConversionConfig, and document why KeepServingStale is required under continuous delivery. Closes #84 --- docs/examples/index.md | 4 ++ docs/gitops/operator-sync.md | 41 ++++++++++++ docs/installation.md | 5 ++ examples/README.md | 3 +- examples/gitops/README.md | 63 +++++++++++++++++++ examples/gitops/argo/README.md | 36 +++++++++++ examples/gitops/argo/configs-application.yaml | 22 +++++++ .../gitops/argo/operator-application.yaml | 31 +++++++++ examples/gitops/configs/kustomization.yaml | 5 ++ examples/gitops/configs/xrd.yaml | 49 +++++++++++++++ .../gitops/configs/xrdconversionconfig.yaml | 18 ++++++ examples/gitops/flux/README.md | 38 +++++++++++ .../gitops/flux/clusters/kustomizations.yaml | 44 +++++++++++++ .../gitops/flux/operator/helmrelease.yaml | 28 +++++++++ .../gitops/flux/operator/helmrepository.yaml | 9 +++ .../gitops/flux/operator/kustomization.yaml | 5 ++ mkdocs.yml | 1 + 17 files changed, 401 insertions(+), 1 deletion(-) create mode 100644 docs/gitops/operator-sync.md create mode 100644 examples/gitops/README.md create mode 100644 examples/gitops/argo/README.md create mode 100644 examples/gitops/argo/configs-application.yaml create mode 100644 examples/gitops/argo/operator-application.yaml create mode 100644 examples/gitops/configs/kustomization.yaml create mode 100644 examples/gitops/configs/xrd.yaml create mode 100644 examples/gitops/configs/xrdconversionconfig.yaml create mode 100644 examples/gitops/flux/README.md create mode 100644 examples/gitops/flux/clusters/kustomizations.yaml create mode 100644 examples/gitops/flux/operator/helmrelease.yaml create mode 100644 examples/gitops/flux/operator/helmrepository.yaml create mode 100644 examples/gitops/flux/operator/kustomization.yaml diff --git a/docs/examples/index.md b/docs/examples/index.md index 76b74f5..0eb9bd7 100644 --- a/docs/examples/index.md +++ b/docs/examples/index.md @@ -20,6 +20,10 @@ isolation, these show a complete, runnable config you can copy and adapt. | [`crossplane-xr-multiversion/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/crossplane-xr-multiversion) | Staged Crossplane XR lifecycle: one-version XRD + ConfigMap Composition, add a spoke, promote the hub (new Composition + retarget `compositionRef`), add `v3`, promote `v3` as the standard, deprecate `v1` (including `convctl migrate-storage` and dropping the version block). GitOps alternative: [`gitops/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/crossplane-xr-multiversion/gitops) + [`convctl generate kyverno`](../cli.md#convctl-generate-kyverno) (`--gitops-engine simulate\|flux\|argo`). | [`FieldRename`](../strategies/field-rename.md) — see the [lifecycle walkthrough](xr-lifecycle.md) | | [`native-crd/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/native-crd) | The same model against a plain Kubernetes CRD, with no Crossplane anywhere. | [`FieldRename`](../strategies/field-rename.md), [`Delete`](../strategies/delete.md) | +Operator install via Flux or Argo (apply order and `driftPolicy`) is +[`examples/gitops/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/gitops) +— see [GitOps operator sync](../gitops/operator-sync.md). + For a single fixture that exercises *every* built-in strategy at once, see the [kitchen-sink walkthrough](kitchen-sink.md). diff --git a/docs/gitops/operator-sync.md b/docs/gitops/operator-sync.md new file mode 100644 index 0000000..c04e068 --- /dev/null +++ b/docs/gitops/operator-sync.md @@ -0,0 +1,41 @@ +# GitOps: installing the operator (Flux / Argo) + +Copy-pasteable trees live in +[`examples/gitops/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/gitops): + +| Tree | Tool | What it shows | +|---|---|---| +| [`flux/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/gitops/flux) | Flux `HelmRelease` + two `Kustomization`s | CRDs + chart, then configs (`dependsOn` + `wait`) | +| [`argo/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/gitops/argo) | Two Argo `Application`s | Same order via sync-waves `0` then `1` | + +Apply order is always: + +```text +CRDs → ConversionWebhookServer (chart) → XRDConversionConfig / CRDConversionConfig +``` + +The controller already health-gates the last step (it will not patch a +target until the assigned `ConversionWebhookServer` is `Available`). The +GitOps annotations exist so Flux/Argo do not report the environment +failed while that gate is still open. + +## `driftPolicy: KeepServingStale` + +A GitOps reconcile is not an atomic multi-object transaction. A hub +promotion changes the schema and the conversion config independently. +During that window they disagree. + +`KeepServingStale` (default, and set explicitly on the sample config) +keeps serving the last known-good plan. `FailClosed` unpatches +conversion the moment they disagree — the failure mode that turns a +routine Flux/Argo retry into an API outage. Use `FailClosed` only when a +human applies both objects in one shot and is watching. + +Details: [examples/gitops/README.md](https://github.com/terasky-oss/declarative-conversion-operator/blob/main/examples/gitops/README.md). + +## Related + +- [Fleet CI](fleet-ci.md) — `convctl test --live` / `diff --live` per cluster +- [Installation](../installation.md) +- XRD lifecycle demo (app GitOps, not operator install): + [`examples/crossplane-xr-multiversion/gitops/`](https://github.com/terasky-oss/declarative-conversion-operator/tree/main/examples/crossplane-xr-multiversion/gitops) diff --git a/docs/installation.md b/docs/installation.md index bdfa1bb..f743d3b 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -151,3 +151,8 @@ helm lint charts/declarative-conversion-operator helm template declarative-conversion-operator charts/declarative-conversion-operator \ --namespace declarative-conversion-system ``` + +For Flux or Argo, use the [GitOps operator sync](gitops/operator-sync.md) +examples (`examples/gitops/flux`, `examples/gitops/argo`). Keep +`driftPolicy: KeepServingStale` on GitOps-managed configs — `FailClosed` +drops conversions while the schema and config reconcile independently. diff --git a/examples/README.md b/examples/README.md index 036510e..76371b4 100644 --- a/examples/README.md +++ b/examples/README.md @@ -5,7 +5,8 @@ conversion config, and `samples/` — everything `convctl` needs offline. [`crossplane-xr-multiversion/`](crossplane-xr-multiversion/) is a **staged** walkthrough (XRD + Composition that writes a ConfigMap) rather than a single end-state snapshot. [`demo.sh`](crossplane-xr-multiversion/demo.sh) runs the -full lifecycle on a live cluster. +full lifecycle on a live cluster. [`gitops/`](gitops/) is Flux/Argo YAML for +installing the operator itself (CRDs → `ConversionWebhookServer` → config). | Example | Story | Strategies | |---|---|---| diff --git a/examples/gitops/README.md b/examples/gitops/README.md new file mode 100644 index 0000000..9ad9aa2 --- /dev/null +++ b/examples/gitops/README.md @@ -0,0 +1,63 @@ +# GitOps: operator apply order (Flux and Argo) + +These trees install the operator from the published Helm chart, then apply +an `XRDConversionConfig`. They are **not** the Crossplane XR lifecycle +demo — that lives in +[`examples/crossplane-xr-multiversion/gitops/`](../crossplane-xr-multiversion/gitops/). + +## Apply order + +1. **CRDs** — the chart's `crds/` directory (`XRDConversionConfig`, + `CRDConversionConfig`, `ConversionWebhookServer`). Helm installs them + on first `helm install`; Flux/Argo must be told to create-or-replace + CRDs on upgrade too, or a later chart bump leaves the cluster on stale + CRDs. +2. **Chart** — manager + the bootstrap `ConversionWebhookServer/default`. + Wait until that CWS is `Available` before treating the install as + ready. The controller will not patch a target XRD/CRD until this gate + passes. +3. **Configs** — `XRDConversionConfig` / `CRDConversionConfig` (and the + XRD/CRD they target). Applying a config before the CWS is ready is + safe: the object stays `Pending` until every health gate passes. GitOps + `wait` / sync-waves still matter so the tool does not report the + environment failed during that window. + +```text +CRDs → ConversionWebhookServer (chart) → XRDConversionConfig +``` + +[`flux/`](flux/) uses two Flux `Kustomization`s (`dependsOn` + `wait`). +[`argo/`](argo/) uses two Argo `Application`s and sync-waves. + +## `driftPolicy` in continuous delivery + +Use **`KeepServingStale`** (the default) on every config a GitOps tool +reconciles. + +GitOps applies the XRD/CRD and the conversion config as **separate +objects**. A hub promotion is two fields that must move together +(`hubVersion` on the config, `referenceable`/`storage` on the schema). +They will disagree for at least one reconcile, often longer if Flux/Argo +retry independently. + +- **`KeepServingStale`** marks the config `Stale` and **keeps serving the + last known-good plan**. Conversions stay up while the two objects + converge. That is the failure mode this default exists to avoid. +- **`FailClosed`** **stops serving conversions** the moment the schema + and config disagree. The apiserver then fails `ConversionReview`s. + Flux/Argo see XRs/CRs as unhealthy, retry, and can widen the outage. + Reserve `FailClosed` for a human-driven apply where both objects change + in one `kubectl` invocation you are watching. + +See [Changing the hub version](https://terasky-oss.github.io/declarative-conversion-operator/configuration/xrdconversionconfig/#changing-the-hub-version). + +## Prerequisites + +cert-manager (and Crossplane, if you apply the sample XRD) must already +be on the cluster. The chart does not vendor either. + +## Sample config + +[`configs/`](configs/) is the [`field-rename`](../field-rename/) XRD plus +its conversion config, with `driftPolicy: KeepServingStale` written +explicitly so a reviewer sees the CD choice. diff --git a/examples/gitops/argo/README.md b/examples/gitops/argo/README.md new file mode 100644 index 0000000..eb2d11b --- /dev/null +++ b/examples/gitops/argo/README.md @@ -0,0 +1,36 @@ +# Argo CD: operator then configs + +Two `Application`s. The operator Application is sync-wave `0`; configs +are wave `1` so Argo does not apply the `XRDConversionConfig` until the +chart (CRDs + `ConversionWebhookServer/default`) has been synced. + +```text +examples/gitops/argo/operator-application.yaml +examples/gitops/argo/configs-application.yaml → examples/gitops/configs +``` + +## Apply + +```console +kubectl apply -f examples/gitops/argo/operator-application.yaml +# Wait for the chart, then: +kubectl wait --for=condition=Available --timeout=180s conversionwebhookserver/default +kubectl apply -f examples/gitops/argo/configs-application.yaml +kubectl get xrdconversionconfig xbuckets-conversion +``` + +With automated sync on both Applications, the wave annotation on the +config Application still orders the first sync. After that, Argo health +on `ConversionWebhookServer` (`Available`) is what keeps a config from +looking "stuck" while the controller health-gates. + +The operator Application uses the OCI Helm chart +(`ghcr.io/terasky-oss/charts` / `declarative-conversion-operator`). +Enable Helm CRD install/upgrade in the Application so chart CRDs are not +install-once. + +## driftPolicy + +Same rule as Flux: `KeepServingStale` on every GitOps-managed config. +`FailClosed` plus Argo auto-sync on a hub flip is an outage — see the +[parent README](../README.md#driftpolicy-in-continuous-delivery). diff --git a/examples/gitops/argo/configs-application.yaml b/examples/gitops/argo/configs-application.yaml new file mode 100644 index 0000000..d867f81 --- /dev/null +++ b/examples/gitops/argo/configs-application.yaml @@ -0,0 +1,22 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: conversion-configs + namespace: argocd + annotations: + argocd.argoproj.io/sync-wave: "1" +spec: + project: default + destination: + server: https://kubernetes.default.svc + namespace: default + syncPolicy: + syncOptions: + - ServerSideApply=true + automated: + prune: true + selfHeal: true + source: + repoURL: https://github.com/TeraSky-OSS/declarative-conversion-operator + targetRevision: main + path: examples/gitops/configs diff --git a/examples/gitops/argo/operator-application.yaml b/examples/gitops/argo/operator-application.yaml new file mode 100644 index 0000000..150c3fa --- /dev/null +++ b/examples/gitops/argo/operator-application.yaml @@ -0,0 +1,31 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: conversion-operator + namespace: argocd + annotations: + argocd.argoproj.io/sync-wave: "0" +spec: + project: default + destination: + server: https://kubernetes.default.svc + namespace: declarative-conversion-system + syncPolicy: + syncOptions: + - CreateNamespace=true + - ServerSideApply=true + automated: + prune: true + selfHeal: true + source: + repoURL: ghcr.io/terasky-oss/charts + chart: declarative-conversion-operator + targetRevision: 0.1.0 + helm: + skipCrds: false + values: | + features: + crossplane: + enabled: true + nativeCRD: + enabled: true diff --git a/examples/gitops/configs/kustomization.yaml b/examples/gitops/configs/kustomization.yaml new file mode 100644 index 0000000..ab75543 --- /dev/null +++ b/examples/gitops/configs/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - xrd.yaml + - xrdconversionconfig.yaml diff --git a/examples/gitops/configs/xrd.yaml b/examples/gitops/configs/xrd.yaml new file mode 100644 index 0000000..88d1c4a --- /dev/null +++ b/examples/gitops/configs/xrd.yaml @@ -0,0 +1,49 @@ +apiVersion: apiextensions.crossplane.io/v2 +kind: CompositeResourceDefinition +metadata: + name: xbuckets.example.org +spec: + scope: Namespaced + group: example.org + names: + kind: XBucket + plural: xbuckets + versions: + - name: v2 + served: true + referenceable: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + storageGB: + type: string + region: + type: string + status: + type: object + properties: + phase: + type: string + - name: v1 + served: true + referenceable: false + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + storageSize: + type: string + region: + type: string + status: + type: object + properties: + phase: + type: string diff --git a/examples/gitops/configs/xrdconversionconfig.yaml b/examples/gitops/configs/xrdconversionconfig.yaml new file mode 100644 index 0000000..c5f2ec4 --- /dev/null +++ b/examples/gitops/configs/xrdconversionconfig.yaml @@ -0,0 +1,18 @@ +apiVersion: terasky.com/v1alpha1 +kind: XRDConversionConfig +metadata: + name: xbuckets-conversion +spec: + targetXRD: + name: xbuckets.example.org + hubVersion: v2 + # Required for GitOps: XRD and this object reconcile independently. + # FailClosed would drop conversions during that window. + driftPolicy: KeepServingStale + spokes: + - version: v1 + rules: + - strategy: FieldRename + fieldRename: + hubPath: spec.storageGB + spokePath: spec.storageSize diff --git a/examples/gitops/flux/README.md b/examples/gitops/flux/README.md new file mode 100644 index 0000000..293e920 --- /dev/null +++ b/examples/gitops/flux/README.md @@ -0,0 +1,38 @@ +# Flux: operator then configs + +Two `Kustomization`s. The second `dependsOn` the first and waits, so Flux +does not declare the environment ready until `ConversionWebhookServer/default` +exists and the sample config can go `Applied`. + +```text +examples/gitops/flux/operator HelmRepository + HelmRelease (CRDs + CWS) +examples/gitops/configs XRD + XRDConversionConfig +``` + +## Apply + +Point a Flux `GitRepository` at this repository (or a fork) and apply +[`clusters/kustomizations.yaml`](clusters/kustomizations.yaml). Adjust +`spec.url` / `spec.path` prefixes if you vendor the tree. + +```console +kubectl apply -f examples/gitops/flux/clusters/kustomizations.yaml +kubectl -n flux-system wait --for=condition=Ready --timeout=10m \ + kustomization/conversion-operator +kubectl wait --for=condition=Available --timeout=180s conversionwebhookserver/default +kubectl -n flux-system wait --for=condition=Ready --timeout=5m \ + kustomization/conversion-configs +kubectl get xrdconversionconfig xbuckets-conversion +``` + +`install.crds: CreateReplace` / `upgrade.crds: CreateReplace` on the +`HelmRelease` is what keeps chart CRDs moving on upgrade. Without that, +Helm's default "CRDs only on first install" leaves GitOps stuck on the +install-time CRDs. + +## driftPolicy + +The sample config sets `KeepServingStale`. Do not switch it to +`FailClosed` under Flux: a hub flip is two objects, and Flux will +reconcile them in an order you do not control. See the +[parent README](../README.md#driftpolicy-in-continuous-delivery). diff --git a/examples/gitops/flux/clusters/kustomizations.yaml b/examples/gitops/flux/clusters/kustomizations.yaml new file mode 100644 index 0000000..89869ff --- /dev/null +++ b/examples/gitops/flux/clusters/kustomizations.yaml @@ -0,0 +1,44 @@ +# Apply after Flux is installed. Point spec.url at the repo that contains +# this tree (this file uses the public operator repo as the example). +apiVersion: source.toolkit.fluxcd.io/v1 +kind: GitRepository +metadata: + name: declarative-conversion-operator + namespace: flux-system +spec: + interval: 10m + url: https://github.com/TeraSky-OSS/declarative-conversion-operator + ref: + branch: main +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: conversion-operator + namespace: flux-system +spec: + interval: 10m + prune: true + wait: true + timeout: 10m + path: ./examples/gitops/flux/operator + sourceRef: + kind: GitRepository + name: declarative-conversion-operator +--- +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: conversion-configs + namespace: flux-system +spec: + interval: 5m + prune: true + wait: true + timeout: 5m + path: ./examples/gitops/configs + dependsOn: + - name: conversion-operator + sourceRef: + kind: GitRepository + name: declarative-conversion-operator diff --git a/examples/gitops/flux/operator/helmrelease.yaml b/examples/gitops/flux/operator/helmrelease.yaml new file mode 100644 index 0000000..9609d14 --- /dev/null +++ b/examples/gitops/flux/operator/helmrelease.yaml @@ -0,0 +1,28 @@ +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: declarative-conversion-operator + namespace: flux-system +spec: + interval: 30m + targetNamespace: declarative-conversion-system + storageNamespace: declarative-conversion-system + install: + crds: CreateReplace + createNamespace: true + upgrade: + crds: CreateReplace + chart: + spec: + chart: declarative-conversion-operator + version: "0.1.0" + sourceRef: + kind: HelmRepository + name: terasky-oss + namespace: flux-system + values: + features: + crossplane: + enabled: true + nativeCRD: + enabled: true diff --git a/examples/gitops/flux/operator/helmrepository.yaml b/examples/gitops/flux/operator/helmrepository.yaml new file mode 100644 index 0000000..bf7c9f4 --- /dev/null +++ b/examples/gitops/flux/operator/helmrepository.yaml @@ -0,0 +1,9 @@ +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: terasky-oss + namespace: flux-system +spec: + type: oci + interval: 1h + url: oci://ghcr.io/terasky-oss/charts diff --git a/examples/gitops/flux/operator/kustomization.yaml b/examples/gitops/flux/operator/kustomization.yaml new file mode 100644 index 0000000..d7fbb03 --- /dev/null +++ b/examples/gitops/flux/operator/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - helmrepository.yaml + - helmrelease.yaml diff --git a/mkdocs.yml b/mkdocs.yml index fd76e1c..12277b8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -88,6 +88,7 @@ nav: - Capacity planning: operations/capacity.md - HPA on conversion QPS: operations/hpa-custom-metrics.md - Fleet CI (many kubecontexts): gitops/fleet-ci.md + - GitOps operator sync (Flux/Argo): gitops/operator-sync.md - CLI Reference: cli.md - Strategy Reference: - strategies/index.md From 03b86049caea28d88a6b12b019dbb392608ffea5 Mon Sep 17 00:00:00 2001 From: vrabbi Date: Sun, 16 Aug 2026 18:18:11 +0300 Subject: [PATCH 5/5] fix: keep fleet test running after a convctl diff delta Treat diff exit 1 as a review artifact unless FAIL_ON_DIFF=1, always run test --live, and wait for ConversionWebhookServer/default on the Flux operator Kustomization. --- docs/cli.md | 6 +++--- docs/gitops/convctl-fleet.gha.yml | 7 +++++++ docs/gitops/convctl-fleet.sh | 16 +++++++++++++--- docs/gitops/fleet-ci.md | 11 ++++++----- .../gitops/flux/clusters/kustomizations.yaml | 5 +++++ 5 files changed, 34 insertions(+), 11 deletions(-) diff --git a/docs/cli.md b/docs/cli.md index 61b99ae..b95da2c 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -456,9 +456,9 @@ This is the tool to run before applying a new or changed `XRDConversionConfig`/` To run the same pair of checks (`diff --live` + `test --live`) against every cluster in a fleet before merge, see [Fleet CI](gitops/fleet-ci.md). -`convctl test --live --contexts east,west -o junit` produces one JUnit -document with a `` per cluster (a cluster that cannot be -reached is an `` suite, not a silent skip). +`convctl test --xrd xrd.yaml --config proposed.yaml --live --contexts east,west -o junit` +produces one JUnit document with a `` per cluster (a cluster +that cannot be reached is an `` suite, not a silent skip). ## Shell completion diff --git a/docs/gitops/convctl-fleet.gha.yml b/docs/gitops/convctl-fleet.gha.yml index b14efa0..58edf64 100644 --- a/docs/gitops/convctl-fleet.gha.yml +++ b/docs/gitops/convctl-fleet.gha.yml @@ -42,13 +42,20 @@ jobs: chmod 600 "${HOME}/.kube/config" - name: convctl diff --live + # Exit 1 is a coverage/claim delta (review artifact). Exit 2 is a + # usage/cluster error and must fail the job. run: | mkdir -p fleet-out + set +e convctl diff --config path/to/xrdconversionconfig.yaml --live \ --context "${{ matrix.context }}" -o json \ > "fleet-out/${{ matrix.context }}.diff.json" + rc=$? + if [ "${rc}" -eq 2 ]; then exit 2; fi + exit 0 - name: convctl test --live + if: ${{ always() }} run: | convctl test --config path/to/xrdconversionconfig.yaml \ --xrd path/to/xrd.yaml --live \ diff --git a/docs/gitops/convctl-fleet.sh b/docs/gitops/convctl-fleet.sh index ae5a5ff..2ac2dc0 100755 --- a/docs/gitops/convctl-fleet.sh +++ b/docs/gitops/convctl-fleet.sh @@ -14,11 +14,14 @@ # KUBECONFIG kubeconfig used with CONTEXTS (kubectl default if unset) # OUT_DIR where per-cluster JUnit/JSON land (default: ./convctl-fleet-out) # SKIP_DIFF set to 1 to run only test --live +# FAIL_ON_DIFF set to 1 to fail the gate on a coverage/claim delta (diff exit 1). +# Default: treat exit 1 as a review artifact; fail only on exit 2. set -euo pipefail CONVCTL="${CONVCTL:-convctl}" OUT_DIR="${OUT_DIR:-./convctl-fleet-out}" SKIP_DIFF="${SKIP_DIFF:-0}" +FAIL_ON_DIFF="${FAIL_ON_DIFF:-0}" die() { echo "error: $*" >&2; exit 2; } @@ -56,11 +59,18 @@ run_one() { echo "=== ${label} ===" if [[ "${SKIP_DIFF}" != 1 ]]; then - if ! "${CONVCTL}" diff --config "${CONVCTL_CONFIG}" --live \ + local diff_rc=0 + "${CONVCTL}" diff --config "${CONVCTL_CONFIG}" --live \ "${kube_flags[@]}" -o json \ - > "${OUT_DIR}/${safe}.diff.json"; then - echo "diff --live failed on ${label}" >&2 + > "${OUT_DIR}/${safe}.diff.json" || diff_rc=$? + if [[ "${diff_rc}" -eq 2 ]]; then + echo "diff --live usage/cluster error on ${label}" >&2 failed=1 + elif [[ "${diff_rc}" -eq 1 && "${FAIL_ON_DIFF}" == 1 ]]; then + echo "diff --live reported a delta on ${label} (FAIL_ON_DIFF=1)" >&2 + failed=1 + elif [[ "${diff_rc}" -eq 1 ]]; then + echo "diff --live reported a delta on ${label} (review ${OUT_DIR}/${safe}.diff.json)" >&2 fi fi if ! "${CONVCTL}" test --config "${CONVCTL_CONFIG}" --live \ diff --git a/docs/gitops/fleet-ci.md b/docs/gitops/fleet-ci.md index aac943d..48aac6b 100644 --- a/docs/gitops/fleet-ci.md +++ b/docs/gitops/fleet-ci.md @@ -75,11 +75,12 @@ strategy: | `convctl diff --live` | Cluster config and the file are equivalent | Any coverage/claim/lossy delta | Usage or cluster error | | `convctl test --live` | Every path passed, or every loss was already `acknowledgeLossy` | Unacknowledged loss or conversion error | Usage or cluster error | -A fleet gate should fail the PR if **any** cluster returns 1 or 2. A -delta on `diff` is not automatically a bug — it is the change you are -about to roll out. Typical pattern: require `test --live` green on every -cluster, and treat `diff --live` as a required review artifact (upload -the JSON) unless you are enforcing "no accidental coverage change." +A fleet gate should fail the PR if **any** cluster's `test --live` +returns 1 or 2, or if `diff --live` returns 2 (usage / cannot reach the +cluster). A `diff` exit 1 is a coverage/claim delta — the change you are +about to roll out — so the reference script and workflow treat it as a +review artifact by default. Set `FAIL_ON_DIFF=1` when you want that +delta to fail the gate. ## Two-cluster check diff --git a/examples/gitops/flux/clusters/kustomizations.yaml b/examples/gitops/flux/clusters/kustomizations.yaml index 89869ff..acc6235 100644 --- a/examples/gitops/flux/clusters/kustomizations.yaml +++ b/examples/gitops/flux/clusters/kustomizations.yaml @@ -22,6 +22,11 @@ spec: wait: true timeout: 10m path: ./examples/gitops/flux/operator + healthChecks: + - apiVersion: terasky.com/v1alpha1 + kind: ConversionWebhookServer + name: default + namespace: declarative-conversion-system sourceRef: kind: GitRepository name: declarative-conversion-operator