Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/cluster-setup-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ jobs:
fi
yq --version

- name: Check version drift
run: bash tools/cluster_setup/check_versions.sh

- name: Run cluster_setup unit tests
run: bash tools/cluster_setup/test_k0s_cluster_with_stack.sh

Expand Down
4 changes: 2 additions & 2 deletions docs/deployment/k0s-quick-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ default repos; `git` and `jq` are. Install each via its own supported method
rather than a single package-manager command:

- `kubectl` — [official binary download](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/),
pinned to `v1.36.1` (matches the k0s version this repo installs by default)
pinned to `v1.36.1` <!-- ver:KUBECTL_VERSION --> (matches the k0s version this repo installs by default)
- `helm` — [install script or binary release](https://helm.sh/docs/intro/install/)
- `yq` — [binary release](https://github.com/mikefarah/yq#install), pinned to
`v4.44.1` (matches the version this repo already relies on elsewhere)
`v4.44.1` <!-- ver:YQ_VERSION --> (matches the version this repo already relies on elsewhere)
- `git`, `jq` — `sudo apt-get install -y git jq` (Ubuntu) or
`sudo dnf install -y git jq` (RHEL 9)
- `crane` — [binary release](https://github.com/google/go-containerregistry/releases),
Expand Down
13 changes: 10 additions & 3 deletions tools/cluster_setup/DEPLOYMENT_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,10 @@ graph TB
|---|---|---|
| Splunk AI Operator | your build | Manages `AIPlatform` CR lifecycle |
| Splunk Operator | 3.0.0 | Manages Splunk Enterprise |
| KubeRay | 1.2.2 | Manages Ray clusters for AI inference |
| cert-manager | v1.13.0 | TLS certificate management |
| KubeRay | 1.2.2 <!-- ver:K0S_KUBERAY_CHART_VERSION --> | Manages Ray clusters for AI inference |
| cert-manager | v1.13.0 <!-- ver:K0S_CERT_MANAGER_VERSION --> | TLS certificate management |
| OTel Operator | latest | Observability |
| NVIDIA Device Plugin | v0.17.3 | Exposes GPUs to Kubernetes |
| NVIDIA Device Plugin | v0.17.3 <!-- ver:K0S_NVIDIA_DEVICE_PLUGIN_VERSION --> | Exposes GPUs to Kubernetes |

### Version Compatibility

Expand Down Expand Up @@ -507,6 +507,8 @@ graph TD
K0S["k0s binary\nv1.36.1+k0s.0 (default) or --k0s-version"]
YQ["yq v4.44.1\nYAML processor"]
end
%% v1.36.1+k0s.0 ver:K0S_VERSION
%% v4.44.1 ver:YQ_VERSION

subgraph IMG["📁 images/ ⭐ pre-loaded OCI image bundles"]
K0SIMG["k0s-images.tar\nk0s control-plane images:\npause · calico · kube-proxy\ncoredns · metrics-server"]
Expand All @@ -518,13 +520,18 @@ graph TD
LP["local-path-provisioner v0.0.24"]
NDP["nvidia-device-plugin v0.17.3"]
end
%% v1.13.0 ver:K0S_CERT_MANAGER_VERSION
%% v0.0.24 ver:K0S_LOCAL_PATH_PROVISIONER_VERSION
%% v0.17.3 ver:K0S_NVIDIA_DEVICE_PLUGIN_VERSION

subgraph CHARTS["📁 charts/"]
PROM["kube-prometheus-stack\n(version resolved at bundle time)"]
OTEL["opentelemetry-operator\n(version resolved at bundle time)"]
KUBERAY["kuberay-operator 1.2.2"]
METALLB["metallb 0.14.8"]
end
%% 1.2.2 ver:K0S_KUBERAY_CHART_VERSION
%% 0.14.8 ver:K0S_METALLB_CHART_VERSION

subgraph PKGS["📁 packages/ (GPU nodes)"]
EPEL["epel-release-latest-9.noarch.rpm"]
Expand Down
48 changes: 24 additions & 24 deletions tools/cluster_setup/K0S_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,11 @@ The script installs everything needed for the AI Platform:
1. **k0s Kubernetes Cluster** — CNCF certified, single-binary Kubernetes
2. **Calico CNI** — High-performance networking with VXLAN
3. **local-path Storage Provisioner** — Default StorageClass for PVCs
4. **Cert-Manager v1.13.0** — Automated certificate management
4. **Cert-Manager v1.13.0** — Automated certificate management <!-- ver:K0S_CERT_MANAGER_VERSION -->
5. **Kube-Prometheus Stack** — Monitoring with Prometheus + Grafana
6. **OpenTelemetry Operator** — Distributed tracing and telemetry
7. **NVIDIA Host Drivers + Device Plugin** — GPU support (RHEL 9)
8. **KubeRay Operator v1.2.2** — Ray cluster management for distributed AI
8. **KubeRay Operator v1.2.2** — Ray cluster management for distributed AI <!-- ver:K0S_KUBERAY_CHART_VERSION -->
9. **Splunk Operator** — Splunk Enterprise management
10. **Splunk AI Platform Operator** — AI platform orchestration (SAIA feature)
11. **AIPlatform CR** — Complete AI deployment with features, scheduling, and secrets
Expand Down Expand Up @@ -105,7 +105,7 @@ brew install kubectl helm git jq yq crane
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg git jq

# pinned to match the k0s version this repo installs by default (v1.36.1+k0s.0) — keep in sync with that version
# pinned to match the k0s version this repo installs by default (v1.36.1+k0s.0) — keep in sync with that version # ver:K0S_VERSION
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
Expand All @@ -114,12 +114,12 @@ sudo apt-get install -y kubectl
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# pinned to match the version this repo already relies on (k0s_cluster_with_stack.sh, airgap_install.sh)
sudo wget https://github.com/mikefarah/yq/releases/download/v4.44.1/yq_linux_amd64 -O /usr/local/bin/yq
sudo wget https://github.com/mikefarah/yq/releases/download/v4.44.1/yq_linux_amd64 -O /usr/local/bin/yq # ver:YQ_VERSION
sudo chmod +x /usr/local/bin/yq

# crane — used by the image-mirroring commands below (see Step 2 — Mirror
# Container Images); no Docker daemon/root/group setup required
curl -fsSL https://github.com/google/go-containerregistry/releases/download/v0.21.9/go-containerregistry_Linux_x86_64.tar.gz -o /tmp/crane.tar.gz
curl -fsSL https://github.com/google/go-containerregistry/releases/download/v0.21.9/go-containerregistry_Linux_x86_64.tar.gz -o /tmp/crane.tar.gz # ver:CRANE_VERSION
tar -xzf /tmp/crane.tar.gz -C /tmp crane
sudo install -o root -g root -m 0755 /tmp/crane /usr/local/bin/crane
rm -f /tmp/crane.tar.gz /tmp/crane
Expand All @@ -144,7 +144,7 @@ sudo dnf install -y git jq
# kubectl — official binary download (https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)
# pinned to match the k0s version this repo installs by default (v1.36.1+k0s.0,
# see DEPLOYMENT_GUIDE.md's Hardware Requirements) — keep in sync with that version
curl -fsSLO "https://dl.k8s.io/release/v1.36.1/bin/linux/amd64/kubectl"
curl -fsSLO "https://dl.k8s.io/release/v1.36.1/bin/linux/amd64/kubectl" # ver:KUBECTL_VERSION
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
rm -f kubectl

Expand All @@ -153,12 +153,12 @@ curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 |

# yq — binary release, pinned to match the version this repo already relies on
# (k0s_cluster_with_stack.sh, airgap_install.sh) — https://github.com/mikefarah/yq#install
sudo curl -fsSL https://github.com/mikefarah/yq/releases/download/v4.44.1/yq_linux_amd64 -o /usr/local/bin/yq
sudo curl -fsSL https://github.com/mikefarah/yq/releases/download/v4.44.1/yq_linux_amd64 -o /usr/local/bin/yq # ver:YQ_VERSION
sudo chmod +x /usr/local/bin/yq

# crane — used by the image-mirroring commands below (see Step 2 — Mirror
# Container Images); no Docker daemon/root/group setup required
curl -fsSL https://github.com/google/go-containerregistry/releases/download/v0.21.9/go-containerregistry_Linux_x86_64.tar.gz -o /tmp/crane.tar.gz
curl -fsSL https://github.com/google/go-containerregistry/releases/download/v0.21.9/go-containerregistry_Linux_x86_64.tar.gz -o /tmp/crane.tar.gz # ver:CRANE_VERSION
tar -xzf /tmp/crane.tar.gz -C /tmp crane
sudo install -o root -g root -m 0755 /tmp/crane /usr/local/bin/crane
rm -f /tmp/crane.tar.gz /tmp/crane
Expand Down Expand Up @@ -359,13 +359,13 @@ images:

operators:
ray:
version: "v1.2.2"
version: "v1.2.2" # ver:K0S_KUBERAY_CHART_VERSION
modelVersion: "v0.3.14-36-g1549f5a"
rayVersion: "2.44.0"
certManager:
installCRDs: true
nvidia:
devicePluginVersion: "v0.17.3"
devicePluginVersion: "v0.17.3" # ver:K0S_NVIDIA_DEVICE_PLUGIN_VERSION

kubernetes:
namespace: ai-platform
Expand Down Expand Up @@ -717,8 +717,8 @@ the staging directory, `./airgap-bundle/airgap-bundle-<timestamp>` by default.
| `NVIDIA_DEVICE_PLUGIN_MANIFEST_URL` | GitHub NVIDIA device plugin URL | `file://<staged>/manifests/nvidia-device-plugin.yml` |
| `PROMETHEUS_CHART_PATH` | `prometheus-community/kube-prometheus-stack` | `<staged>/charts/kube-prometheus-stack-72.3.0.tgz` |
| `OTEL_CHART_PATH` | `open-telemetry/opentelemetry-operator` | `<staged>/charts/opentelemetry-operator-0.80.0.tgz` |
| `KUBERAY_CHART_PATH` | `kuberay/kuberay-operator` | `<staged>/charts/kuberay-operator-1.2.2.tgz` |
| `METALLB_CHART_PATH` | `metallb/metallb` | `<staged>/charts/metallb-0.14.8.tgz` |
| `KUBERAY_CHART_PATH` | `kuberay/kuberay-operator` | `<staged>/charts/kuberay-operator-1.2.2.tgz` <!-- ver:K0S_KUBERAY_CHART_VERSION --> |
| `METALLB_CHART_PATH` | `metallb/metallb` | `<staged>/charts/metallb-0.14.8.tgz` <!-- ver:K0S_METALLB_CHART_VERSION --> |

See [Air-Gapped Deployment](#air-gapped-deployment) for the full air-gap workflow.

Expand Down Expand Up @@ -1391,10 +1391,10 @@ cd tools/cluster_setup

| Category | Contents |
|---|---|
| Binaries | `k0s v1.36.1+k0s.0` (default; override with `--k0s-version`), `yq v4.44.1` |
| Binaries | `k0s v1.36.1+k0s.0` <!-- ver:K0S_VERSION --> (default; override with `--k0s-version`), `yq v4.44.1` <!-- ver:YQ_VERSION --> |
| **Image bundles** (`images/`) | **`k0s-images.tar`** — k0s control-plane images (pause, Calico, kube-proxy, CoreDNS, metrics-server); **`addon-images.tar`** — add-on component images (cert-manager, kube-prometheus-stack, kuberay, MetalLB, OTel, NVIDIA device plugin, busybox). Both built automatically and staged to `/var/lib/k0s/images/` on every node at install time. |
| Manifests | `cert-manager v1.13.0`, `local-path-provisioner v0.0.24`, `nvidia-device-plugin v0.17.3` |
| Helm charts | `kube-prometheus-stack` (version captured at download time), `opentelemetry-operator` (version captured at download time), `kuberay-operator 1.2.2`, `metallb 0.14.8` |
| Manifests | `cert-manager v1.13.0` <!-- ver:K0S_CERT_MANAGER_VERSION -->, `local-path-provisioner v0.0.24` <!-- ver:K0S_LOCAL_PATH_PROVISIONER_VERSION -->, `nvidia-device-plugin v0.17.3` <!-- ver:K0S_NVIDIA_DEVICE_PLUGIN_VERSION --> |
| Helm charts | `kube-prometheus-stack` (version captured at download time), `opentelemetry-operator` (version captured at download time), `kuberay-operator 1.2.2` <!-- ver:K0S_KUBERAY_CHART_VERSION -->, `metallb 0.14.8` <!-- ver:K0S_METALLB_CHART_VERSION --> |
| GPU packages | `packages/nvidia-closure/` — a complete offline dnf repo (driver, DKMS, gcc/make toolchain, container toolkit, `kernel-devel`/`kernel-headers` per GPU node kernel); PyYAML wheel (all nodes) |
| Node packages | `packages/node-closure/` — `kernel-modules-extra` for every node kernel that lacks `xt_conntrack` (RHEL 10 keeps kube-proxy's netfilter modules there). Only present when a node needs it; with `--download-only` and no `--config`, pass `--node-hosts` (or `--node-kernels`) so the nodes get probed. |
| Metadata | `bundle-versions.txt`, `container-images.txt`, `airgap-env.sh`, `checksums.sha256` |
Expand Down Expand Up @@ -1760,8 +1760,8 @@ These variables are set automatically by the air-gap staging step. Set them manu
|---|---|---|
| `PROMETHEUS_CHART_PATH` | _(not set — uses remote repo)_ | `<staged>/charts/kube-prometheus-stack-<ver>.tgz` |
| `OTEL_CHART_PATH` | _(not set — uses remote repo)_ | `<staged>/charts/opentelemetry-operator-<ver>.tgz` |
| `KUBERAY_CHART_PATH` | _(not set — uses remote repo)_ | `<staged>/charts/kuberay-operator-1.2.2.tgz` |
| `METALLB_CHART_PATH` | _(not set — uses remote repo)_ | `<staged>/charts/metallb-0.14.8.tgz` |
| `KUBERAY_CHART_PATH` | _(not set — uses remote repo)_ | `<staged>/charts/kuberay-operator-1.2.2.tgz` <!-- ver:K0S_KUBERAY_CHART_VERSION --> |
| `METALLB_CHART_PATH` | _(not set — uses remote repo)_ | `<staged>/charts/metallb-0.14.8.tgz` <!-- ver:K0S_METALLB_CHART_VERSION --> |

**GPU Node OS Package URLs:**

Expand All @@ -1783,10 +1783,10 @@ These variables are set automatically by the air-gap staging step. Set them manu

```bash
# Use an internal Helm chart mirror for metallb only
export METALLB_CHART_PATH="/shared/charts/metallb-0.14.8.tgz"
export METALLB_CHART_PATH="/shared/charts/metallb-0.14.8.tgz" # ver:K0S_METALLB_CHART_VERSION

# Use an internal mirror for NVIDIA device plugin manifest only
export NVIDIA_DEVICE_PLUGIN_MANIFEST_URL="https://manifests.internal/nvidia-device-plugin-v0.17.3.yml"
export NVIDIA_DEVICE_PLUGIN_MANIFEST_URL="https://manifests.internal/nvidia-device-plugin-v0.17.3.yml" # ver:K0S_NVIDIA_DEVICE_PLUGIN_VERSION

CONFIG_FILE=./my-config.yaml ./k0s_cluster_with_stack.sh install
```
Expand Down Expand Up @@ -2250,7 +2250,7 @@ If `yq` is not installed or cannot parse the selected artifact profile, the down
ERROR: yq failed to parse './model_artifacts_configs_unquantized.yaml' — check that yq is installed and the file is valid YAML.
```

Install yq: `sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/download/v4.44.1/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq`
Install yq: `sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/download/v4.44.1/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq` <!-- ver:YQ_VERSION -->

#### Re-stage a single model without restarting from scratch

Expand Down Expand Up @@ -2426,15 +2426,15 @@ The script downloads various binaries, manifests, Helm charts, OS packages, and
| What | URL / Source |
|------|-------------|
| Public IP detection | `https://checkip.amazonaws.com`, `https://ipinfo.io/ip`, `https://api.ipify.org` |
| cert-manager manifest | `https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml` |
| cert-manager manifest | `https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml` <!-- ver:K0S_CERT_MANAGER_VERSION --> |
| NVIDIA k8s device plugin | `https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/<version>/deployments/static/nvidia-device-plugin.yml` |
| local-path-provisioner | `https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.24/deploy/local-path-storage.yaml` |
| local-path-provisioner | `https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.24/deploy/local-path-storage.yaml` <!-- ver:K0S_LOCAL_PATH_PROVISIONER_VERSION --> |
| Prometheus Helm repo | `https://prometheus-community.github.io/helm-charts` |
| kube-prometheus-stack chart | `prometheus-community/kube-prometheus-stack` (via `helm install`) |
| OpenTelemetry Helm repo | `https://open-telemetry.github.io/opentelemetry-helm-charts` |
| OpenTelemetry Operator chart | `open-telemetry/opentelemetry-operator` (via `helm install`) |
| KubeRay Helm repo | `https://ray-project.github.io/kuberay-helm/` |
| KubeRay Operator chart | `kuberay/kuberay-operator` version `1.2.2` (via `helm install`) |
| KubeRay Operator chart | `kuberay/kuberay-operator` version `1.2.2` <!-- ver:K0S_KUBERAY_CHART_VERSION --> (via `helm install`) |

### Downloads on All Nodes via SSH

Expand Down Expand Up @@ -2469,7 +2469,7 @@ These images are pulled from registries when pods are scheduled. Pre-pull for ai
| Splunk Enterprise | ECR or configured registry |
| Splunk Operator | `docker.io/splunk/splunk-operator:3.0.0` |
| Prometheus, Grafana, Alertmanager | Pulled by kube-prometheus-stack Helm chart |
| KubeRay Operator | `quay.io/kuberay/operator:v1.2.2` |
| KubeRay Operator | `quay.io/kuberay/operator:v1.2.2` <!-- ver:K0S_KUBERAY_CHART_VERSION --> |
| OpenTelemetry Operator | Pulled by opentelemetry-operator Helm chart |
| cert-manager (controller, webhook, cainjector) | Pulled by cert-manager manifest |
| NVIDIA device plugin | Pulled by DaemonSet manifest |
Expand Down
17 changes: 9 additions & 8 deletions tools/cluster_setup/airgap_install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,19 @@

set -euo pipefail

# ── Versions (keep in sync with k0s_cluster_with_stack.sh) ─────────────────
YQ_VERSION="v4.44.1"
CERT_MANAGER_VERSION="v1.13.0"
LOCAL_PATH_PROVISIONER_VERSION="v0.0.24"
NVIDIA_DEVICE_PLUGIN_VERSION="v0.17.3"
METALLB_CHART_VERSION="0.14.8"
KUBERAY_CHART_VERSION="1.2.2"
# ── Versions (keep in sync with k0s_cluster_with_stack.sh; source of truth:
# tools/cluster_setup/versions.env — run check_versions.sh after bumping) ─
YQ_VERSION="v4.44.1" # ver:YQ_VERSION
CERT_MANAGER_VERSION="v1.13.0" # ver:K0S_CERT_MANAGER_VERSION
LOCAL_PATH_PROVISIONER_VERSION="v0.0.24" # ver:K0S_LOCAL_PATH_PROVISIONER_VERSION
NVIDIA_DEVICE_PLUGIN_VERSION="v0.17.3" # ver:K0S_NVIDIA_DEVICE_PLUGIN_VERSION
METALLB_CHART_VERSION="0.14.8" # ver:K0S_METALLB_CHART_VERSION
KUBERAY_CHART_VERSION="1.2.2" # ver:K0S_KUBERAY_CHART_VERSION

# Pinned to the release validated in DEPLOYMENT_GUIDE.md's Hardware Requirements
# (v1.36.1, containerd 2.x). Override with --k0s-version for a different
# release; "latest" resolves the newest k0s release at run time.
K0S_VERSION="${K0S_VERSION:-v1.36.1+k0s.0}"
K0S_VERSION="${K0S_VERSION:-v1.36.1+k0s.0}" # ver:K0S_VERSION

# Target OS for GPU node driver packages: rhel9/rhel10 (RPM) or ubuntu24 (deb).
# 'auto' probes the GPU nodes over SSH — the installer delegates here without
Expand Down
68 changes: 68 additions & 0 deletions tools/cluster_setup/check_versions.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# Fails if a version hardcoded in scripts/docs disagrees with versions.env.
#
# Every pinned version must be tagged inline with `ver:<KEY>` immediately
# after the version string, e.g.:
# YQ_VERSION="v4.44.1" # ver:YQ_VERSION
# pinned to `v4.44.1` <!-- ver:YQ_VERSION -->
#
# This script finds every `ver:<KEY>` marker in the repo, and checks that the
# version string named by KEY in versions.env also appears on that line.
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
VERSIONS_FILE="${SCRIPT_DIR}/versions.env"

[[ -f "${VERSIONS_FILE}" ]] || { echo "missing ${VERSIONS_FILE}" >&2; exit 1; }

declare -A EXPECTED
while IFS='=' read -r key value; do
[[ -z "${key}" || "${key}" == \#* ]] && continue
EXPECTED["${key}"]="${value}"
done < "${VERSIONS_FILE}"

mismatches=0
checked=0

# grep -rn output is "path:line_no:content" — content may itself contain ':',
# so split only on the first two colons.
while IFS= read -r hit; do
file="${hit%%:*}"
rest="${hit#*:}"
line_no="${rest%%:*}"
content="${rest#*:}"

# Require a non-identifier char (or start of line) before "ver:" so this
# doesn't false-match inside words like "nvidia-driver:latest-dkms".
[[ "${content}" =~ (^|[^A-Za-z0-9_-])ver:([A-Za-z0-9_]+) ]] || continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate every marker found on a line

This regex extracts only the first ver:<KEY> occurrence from each grep result. Several newly tagged documentation lines contain multiple markers, such as the binaries row in K0S_README.md:1394; changing its second YQ_VERSION value to v9.99.9 still produces “No version drift” because only the preceding K0S_VERSION marker is checked. Iterate through all marker matches in each line rather than using a single BASH_REMATCH.

Useful? React with 👍 / 👎.

key="${BASH_REMATCH[2]}"

if [[ -z "${EXPECTED[${key}]+x}" ]]; then
echo "UNKNOWN KEY: ${file}:${line_no} references ver:${key}, not present in versions.env" >&2
mismatches=$(( mismatches + 1 ))
continue
fi

expected="${EXPECTED[${key}]}"
if ! grep -qF -- "${expected}" <<<"${content}"; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject version values that merely contain the expected prefix

The fixed-string containment test accepts a different version whenever it starts with the expected value. For example, with YQ_VERSION=v4.44.10 in versions.env, a tagged site containing v4.44.100 passes this check. This can silently miss ordinary patch-version drift, so the match needs version-token boundaries while still permitting formats such as a leading v or an embedded URL.

Useful? React with 👍 / 👎.

echo "MISMATCH: ${file}:${line_no} expected '${expected}' (versions.env:${key}) — got: ${content}" >&2
mismatches=$(( mismatches + 1 ))
fi
checked=$(( checked + 1 ))
done < <(grep -rn 'ver:[A-Za-z0-9_]\+' \
--include='*.sh' --include='*.md' \
"${REPO_ROOT}/tools/cluster_setup" "${REPO_ROOT}/docs/deployment" 2>/dev/null)
Comment on lines +53 to +55

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cover unmarked runtime pins with drift validation

Because this scan only discovers existing ver: markers, it cannot enforce the stated requirement that every pinned version is checked. For example, the online k0s installer still hardcodes the unmarked cert-manager URL at k0s_cluster_with_stack.sh:3337 and KubeRay versions at lines 4844–4851. Bumping K0S_CERT_MANAGER_VERSION and every marked occurrence to v1.14.0 still makes this check pass while the online installer fetches v1.13.0, allowing the online and air-gapped paths to drift.

Useful? React with 👍 / 👎.

Comment on lines +53 to +55

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Propagate failures from the marker scan

The grep runs in a process substitution, so its nonzero status is not propagated, and stderr is discarded. If either configured scan root is missing, renamed, or unreadable while the other still contains markers, the script reports success without checking that entire scope; for example, removing docs/deployment still yields “Checked 50 marker(s). No version drift.” Capture and validate the scan command's status instead of treating partial output as a complete scan.

Useful? React with 👍 / 👎.


if (( checked == 0 )); then
echo "No ver:<KEY> markers found — nothing checked. Is the marker convention still in use?" >&2
exit 1
fi

echo "Checked ${checked} marker(s)."
if (( mismatches > 0 )); then
echo "${mismatches} version drift issue(s) found." >&2
exit 1
fi

echo "No version drift."
Loading