Skip to content

refactor(scripts): move ops and docs scripts by audience, gate terramate script paths - #2076

Open
Smana wants to merge 19 commits into
mainfrom
refactor/scripts-ops-docs
Open

Smana wants to merge 19 commits into
mainfrom
refactor/scripts-ops-docs

Conversation

@Smana

@Smana Smana commented Sep 21, 2026

Copy link
Copy Markdown
Owner

PR 2 of 3 in the scripts/ restructure. It moves the day-2 operations scripts into scripts/ops/<area>/ and the docs generators into scripts/docs/. A new gate goes in first: it fails CI when any script path that opentofu or terramate executes doesn't exist. PR 1 (#2061) is merged; PR 3 (provision/) is still to come.

flowchart LR
  subgraph before["after PR 1"]
    R["scripts/ root<br/>32 files"]
  end
  subgraph after["after PR 2"]
    O["ops/aws · gcp · k8s · demo · teardown<br/>17 scripts"]
    D["docs/<br/>3 generators"]
    C["ci/check-rebased.sh"]
    P["root: 8 files<br/>provision-bound (PR 3)"]
  end
  R ==> O & D & C & P
  G["test-terramate-script-refs.sh<br/>88 executed refs, 0 unresolved"] -.->|gates| O
  classDef new fill:#d4edda,stroke:#28a745
  classDef pending fill:#f8f9fa,stroke:#adb5bd,stroke-dasharray:4
  class O,D,C,G new
  class P pending
Loading

📋 Design

What changed

scripts/ci/tests/test-terramate-script-refs.sh New gate. It resolves every script mention on an executed line in opentofu/**/*.{tf,tm.hcl,tfvars}: 88 refs, floor 80, .yaml included, .terraform/ excluded. A shape it can't resolve, or a ${…} prefix outside its allowlist, fails loudly instead of being skipped. These paths run at apply and destroy time, and no CI job executes them.
scripts/ops/{aws,gcp,k8s,demo}/ 13 scripts. The prefix drops where it repeats the directory: aws-sweep-orphaned-volumes.sh becomes ops/aws/sweep-orphaned-volumes.sh.
scripts/ops/teardown/ teardown.sh, the supported teardown, plus the 3 helpers terramate destroy calls
scripts/docs/ export-diagrams.sh, diagram-icons.py, build-og-card.html
scripts/ci/check-rebased.sh the pre-push hook, with .pre-commit-config.yaml updated
task --list 15 ops:*/docs:* entries that work from any directory. Destroy-time helpers are deliberately not indexed.
scripts/ci/tests/test-script-paths.sh It now tells a sibling path (../../lib/…) from a root climb, and fails an under-climb that lands on a name the repo root also has (nested AGENTS.md, scripts/README.md).

The design's Deletions table was overruled for teardown.sh and aws-sweep-controller-orphans.sh. Both have "no caller" only because a human is the caller: teardown.sh is the supported teardown (#1970, #1976), and the sweep is its recovery step. Owner decision, 2026-09-21. Nothing is deleted in this PR.

⚠️ Merge gate: read before merging

The plan makes terramate script run preview on both clouds a merge gate (owner decision, 2026-09-21). The whole-branch review then found that preview runs none of the moved scripts. Counting terramate script info lines that name a moved script:

Script Lines
preview 0
drift detect 0
deploy 3
destroy 28

The design's preview evidence fits PR 3, where helm-release-present.sh runs at plan time. For this PR, the reviewer recommends different evidence:

  • the two static gates, which cover every executed path;
  • the first real deploy and task ops:teardown after merge, whose logs should be watched for No such file.

I left the gate as you set it. Please choose:

  • Run the preview on both clouds, or accept the static gates plus a watched first deploy and teardown in its place

Evidence

Run on this branch, rebased on main @ 1c987b17. CI runs the render.

$ bash scripts/ci/tests/test-terramate-script-refs.sh
88 script reference(s) on executed opentofu/terramate lines checked; 0 failed
$ bash scripts/ci/tests/test-script-paths.sh
11 roots, 4 relative paths, 21 sources, 16 subjects checked; 0 failed
$ task ci:test
23 passed, 1 skipped, 0 failed        # SKIP test-vector-vrl  missing: vector
$ task ci:links / ci:doc-paths / ci:doc-claims / ci:idp-topology   # all exit 0
$ task --list | grep -cE '^\* (ops|docs):'
15

The refs gate was proved against fixtures:

  • moving terramate-destroy-confirm.sh without rewriting gives 15 failed;
  • breaking helm-release-present.sh gives 2 failed;
  • an empty tree trips the floor;
  • all 5 previously silent shapes fail;
  • ${path.root}/… fails as an unrecognised prefix.

All 13 symlinks are intact, no workflow changed, and job names are unchanged.

Rulings made during implementation

All are recorded in the SDD ledger.

  • P3: pacing. The gate got the full protocol; the moves went as one batch with one review.
  • R1: keep and move the two tools the design listed for deletion (owner).
  • R2: the gate reads .yaml too (88, not 87).
  • R3 and R5: the paths gate classifies sibling, root and ambiguous paths instead of treating every /.. as a root climb.
  • R4: the # shellcheck source= lines stay root-relative, because CI runs shellcheck from the root.
  • R6: this merge-gate note.
  • R7: don't index ops:k8s:reclaim-csi-volumes. It deletes every PVC on the current context without a prompt.

Follow-ups (not in this PR)

  • teardown.sh's ${ROOT}/scripts/ops/aws/… sweep calls are not gated. The paths gate's check 3 covers only HERE/SCRIPT_DIR.
  • The paths gate classifies only the first /.. climb on a line.
  • Neither gate has a committed negative-fixture suite. The fixtures live in the plan and the reviews.
  • scripts/ci/validate-manifests.sh:20 and flux-schema/gen-catalog.sh:37 have # shellcheck source= paths that don't resolve from the root (SC1091, hidden at -S warning).
  • Known limits of the refs gate:
    • a multi-line /* */ block would be read as code, which fails loud, and there are 0 today;
    • a script path inside a Terramate bundle.tm.yml is not scanned, and there are no bundles today.

…cture

ops/ and docs/ move under a new gate that resolves every script path on
an executed .tf/.tm.hcl line: 87 today, 30 of them to scripts this PR
moves. Every expected count was measured by simulating the move.
87 references on executed lines, 30 of them to scripts this PR moves.
They run at apply and destroy time and no CI job executes them; neither
test-script-paths.sh nor verify-doc-paths.sh reads a .tf or .tm.hcl.
…o ops/

The design listed both for deletion as having no caller. A human is the
caller: teardown.sh is the supported teardown path (#1970, #1976) and
the sweep is its recovery step. Owner decision, 2026-09-21.
aws/, gcp/, k8s/, demo/. The cloud prefix drops where it repeats the
directory. eks-prepare-destroy.sh's call to reclaim-csi-volumes.sh
crosses directories now and is corrected by hand: no gate sees a path
passed to exec.
teardown.sh is the supported teardown entry point; the other three are
what terramate destroy calls (22 of the 34 references). The terramate
reference gate went from 22 failed to 0.
Spec criterion 2: every entry point has a one-line description.
Only teardown.sh is indexed in ops/teardown/; terramate calls the rest.
Check 1 flagged any dirname+.. idiom that didn't resolve to the repo
root, which made every correct Task 4/5 fix false-positive: climbing
to a sibling directory (lib/, k8s/) isn't a root climb. Classify by
what follows the climb -- a named path segment means "sibling",
checked for existence; anything else (the climb ends the path, or a
variable follows it) means "root", checked against the two markers as
before.

This also means eks-prepare-destroy.sh's exec of
../k8s/reclaim-csi-volumes.sh is now gated -- 169b2493 said no gate
saw it; it does now, via the new "relative paths" count.

11 roots, 4 relative paths, 21 sources, 16 subjects checked; 0 failed.
The ops/{aws,gcp,k8s} row said "also called from terramate destroy
scripts", but eks-recycle-bootstrap-nodes.sh and adopt-workforce-pool.sh
run on every deploy, not just destroy. Named both, and corrected the
closing note: those two already moved to ops/, ahead of the rest of
the apply-time scripts still waiting on provision/.
Round 1's sibling branch trusted plain existence, but AGENTS.md and
README.md both nest at several depths: a climb that stops one level
short of the root can land on a decoy of the same name and pass by
accident, exactly like the wrong-depth case it's meant to catch.

Before the existence check, look at the climb's first path segment.
If it also exists at the repo root and the climb itself didn't land
there, fail loudly instead of trusting the coincidence.

11 roots, 4 relative paths, 21 sources, 16 subjects checked; 0 failed.
…an't resolve

The gate only ever extracted what its inner regex matched, so an
unbraced variable, a single-quoted path, a bare "../.." climb, or a
$(git rev-parse ...) prefix silently checked 0 refs and passed. Count
the outer "scripts/..." mentions per line against what the inner
regex extracted; fewer extracted fails as an unresolvable shape.

Also close the trust gap on the other side: a ${...}/$${...} prefix
now resolves only as ${path.module}, ${terramate.root.path.fs.absolute}
or $${ROOT} -- the three this codebase actually uses. Anything else
(${path.root}, ${terramate.stack.path.absolute}) failed silently
resolved against $ROOT/scripts/ before; it now fails as an
unrecognised prefix instead of guessing.

--exclude-dir=.terraform on the outer grep: vendored modules there are
gitignored and inflated the count locally (94 vs 88 in CI).

88 checked; 0 failed on the tree, unchanged.
It is a destroy-time helper, not a day-2 entry point: gke destroy and
eks-prepare-destroy.sh call it directly, with no prompt, and it
deletes every PVC and CNPG cluster in the current kube context. The
file's own rule already excludes destroy-time helpers -- this one was
missed. task --list now shows 15 ops/docs tasks, not 16.
"Compute it from a REPO_ROOT/SCRIPTS variable" was itself wrong: a
SCRIPTS=.../../.. variable climbs the same ambiguous way and fails
the root check it's supposed to satisfy. The only fix that actually
resolves is climbing to REPO_ROOT and naming the path from there.
…ables

M5: the closing paragraph repeated the ops row and reversed the
causality (named the two deploy-time scripts as if they were the
exception rather than the rule). Replaced with the plain fact:
apply-time provisioning scripts still sit at the root of scripts/,
moving to provision/ later.

M6: the terramate-refs row described the gate's pre-I1 behavior
(exists or is silently skipped). After I1 every mention on an executed
line is resolved or fails loudly, and .tfvars is one of the three
extensions it reads.
- ops:gcp:adopt-workforce-pool needs an initialised stack directory
  as cwd for its tofu-import half; without dir:, that half always
  skipped. Set dir: to the workforce-identity stack.
- adopt-workforce-pool.sh's usage said "from the stack directory" but
  gave a repo-root path; gave both the from-stack-dir and from-anywhere
  forms instead.
- commands.md named the renamed scripts by bare name, which
  verify-doc-paths.sh cannot check and which means nothing on their
  own (load.sh, reclaim-csi-volumes.sh, ...). Full scripts/ops/... and
  scripts/docs/... paths. Also: export-diagrams.sh writes SVG, not PNG.
- docs/architecture/AGENTS.md's "after any diagram change" block lost
  its column alignment when export-diagrams.sh moved into scripts/docs/.
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rendered manifest diff — this PR vs main (desired state)

No changes to the rendered desired state. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant