ci(mirror): fail-closed publish guard + publisher for the public deliverable mirror

[LukasWodka]

Summary

This repository is becoming the private development repo of the CLI; its public face will be a separate, deliverable-only mirror: README/LICENSE/user docs on its default branch and each release with its assets (signed binaries, SHA256SUMS, the two installers). No Go source ever leaves. This PR adds the fail-closed guard and publisher that is the only path from here to that mirror:

• .publish-include — the explicit allowlist of what MAY ship on the mirror's branch: README.md, LICENSE, docs/*.md (one level, so docs/rfcs/ stays home). Tracked files only.
• .publish-forbidden — [paths] that must never appear in the staged tree or among the release assets even if allowlisted by mistake (*.go, go.mod, go.sum, Makefile, test suites, CI, workflows, RFC docs, tooling, credential-shaped files), and two tiers of string needles matched over every staged text file: [strings-refuse] (mailboxes other than the allowlisted support one, arn:aws:, 12-digit ECR account ids) refuses on a hit; [strings-report] (internal ticket/RFC identifiers, non-production tracebloc hostnames) is counted and printed — per-needle totals and the ten most-hit files — and refuses only under --strict. [allow] spares the one public mailbox. The parser refuses the list itself (exit 2) on an unknown section header, a needle listed in both tiers, or an empty [strings-refuse] — a guard with nothing to refuse is misconfigured. Customer and tenant identifiers are deliberately not committed — this file is public — they arrive at publish time from a secret (PUBLISH_FORBIDDEN_TENANTS), join the refuse tier, and the guard refuses to run the scan without them.
• scripts/publish-guard.sh — stages the allowlisted tree and the release assets into a clean directory and runs four guards: allowlist, forbidden paths, forbidden strings (both tiers; --strict promotes the report tier), gitleaks detect --no-git --redact. Exit 0 only when all four ran and passed; 1 = refused (names the guard, the tier, the rule and the file:line — never the matched text); 2 = could not tell (unreadable/empty/malformed lists, zero tracked files, an allowlist matching nothing, a symlink, a missing or crashed scanner, a non-empty output dir). Binaries are opaque to the string scan and counted as such; their integrity is the release's own SHA256SUMS + cosign signatures, which travel with them.
• scripts/publish-mirror.sh — target (refuses an unset mirror, a malformed name, or a mirror equal to this repository), tree (clone / replace / commit / plain push, never force), release (creates the tag at the pushed commit with every asset; refuses to overwrite an existing release).
• .github/workflows/mirror-publish.yml — runs after Release completes (not on release: published, which fires before all eight binaries and SHA256SUMS are attached), plus workflow_dispatch with dry-run (default true) that runs every guard and prints the staged file list without pushing, and strict (default false) that passes --strict; the repository variable PUBLISH_STRICT=true does the same for every run, including the workflow_run path that has no inputs. The mirror is named by the MIRROR_REPO variable with no default: until the mirror exists the job refuses to publish. Publishing uses a GitHub App installation token scoped to the mirror only.
• build.yml Installer job shellchecks the two scripts and runs both harnesses; scripts/RELEASE_CHECKLIST.md gets the new automated step.

What a dry run prints today

On the current tree, with install.sh / install.ps1 / SHA256SUMS staged as assets and --extra-forbidden fed a placeholder tenant needle, the allowlist stages 5 of 526 tracked files (README, LICENSE, three docs), [forbidden-paths] is clean, [gitleaks] is clean, and [forbidden-strings] reports exit 0:

• refuse tier: 0 hits across its 4 needles (mailbox rule, arn:aws:, ECR account id, the placeholder tenant). Nothing in the deliverable or the installers trips the hard tier today.
• report tier: 26 hits in 3 files across its 10 needles — RFC-0 17, backend# 9; no hostname hits. Most-hit files: install.ps1 15, install.sh 9 (release assets), README.md 2.

The same run with --strict exits 1 and refuses all 26 lines under [strings-report (strict)]. A guard that could never pass was a guard nobody would arm; this one passes on the hard rules today and keeps the soft backlog measured until decision 3 below is taken. Nothing here gates a PR; the workflow only runs after a release or on dispatch.

Related

Part of the public/private repository split.

Type of change

• Feature
• Security / hardening

Test plan

• bash scripts/tests/publish-guard-verify.sh (43 checks locally, 42 in CI where the real-gitleaks case skips; was 34) and scripts/tests/publish-mirror-verify.sh (16): same harness shape as install-verify.sh. Every guard is mutation-proved against a fixture repository whose allowlist/forbidden lists are written in the harness, and each refusal is asserted on the guard name, the tier and the offending path or needle. Tier cases: a report-tier hit alone → exit 0 with the per-needle count, the most-hit table and the report file written; the same hit under --strict → exit 1 naming [strings-report (strict)]; a refuse-tier hit → exit 1 naming [strings-refuse]; both in one run → refused and still counted; the table sums needles per file, largest first, ten rows at most. List cases: a needle in both tiers → exit 2 naming the duplicate; the retired [strings] header (and a header with a space) → exit 2 naming the unknown section for both scans; an empty [strings-refuse], absent or present-but-empty, → exit 2, judged before the private needles join. Fail-closed cases as before: empty/missing allowlist, allowlist matching nothing, empty/missing forbidden list, empty/missing private needle list, missing/crashing scanner, non-empty output dir, symlink, empty assets. The committed lists get their own cases with inputs written in the harness: every refuse-tier needle refused by name, every report-tier needle counted by name and refused under --strict (the summary's needle counts are asserted, so a needle added without a planted input reddens the harness), every forbidden path class by name, and the real tree now asserted exit 0 — the refuse tier holds on the actual deliverable. The guard script is byte-identical to the client repo's copy, where each new check was mutation-proved by breaking it and watching only its named tests redden. tree runs against real bare repos over file://; release against a recording gh shim.
• shellcheck --severity=warning on both scripts, --severity=error on both harnesses, bash -n, actionlint on both workflows, go build ./..., bash scripts/check-style.sh, bash scripts/check-tool-pins.sh, scripts/file-budget.sh, the org house-rules.sh --all and pipefail-early-close.sh (shell + yaml scopes): all clean.
• bash scripts/publish-guard.sh --source . --out /tmp/stage --assets <install.sh, install.ps1, SHA256SUMS> on the real tree: verdict above.

Open decisions

1. Name the mirror repository — create it, install the release-train App on it, set the MIRROR_REPO variable. Until then every real run refuses at the target check.
2. Populate PUBLISH_FORBIDDEN_TENANTS — one extended regex per line. The guard refuses to run the string scan while the secret is empty.
3. Strip or accept the report tier — flip strict when decided. The report tier (internal ticket/RFC identifiers, non-production hostnames) is measured on every run, refused on none. The day the call is "strip", set the repository variable PUBLISH_STRICT=true (or dispatch with strict: true) and the same needles refuse; the day it is "accept", delete the needles from [strings-report]. Nothing else in the guard changes either way.

Deployment notes

• Set the Actions variable MIRROR_REPO (bare repo name in this organisation) only once the mirror repository exists. Until then every real run refuses at the target check; dry runs work.
• Set the secret PUBLISH_FORBIDDEN_TENANTS (one extended regex per line) before the first dry run is expected to pass — an empty list is refused as "could not tell".
• PUBLISH_STRICT (repository variable) is unset: the report tier is measured, not enforced. Setting it to true arms it for every run.
• RELEASE_TRAIN_APP_ID / RELEASE_TRAIN_APP_PRIVATE_KEY already exist at the organisation level; the App must be installed on the mirror repository.

Checklist

• Tests added / updated and passing locally
• Docs updated if behavior or config changed (scripts/RELEASE_CHECKLIST.md, workflow header, list file headers)
• No secrets / credentials in the diff
• For security-sensitive paths: appropriate reviewer requested
• go build ./..., go vet, and the Lint job's checks pass locally
• Terminal output follows STYLE.md; bash scripts/check-style.sh passes

---

Note

High Risk
Introduces release-time publishing to an external mirror with secret-backed string rules and write tokens; mistakes could leak dev content or push to the wrong repo, though guards and refusals are designed fail-closed.

Overview
Adds the only automated path from this private dev repo to a public, deliverable-only mirror: README/LICENSE/docs/*.md on the default branch plus mirrored GitHub releases (binaries, installers, checksums)—no Go source.

Policy and scripts: New .publish-include / .publish-forbidden (path blocks, refuse vs report string tiers, private tenant needles via secret). scripts/publish-guard.sh stages allowlisted tracked files and release assets, then runs four fail-closed guards (allowlist, forbidden paths, forbidden strings + optional --strict, pinned gitleaks). scripts/publish-mirror.sh validates the mirror target, plain-pushes the staged tree, and creates releases without overwriting existing tags.

Workflow: .github/workflows/mirror-publish.yml runs after Release completes (not on release: published), supports workflow_dispatch with dry-run default true, fetches release tags as read-only data at a pinned SHA, gates default-branch updates to the newest stable release (prereleases/older stables only add releases), and uses a GitHub App token scoped to MIRROR_REPO.

CI/docs: build.yml shellchecks the new scripts and runs three hermetic harnesses (publish-guard-verify, publish-mirror-verify, mirror-publish-workflow-verify). scripts/RELEASE_CHECKLIST.md documents the mirror step.

^{Reviewed by Cursor Bugbot for commit 0602d21. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[LukasWodka]

bugbot run

[LukasWodka]

bugbot run

[shujaatTracebloc]

Reviewed at a0d4231 (a0d4231 landed while I was writing a review of b54b5b5; this is against the new head).

What a0d4231 fixed — verified in the diff

• Untrusted checkout (CodeQL): single actions/checkout with no ref; the tag is fetched --no-tags --depth 1 into a detached $RUNNER_TEMP worktree and refused unless it resolves to workflow_run.head_sha (API commit for a dispatch) and the release's tagName matches. Guard, publisher and policy lists run from github.sha. Structural fix, and the current guard always runs.
• Prerelease overwrote the mirror: publish_tree=false on isPrerelease, tree push gated, release still mirrored --prerelease and pinned via the keep step (refuses on an empty mirror). Correct. This defect was only threaded on tracebloc/client#1034 but the code here was identical, so good that it moved in lockstep.
• $(…) swallowed refusals: --output FILE on target/tree, call sites run directly. Correct.
• gitleaks curl --tlsv1.2.

Still blocking

1. Installer (shell) is red on this head — the new scripts/tests/mirror-publish-workflow-verify.sh fails its own src case:

   fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
   FAIL src fetch (rc=1): … tag 'v1.2.3' resolves to 0503cb42… but the release was cut at HEAD
   mirror-publish-workflow-verify: 16 passed, 1 failed
   

   The fixture hands the step body the literal string HEAD as EXPECT_SHA from a directory that is not a git repo, so the (correct) sha comparison in the step refuses. Resolve the fixture's sha with git -C <origin> rev-parse HEAD before exporting it. The other 16 cases, including the four shape mutations, pass — the workflow logic is right; the test is wrong.

2. Tenant needles are printed to a public log — scripts/publish-guard.sh is byte-identical to the client PR on both heads (blob ff7349e), so the finding Bugbot threaded there at :394 applies here verbatim and tracebloc/cli is public: refuse/note at :390/:394 and GREP_ERR at :358/:370 interpolate the needle pattern into stdout, which the guard step tees into $GITHUB_STEP_SUMMARY. The --extra-forbidden needles are the customer identifiers the PUBLISH_FORBIDDEN_TENANTS secret exists to keep off this public repo; one hit publishes them on the run page, contradicting .publish-forbidden's "never the matched text". Print private needles as tier + ordinal or hash, never the pattern; scripts/tests/publish-guard-verify.sh will need the matching assertion, plus one that plants an extra-forbidden needle and asserts the pattern is absent from the output.

3. Test, bugbot / review and Bugbot are still pending on this head.

Will approve once the guard stops printing private needles and the head is green with Bugbot clean. Keep the shared scripts in lockstep with tracebloc/client#1034.

[LukasWodka]

Thanks — both blockers accepted.

• Fixture HEAD: already fixed on the current head (test(mirror): pin the fixture origin's HEAD so the tag-fetch case runs on a fresh runner) — the fixture resolves the sha with rev-parse before exporting it; CI on this head is the proof and is still running.
• Private needles printed to a public log: real, and not yet fixed — refuse/note and the GREP_ERR paths still interpolate the pattern. Next push: private (--extra-forbidden) needles print as tier + ordinal, never the pattern; committed-tier needles keep their name; the split assertion and a plant-a-private-needle-and-assert-absent case go in with it, in lockstep on both repos.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[LukasWodka]

bugbot run

[LukasWodka]

Status at 58571a5, against the three blockers:

1. Fixture HEAD — fixed in 512223b (the fixture pins the bare origin's HEAD to main and resolves the sha with rev-parse); Installer (shell) was green on 0528177.
2. Private needles printed to a public log — fixed in 0528177. needle_hits takes a shown name: a committed needle keeps its pattern, a needle from --extra-forbidden is private needle #N in every refuse/note/GREP_ERR line and in publish-guard-report.txt. publish-guard-verify.sh plants planted-tenant as a private needle, asserts the refusal names private needle #1 + the file:line, and greps the whole output and the report for the pattern — absent. scripts/publish-guard.sh is blob ef1aae2 here and on the client PR's current head; the two stay in lockstep.
3. Bugbot on 0528177 (an older stable tag replacing the mirror's default branch) — fixed in 58571a5: the tree is pushed only when the tag is the repo's releases/latest; anything else mirrors its release alone, pinned to the default branch's head, and an unreadable latest refuses. Tested in mirror-publish-workflow-verify.sh incl. a plan-body mutation. CI and Bugbot are running on this head.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 58571a5. Configure here.}

[saqlainsyed007]

Independent correctness pass on 58571a5 (I have not reviewed this PR before). No new defects found.

The guard is genuinely fail-closed: exit 0 only when all four guards ran and passed (RAN == GUARDS_EXPECTED self-check); every unreadable/empty/malformed list, unknown section, empty refuse tier, allowlist-matches-nothing, symlink, missing/crashed scanner, and non-empty out dir maps to "could not tell" (exit 2); and gitleaks' --exit-code 9 keeps a scanner crash from being read as clean. The publisher refuses an unset/malformed/self-pointing mirror and never force-pushes. Correct design for a public-mirror publish path.

@shujaatTracebloc's three blocking points all read as addressed on this head:

1. Installer (shell) is now green.
2. Private (--extra-forbidden) needles are named private needle #N, and needle_hits reduces hits.txt to file:line only, so the tenant patterns and matched text never reach the public run log / step summary.
3. Test, bugbot / review and Cursor Bugbot are all green/clean on this head.

Not approving — this is @shujaatTracebloc's standing change-request and only he can clear it. @shujaatTracebloc, over to you to confirm and re-review.

[LukasWodka]

bugbot run

[saqlainsyed007]

Re-reviewed on 0602d21 (delta since my 58571a5 pass: the null/non-boolean isPrerelease refusal + the errexit-safe guard-summary capture).

Both changes are correct and strictly tighten the fail-closed posture:

• The case "$PRERELEASE" in true|false) gate means only an explicit false arms the mirror tree-push; a missing or malformed value is refused before the newest-release question is asked. Right direction.
• The guard-tree step runs under set -uo pipefail, so bash publish-guard.sh … | tee … || rc=$?; exit "$rc" propagates the guard's own exit status through tee and re-raises it — a refusal reaches $GITHUB_STEP_SUMMARY and still fails the step. Not a fail-open.
• The new workflow-verify cases are mutation-proof and derived: the null-isPrerelease case asserts the exact ::error:: line, an empty $GITHUB_OUTPUT, and that releases/latest is never consulted; the guard-errexit case runs the body under the same errexit CI uses.

No new defects. This remains @shujaatTracebloc's standing change-request and only he can clear it — @shujaatTracebloc, the new head looks clean to me, over to you.