PF-192: Pipeline job running > hour fails with expired creds

[gruntwork-ci]

Implements plan for Linear issue PF-192.

Plan source: https://github.com/gruntwork-io-team/linear-claude-plans/blob/main/plans/PF-192.md

---

Claude's implementation summary

Implementation complete across four repos.

## `pipelines-gitlab-init`

- **New `scripts/lib/credentials.sh`** — sourceable library with no side effects on source:
  - `pipelines_mint_gruntwork_read_token <log>` — exchanges `APERTURE_OIDC_TOKEN` for a Gruntwork read token, echoes it on stdout. Writes the node script's output to a `mktemp` file (removed after sourcing) instead of `credentials.sh` in the repo root, and saves/restores `set -x` so debug/trace logging never prints token material.
  - `pipelines_verify_gruntwork_read_token <token> <log>` — the GitHub API probe, returning a status instead of exiting.
  - `pipelines_publish_gruntwork_read_token <token> <source>` — exports the token plus a new `PIPELINES_GRUNTWORK_READ_TOKEN_SOURCE` (`aperture` or `ci_variable`) and rewrites (rather than appends) both keys in `$GITLAB_ENV` and `build.env`. `build.env` is only touched when it already exists, so a refresh in an execute job doesn't drop a file into the customer's checked-out repo.
- **New `scripts/refresh-credentials.sh`** — standalone entrypoint designed to be sourced. No `exec`, no global `set -e`/`set -u`, no `glab` calls, no note collapsing, no `pipelines-gitlab-actions` clone, no CLI install. Skips refresh when the token came from a CI variable or when `APERTURE_OIDC_TOKEN`/`API_BASE_URL` are absent; on a mint failure it falls back to the inherited token (and only hard-fails when there is no token at all). Works both sourced and executed.
- **`scripts/init.sh`** — now sources the library and calls into it, preserving today's precedence (customer CI variable wins and is verified; otherwise mint from Aperture). Tracing is disabled across the whole token section and around the token-bearing `git clone` URL.
- **`scripts/pipelines-credentials.mjs`** — output path now comes from `PIPELINES_CREDENTIALS_OUTPUT_FILE` (default unchanged) and uses `writeFileSync` instead of `appendFileSync`; the inaccurate dotenv comment is replaced.
- **`README.md`** — documents the scripts and the refresh pattern.

Verified with `bash -n`, `shellcheck -x` (clean), `node --check`, and functional smoke tests covering: fresh mint through a sourced call, stale-key replacement in both env files, `ci_variable` skip, mint failure with and without an inherited token, direct execution, and a `set -x` run confirming no token appears in trace output.

## `pipelines`

- `internal/actions/gitlabyml/gitlabdynamicchildren.go` — new `defaultApertureOIDCAudience` const and `resolveApertureOIDCAudience()` (overridable via `APERTURE_OIDC_AUDIENCE`, mirroring `CI_JOB_IMAGE`); `TemplateJobData` gains `ApertureOIDCAudience`.
- `internal/actions/gitlabyml/gitlabinitdynamicchildren.go` — same audience threaded into the reinit template data.
- All six job templates (`execute`, `provision_new_accounts`, `baseline_core_accounts`, `baseline_new_accounts`, `setup_delegated_repo`, `reinit`) now declare an `APERTURE_OIDC_TOKEN` id_token and clone `pipelines-gitlab-init` + source `refresh-credentials.sh` immediately before the `pipelines-gitlab-actions` clone.
- Tests: all nine golden YAML blocks in `gitlabdynamicchildren_test.go` updated; new `TestCustomApertureOIDCAudience`; new `gitlabinitdynamicchildren_test.go` with a golden `TestReinit` (the reinit path previously had no test) and `TestReinitCustomApertureOIDCAudience`.
- `ARCHITECTURE.md` — new cheat-sheet row for the generated GitLab child-pipeline YAML surface.

**`go test` could not be run here**: the module depends on the private `github.com/gruntwork-io-team/pipelines-hooks`, which is neither cached nor fetchable in this environment (no GitHub credentials), so the package fails at `setup failed` before compiling. To compensate I rendered every template through a standalone stdlib-only harness using the identical funcmap and data shapes; this caught and fixed a `{{ ..ApertureOIDCAudience }}` scoping error in three templates, and the golden strings in the tests were produced from that harness's exact byte output. `gofmt` is clean. The Go changes still need a real `go test ./internal/...` run in CI.

## `gitlab-pipelines-workflows`

- New `.refresh_credentials` anchor (clone `pipelines-gitlab-init`, source `refresh-credentials.sh`), used as the first `script:` entry of `status check` ahead of its clone. `Drift Detection` / `Unlock Unit` / `Unlock All` are untouched since they already run `*init_pipelines` first.
- `pipelines_gitlab_init_ref` default bumped `v1.1.3` → `v1.2.0`.
- YAML parses and the anchor expands correctly (nested list, same shape as the existing `*init_pipelines` usage).

**Not done:** `pipelines_binary_version` is still `v0.58.0`. The plan calls for bumping it to the new CLI release, but that version doesn't exist yet and pinning a guessed tag would break the workflow. Per the plan's own release ordering (`pipelines-gitlab-init` tag → CLI release → workflows bump), this pin must be bumped to the actual CLI release before the child-pipeline half of the fix reaches customers. As it stands this PR fixes `status check` only.

## `docs`

- `docs/2.0/docs/pipelines/architecture/security-controls.md` — the GitLab tab of "Token Strategy" now states that Gruntwork GitHub access tokens expire after one hour and are minted per job from the job's OIDC identity, and that a customer-supplied `PIPELINES_GRUNTWORK_READ_TOKEN` CI variable is used as-is and never refreshed.
- `ci-workflows.md` left alone: its GitLab tab doesn't enumerate jobs or steps, so there's nothing to keep in sync.

All repos in the plan's `target_repos` were available; nothing was skipped for a missing repo.

---

Related PRs in this change set

• pipelines: https://github.com/gruntwork-io-team/pipelines/pull/666
• pipelines-gitlab-init: PF-192: Pipeline job running > hour fails with expired creds pipelines-gitlab-init#11
• gitlab-pipelines-workflows: https://gitlab.com/gruntwork-io/pipelines-workflows/-/merge_requests/71

Summary by CodeRabbit

• Documentation
  • Clarified that automatically generated GitHub access tokens expire after one hour.
  • Documented that each job obtains its own token through its OIDC identity.
  • Clarified that user-provided access tokens are used unchanged, with expiration managed externally.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview	Aug 7, 2026 9:39pm

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3f0f9237-6479-4beb-b211-ca6048a7d650

📥 Commits

Reviewing files that changed from the base of the PR and between a13cbc1 and 299bfc1.

📒 Files selected for processing (1)

• docs/2.0/docs/pipelines/architecture/security-controls.md

🚧 Files skipped from review as they are similar to previous changes (1)

• docs/2.0/docs/pipelines/architecture/security-controls.md

---

📝 Walkthrough

Walkthrough

The security controls documentation describes one-hour, per-job runtime-generated Gruntwork GitHub tokens. It also clarifies that user-provided PIPELINES_GRUNTWORK_READ_TOKEN values remain unchanged.

Changes

Token lifetime documentation

Layer / File(s)	Summary
Document token lifetime behavior docs/2.0/docs/pipelines/architecture/security-controls.md	Documents per-job OIDC token generation, one-hour expiration, and externally controlled lifetimes for user-provided tokens.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the primary issue addressed: pipeline jobs running longer than one hour due to expired credentials.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude-pr/PF-192-31202520184

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/2.0/docs/pipelines/architecture/security-controls.md`:
- Line 66: Update the sentence describing Gruntwork GitHub access tokens to
hyphenate “short-lived” when it modifies “tokens,” leaving the rest of the
content unchanged.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5577e4ee-5e27-4002-8521-1934bd21450d

📥 Commits

Reviewing files that changed from the base of the PR and between 8904ebf and a13cbc1.

📒 Files selected for processing (1)

• docs/2.0/docs/pipelines/architecture/security-controls.md