diff --git a/.claude/skills/testing-github-actions/SKILL.md b/.claude/skills/testing-github-actions/SKILL.md new file mode 100644 index 0000000..64122c6 --- /dev/null +++ b/.claude/skills/testing-github-actions/SKILL.md @@ -0,0 +1,142 @@ +--- +name: testing-github-actions +description: > + Use this skill when developing or changing a composite action in this repo + (nsheaps/github-actions) and you need to PROVE it works end-to-end before + merging — especially when a new input/output or behavior was added. Covers + pinning a consumer workflow (e.g. in nsheaps/.github) at the action's feature + branch or commit SHA, triggering the run, and verifying the new code actually + executed. Recall it whenever a consumer workflow "passed" but you're not sure + it exercised your changes, or when you see/expect an "Unexpected input(s)" + warning. Triggers: "test the action", "prove the action works", "why did the + workflow pass", "test before merge", "pin action to branch/SHA". +--- + +# Testing GitHub Actions cross-repo + +Actions in this repo are referenced by **other repos** (e.g. `nsheaps/.github`, +`nsheaps/.org`) as `uses: nsheaps/github-actions/.github/actions/@main`. +That `@main` is the crux of testing: a consumer workflow pinned to `@main` runs +the action **as it exists on `main` right now** — i.e. the _pre-merge_ version, +not the code on your branch. Pushing changes to a feature branch does nothing +to what `@main` consumers see. + +To test action changes end-to-end you must point a consumer at your branch/SHA. + +## The trap: a green check does NOT mean your code ran + +GitHub Actions **silently ignores undefined inputs** to an action. If you pass +an input the resolved action version doesn't declare, you get a non-fatal +annotation: + +``` +Warning: Unexpected input(s) 'mode', valid inputs are ['token', 'owner', ...] +``` + +The step still runs and **succeeds** — using only the inputs the action does +declare. So if you add a `mode: export` input on a branch, then run a consumer +pinned to `@main` (which has no `mode` yet), the job goes green while running +the _old_ code path and ignoring `mode` entirely. The green check is a false +positive — it proves nothing about your change. + +> Real example from this repo: `.github`'s `apply-repo-settings.yaml` passed +> `mode: export` to `apply-repo-settings@main` before the `mode` input was +> merged. The export job "passed" — but `@main` had no `mode` input, so it +> silently ran the old **apply** path. Nothing exported. + +**Rule of thumb:** if you changed an action's `inputs`/`outputs`/behavior, a +consumer pinned to `@main` cannot validate it. Pin to your ref first. + +## How to test against your branch + +1. **Push your action changes** to its feature branch in this repo + (`nsheaps/github-actions`). Note the branch name or the commit SHA. + +2. **Point a consumer workflow at that ref.** In the consuming repo (often + `nsheaps/.github`), edit every `uses:` line for the action under test: + + ```yaml + # from: + - uses: nsheaps/github-actions/.github/actions/apply-repo-settings@main + # to (branch): + - uses: nsheaps/github-actions/.github/actions/apply-repo-settings@claude/zealous-heisenberg-wfaxhk + # or (immutable, preferred for a definitive proof): + - uses: nsheaps/github-actions/.github/actions/apply-repo-settings@ + ``` + + A branch ref re-resolves on every run (good while iterating). A SHA is + immutable (good for a final, reproducible proof). Reusable workflows + (`uses: org/repo/.github/workflows/x.yaml@ref`) pin the same way. + +3. **Trigger the consumer.** Use whatever the workflow listens for — push to a + branch, `workflow_dispatch`, or `repository_dispatch`. Prefer a non-default + branch / dry-run input so a test run can't mutate production state. + +4. **Watch the run and read the logs** (see "Observing runs" below). Don't + trust the green ✓ alone — open the step and confirm the new behavior. + +5. **Verify your inputs were actually accepted** — the key check: + - **No `Unexpected input(s)` warning** for your new inputs → the resolved + action version declares them → your version ran. + - Look for a log line that only your new code path emits (add one if + needed, e.g. `echo "mode=$MODE"`), and confirm the expected branch ran. + - Check the step's `outputs` if your change added one. + +6. **Revert the pin before merging the consumer.** Change every `uses:` back + to `@main` (or the next released tag). Shipping a consumer pinned to a + feature branch is a landmine — the branch gets deleted and the consumer + breaks. Make reverting part of the consumer PR's final commit. + +## Order of operations across repos + +Because consumers reference `@main`, the action change must land first: + +1. Test the action on its branch via a pinned consumer (steps above). +2. Merge the **action** PR (`nsheaps/github-actions`) to `main`. +3. Re-point consumers to `@main`, confirm green, merge the **consumer** PRs + (`nsheaps/.github`, which then syncs to managed repos including `.org`). + +Never merge a consumer that depends on unreleased action behavior while it's +still pinned to `@main` — it'll run stale code in production. + +## Observing runs + +Authenticated access is needed to read run logs. With `gh` + a token +(`GH_TOKEN`), from any repo: + +```bash +# latest runs of a workflow +gh run list --repo nsheaps/.github --workflow apply-repo-settings.yaml --limit 5 +# watch a run to completion +gh run watch --repo nsheaps/.github +# full logs (grep for your markers / warnings) +gh run view --repo nsheaps/.github --log | grep -iE 'unexpected input|mode=|export' +``` + +In web sessions the git remote is a local proxy — use `gh api --hostname +github.com ...` or the GitHub MCP tools instead of `gh run` if `gh` isn't +configured for github.com. If no token is available, fall back to the run's +**Actions** page in the GitHub UI and read the step logs there. + +## Quick checklist + +- [ ] Action change pushed to its branch; SHA/branch noted. +- [ ] Consumer's `uses:` repinned from `@main` to the branch/SHA (every occurrence). +- [ ] Run triggered on a safe branch / with dry-run where possible. +- [ ] Logs show **no** `Unexpected input(s)` warning for new inputs. +- [ ] A log line / output unique to the new code path is present. +- [ ] Consumer repinned back to `@main` before its PR merges. +- [ ] Action PR merged before consumer PR. + +## Gotchas + +- **Undefined inputs never fail** — only warn. Absence of the warning is your + signal the right version ran; presence means you're hitting a stale ref. +- **Composite actions are fetched at the pinned ref**, including their own + internal `uses:` and scripts. Editing a script on your branch only takes + effect for consumers pinned to that branch. +- **`@main` is a moving target** — a consumer test that passed yesterday can + break when `main` moves. Pin to a SHA for a stable proof. +- **Don't dry-run-only and call it done** — `dry-run: true` proves the action + parses inputs and reaches its logic, but not that the real API + calls/side-effects work. Do at least one non-dry-run on a throwaway target. diff --git a/.editorconfig b/.editorconfig index f0b85f6..7eee1bc 100644 --- a/.editorconfig +++ b/.editorconfig @@ -16,3 +16,6 @@ trim_trailing_whitespace = true [*.sh] indent_size = 2 + +[*.py] +indent_size = 4 diff --git a/.github/actions/apply-repo-settings/README.md b/.github/actions/apply-repo-settings/README.md index a8626ba..312fb23 100644 --- a/.github/actions/apply-repo-settings/README.md +++ b/.github/actions/apply-repo-settings/README.md @@ -4,26 +4,90 @@ An ephemeral, in-workflow alternative to the [repository-settings GitHub App](ht Reads `.github/settings.yml` from the current repo and applies the supported sections to the target repo via the GitHub API, using a GitHub App token (so the action runs only when invoked — no always-on bot, no third-party service). +It also runs in reverse: with `mode: export` it reads the repo's **live** state and writes it back into `settings.yml`, so manual changes made in the GitHub UI can be captured back into source control (see [Reverse sync](#reverse-sync-mode-export) below). + +Supported sections: `repository`, `rulesets`, `labels`, `collaborators`, `teams`. **Apply is non-destructive for every section** — it creates/updates what's listed but never deletes rulesets/labels or revokes collaborator/team access that exists on the repo but isn't in `settings.yml`. (A future opt-in prune mode will let `settings.yml` be authoritative per section.) Not modeled: `environments` and classic branch protection (`branches:`) — this org uses rulesets. + ## Why not self-host the upstream app? -The upstream app is a Probot webhook server — it's designed to listen for `push` events on a long-running process. Adapting it for ephemeral one-shot runs is more invasive than building a minimal applier from scratch. This action covers the two sections we actually use (`repository:` config and `rulesets:`) in ~170 lines of bash + `gh api`. Other sections (`labels`, `collaborators`, `teams`, `environments`, legacy `branches`) are not yet implemented — labels are managed via a separate sync today. +The upstream app is a Probot webhook server — it's designed to listen for `push` events on a long-running process. Adapting it for ephemeral one-shot runs is more invasive than building a minimal applier from scratch. This action covers the sections we use in bash + `gh api`. The `labels` section **replaces** the previous `github-label-sync` flow — org-standard labels now live in the org settings template and flow through the same per-repo `settings.yml` pipeline as everything else. ## Inputs -| Input | Required | Default | Description | -| --------------- | -------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------- | -| `token` | yes | — | GitHub token with `Administration: write` on the target repo (typically from `checkout-as-app` or `github-app-auth`). | -| `owner` | no | current owner | Target repo owner. | -| `repo` | no | current repo | Target repo name. | -| `settings-file` | no | `.github/settings.yml` | Path to the YAML to apply. | -| `dry-run` | no | `false` | Print what would change without applying. | -| `sections` | no | `repository,rulesets` | Comma-separated section names to apply. | +| Input | Required | Default | Description | +| --------------- | -------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | +| `token` | yes | — | GitHub token with `Administration: write` on the target repo (typically from `checkout-as-app` or `github-app-auth`). | +| `owner` | no | current owner | Target repo owner. | +| `repo` | no | current repo | Target repo name. | +| `settings-file` | no | `.github/settings.yml` | Path to the YAML to apply (or write, in export mode). | +| `dry-run` | no | `false` | Print what would change without applying / writing. | +| `mode` | no | `apply` | `apply` (file → repo) or `export` (repo → file, reverse sync). | +| `sections` | no | `repository,rulesets,labels,collaborators,teams` | Comma-separated section names to sync. | The action does **not** do any templating or placeholder substitution on `settings-file` — it applies the YAML as-is. If you need env-var-based substitution (e.g. resolving a GitHub App ID into `bypass_actors[].actor_id`), render the file upstream (e.g. with `envsubst`) before invoking this action. ## Outputs -- `summary` — JSON object: `{ repository, rulesets_created, rulesets_updated, rulesets_unchanged }`. +- `summary` — JSON object. In `apply` mode: `{ repository, rulesets_created, rulesets_updated, rulesets_unchanged, labels_applied, collaborators_applied, teams_applied }`. In `export` mode: `{ mode, changed, sections }`. +- `changed` — export mode only: `"true"` if `settings-file` was modified, else `"false"`. Empty in apply mode. + +## Reverse sync (`mode: export`) + +`mode: export` flips the direction: instead of pushing the YAML to the repo, it reads the repo's live state from the API and **deep-merges it into** `settings-file`, then reports whether the file changed via the `changed` output. It lets a workflow capture changes made directly in the GitHub UI back into source control instead of letting them drift. + +The merge (`merge_live.py`, ruamel) is **additive and lossless**, so `settings-file` becomes a faithful, growing record of the repo: + +- The file **wins on existing scalars** — export never overwrites a value already in the file. +- Live state only **adds what's missing**: new top-level keys, and within identity-matched list items (rulesets by `name`, rules by `type`, `bypass_actors` by `(actor_id, actor_type)`, labels by `name`, collaborators by `username`, teams by `name`) it adds missing sub-keys and list entries (e.g. a newly-added bypass actor or collaborator). +- New list items (e.g. a ruleset created in the UI) are **appended**; **nothing already in the file is removed** — entries not yet applied to the repo survive. +- **Comments and formatting are preserved.** + +Per-section live capture: + +- **`repository`** → the full settable key set (`has_issues`, `allow_*`, `default_branch`, `description`, `homepage`, `topics`, `private`, merge-commit settings, …; nulls dropped). No longer narrowed to keys already in the file. +- **`rulesets`** → every ruleset, normalized to `name`/`target`/`enforcement`/`conditions`/`bypass_actors`/`rules`. +- **`labels`** → every label as `{ name, color, description }`. +- **`collaborators`** → **direct** collaborators (`?affiliation=direct`; org-inherited access left alone) as `{ username, permission }`, normalizing `role_name` (`read`→`pull`, `write`→`push`). +- **`teams`** → teams with repo access as `{ name, permission }` (`name` = team slug). + +Classic branch protection (legacy `branches:`) and `environments` are **not** exported. The file is rewritten only when the merge actually changes content, so an up-to-date file is left untouched. + +### Auto-capturing live drift + +Run export from a workflow, then commit whatever it captured. The canonical wiring triggers export on a push that changes the workflow file (so the captured `settings.yml` lands in the same branch/PR as a preview) and on a weekly schedule (to catch UI ruleset edits that fire no push): + +```yaml +on: + push: + paths: ['.github/workflows/apply-repo-settings.yaml'] + schedule: + - cron: '0 0 * * 0' + +jobs: + export: + runs-on: ubuntu-latest + steps: + - uses: nsheaps/github-actions/.github/actions/checkout-as-app@main + id: checkout + with: + app-id: ${{ secrets.AUTOMATION_GITHUB_APP_ID }} + private-key: ${{ secrets.AUTOMATION_GITHUB_APP_PRIVATE_KEY }} + - uses: nsheaps/github-actions/.github/actions/apply-repo-settings@main + id: export + with: + token: ${{ steps.checkout.outputs.token }} + mode: export + sections: rulesets + - if: steps.export.outputs.changed == 'true' + run: | + git add .github/settings.yml + git commit -m "chore: sync rulesets into settings.yml" + git push origin "HEAD:${{ github.ref_name }}" +``` + +> There is no `repository_ruleset` Actions trigger, so pure UI ruleset edits (with no corresponding push) are caught by the weekly schedule. For immediate ruleset-edit-driven exports, dispatch this action from an org-level ruleset webhook via `repository_dispatch`. + +The full wired-up workflow (push/schedule gating + commit-back) lives at [`nsheaps/.github`'s `apply-repo-settings.yaml` template](https://github.com/nsheaps/.github/blob/main/ansible/templates/.github/workflows/apply-repo-settings.yaml). ## Setting up the GitHub App @@ -85,11 +149,15 @@ jobs: - if no ruleset with that name exists → `POST /repos/{owner}/{repo}/rulesets` - if one exists and content matches → no-op (logged as `unchanged`) - if one exists and content differs → `PUT /repos/{owner}/{repo}/rulesets/{id}` +- **`labels:` list** → `POST` to create, `PATCH /repos/{owner}/{repo}/labels/{name}` to update color/description. +- **`collaborators:` list** → `PUT /repos/{owner}/{repo}/collaborators/{username}` with `{ permission }`. +- **`teams:` list** → `PUT /orgs/{owner}/teams/{slug}/repos/{owner}/{repo}` with `{ permission }`. -Rulesets that exist on the repo but are absent from the YAML are **not deleted** (safer default — repos may have UI-created rulesets we don't want to wipe). If you want destructive sync, add a `--prune` mode in a follow-up. +**Nothing is ever deleted or revoked.** Rulesets, labels, collaborators, and teams that exist on the repo but are absent from the YAML are left untouched (safer default — repos may have UI-created rulesets/labels, and we never want to silently revoke access). A future opt-in prune mode (tracked by `TODO(prune)` markers in `action.sh`) will make `settings.yml` authoritative per section once export is proven. ## Limitations - The upstream repository-settings app reads from the default branch only. This action reads from the workflow's checkout, so it works on any branch — useful for testing changes in a PR before merging. -- No support yet for `labels`, `collaborators`, `teams`, `environments`. Those are tracked separately. +- `environments` and classic branch protection (`branches:`) are not supported. +- `collaborators`/`teams` apply needs the App to have `Administration: write` (and org `Members` for teams). Export only reads. - `bypass_actors[].actor_id` for `RepositoryRole` must be the role's numeric ID (community-documented: 1=read, 2=triage, 3=write, 4=maintain, 5=admin). Custom roles have user-assigned IDs. diff --git a/.github/actions/apply-repo-settings/__pycache__/merge_live.cpython-311.pyc b/.github/actions/apply-repo-settings/__pycache__/merge_live.cpython-311.pyc new file mode 100644 index 0000000..0ee1843 Binary files /dev/null and b/.github/actions/apply-repo-settings/__pycache__/merge_live.cpython-311.pyc differ diff --git a/.github/actions/apply-repo-settings/action.sh b/.github/actions/apply-repo-settings/action.sh index 52f778d..a6822ce 100755 --- a/.github/actions/apply-repo-settings/action.sh +++ b/.github/actions/apply-repo-settings/action.sh @@ -1,13 +1,40 @@ #!/usr/bin/env bash -# Apply Repo Settings — reads SETTINGS_FILE and applies the supported -# top-level sections to {OWNER}/{REPO} via the GitHub API. +# Apply Repo Settings — syncs the supported top-level sections between +# SETTINGS_FILE and {OWNER}/{REPO} via the GitHub API. +# +# Direction is controlled by MODE: +# apply (default) — read SETTINGS_FILE and apply it to the repo. +# export — read the repo's live state and deep-merge it INTO +# SETTINGS_FILE (reverse sync). The merge is additive and +# lossless (see merge_live.py): the file wins on existing +# scalars, live only ADDS missing keys/list-items, comments +# are preserved, and nothing already in the file is removed. +# A calling workflow can then commit the result. # # Supported sections: -# repository → PATCH /repos/{owner}/{repo} -# rulesets → POST/PUT/DELETE /repos/{owner}/{repo}/rulesets +# repository → apply: PATCH /repos/{owner}/{repo} +# export: GET /repos/{owner}/{repo} (full settings key set) +# rulesets → apply: POST/PUT /repos/{owner}/{repo}/rulesets +# export: GET /repos/{owner}/{repo}/rulesets (deep-merged) +# labels → apply: POST/PATCH /repos/{owner}/{repo}/labels +# export: GET /repos/{owner}/{repo}/labels (ALL labels) +# collaborators → apply: PUT /repos/{owner}/{repo}/collaborators/{user} +# export: GET /repos/{owner}/{repo}/collaborators?affiliation=direct +# teams → apply: PUT /orgs/{org}/teams/{slug}/repos/{owner}/{repo} +# export: GET /repos/{owner}/{repo}/teams +# +# PRUNE BEHAVIOR: BOTH directions are **non-destructive**. apply creates/updates +# what's in SETTINGS_FILE but never deletes rulesets/labels or revokes +# collaborators/teams absent from it; export only ADDS to SETTINGS_FILE and +# never removes entries (so pending settings not yet on the repo survive). This +# matches the github-label-sync (--allow-added-labels) behavior the labels +# section replaces, and avoids auto-revoking access. TODO(prune): once export is +# proven to capture full live state reliably, add an opt-in prune mode so +# SETTINGS_FILE can be made authoritative per-section. # -# Not supported (yet): labels, collaborators, teams, environments, branches. -# Those are handled by other workflows in nsheaps/.github today. +# Not modeled: environments, and classic branch protection (the `branches:` +# section / legacy protection API) — this org uses rulesets, so export captures +# rulesets, not classic protection. set -euo pipefail @@ -16,11 +43,18 @@ set -euo pipefail : "${REPO:?REPO required}" : "${SETTINGS_FILE:=.github/settings.yml}" : "${DRY_RUN:=false}" -: "${SECTIONS:=repository,rulesets}" +: "${MODE:=apply}" +: "${SECTIONS:=repository,rulesets,labels,collaborators,teams}" if [[ ! -f "$SETTINGS_FILE" ]]; then - echo "::error file=$SETTINGS_FILE::settings file not found" - exit 1 + if [[ "$MODE" == "export" ]]; then + # Export can bootstrap a settings file from the repo's live state. + mkdir -p "$(dirname "$SETTINGS_FILE")" + printf '# Generated by apply-repo-settings (export mode).\n' > "$SETTINGS_FILE" + else + echo "::error file=$SETTINGS_FILE::settings file not found" + exit 1 + fi fi log() { echo "::group::$*"; } @@ -98,6 +132,14 @@ want_section() { [[ ",$SECTIONS," == *",$target,"* ]] } +# URL-encode a path segment (label names can contain spaces, etc.). +_uri() { jq -rn --arg s "$1" '$s|@uri'; } + +# Normalize a GitHub permission/role_name to a PUT-compatible value +# (the collaborators/teams PUT APIs want pull/push/admin/maintain/triage; +# the read endpoints report read/write for the first two). +_norm_perm() { jq -rn --arg p "$1" '{"read":"pull","write":"push"}[$p] // $p'; } + ############################################################################### # repository: PATCH /repos/{owner}/{repo} ############################################################################### @@ -184,23 +226,239 @@ apply_rulesets() { endlog } +############################################################################### +# labels: create/update by name. Non-destructive — extra repo labels are kept +# (replaces github-label-sync --allow-added-labels). TODO(prune): opt-in delete. +############################################################################### +apply_labels() { + log "labels" + local count + count="$(yq '.labels | length // 0' "$SETTINGS_FILE")" + if [[ "$count" == "0" || "$count" == "null" ]]; then + info "no labels block, skipping"; endlog; return + fi + + local existing + existing="$(gh api --paginate "/repos/${OWNER}/${REPO}/labels" --jq '[.[] | {name, color, description}]')" + info "found $(echo "$existing" | jq 'length') existing label(s)" + + local i + for ((i=0; i/dev/null + else + info "update label: $name" + [[ "$DRY_RUN" == "true" ]] || \ + api PATCH "/repos/${OWNER}/${REPO}/labels/${name_enc}" "$body" >/dev/null + fi + LABELS_APPLIED+=("$name") + done + endlog +} + +############################################################################### +# collaborators: PUT each user's permission. Non-destructive — never removes +# collaborators absent from the file (no silent access revocation). +# TODO(prune): opt-in removal once export is proven. +############################################################################### +apply_collaborators() { + log "collaborators" + local count + count="$(yq '.collaborators | length // 0' "$SETTINGS_FILE")" + if [[ "$count" == "0" || "$count" == "null" ]]; then + info "no collaborators block, skipping"; endlog; return + fi + + local i + for ((i=0; i/dev/null + COLLAB_APPLIED+=("$user") + done + endlog +} + +############################################################################### +# teams: PUT each team's repo permission (org endpoint). Non-destructive. +# TODO(prune): opt-in removal once export is proven. +############################################################################### +apply_teams() { + log "teams" + local count + count="$(yq '.teams | length // 0' "$SETTINGS_FILE")" + if [[ "$count" == "0" || "$count" == "null" ]]; then + info "no teams block, skipping"; endlog; return + fi + + local i + for ((i=0; i/dev/null + TEAMS_APPLIED+=("$slug") + done + endlog +} + +############################################################################### +# export: merge the repo's LIVE state INTO SETTINGS_FILE (reverse sync). +# +# Builds a "source" JSON of the live repo state (in the settings.yml schema) for +# the requested sections, then deep-merges it into the existing file via +# merge_live.py (ruamel, comment-preserving). The merge is ADDITIVE and lossless: +# the file wins on existing scalars, live only ADDS missing keys/list-items, and +# nothing already in the file is removed (so pending settings not yet applied to +# the repo survive). The file is rewritten only when content actually changes. +############################################################################### + +# Full set of repository settings keys we capture (drops nulls). This is the +# *complete* settable set — export no longer narrows to keys already in the file, +# so settings.yml can represent the full repo configuration. +_REPO_KEYS='{has_issues, has_projects, has_wiki, has_downloads, is_template, default_branch, allow_squash_merge, allow_merge_commit, allow_rebase_merge, allow_auto_merge, allow_update_branch, delete_branch_on_merge, squash_merge_commit_title, squash_merge_commit_message, merge_commit_title, merge_commit_message, description, homepage, topics, private, web_commit_signoff_required, archived}' + +# Build the live-state source object (only requested sections) on stdout. +build_export_source() { + local obj='{}' + + if want_section "repository"; then + local live rep + live="$(api GET "/repos/${OWNER}/${REPO}")" + rep="$(jq -c "$_REPO_KEYS | with_entries(select(.value != null))" <<<"$live")" + obj="$(jq -c --argjson v "$rep" '.repository = $v' <<<"$obj")" + fi + + if want_section "rulesets"; then + local list arr='[]' id full norm + list="$(gh api --paginate "/repos/${OWNER}/${REPO}/rulesets" --jq '[.[].id]')" + for id in $(echo "$list" | jq -r '.[]'); do + full="$(gh api "/repos/${OWNER}/${REPO}/rulesets/${id}")" + norm="$(echo "$full" | jq '{ + name, target, enforcement, conditions, + bypass_actors: [ (.bypass_actors // [])[] | {actor_id, actor_type, bypass_mode} ], + rules: [ (.rules // [])[] | {type} + (if has("parameters") and .parameters != null then {parameters} else {} end) ] + }')" + arr="$(jq -c --argjson i "$norm" '. + [$i]' <<<"$arr")" + done + obj="$(jq -c --argjson v "$arr" '.rulesets = $v' <<<"$obj")" + fi + + if want_section "labels"; then + local arr + arr="$(gh api --paginate "/repos/${OWNER}/${REPO}/labels" --jq '[.[] | {name, color, description}]')" + obj="$(jq -c --argjson v "$arr" '.labels = $v' <<<"$obj")" + fi + + if want_section "collaborators"; then + local arr + arr="$(gh api --paginate "/repos/${OWNER}/${REPO}/collaborators?affiliation=direct" \ + --jq '[.[] | {username: .login, permission: (.role_name // "push")}]')" + arr="$(jq -c 'map(.permission |= ({"read":"pull","write":"push"}[.] // .))' <<<"$arr")" + obj="$(jq -c --argjson v "$arr" '.collaborators = $v' <<<"$obj")" + fi + + if want_section "teams"; then + local arr + arr="$(gh api --paginate "/repos/${OWNER}/${REPO}/teams" \ + --jq '[.[] | {name: .slug, permission: (.permission // "push")}]')" + arr="$(jq -c 'map(.permission |= ({"read":"pull","write":"push"}[.] // .))' <<<"$arr")" + obj="$(jq -c --argjson v "$arr" '.teams = $v' <<<"$obj")" + fi + + echo "$obj" +} + +_ensure_ruamel() { + python3 -c 'import ruamel.yaml' 2>/dev/null && return 0 + info "installing ruamel.yaml..." + python3 -m pip install --quiet --disable-pip-version-check ruamel.yaml >/dev/null 2>&1 \ + || python3 -m pip install --quiet --break-system-packages ruamel.yaml >/dev/null 2>&1 \ + || { echo "::error::failed to install ruamel.yaml (needed for export merge)"; return 1; } +} + +run_export() { + echo "Exporting $OWNER/$REPO live state into $SETTINGS_FILE (dry-run=$DRY_RUN, sections=$SECTIONS)" + local script_dir + script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + + _ensure_ruamel + + log "build live source" + local source_json + source_json="$(build_export_source)" + if [[ "$DRY_RUN" == "true" ]]; then + echo "$source_json" | jq '.' + fi + endlog + + log "merge into $SETTINGS_FILE" + local write_flag="" out + [[ "$DRY_RUN" == "true" ]] || write_flag="--write" + out="$(echo "$source_json" | python3 "$script_dir/merge_live.py" --file "$SETTINGS_FILE" $write_flag)" + echo "$out" + CHANGED="false" + [[ "$out" == *"changed=true"* ]] && CHANGED="true" + endlog + + summary="$(jq -nc --arg changed "$CHANGED" --arg sections "$SECTIONS" \ + '{mode: "export", changed: $changed, sections: ($sections | split(","))}')" + echo "Summary: $summary" + { + echo "summary=$summary" + echo "changed=$CHANGED" + } >> "$GITHUB_OUTPUT" + + { + echo "## Apply Repo Settings (export) — \`$OWNER/$REPO\`" + echo + echo "- **changed**: \`$CHANGED\`" + echo "- **sections**: \`$SECTIONS\`" + echo + echo "Target: \`$SETTINGS_FILE\` · Dry run: \`$DRY_RUN\` (merge is additive — the file is never narrowed)" + } >> "$GITHUB_STEP_SUMMARY" +} + ############################################################################### # main ############################################################################### +if [[ "$MODE" == "export" ]]; then + run_export + exit 0 +fi + CREATED=() UPDATED=() UNCHANGED=() +LABELS_APPLIED=() +COLLAB_APPLIED=() +TEAMS_APPLIED=() REPO_CHANGED="false" echo "Applying $SETTINGS_FILE to $OWNER/$REPO (dry-run=$DRY_RUN, sections=$SECTIONS)" -if want_section "repository"; then - apply_repository -fi - -if want_section "rulesets"; then - apply_rulesets -fi +if want_section "repository"; then apply_repository; fi +if want_section "rulesets"; then apply_rulesets; fi +if want_section "labels"; then apply_labels; fi +if want_section "collaborators"; then apply_collaborators; fi +if want_section "teams"; then apply_teams; fi # Summary summary="$(jq -nc \ @@ -208,7 +466,10 @@ summary="$(jq -nc \ --argjson created "$(printf '%s\n' "${CREATED[@]:-}" | jq -R . | jq -s 'map(select(length>0))')" \ --argjson updated "$(printf '%s\n' "${UPDATED[@]:-}" | jq -R . | jq -s 'map(select(length>0))')" \ --argjson unchanged "$(printf '%s\n' "${UNCHANGED[@]:-}" | jq -R . | jq -s 'map(select(length>0))')" \ - '{repository: $repo_changed, rulesets_created: $created, rulesets_updated: $updated, rulesets_unchanged: $unchanged}' + --argjson labels "${#LABELS_APPLIED[@]}" \ + --argjson collaborators "${#COLLAB_APPLIED[@]}" \ + --argjson teams "${#TEAMS_APPLIED[@]}" \ + '{repository: $repo_changed, rulesets_created: $created, rulesets_updated: $updated, rulesets_unchanged: $unchanged, labels_applied: $labels, collaborators_applied: $collaborators, teams_applied: $teams}' )" echo "Summary: $summary" @@ -222,6 +483,9 @@ echo "summary=$summary" >> "$GITHUB_OUTPUT" echo "- **rulesets created**: ${#CREATED[@]}${CREATED:+: ${CREATED[*]}}" echo "- **rulesets updated**: ${#UPDATED[@]}${UPDATED:+: ${UPDATED[*]}}" echo "- **rulesets unchanged**: ${#UNCHANGED[@]}${UNCHANGED:+: ${UNCHANGED[*]}}" + echo "- **labels applied**: ${#LABELS_APPLIED[@]}" + echo "- **collaborators applied**: ${#COLLAB_APPLIED[@]}" + echo "- **teams applied**: ${#TEAMS_APPLIED[@]}" echo echo "Source: \`$SETTINGS_FILE\` · Dry run: \`$DRY_RUN\` · Sections: \`$SECTIONS\`" } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/actions/apply-repo-settings/action.yml b/.github/actions/apply-repo-settings/action.yml index 1abe130..5f4d9d1 100644 --- a/.github/actions/apply-repo-settings/action.yml +++ b/.github/actions/apply-repo-settings/action.yml @@ -30,16 +30,31 @@ inputs: required: false default: 'false' + mode: + description: | + Direction of the sync: + apply — read settings-file and apply it to the repo (default). + export — read the repo's live config/rulesets from the API and write + them back into settings-file (reverse sync). A calling + workflow can then commit the result to capture changes made + in the GitHub UI back into source control. + required: false + default: 'apply' + sections: - description: 'Comma-separated list of top-level keys to apply. Defaults to "repository,rulesets". Add others (labels, collaborators, teams) as we extend support.' + description: 'Comma-separated list of top-level keys to sync. Supported: repository, rulesets, labels, collaborators, teams. Apply is non-destructive (never deletes/revokes) for every section.' required: false - default: 'repository,rulesets' + default: 'repository,rulesets,labels,collaborators,teams' outputs: summary: - description: 'JSON summary of changes applied. Keys: repository, rulesets_created, rulesets_updated, rulesets_unchanged.' + description: 'JSON summary of changes. In apply mode keys are: repository, rulesets_created, rulesets_updated, rulesets_unchanged, labels_applied, collaborators_applied, teams_applied. In export mode: mode, changed, sections.' value: ${{ steps.apply.outputs.summary }} + changed: + description: 'In export mode: "true" if settings-file was modified, else "false". Empty in apply mode.' + value: ${{ steps.apply.outputs.changed }} + runs: using: 'composite' steps: @@ -93,5 +108,6 @@ runs: REPO: ${{ inputs.repo }} SETTINGS_FILE: ${{ inputs.settings-file }} DRY_RUN: ${{ inputs.dry-run }} + MODE: ${{ inputs.mode }} SECTIONS: ${{ inputs.sections }} run: ${{ github.action_path }}/action.sh diff --git a/.github/actions/apply-repo-settings/merge_live.py b/.github/actions/apply-repo-settings/merge_live.py new file mode 100644 index 0000000..b99245f --- /dev/null +++ b/.github/actions/apply-repo-settings/merge_live.py @@ -0,0 +1,150 @@ +#!/usr/bin/env python3 +"""Merge live repo state (JSON on stdin) INTO an existing settings YAML file. + +Used by apply-repo-settings *export* mode. The file is the merge TARGET and the +live state is the SOURCE; semantics are deliberately additive and lossless: + + - mappings: recurse + - scalars: TARGET wins (the file's value is kept; export + never overwrites a value already in the file) + - identity-keyed lists: match items by identity key, recurse into matches + (so missing sub-keys / list entries — e.g. a new + bypass actor or collaborator — are added), append + source-only items to the END, and KEEP every + target-only item (things in the file not yet on + the repo are never removed) + - other lists: concat + dedupe + +Comments and formatting are preserved via ruamel.yaml round-trip. The file is +rewritten ONLY when the merge actually changes content (and only with --write), +so an up-to-date file is left byte-for-byte untouched. + +Prints `changed=true` or `changed=false`. +""" +from __future__ import annotations + +import argparse +import io +import json +import sys + +from ruamel.yaml import YAML +from ruamel.yaml.comments import CommentedMap + +# (path-tuple-pattern, identity-key) — "*" matches any single segment. +IDENTITY_KEYS: list[tuple[tuple[str, ...], object]] = [ + (("rulesets",), "name"), + (("rulesets", "*", "rules"), "type"), + (("rulesets", "*", "bypass_actors"), ("actor_id", "actor_type")), + (("labels",), "name"), + (("collaborators",), "username"), + (("teams",), "name"), +] + + +def _path_match(pattern: tuple[str, ...], path: tuple[str, ...]) -> bool: + return len(pattern) == len(path) and all( + p == "*" or p == q for p, q in zip(pattern, path) + ) + + +def _lookup_identity(path: tuple[str, ...]): + for pattern, key in IDENTITY_KEYS: + if _path_match(pattern, path): + return key + return None + + +def _ident(item, key) -> tuple: + if isinstance(key, str): + return (item.get(key),) + return tuple(item.get(k) for k in key) + + +def deep_merge(source, target, path: tuple[str, ...] = ()): + """Merge source into target (target wins on scalars). Mutates target.""" + if source is None: + return target + if target is None: + return source + + if isinstance(source, dict) and isinstance(target, dict): + for k, sv in source.items(): + if k in target: + target[k] = deep_merge(sv, target[k], path + (k,)) + else: + target[k] = sv + return target + + if isinstance(source, list) and isinstance(target, list): + key = _lookup_identity(path) + if key is None: + for item in source: + if item not in target: + target.append(item) + return target + index_by_id = { + _ident(item, key): i + for i, item in enumerate(target) + if isinstance(item, dict) + } + for s_item in source: + if not isinstance(s_item, dict): + if s_item not in target: + target.append(s_item) + continue + sid = _ident(s_item, key) + if sid in index_by_id: + idx = index_by_id[sid] + target[idx] = deep_merge(s_item, target[idx], path + ("*",)) + else: + target.append(s_item) + return target + + # scalar / mismatched types: target wins (never overwrite the file's value). + return target + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("--file", required=True, help="settings YAML file (merge target)") + ap.add_argument("--write", action="store_true", help="write the merged result back") + args = ap.parse_args() + + source = json.load(sys.stdin) + + yaml = YAML() + yaml.preserve_quotes = True + yaml.width = 4096 # don't wrap long descriptions + # Match the org_settings_merge.py canonical style so re-serializing a file + # produced by that merger doesn't reindent (which would be spurious drift). + yaml.indent(mapping=2, sequence=4, offset=2) + + try: + with open(args.file, encoding="utf-8") as fh: + target = yaml.load(fh) + except FileNotFoundError: + target = None + if target is None: + target = CommentedMap() + + def dump(obj) -> str: + buf = io.StringIO() + yaml.dump(obj, buf) + return buf.getvalue() + + before = dump(target) # round-tripped original (formatting-neutral baseline) + merged = deep_merge(source, target) + after = dump(merged) + + changed = before != after + if changed and args.write: + with open(args.file, "w", encoding="utf-8") as fh: + fh.write(after) + + print("changed=true" if changed else "changed=false") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/.github/actions/apply-repo-settings/merge_live_test.py b/.github/actions/apply-repo-settings/merge_live_test.py new file mode 100644 index 0000000..36faed7 --- /dev/null +++ b/.github/actions/apply-repo-settings/merge_live_test.py @@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +"""Unit tests for merge_live.py (apply-repo-settings export deep-merge). + +Run: python3 merge_live_test.py (requires ruamel.yaml) +""" +import json +import os +import subprocess +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) + +from merge_live import deep_merge # noqa: E402 + +MERGE = os.path.join(HERE, "merge_live.py") + + +def merge_file(yaml_text, source_obj, write=True): + """Run merge_live.py against yaml_text with source_obj; return (changed, new_text).""" + with tempfile.NamedTemporaryFile("w", suffix=".yml", delete=False) as fh: + fh.write(yaml_text) + path = fh.name + try: + args = [sys.executable, MERGE, "--file", path] + if write: + args.append("--write") + out = subprocess.run( + args, input=json.dumps(source_obj), capture_output=True, text=True, check=True + ).stdout + changed = "changed=true" in out + with open(path, encoding="utf-8") as fh: + return changed, fh.read() + finally: + os.unlink(path) + + +class DeepMergeLogic(unittest.TestCase): + def test_scalar_target_wins(self): + # file (target) value is kept; live (source) never overwrites it. + out = deep_merge({"has_issues": False}, {"has_issues": True}) + self.assertEqual(out["has_issues"], True) + + def test_adds_missing_key(self): + out = deep_merge({"a": 1, "b": 2}, {"a": 9}) + self.assertEqual(out, {"a": 9, "b": 2}) + + def test_nested_mapping_recurse(self): + out = deep_merge( + {"repository": {"has_issues": False, "default_branch": "main"}}, + {"repository": {"has_issues": True}}, + ) + self.assertEqual(out["repository"], {"has_issues": True, "default_branch": "main"}) + + def test_rulesets_identity_merge_append_and_keep(self): + source = { + "rulesets": [ + { + "name": "protect", + "bypass_actors": [ + {"actor_id": 5, "actor_type": "RepositoryRole"}, + {"actor_id": 99, "actor_type": "Integration"}, + ], + "rules": [{"type": "deletion"}, {"type": "non_fast_forward"}], + }, + {"name": "new-from-ui", "target": "branch"}, + ] + } + target = { + "rulesets": [ + {"name": "pending-only", "target": "branch"}, # file-only, must stay + { + "name": "protect", + "bypass_actors": [{"actor_id": 5, "actor_type": "RepositoryRole"}], + "rules": [{"type": "deletion"}], + }, + ] + } + out = deep_merge(source, target) + names = [r["name"] for r in out["rulesets"]] + # pending-only kept, protect kept in place, new-from-ui appended at end. + self.assertEqual(names, ["pending-only", "protect", "new-from-ui"]) + protect = out["rulesets"][1] + actor_ids = sorted(a["actor_id"] for a in protect["bypass_actors"]) + self.assertEqual(actor_ids, [5, 99]) # missing actor added + rule_types = sorted(r["type"] for r in protect["rules"]) + self.assertEqual(rule_types, ["deletion", "non_fast_forward"]) # missing rule added + + def test_labels_collaborators_teams_identity(self): + out = deep_merge( + { + "labels": [{"name": "bug", "color": "ff0000"}, {"name": "new", "color": "00ff00"}], + "collaborators": [{"username": "alice", "permission": "admin"}], + "teams": [{"name": "core", "permission": "push"}], + }, + { + "labels": [{"name": "bug", "color": "d73a4a"}], # color kept (target wins) + "collaborators": [{"username": "alice", "permission": "pull"}], + "teams": [{"name": "core", "permission": "pull"}], + }, + ) + labels = {label["name"]: label["color"] for label in out["labels"]} + self.assertEqual(labels["bug"], "d73a4a") # target wins + self.assertEqual(labels["new"], "00ff00") # appended + self.assertEqual(out["collaborators"][0]["permission"], "pull") # target wins + self.assertEqual(out["teams"][0]["permission"], "pull") # target wins + + def test_target_only_never_removed(self): + out = deep_merge({"labels": []}, {"labels": [{"name": "keep", "color": "x"}]}) + self.assertEqual([label["name"] for label in out["labels"]], ["keep"]) + + +class CliBehavior(unittest.TestCase): + def test_changed_true_when_adding(self): + changed, text = merge_file( + "repository:\n has_issues: true\n", + {"repository": {"has_issues": False, "default_branch": "main"}}, + ) + self.assertTrue(changed) + self.assertIn("default_branch: main", text) + self.assertIn("has_issues: true", text) # not overwritten to false + + def test_changed_false_and_idempotent(self): + yaml_text = ( + "repository:\n has_issues: true\n" + "labels:\n - name: bug\n color: \"d73a4a\"\n" + ) + changed, text = merge_file( + yaml_text, + {"repository": {"has_issues": True}, "labels": [{"name": "bug", "color": "d73a4a"}]}, + ) + self.assertFalse(changed) + self.assertEqual(text, yaml_text) # byte-for-byte untouched when nothing to add + + def test_dry_run_does_not_write(self): + yaml_text = "repository:\n has_issues: true\n" + changed, text = merge_file( + yaml_text, {"repository": {"default_branch": "main"}}, write=False + ) + self.assertTrue(changed) # would change + self.assertEqual(text, yaml_text) # but file untouched (no --write) + + def test_comments_preserved(self): + yaml_text = ( + "# top-of-file comment\n" + "rulesets:\n" + " # keep this ruleset comment\n" + " - name: protect # inline\n" + " enforcement: active\n" + ) + changed, text = merge_file( + yaml_text, + {"rulesets": [{"name": "protect", "enforcement": "active", "target": "branch"}]}, + ) + self.assertTrue(changed) # added target: branch + self.assertIn("# top-of-file comment", text) + self.assertIn("# keep this ruleset comment", text) + self.assertIn("# inline", text) + self.assertIn("target: branch", text) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/workflows/check.yaml b/.github/workflows/check.yaml index 4d6e6fd..dbd04ef 100644 --- a/.github/workflows/check.yaml +++ b/.github/workflows/check.yaml @@ -117,3 +117,18 @@ jobs: id: grype - uses: ./.github/actions/lint-gitleaks id: gitleaks + + test: + name: Test + runs-on: ubuntu-latest + steps: + - name: Checkout Code + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - name: Install mise + uses: jdx/mise-action@v2 + + - name: Run action unit tests + run: mise run test diff --git a/.mise/tasks/test b/.mise/tasks/test new file mode 100755 index 0000000..9e75c2c --- /dev/null +++ b/.mise/tasks/test @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +#MISE description="Run action unit tests (apply-repo-settings merge_live)" +set -euo pipefail + +echo "Installing test deps (ruamel.yaml)..." +python3 -m pip install --quiet --disable-pip-version-check ruamel.yaml \ + || python3 -m pip install --quiet --break-system-packages ruamel.yaml + +echo "Running apply-repo-settings merge_live tests..." +python3 .github/actions/apply-repo-settings/merge_live_test.py diff --git a/mise.toml b/mise.toml index b040e19..0c8a591 100644 --- a/mise.toml +++ b/mise.toml @@ -1,5 +1,6 @@ [tools] node = "lts" +python = "3.12" # for action unit tests (merge_live) # Format tools editorconfig-checker = "latest"