Skip to content

ci: audit orphaned Actions workflow identities - #203

Draft
seonghobae wants to merge 15 commits into
mainfrom
fix/orphan-workflow-registry-audit-20260812
Draft

ci: audit orphaned Actions workflow identities#203
seonghobae wants to merge 15 commits into
mainfrom
fix/orphan-workflow-registry-audit-20260812

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Purpose

Progress issue #202 with a read-only recurrence detector for GitHub Actions registry identities that remain active after their repository workflow source disappears. This PR does not disable workflows, add workflow-write authority, mint credentials, or restore historical one-shot workflow sources.

Exact current identity

  • protected/live base and PR-base snapshot: 7faf7a3b8a47980113982914000e724ab6a6cda5;
  • exact current head: 9ac4bc2de5bdba2e92be184e46a42ef55627bd77;
  • state: open / Draft / mergeable;
  • current delta is exactly scripts/ci/audit_workflow_registry.py, tests/test_workflow_registry_audit.py, tests/test_workflow_registry_partial_transfer.py, and tests/test_workflow_registry_state_integrity.py;
  • formal reviews: none;
  • unresolved review threads: none.

Implemented audit contract

The auditor is finite and read-only. It:

  • distinguishes present repository workflows, active orphan identities, disabled absent identities, GitHub-owned dynamic workflows, and workflow paths reserved by exact open-PR snapshots;
  • accepts only the exact documented workflow lifecycle states active, deleted, disabled_fork, disabled_inactivity, and disabled_manually; an unknown provider/future state fails closed instead of being silently treated as disabled;
  • binds output to one expected protected default-branch SHA and records observation time;
  • emits workflow ID/path/state/classification plus pagination receipts;
  • bounds GitHub response size, request time, workflow pagination, open-PR pagination, and PR-file pagination;
  • rejects malformed/noncanonical workflow identities, duplicate workflow IDs/PR numbers, incomplete/changing pagination, branch movement, registry movement, PR-reservation movement, malformed JSON, non-200 responses, permission/network failures, partial transfers, and unsafe path forms;
  • converts partial-transfer and ordinary network failures into one generic non-leaking AuditError whose __cause__ and __context__ are both absent;
  • rechecks the registry, open-PR reservation snapshot, and protected branch before emitting a result;
  • returns nonzero while active orphan identities remain.

It never disables a workflow, updates a branch, comments on a PR, requests a reviewer, publishes status, or changes repository state.

Test-first lineage added on this lane

Partial-transfer boundary

  • test-only RED 06f5fbf3a96f77bae9ba2e8bd674813773a761e9;
  • RED CI 31662731204, Python 3.14 job 94330765367: 1 failed, 1043 passed; raw IncompleteRead(11 bytes read, 100 more expected) escaped;
  • first fix d936687d94ec26911854ceac8cb87aba1e173909 caught the transfer failure;
  • second RED CI 31663075087, Python 3.14 job 94331810126: 1 failed, 1043 passed; the generic AuditError still retained IncompleteRead in __context__;
  • provenance-safe narrow fix 7d3edee15b184c308b7fa77670a239218989c57e records a bounded failure classification inside the handler and raises a fresh generic error only after leaving the active exception context.

Unknown lifecycle state boundary

A further audit found that any non-empty state string other than active was previously classified as disabled_absent. A future or malformed provider state could therefore be misrepresented as safely disabled.

  • test-only RED ee0a630d7f94f8f61d7a7c56e6e4b95799de3ff4;
  • RED CI 31663714501, Python 3.11 job 94333748752: exact checkout and Ruff passed; pytest reported 1 failed, 1044 passed because provider_future_state did not raise;
  • narrow fix / exact current head 9ac4bc2de5bdba2e92be184e46a42ef55627bd77 introduces one exact lifecycle-state validator and applies it in both the final audit and registry recheck snapshot.

Exact-current-head proof

On unchanged 9ac4bc2de5bdba2e92be184e46a42ef55627bd77:

  • repository CI 31663948791: completed success across wheel/sdist package acceptance and Python 3.10, 3.11, 3.12, 3.13, and 3.14;
  • Python 3.14 job 94334452103: checked out exact 9ac4bc2..., ran CPython 3.14.7, Ruff, 1045 passed, product guard and compileall successfully;
  • exact owned-production coverage remained 1758/1758 statements and 588/588 branches, zero misses/partials;
  • package archive verification, checksums and installed-wheel smoke succeeded;
  • SAST Semgrep 31663948803, Security Scan 31663948798, dependency-review job/check 94334454133, and required Strix run 31663947448 / check 94334450406 remain queued at the latest refetch. Queued evidence is non-passing.

Remaining acceptance

  1. Keep Draft while any exact-head required workflow is queued, skipped, unavailable, wrapper-only, or otherwise non-substantive.
  2. Require the Security Scan's actual immutable-pinned Dependency review action to execute and succeed. Aggregate wrapper green with that action skipped remains non-passing; the root fail-closed repair is read-only central PR ContextualWisdomLab/.github#897 or a freshly verified protected successor.
  3. Require an actual exact-head Strix semantic report and a current semantic/source review with zero valid unresolved findings; model/check/status transport alone is not approval or review evidence.
  4. Immediately before integration refetch exact head, live base, ancestry, live ruleset, every required workflow, reviews/findings/threads, and writer evidence.
  5. Merge only an unchanged accepted tree under normal governance, then use the protected auditor as immutable recurrence evidence for issue [Fleet incident] Disable orphaned implementation, release, and PR-integration workflow identities #202's separately authorized control-plane cleanup.

Direct lifecycle disablement remains a separate operator/control-plane action. Do not reintroduce removed YAML, create a self-deleting cleanup workflow, invent a PAT, weaken a required gate, or disable workflows by display name.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 14 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5759cab5-f0e5-4223-b551-430ff41acfb5

📥 Commits

Reviewing files that changed from the base of the PR and between 7faf7a3 and 9ac4bc2.

📒 Files selected for processing (4)
  • scripts/ci/audit_workflow_registry.py
  • tests/test_workflow_registry_audit.py
  • tests/test_workflow_registry_partial_transfer.py
  • tests/test_workflow_registry_state_integrity.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 12, 2026 23:14
@seonghobae
seonghobae marked this pull request as draft August 13, 2026 01:04

Copy link
Copy Markdown
Contributor Author

@coderabbitai review exact head 9ac4bc2de5bdba2e92be184e46a42ef55627bd77, focusing on read-only workflow-registry classification, bounded pagination, partial-transfer non-leakage, and lifecycle-state fail-closed behavior.

Copy link
Copy Markdown
Contributor Author

@opencode-agent review exact head 9ac4bc2de5bdba2e92be184e46a42ef55627bd77 and submit a formal verdict on registry identity, pagination consistency, unknown-state handling, and generic non-leaking failure semantics.

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review independently review exact head 9ac4bc2de5bdba2e92be184e46a42ef55627bd77 after a qualifying primary verdict and submit a formal current-head decision.

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

expected_head='9ac4bc2de5bdba2e92be184e46a42ef55627bd77'
actual_head="$(git rev-parse HEAD)"
printf 'expected_head=%s\nactual_head=%s\n' "$expected_head" "$actual_head"
git diff --no-ext-diff --unified=80 "${expected_head}^" "${expected_head}" -- \
  scripts/ci/audit_workflow_registry.py \
  tests/test_workflow_registry_audit.py \
  tests/test_workflow_registry_partial_transfer.py \
  tests/test_workflow_registry_state_integrity.py

printf '\n== Production module outline ==\n'
ast-grep outline scripts/ci/audit_workflow_registry.py --items all --view expanded

printf '\n== Focused test outlines ==\n'
ast-grep outline tests/test_workflow_registry_audit.py --items all --view expanded
ast-grep outline tests/test_workflow_registry_partial_transfer.py --items all --view expanded
ast-grep outline tests/test_workflow_registry_state_integrity.py --items all --view expanded

printf '\n== Focused implementation symbols ==\n'
rg -n -C 8 -e '^(def|class) ' \
  -e 'urlopen|IncompleteRead|HTTPError|URLError|AuditError' \
  -e 'pagination|page|workflow|lifecycle|state|active|disabled' \
  scripts/ci/audit_workflow_registry.py

printf '\n== Focused test assertions ==\n'
rg -n -C 6 -e 'IncompleteRead|__cause__|__context__' \
  -e 'future_state|lifecycle|disabled_absent|active_orphan' \
  -e 'pagination|page|partial' \
  tests/test_workflow_registry_audit.py \
  tests/test_workflow_registry_partial_transfer.py \
  tests/test_workflow_registry_state_integrity.py

Length of output: 71186


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Exact changed-file scope =='
git diff-tree --no-commit-id --name-status -r 9ac4bc2de5bdba2e92be184e46a42ef55627bd77

printf '%s\n' '== Live audit control flow =='
sed -n '124,570p' scripts/ci/audit_workflow_registry.py

printf '%s\n' '== Complete focused tests =='
cat -n tests/test_workflow_registry_audit.py
cat -n tests/test_workflow_registry_partial_transfer.py
cat -n tests/test_workflow_registry_state_integrity.py

printf '%s\n' '== HTTP request construction and potential mutation surface =='
ast-grep run --lang python \
  --pattern 'urllib.request.Request($URL, headers=$HEADERS, method=$METHOD)' \
  scripts/ci/audit_workflow_registry.py
rg -n -C 3 \
  -e 'method\s*=' \
  -e 'actions/workflows|branches/|pulls/|contents/' \
  -e '\b(POST|PUT|PATCH|DELETE)\b' \
  scripts/ci/audit_workflow_registry.py

printf '%s\n' '== State validation and classification call paths =='
rg -n -C 5 \
  -e '_require_workflow_state\s*\(' \
  -e 'classification\s*=' \
  -e 'collect_registry_pages\s*\(' \
  -e '_collect_open_pr_workflow_snapshot\s*\(' \
  scripts/ci/audit_workflow_registry.py

Length of output: 45409


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Final lines with end-of-line markers =='
tail -n 20 tests/test_workflow_registry_audit.py | sed -n 'l'

printf '%s\n' '== File length and final bytes =='
wc -l tests/test_workflow_registry_audit.py
tail -c 160 tests/test_workflow_registry_audit.py | od -An -tx1c

printf '%s\n' '== Standard-library tokenization check; repository code is not executed =='
python - <<'PY'
from pathlib import Path
import io
import tokenize

path = Path("tests/test_workflow_registry_audit.py")
source = path.read_bytes()
try:
    list(tokenize.tokenize(io.BytesIO(source).readline))
except (tokenize.TokenError, IndentationError, SyntaxError) as exc:
    print(f"tokenization_error={type(exc).__name__}: {exc}")
else:
    print("tokenization_ok")
PY

Length of output: 2111


@seonghobae: I reviewed exact head 9ac4bc2de5bdba2e92be184e46a42ef55627bd77.

I found no valid findings in the requested scope.

  • The auditor uses GET requests only. It does not expose a workflow mutation path.
  • Registry, open-PR, and PR-file collection have explicit pagination bounds. Incomplete, inconsistent, duplicate, and changing pagination fails closed.
  • IncompleteRead becomes a generic AuditError after the exception handler. The focused regression test verifies that __cause__ and __context__ are both None.
  • _require_workflow_state allows only the five documented lifecycle states. Both final classification and registry recheck validation use it.
  • Classification keeps present repository workflows, active PR reservations, active orphans, disabled absent identities, and dynamic workflows separate.
  • The audit rechecks pull-request reservations, registry identity, and the protected branch before output.

The last verification command could not run od, but the completed file-end output confirmed that the focused test file closes correctly at Line 328.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant