Skip to content

Fork review gate: enforce the owner's rules (#82, #99, #100) - #943

Merged
scuttletoby merged 5 commits into
mainfrom
fork-ci/review-gate-enforce
Nov 22, 2026
Merged

scuttletoby merged 5 commits into
mainfrom
fork-ci/review-gate-enforce

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 22, 2026 •

Copy link
Copy Markdown
Owner

Part of #82, part of #99, part of #100 (see "Why not Fixes" below).

What

The review block of .github/fork-pr-queue.json goes from shadow to enforce, for exactly the rules the owner decided on 2026-11-14:

One rule change, found by the first dry run. .github/** made two bookkeeping lists critical: .github/fork-seams.json (seam announcements) and .github/fork-ci-tests.txt (the test list). So the gate rule names the .github files one by one: .github/*/*/** (every subfolder), *.md, *.td, *.yml, *.yaml and each JSON or text setting file. A test walks .github and fails when a new top-level file is in neither the gate rule nor the two-entry bookkeeping list.

What enforce does on the fork today: nothing yet. No fork workflow calls the head gate. There is no gate App (#81) and no head-gate environment, and the main ruleset does not require head_check. The switch puts the config in its decided end state, so the gate enforces it from its first run once those parts exist.

Fix round 1 (after the merge-path and state lens reports)

  1. Every rule hit gives its lenses (merge-path major). Before, review_judge.judge took the lenses of the heaviest rule hit only. So app code plus a gate file needed merge-path + state and no code: the app code went unread by its own lens, against Owner question: extra reviewer for workflow and gate changes? #82's "one extra review". Now the weight is still the heaviest hit (unchanged for every other user of it: min_lenses, the summary line, the queue), but the required lenses are the union over every rule hit, in rule order. The TypeScript mirror (apps/server/src/fork/reviewRules/plan.ts) and the shared vectors (review.vectors.json) change the same way.
    • A protected path (the config, the lens prompts, .github/workflows/**) counts for the first critical rule and for the rule it matches itself. Under a catch-all normal rule that rule's lens is now also required. This only ever adds lenses; on the fork the protected paths sit in the gate rule, so nothing extra.
    • Release notes entry; the union reaches callers in the release after ci/v22.
  2. fork-kit/review/** and fork-kit/gh-apps/** join the gate rule. fork-kit/review is the lens-format checker and the lens skill: it decides what a lens report looks like, which is what the gate reads (Owner question: should the merge gate block a PR when a reviewer reported a blocker or major finding? #99). fork-kit/gh-apps handles the gate App's key and tokens (Owner question: one gate App for both repos, or one each? #81): a change there changes who can post as the gate. A new test lists every fork-kit folder as gate or not and fails when a new folder is in neither list, the same drift pattern as .github.
  3. Old releases fail closed. The config now writes "override_authors": []. Checked by running the old releases' head_check on this config:
  4. Rollback. README "Roll back" and this body: rollback = the "mode" line plus the two tests that pin it (ForkConfigRules.test_it_is_in_enforce in head_check.test.py, ForkConfig.test_it_parses_in_enforce_with_the_owners_numbers_and_no_person_yet in review_judge.test.py). The rollback PR changes the config, so it is judged by the enforce rules of its base: it needs clean merge-path and state reports within the round cap. New test test_the_rollback_to_shadow_needs_the_gate_lenses. If the gate itself is broken, the owner turns off the ruleset entry for head_check (an owner step).
  5. Owner question: may your own GitHub login leave the agent machine? #78 note. README: until agents stop posting as the owner (Owner question: may your own GitHub login leave the agent machine? #78), the round cap and the "pushed after the review" refusal cannot be passed by a person's override. The only ways out are a new clean report on the current head or a new PR. They become a person's decision only after Owner question: may your own GitHub login leave the agent machine? #78.

Fix round 2 (after the round-2 merge-path and state lens reports)

Both round-2 reports said "not safe" only because CI was red; all five round-1 fixes held.

  1. Three stale tests updated to the union (the CI failures). plan.test.py test_a_critical_path_needs_two_independent_lenses: the kit's plan for a storage file plus an app file now lists data-1, data-2, code, and checks each lens's prompt line. review_guide.test.py test_findings_rounds_and_override_authors_come_after_the_same_release: expects the moved sentence "are in ci/v21 and later", with the version worked out from the fork-ci: enforce review findings, rounds and critical gate files #740 release note. test_the_enforce_pull_request_needs_only_the_input_lenses: the config alone now needs data-1, data-2, code (it also counts for the catch-all), and the test checks both rules hit.
  2. Release floor. pr-queue README step 1, the fork paragraph and the Owner question: extra reviewer for workflow and gate changes? #82 release-note upgrade text: a caller of a config in enforce pins the first release after ci/v22 or later; ci/v21 and ci/v22 accept the config but judge mixed changes the old, looser way. The fork's gate caller (Owner question: one gate App for both repos, or one each? #81) starts at that floor. New test test_a_caller_in_enforce_pins_the_release_with_every_rule_hit keeps these sentences in step with the release note's "First in the release after" version.
  3. README step 5. The enforce pull request asks for the critical lenses plus the lenses of every other rule its paths hit. The test checks this sentence.
  4. docs/fork/gate-trust.md joins the gate rule. It holds the owner's paste commands for the gate App's key and tokens, a gate setting like fork-kit/gh-apps. The rest of docs/fork stays normal with the code lens; no light, no-lens rule was added for docs/fork or .github/fork-ci-tests.txt. Tests: test_only_the_gate_trust_page_of_docs_fork_is_critical and the critical-path list in review_judge.test.py; the head gate test test_a_workflow_or_gate_change_needs_the_extra_review in head_check.test.py.

Not done (outside this round): one release-note line saying an old T3 build plans mixed changes without code, so the gate asks for one more lens run (the state lens's minor 2; fails closed).

State question for SkyRPG. The same action reads SkyRPG's .github/pr-queue.json. It has no review block, so the path rules are off there and the union changes none of its verdicts. Checked read-only.

Dry run: the newest 20 merged fork pull requests

How it was run: the gate's own head_check.decide(), with the config forced to each mode. Base = the squash commit's first parent, head = the PR head as merged. Comments and reviews came from gh (read-only). "Before" = this PR's previous head (0795fae), "after" = this head.

PR Paths judged before → after Enforce before → after Why (after) Right?
#940 critical [merge-path, state] → critical [+ code] pass → refuse fork-kit/heavy, model-guard, pc-agents app files; only merge-path and state read it, no code yes: the fix
#939 critical → critical [+ code] refuse merge-path reported a major: ("Fixes" wording), never cleared (#99) yes
#938 normal [code] pass yes
#928 normal [code] refuse state says NOT SAFE (#99); shadow refuses too yes
#927 normal [code] refuse the lenses read different heads; shadow refuses too yes
#922 critical → critical [+ code] refuse code says NOT SAFE (#99); shadow refuses too yes
#920 critical [merge-path, state] → critical [+ code] pass → refuse desk and pc-agents app files; no code report yes: the fix
#917 normal [code] pass yes
#912 normal [code] pass yes
#909 normal [code] refuse the head is not the change read; shadow refuses too yes
#907 normal [code] pass yes
#874 critical → critical [+ code] refuse only merge-path; state and code missing yes
#847 normal [code] refuse lens b says NOT SAFE; the owner-login override does not count (#78) yes
#845 normal [code] → critical [merge-path, state] refuse changes fork-kit/review (now gate); lens named a, round 3 without approach yes
#843 normal [code] refuse only state; code missing yes
#840 normal [code] refuse old free-format lens comments; shadow refuses too yes
#835 normal [code] → critical [merge-path, state, code] refuse changes fork-kit/gh-apps (now gate); old free-format comments; shadow refuses too yes
#833 critical → critical [+ code] refuse no lens comment at all yes
#832 normal [code] refuse no lens comment at all yes
#827 normal [code] refuse lenses security, state; the config names code yes

Two verdicts flip (#940, #920), both because app code was read only by the gate lenses. Fix round 2 changes no row: none of the 20 changed docs/fork/gate-trust.md (#939, #928, #927, #874 and #827 change other docs/fork pages, which stay normal). Shadow refuses 6 of the 20 (#928 #927 #922 #909 #840 #835); they merged because the fork runs no gate.

Cost, kept on purpose: a gate change that also adds a line to .github/fork-ci-tests.txt or touches docs/fork now needs code too (#874, #835). No light, no-lens rule for them: docs/fork holds runbooks the owner and agents paste from, so a changed command would merge unread. A light rule for exactly .github/fork-ci-tests.txt would be safe, but is not part of this PR.

Tests

Each fix has a test that fails without it:

State question

  • The switch lands while PRs are open. Nothing reads the fork's config in CI today, so no open PR gets stuck. Once a gate caller exists, the gate judges each open PR at its next run (dispatch the backfill right after). A PR with old free-format comments or old lens names then fails, and needs new reports in the lens: <config name> · reviewer: … · read at: <head sha> format on its current head. A PR that mixes app code and gate files now also needs a code report.
  • An older release reads the config. ci/v20 and older refuse it (fail closed, checked above); ci/v21+ accept it. The union needs the release after ci/v22; ci/v21 and ci/v22 still take the heaviest rule's lenses only, so the README sets the caller floor at the first release after ci/v22.
  • It runs again / stops half-way. A config change judges nothing by itself. A run that stops leaves the previous verdict, and the next event or a backfill judges again.
  • Rollback to shadow. See fix 4: the mode line plus its two tests, judged by the gate lenses.

Why not "Fixes"

Each issue is covered as config plus tests. On the fork, though, nothing enforces it until a gate caller, the shared gate App (#81) and a ruleset entry for head_check exist. The "a person's override" half of #99 and #100 waits on #78.

Docs

  • fork-ci/actions/pr-queue/README.md: "How the gate judges" (the union), step 1 and step 5 (caller floor: the first release after ci/v22; the fork-kit gate folders and docs/fork/gate-trust.md and why; the enforce PR asks for every rule's lenses; the Owner question: may your own GitHub login leave the agent machine? #78 note), step 6 (the fork's rollback).
  • fork-ci/actions/head-check/README.md: "Lenses by changed path" says every rule hit gives its lenses.
  • fork-ci/RELEASE-NOTES.md: new entry with an upgrade note (including the caller floor).

General vs project

  • General: the union rule in review_judge.py, its TS mirror and the shared vectors apply to every project that uses the review rules. The README notes and the drift-test pattern (walk the folder, every entry sorted) apply to any project.
  • Project (the fork as a user of its own kit): .github/fork-pr-queue.json and the tests that pin what it decides for the fork.

Checks run

At head 20ab00c (fix round 2), every test that reads the review config, the README or the plan:

  • review_judge.test.py: 54 OK
  • review_config.test.py: 51 OK
  • review_glob.test.py: 8 OK
  • lens.test.py: 80 OK
  • pr_queue.test.py: 419 OK; pr_queue_laws.test.py: 3 OK
  • head_check.test.py: 262 OK, 1 skipped
  • head_gate.test.py: 108 OK
  • review_guide.test.py: 34 OK
  • fork-kit/review/tests: plan.test.py 27 OK, comments.test.py 36 OK, eval.test.py 28 OK
  • fork-contracts.test.py: 716 ok; fork-owned-paths.test.py: 52 ok
  • vitest apps/server/src/fork/reviewRules: 60 passed (2 files)
  • vp fmt --check on the config, the release notes and the pr-queue README: clean
  • fork-gate-trust-docs.test.py: not run locally (that page did not change); CI runs it
  • actionlint: not run, because no workflow file changed

🤖 Generated with Claude Code

The review block of .github/fork-pr-queue.json moves from shadow to enforce:
workflow and gate-setting files need the merge-path and state lenses (#82),
an open blocker or major fails the head (#99), the approach lens joins from
round 3 and round 5 needs a person (#100). No override_authors yet (#78).

A dry run over the newest merged pull requests showed the old .github/**
rule made seam announcements and test-list lines critical, so plain app
changes needed the gate lenses and no code lens. The gate rule now names the
.github files one by one; the seam list and the test list stay normal, and a
test fails when a new .github file is in neither list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

lens: merge-path · reviewer: lens agent (Opus 5.5) · read at: 0795fae
verdict: not safe to merge
covered: the diff (.github/fork-pr-queue.json, both test files, the pr-queue README); the owner decisions on #82, #99, #100 and #78; review_judge.judge and review_glob (dialect: * stays in one segment and matches dot-names, case-sensitive, ** takes zero or more segments); renames and deletes (both paths are judged); every reader of the config (no fork workflow calls fork-head-gate.yml or fork-pr-queue.yml; the main ruleset requires fork-ci-result, "No database schema changes" and "Outside pull request", not head_check; the SkyRPG queue reads its own config file); the test list and seam list now marked normal.
checks: ran review_judge.test.py (51 OK) and head_check.test.py (258 OK, 1 skipped). Added an unsorted .github/fork-new-gate.json: the new sorting test fails with the right message, and the "Fork action and fleet walls tests" job that runs it has no path filter, so it runs on every same-repo pull request. Probe on the fork's real config: an app change plus a one-line edit to .github/SECURITY.md is judged critical with lenses merge-path and state only; a code lens report with an open major that was already posted still fails the head.
state: runs again: the config judges nothing by itself and the same head gets the same verdict. Stopped half-way: the previous verdict stays. Existing state: no gate caller and no ruleset entry, so open pull requests and main's merges and releases are not blocked now. Rollback: the tests pin enforce, so a rollback pull request must also change two tests (minor 2).
blocker: none
major: a change that touches any gate file no longer needs the code lens for the rest of its diff, so the config is looser than #82 ("one extra review"). Example: an app change plus one line in .github/SECURITY.md passes with merge-path and state alone; nobody has to read the app code for correctness. The judge takes lenses only from the highest weight hit (fork-ci/actions/pr-queue/review_judge.py:172), and the gate rule lists merge-path and state (.github/fork-pr-queue.json:41). The pull request uses this same argument ("the gate lenses alone never read app code") to make the seam list and test list normal, but the general case stays open, and review_judge.test.py pins it for app plus seam list plus ci.yml. One possible fix in general code: require the lenses of every rule that matched, which gives code plus merge-path plus state on mixed changes and leaves pure gate or pure app changes as they are. Or the owner decides that the gate lenses replace code on mixed changes, and the README says so.
minor: 1. fork-kit/** stays normal (.github/fork-pr-queue.json:39 only adds fork-ci/), and so do fork-kit/gh-apps (where the gate App's key is kept, from the gate-trust design) and fork-kit/review (the lens-format checker). Both are arguably "gate settings" under #82. The old rule had the same gap, so this is a follow-up, not caused by this pull request. 2. Rolling back to shadow is not "one PR sets mode shadow": head_check.test.py:1589 and review_judge.test.py:451 assert enforce and fail until they change too (that change is critical anyway). The README "Roll back" step and the PR body's State question should say so. 3. I checked the newly normal files: changing the test list or the seam list cannot make fork-ci-result pass for a gate test that did not run. An unlisted test, a test that was listed and then deleted, a missing run record, or a run by file that does not name the test all fail, and a may skip line lives in the test file under fork-ci/ or .github/scripts (critical). So their normal weight is fine.

@scuttletoby

Copy link
Copy Markdown
Owner Author

lens: state · reviewer: state lens agent (Opus 5.5) · read at: 0795fae
covered: the diff (.github/fork-pr-queue.json, the README "Turning it on" step 5, the new tests in review_judge.test.py and head_check.test.py); the owner decisions on issues 82, 99, 100 and 78; review_config.py, review_glob.py, head_check.py and README sections "Findings and rounds", "Lens comments" and "Turning it on" at this head; review_config.py and head_check.py at the tags ci/v14 to ci/v22.
checks: ran review_judge.test.py (51 OK) and head_check.test.py (258 OK, 1 skipped) at this head; mutation: removed .github/fork-upstream.json from the gate rule, and test_every_github_file_is_critical_but_the_two_bookkeeping_lists failed naming that file (restored after); read the glob dialect: .github/*/*/** matches every file in a subfolder (** takes zero segments), and the drift test judges a subfolder as <folder>/x; checked ci/v15 to ci/v20 head_check.py: no mention of findings or rounds.
state: nothing reads this config in CI today, so merging changes no verdict. Once a gate caller exists: an older release (ci/v15 to ci/v20) parses the config and enforces the path rules (82), but silently does not judge findings or rounds (99, 100): it fails open, with no summary row saying so (minor 1). ci/v14 or older refuses the block (fail closed, the README owner step). Runs again: a config change judges nothing; a re-run of the gate on the same head gives the same answer. Stopped half-way: the previous verdict stays until the next event or a backfill. Open pull requests with old free-format lens comments fail at their next gate run; their ways out are delete, not a lens:, or an override (an override from the owner's login still covers a read, only clearing findings and the cap needs a listed person). A review older than a push fails ("not the change read"), unless an override for the new head vouches for it (minor 3). Rollback to shadow: the rollback pull request changes the config, so it needs clean merge-path and state reports within 4 rounds; if the gate itself is broken so that every head fails, that pull request fails too and only the README owner step (a direct push of the revert) gets out. Round counting: edited and hidden lens comments still count, so editing cannot lower the round; deleting can (minor 2).
blocker: none
major: none
minor: 1. Older releases fail open on 99 and 100. ci/v15 to ci/v20 know the keys rounds and findings (KEYS in their review_config.py) but their head_check.py never reads them, so a caller pinned to one of them, or a rollback that pins back to one (README step 6, fork-ci/actions/pr-queue/README.md:419), runs this "enforce" config with no findings or round judging and no row saying so. The PR body says "a caller must pin ci/v21 or later", but the README paragraph this PR adds (README.md:400-417) and step 6 do not: step 6 only warns about releases that refuse a key. Cheapest fix: write "override_authors": [] in the fork's config, which ci/v20 and older refuse as an unknown key, so they fail closed instead of open (the test test_it_is_in_enforce asserts the key is absent and would change). Or at least add the ci/v21 floor to step 6 and to the fork paragraph.
minor: 2. The round cap (100) does not hold against lens authors on the fork. Rounds count from the lens comments that still exist; a deleted comment is gone and only lens authors can delete one (README.md:269). On the fork every lens author is the owner's login, which every agent posts from (78), so an agent can delete old round comments, or close the pull request and open a new one, and start again at round 1. This is the gate's design, not this diff, but the PR body's "past round 4 only a person's override passes" is true only once 78 is done. Worth one sentence in the README fork paragraph, or on issue 100.
minor: 3. A push after the review: the head fails, unless an override for the new head (prefix override for head, association OWNER) vouches for it. That override does not need a listed person: with 78 open, any agent's override lets a pushed head pass on the old reads, while open findings still block (head_check.py:213 via lens.Review.reviewed_head). By design (the nit-only quick check), and the README says it, but it means "only a person decides" in this PR covers findings and the cap, not unread pushes.
minor: 4. No test runs the rollback pull request itself: a base in enforce and a head that sets "mode": "shadow", which should need merge-path and state. It follows from the config being a built-in protected path and from test_workflow_queue_gate_config_and_lens_files_are_critical (the config is in its list), so this is only a gap in the head_check.test.py ForkConfigRules class, which covers the base-in-shadow side only.
verdict: safe to merge

scuttletoby and others added 2 commits November 22, 2026 11:27
…; fail closed on old releases

- review_judge.judge and T3's planReview: the required lenses are those of every rule a path counted for, whatever
  its weight (#82: one extra review). An app change plus a gate file needs code, merge-path and state; before, the
  gate lenses alone passed it. The weight stays the heaviest hit. Shared vectors updated.
- The fork config's gate rule adds fork-kit/review/** and fork-kit/gh-apps/**; a test sorts every fork-kit folder.
- override_authors written out empty: ci/v20 and older refuse the config (fail closed) instead of enforcing the
  path rules without findings and rounds; ci/v21 and later accept it.
- README: the rule, the ci/v21 floor, the fork's rollback (config line plus two tests, judged by the gate lenses),
  and what stays an agent's say until #78. Release notes entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

lens: state r2 · reviewer: state lens agent (Opus 5.5) · read at: 8eee623
verdict: safe to merge
covered: the full diff since main (the config, review_judge.py and its TS mirror plan.ts, review.vectors.json, both test files, the pr-queue and head-check READMEs, RELEASE-NOTES.md); the PR body and its new dry-run table; the owner decisions on issues 82, 99, 100 and 78; both round-1 lens reports on this PR; review_config.py, review_judge.py and head_check.py at ci/v20, ci/v21 and ci/v22; who calls fork-head-gate.yml and fork-pr-queue.yml (nobody in .github/workflows); fork-kit/gh-apps and fork-kit/review imports (nothing from a normal fork-kit folder).
checks: ran review_judge.test.py (53 OK) and head_check.test.py (262 OK, 1 skipped) at this head. Mutation: put the old heaviest-weight-only lens line back in review_judge.py, and review_judge.test.py failed 15 times (restored after). Added an unsorted folder fork-kit/zz-new, and test_every_fork_kit_folder_is_sorted failed and named it (removed after). Parsed this config with ci/v20's config.py: refused, "review: unknown key 'override_authors'"; head_check at ci/v20 turns that ConfigError into a problem, so the head fails. ci/v21 and ci/v22 parse it in enforce. Judged an app file plus .github/SECURITY.md with ci/v22's review_judge: critical, lenses merge-path, state, with no code. Did not run vitest on apps/server/src/fork/reviewRules: its test file did not change. I read the plan.ts filter instead, and it matches the Python line.
state: runs again: the config judges nothing by itself, and the same head gets the same verdict. Stopped half-way: the previous verdict stays until the next event or a backfill. Existing state: no fork workflow calls the head gate or the queue, and the main ruleset does not require head_check, so merging blocks no open pull request today. Old releases: ci/v20 and older now refuse the config, so they fail closed with a clear message (round 1's fail-open gap is closed). ci/v21 and ci/v22 accept it but judge mixed changes the old, looser way (minor 1). Open pull requests whose lens reports predate a push fail with "not the change read" until new reports land on the current head; an override from the owner's login vouches for the old reads (README note, issue 78). Rollback: the PR changes the config plus two tests under fork-ci, all of them in the gate rule, so it needs clean merge-path and state reports (new test). If the gate itself is broken, the owner turns off the ruleset entry. Edited or hidden lens comments read nothing and block. A deleted one lowers the round count; the README now says this holds until issue 78.
round 1, fixed: merge-path's finding that the code lens was dropped on mixed changes; the required lenses are now the union of every rule hit, in Python, TS and the shared vectors, and the mutation above shows the tests catch it.
round 1, fixed: fork-kit/review and fork-kit/gh-apps are in the gate rule, with a test that fails on an unsorted new folder.
round 1, fixed: ci/v15 to ci/v20 judged findings and rounds open; the empty override_authors makes them refuse the config, and the README sets the ci/v21 floor.
round 1, fixed: the rollback steps name the mode line and the two tests that pin it, and test_the_rollback_to_shadow_needs_the_gate_lenses covers the rollback PR.
round 1, fixed: the issue 78 notes on the round cap (deletes) and on pushed heads (an override vouches for old reads) are in the README fork paragraph.
blocker: none
major: none
minor: 1. ci/v21 and ci/v22 are still allowed but looser than issue 82. The fork paragraph says "A caller of this config pins ci/v21 or later" (fork-ci/actions/pr-queue/README.md, fork paragraph under step 5). A caller pinned to ci/v21 or ci/v22 takes the lenses of the heaviest rule only: an app change plus a gate file passes with merge-path and state, and no code lens reads the app code. That is the round-1 major again, silently, with only the summary row's lens list to show it. No caller exists yet, and step 1 says to take the newest release, so nothing is open today. The cheapest fix: the fork paragraph says the fork's caller pins the first release after ci/v22 (the one that carries this change), not ci/v21. A follow-up issue is fine.
minor: 2. A T3 still on the old build plans mixed changes with merge-path and state only (plan.ts), so it starts no code lens. A gate on a new release then refuses that head. This fails closed and needs one more lens run. Worth one line in the release note's upgrade text, no more.

@scuttletoby

Copy link
Copy Markdown
Owner Author

lens: state r2 · reviewer: state lens agent (Opus 5.5) · read at: 8eee623
verdict: not safe to merge
note: this report replaces my earlier round-2 state report on this head, which missed the CI failures below.
covered: the full diff since main (the config, review_judge.py and its TS mirror plan.ts, review.vectors.json, both test files, the pr-queue and head-check READMEs, RELEASE-NOTES.md); the PR body and its new dry-run table; the owner decisions on issues 82, 99, 100 and 78; both round-1 lens reports on this PR; review_config.py, review_judge.py and head_check.py at ci/v20, ci/v21 and ci/v22; who calls fork-head-gate.yml and fork-pr-queue.yml (nobody in .github/workflows); fork-kit/gh-apps and fork-kit/review imports (nothing from a normal fork-kit folder).
checks: read the CI logs of this head: "Fork action and fleet walls tests" and "Fork kit tests 1" fail. Ran review_judge.test.py (53 OK) and head_check.test.py (262 OK, 1 skipped) at this head. Mutation: put the old heaviest-weight-only lens line back in review_judge.py, and review_judge.test.py failed 15 times (restored after). Added an unsorted folder fork-kit/zz-new, and test_every_fork_kit_folder_is_sorted failed and named it (removed after). Parsed this config with ci/v20's config.py: refused, "review: unknown key 'override_authors'"; head_check at ci/v20 turns that ConfigError into a problem, so the head fails. ci/v21 and ci/v22 parse it in enforce. Judged an app file plus .github/SECURITY.md with ci/v22's review_judge: critical, lenses merge-path, state, with no code. Did not run vitest on apps/server/src/fork/reviewRules: its test file did not change. I read the plan.ts filter instead, and it matches the Python line.
state: runs again: the config judges nothing by itself, and the same head gets the same verdict. Stopped half-way: the previous verdict stays until the next event or a backfill. Existing state: no fork workflow calls the head gate or the queue, and the main ruleset does not require head_check, so merging blocks no open pull request today. Old releases: ci/v20 and older now refuse the config, so they fail closed with a clear message (round 1's fail-open gap is closed). ci/v21 and ci/v22 accept it but judge mixed changes the old, looser way (minor 1). Open pull requests whose lens reports predate a push fail with "not the change read" until new reports land on the current head; an override from the owner's login vouches for the old reads (README note, issue 78). Rollback: the PR changes the config plus two tests under fork-ci, all of them in the gate rule, so it needs clean merge-path and state reports (new test). If the gate itself is broken, the owner turns off the ruleset entry. Edited or hidden lens comments read nothing and block. A deleted one lowers the round count; the README now says this holds until issue 78.
round 1, fixed: merge-path's finding that the code lens was dropped on mixed changes; the required lenses are now the union of every rule hit, in Python, TS and the shared vectors, and the mutation above shows the tests catch it.
round 1, fixed: fork-kit/review and fork-kit/gh-apps are in the gate rule, with a test that fails on an unsorted new folder.
round 1, fixed: ci/v15 to ci/v20 judged findings and rounds open; the empty override_authors makes them refuse the config, and the README sets the ci/v21 floor.
round 1, fixed: the rollback steps name the mode line and the two tests that pin it, and test_the_rollback_to_shadow_needs_the_gate_lenses covers the rollback PR.
round 1, fixed: the issue 78 notes on the round cap (deletes) and on pushed heads (an override vouches for old reads) are in the README fork paragraph.
blocker: none
major: three tests outside the changed files still expect the old behaviour, so two CI jobs fail on this head and fork-ci-result cannot pass. (a) fork-ci/actions/head-check/review_guide.test.py Releases.test_findings_rounds_and_override_authors_come_after_the_same_release still looks for the README sentence "Findings, rounds and override_authors are in the first release after ci/v20", which this PR rewrote to "ci/v21 and later". (b) In the same file, StoppedHalfWay.test_the_enforce_pull_request_needs_only_the_input_lenses expects the config file alone to judge as critical with data-1 and data-2; with the union it is data-1, data-2 and code, because the config also counts for its own catch-all rule. (c) fork-kit/review/tests/plan.test.py Plan.test_a_critical_path_needs_two_independent_lenses expects "lenses: data-1, data-2"; the kit's plan now prints data-1, data-2, code for a storage file plus an app file. The PR body's "Checks run" lists neither file. Fix: update the three expectations. (b) also guards a state claim: pr-queue README step 5 says the enforce pull request should "ask for the critical lenses". Under the union, once the base is in enforce, a protected path under a catch-all also needs the catch-all's lens, so step 5 should say "the critical lenses and the lenses of the rules its paths hit".
minor: 1. ci/v21 and ci/v22 are still allowed but looser than issue 82. The fork paragraph says "A caller of this config pins ci/v21 or later" (fork-ci/actions/pr-queue/README.md, fork paragraph under step 5). A caller pinned to ci/v21 or ci/v22 takes the lenses of the heaviest rule only: an app change plus a gate file passes with merge-path and state, and no code lens reads the app code. That is the round-1 major again, silently, with only the summary row's lens list to show it. No caller exists yet, and step 1 says to take the newest release, so nothing is open today. The cheapest fix: the fork paragraph says the fork's caller pins the first release after ci/v22 (the one that carries this change), not ci/v21. A follow-up issue is fine.
minor: 2. A T3 still on the old build plans mixed changes with merge-path and state only (plan.ts), so it starts no code lens. A gate on a new release then refuses that head. This fails closed and needs one more lens run. Worth one line in the release note's upgrade text, no more.

@scuttletoby

Copy link
Copy Markdown
Owner Author

lens: merge-path r2 · reviewer: lens agent (Opus 5.5) · read at: 8eee623
verdict: not safe to merge
covered: the full diff against main (the config, review_judge.py, the TS mirror plan.ts, the shared vectors, both test files, the pr-queue and head-check READMEs, the release notes); the PR body and its dry-run table; the owner decisions on issues 82, 99, 100 and 78; both round-1 lens reports; review_judge.judge (first matching rule takes a path, a protected path also counts for the first critical rule); every reader of the config (fork-head-gate.yml and fork-pr-queue.yml are workflow_call only and nothing calls them; the main ruleset requires fork-ci-result, "No database schema changes" and "Outside pull request", not head_check; the only environment is fork-push, used by fork-follow and fork-ci-release, which push directly and never pass a PR gate); lensStart.ts and the kit's plan (both use the same judge, so they plan the union too); the head's CI results.
checks: ran review_judge.test.py (53 OK) and head_check.test.py (262 OK, 1 skipped) at this head. Mutations: put back the old heaviest-weight-only lens line in review_judge.py: 15 failures; added an unsorted .github/CODEOWNERS and a new fork-kit/newkit folder: both drift tests fail with the right message (restored after). The drift tests run in the "Workflow lint and upstream drift" job, which has no path filter on same-repo PRs. Ran the config through the old releases' parser: ci/v20 refuses (unknown key override_authors), ci/v15 to ci/v19 do not know the key either, ci/v21 and ci/v22 accept. Read the CI logs of the two failed jobs at this head. vitest not run locally (no node_modules in the temp worktree); the Test Server jobs passed on CI.
state: runs again: the config judges nothing by itself and the same head gets the same verdict. Stopped half-way: the previous verdict stays until the next event or a backfill. Existing state: no gate caller, no head-gate environment and no ruleset entry for head_check, so the switch blocks no open PR and nothing in main's release or merge chain (fork-follow and fork-ci-release push through the fork-push environment, not through a PR). The #940 and #920 flips are dry-run verdicts of merged PRs; once a gate exists, an open PR that mixes app code and gate files needs a code report at its next run (backfill). Old state: ci/v20 and older refuse the config (fail closed); ci/v21 and ci/v22 accept it but still require only the heaviest rule's lenses (minor 1). Rollback: the mode line plus the two pinning tests, all gate paths, so merge-path and state only; the new rollback test checks it.
round 1, fixed: a gate file plus app code no longer passes without the code lens. The union is in review_judge.py:174 and plan.ts:169, the shared vectors moved with it, and head_check.test.py and review_judge.test.py pin app plus .github/SECURITY.md, a workflow, fork-ci and fork-kit/review.
round 1, fixed: minor 1, fork-kit/review and fork-kit/gh-apps are now in the gate rule, with a drift test over every fork-kit folder. gh-apps imports only its own lib/ghapps/common, not fork-kit/common.
round 1, fixed: minor 2, the README "Roll back" step and the PR body now name the two tests a rollback must change.
round 1, fixed (state lens minor 1): override_authors is written out empty, so ci/v20 and older fail closed; checked against the tagged code above.
round 1, fixed (state lens minors 2 and 3): the README notes that the round cap and the pushed-after-review override are not yet a person's decision until issue 78.
blocker: none
major: the head is red: fork-ci-result is a failure because three tests the PR did not update still pin the old behaviour, and the body's "Checks run" list leaves them out. (1) fork-kit/review/tests/plan.test.py:99 test_a_critical_path_needs_two_independent_lenses expects "lenses: data-1, data-2"; the kit's plan now says "data-1, data-2, code" (job "Fork kit tests 1"). (2) fork-ci/actions/head-check/review_guide.test.py:195 test_findings_rounds_and_override_authors_come_after_the_same_release still expects the README sentence "in the first release after ci/v20" that this PR rewrote to "ci/v21 and later". (3) review_guide.test.py:413 test_the_enforce_pull_request_needs_only_the_input_lenses expects ('critical', ('data-1', 'data-2')) for a change to the config alone; with the union it is (..., 'code'), because in review.recommended.json the config sits under the catch-all (jobs "Fork action and fleet walls tests"). Fix: update the three tests, and see minor 2 for the README sentence behind (3). Run the whole fork-ci and fork-kit/review test sets before the next push, not only the two changed files.
minor: 1. ci/v21 and ci/v22 accept this config but require only the heaviest rule's lenses, so a caller pinned to either still passes app code plus a gate file on merge-path and state alone, with no warning row. The README says so; nothing fails closed. When the gate caller lands (issue 81), pin the first release after ci/v22, or bump something the old releases refuse. No caller exists today, so nothing is open now.
minor: 2. pr-queue README step 5 still says the enforce PR should "ask for the critical lenses". With the union, a project whose config file sits under the catch-all rule (as in review.recommended.json) needs the catch-all's lens too once in enforce. Same for the step 6 rollback text in general (the fork's own rollback is fine: all its paths are gate paths).
minor: 3. docs/fork/gate-trust.md holds the owner's paste commands that set up the gate App's key and tokens, and fork-gate-trust-docs.test.py runs them against a fake gh. It is as much a gate setting as fork-kit/gh-apps, but stays normal (code lens only). Not caused by this PR; a candidate for the gate rule as a follow-up.
minor: 4. On the writer's open question (a light, no-lens rule for .github/fork-ci-tests.txt and docs/fork): safe for fork-ci-tests.txt alone, a hole for docs/fork. The test list is checked by CI against the tree and the run records: an unlisted, deleted-after-listing, or record-less test fails, run-by is limited by RUN_ELSEWHERE in gate code, and may-skip lines live in the test files under gate paths. So even a PR that changes only that list, which would then need no lens at all (min_lenses light is 0), cannot make a gate test stop running. docs/fork is different: gate-trust.md and host-tools.md are runbooks the owner and agents paste from, so a no-lens rule would let a changed owner command merge unread. If the extra code read on gate PRs is worth saving, add a light rule for exactly .github/fork-ci-tests.txt (it matches no gate glob, so its place in the rule order does not matter) and leave docs/fork normal.

scuttletoby and others added 2 commits November 22, 2026 11:59
…he gate rule

- plan.test.py, review_guide.test.py: expect the union of every rule hit
  (data-1, data-2, code) and the moved step 1 sentence.
- pr-queue README and release notes: a caller of a config in enforce pins the
  first release after ci/v22; ci/v21 and ci/v22 judge mixed changes the old,
  looser way. The fork's gate caller (#81) starts at that floor. A test keeps
  the sentences in step with the release notes.
- Step 5: ask for the critical lenses plus the lenses of every other rule the
  enforce pull request's paths hit.
- docs/fork/gate-trust.md (the owner's commands for the gate App's key and
  tokens) joins the gate rule; the rest of docs/fork stays normal. Tests in
  review_judge.test.py and head_check.test.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby
scuttletoby merged commit ff99cd1 into main Nov 22, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant