Skip to content

fix(noema): add standalone-CLI import fallback to noema_review_gate.py - #1503

Open
seonghobae wants to merge 5 commits into
mainfrom
fix/noema-review-gate-standalone-import
Open

fix(noema): add standalone-CLI import fallback to noema_review_gate.py#1503
seonghobae wants to merge 5 commits into
mainfrom
fix/noema-review-gate-standalone-import

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • PR fix(review): require substantive agent verdicts #1497 (4a5dfd82) added an unconditional
    from scripts.ci.opencode_review_normalize_output import changed_file_is_material
    at module scope in noema_review_gate.py. Under bare-script invocation
    (python3 scripts/ci/noema_review_gate.py ..., no PYTHONPATH set),
    sys.path[0] is the script's own directory (scripts/ci/), not the
    repository root, so the absolute import always raised
    ModuleNotFoundError: No module named 'scripts'.
  • Live evidence of the real breakage: contextual-orchestrator PR fix(review): publish substantive OpenCode LLM evidence #946,
    run 33370760438, job noema-review, failed with exactly that traceback.
  • Important update / correction to scope: by the time this branch was
    pushed, fix(noema): run review gate as module #1501 (c8cc68a3, "fix(noema): run review gate as module") had
    already merged to main
    and fixed the immediate breakage by changing
    the noema-review.yml call site to python3 -m scripts.ci.noema_review_gate
    instead of the bare-script form. That module invocation also makes the
    absolute import resolve correctly, so the org-wide outage is already
    remediated on main as of fix(noema): run review gate as module #1501
    — this PR is not fixing a currently
    ongoing outage.
  • This PR is a complementary defense-in-depth fix: it applies the same
    if __package__: ... else: ... conditional-import fallback already used
    by noema_review_handoff.py (which faces the identical "must work both
    as a package import and as a bare script" requirement) to
    noema_review_gate.py's one absolute import. That way the script itself
    is robust to being invoked either way — by this workflow, by any other
    repo's tooling, or by a human debugging it directly — rather than relying
    solely on every call site remembering to use -m. Confirmed there is
    exactly one such absolute import in the file (git show 4a5dfd82 -- scripts/ci/noema_review_gate.py shows it's the only one fix(review): require substantive agent verdicts #1497 added, and
    a full read of the import block confirms no others exist).

Changes

  • scripts/ci/noema_review_gate.py: wrap the changed_file_is_material
    import in the established if __package__: ... else: ... fallback.
  • tests/test_noema_review_gate.py: new regression test
    test_standalone_cli_runs_from_repo_root_without_pythonpath that runs
    python3 scripts/ci/noema_review_gate.py --help as a subprocess from the
    repository root with PYTHONPATH explicitly cleared — reproducing the
    exact production invocation shape and the exact failure mode, and proving
    it no longer occurs. Modeled on noema_review_handoff.py's existing
    test_standalone_cli_starts_outside_repository_root test.
  • CHANGELOG.md: ## Unreleased entry describing the fix and the
    already-merged fix(noema): run review gate as module #1501 remediation it complements.

Verification

coverage run -m pytest tests && coverage report --show-missing
# 2126 passed, 1 skipped, 21 subtests passed
# TOTAL coverage: 100% (scripts/ci/noema_review_gate.py: 100%)

interrogate
# RESULT: PASSED (minimum: 100.0%, actual: 100.0%)

git diff --check
# clean

Also manually reproduced the pre-fix bug in an isolated scratch copy of the
4a5dfd82 version of the file (ModuleNotFoundError: No module named 'scripts') and confirmed both invocation styles work post-fix:

$ unset PYTHONPATH && python3 scripts/ci/noema_review_gate.py --help   # bare script
usage: noema_review_gate.py [-h] --repo REPO --pr-number PR_NUMBER
$ python3 -m scripts.ci.noema_review_gate --help                       # module
usage: noema_review_gate.py [-h] --repo REPO --pr-number PR_NUMBER

Notes for reviewers

  • Because fix(noema): run review gate as module #1501 already fixed the live-outage aspect, there is no urgency
    here and no bypass-merge request — normal review is fine. Flagging
    only for awareness: this PR's own Noema/OpenCode review checks run the
    base branch's trusted copy of noema_review_gate.py and
    noema-review.yml per this repo's pull_request_target trust-boundary
    design (documented in CLAUDE.md/AGENTS.md), and the base branch
    already has fix(noema): run review gate as module #1501's module-invocation fix, so review of this PR should
    run cleanly rather than hitting the original bug.
  • This is a .github-central-repo-only change (no workflow YAML edits);
    it does not touch pr-review-fix-scheduler.yml or any product-specific
    hourly caller.

Generated by Claude Code


Devin Review

PR #1497 (4a5dfd8) added an unconditional
`from scripts.ci.opencode_review_normalize_output import
changed_file_is_material` at module scope in noema_review_gate.py. Under
bare-script invocation (`python3 scripts/ci/noema_review_gate.py ...`,
with no PYTHONPATH set), sys.path[0] is the script's own directory
(scripts/ci/), not the repository root, so the absolute import always
raised `ModuleNotFoundError: No module named 'scripts'`. This was
confirmed live in contextual-orchestrator PR #946, run 33370760438, job
`noema-review`.

Note: by the time this branch was pushed, PR #1501 (c8cc68a) had
already landed on main and fixed the immediate breakage by changing the
noema-review.yml call site to invoke the script as a module
(`python3 -m scripts.ci.noema_review_gate`), which also makes the
absolute import resolve correctly. This commit is a complementary
defense-in-depth fix: it applies the same `if __package__: ... else: ...`
conditional-import fallback already used by noema_review_handoff.py for
the identical dual-invocation requirement, so the script itself is
robust to being invoked either as a package module or as a bare script
by any current or future caller (this workflow, another repo's tooling,
or manual debugging), rather than depending solely on every call site
remembering to use `-m`.

Adds a regression test that runs
`python3 scripts/ci/noema_review_gate.py --help` as a subprocess from
the repository root with PYTHONPATH cleared, reproducing the exact
production failure mode pre-fix and proving it no longer occurs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d6e9ff6a-5414-4426-a993-1b4494266e0c


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

devin-ai-integration[bot]

This comment was marked as resolved.

…ixing the outage

Devin's review correctly flagged that the changelog entry read as if the
bare-script ModuleNotFoundError were still a live outage. PR #1501 already
fixed the active incident (workflow now invokes the script via `python3
-m`); this PR's import-level fallback is complementary hardening, not an
outage fix. Reword to say so plainly.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fc9ff367: reworded the CHANGELOG entry to state plainly that PR #1501 already fixed the live outage (by switching noema-review.yml to python3 -m scripts.ci.noema_review_gate ...), and that this PR's if __package__ fallback is complementary defense-in-depth rather than an active-outage fix.


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

Devin flagged that "contextual-orchestrator PR #946" doesn't follow this
org's binding cross-repo reference convention (owner/repo#num or a full
URL), so GitHub can't create the cross-repository link.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6ccf913e: switched to the full ContextualWisdomLab/contextual-orchestrator#946 cross-repo reference so GitHub can link/timeline it correctly.


Generated by Claude Code

… gate test

assert_opencode_review_uses_codegraph_and_contextual_orchestrator's awk
range `/^  required-workflow-bootstrap:$/,/^[^ ]/` never terminates in this
file, since job keys are always 2-space indented and no truly-unindented
line exists anywhere in the jobs: section. This silently pulled every job
after required-workflow-bootstrap into the "must have no if:" check,
tripping on an unrelated, legitimate if: condition on a later job's step
and failing this required check on every open .github-repo PR.
required-workflow-bootstrap itself has always had zero if: conditions --
only the test's own job-scoping was broken. Replace the range with an
explicit state machine that starts at the bootstrap job header and stops
at the next 2-space-indented job key.

Verified: `bash scripts/ci/test_strix_quick_gate.sh` now passes (previously
failed with exactly the false-positive record_failure this fix removes);
full suite (2125 passed, 1 skipped, 21 subtests) and `git diff --check`
clean.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ported the same exact-head-path-policy/Strix quick-gate fix here as on #1500/#1502 — this check was failing on every open .github-repo PR due to a job-scoping bug in test_strix_quick_gate.sh itself (root cause and canonical fix in #1506, a concurrent session's PR; my own equivalent #1505 is now closed as a duplicate). Since the check runs on plain pull_request: against each PR's own head branch, the fix has to be ported into each open PR individually rather than relying on main alone. Pushed as 76750f1f (resolved a CHANGELOG merge conflict with this PR's own entry).


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

opencode-review failed on this head (76750f1f) after exhausting its full 90-minute active-dispatch-and-poll window (job 99469648851, 11:52:11Z13:23:30Z) with no opencode-agent verdict — not caused by this PR's own diff (a standalone-import fallback in noema_review_gate.py).

This isn't the same specific mechanism I root-caused on #1500 (documented in docs/product-technical-gap-baseline.md's 2026-08-31 entry): this PR has zero unresolved review threads, so pr_review_merge_scheduler.py's unresolved_thread_count() gate isn't what's blocking dispatch here. I haven't fully traced why the dispatch-and-poll mechanism (#1497) didn't produce a verdict for this specific PR within 90 minutes — the other investigation surfaced evidence of significant pr-review-merge-scheduler.yml runner-queue contention (a comparable run for #1500 queued ~42 minutes before even starting, against 45,600+ historical runs of that workflow), which could independently explain an exhausted poll window without any single decision-logic gate being at fault. Recording this as an open question under the same gap-baseline entry rather than guessing at a fix.

Triggering the one allowed re-run now since the underlying cause may be transient runner contention rather than a deterministic block.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

strix failed on this head (4c30269e) with "Default-branch repository_dispatch Strix evidence failed." Not caused by this PR's diff: the dispatched evidence-producing run itself (actions/runs/33400353198, a repository_dispatch run against main) has been sitting in GitHub's own queued state since 14:02:56Z — over 90 minutes without ever starting — so the polling step that sets this commit status gave up waiting on a run that never got a runner, not a run that actually failed.

This is the third required check I've now seen hit this exact class of failure today across different PRs: noema-review's dispatch-and-poll (#1415), opencode-review's dispatch-and-poll (#1500/#1502/#1503 earlier), and now strix's. All three share the same shape — a required check dispatches a repository_dispatch run against main and polls for evidence within a bounded window, and all three are timing out because the dispatched run itself can't get a runner in time, not because of any bug in the reviewed diff. This looks like genuine org-wide GitHub Actions runner-queue contention right now, not a code defect I can fix from this PR.

Not re-running: the underlying dispatched run is still queued, not failed, so re-triggering would just add another run to the same congested queue. Keeping this PR watched rather than treating it as resolved.


Generated by Claude Code

seonghobae pushed a commit that referenced this pull request Aug 31, 2026
noema-review (#1415), opencode-review (#1500/#1502/#1503), and strix
(#1503) all independently timed out today with the identical shape: a
required check dispatches a repository_dispatch run against main, then
polls for evidence; the dispatched run sat queued (never picked up by a
runner) for well over an hour, so the poller gave up and reported
failure. Documented as an infrastructure capacity question, not a
per-PR code defect.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX
@seonghobae

Copy link
Copy Markdown
Contributor Author

Contextual-Orchestrator를 같이 손보든 어쩌든 해결하세요. Bypass merge 필요하면 가능 (chicken and eggs 상황이라면)

seonghobae added a commit that referenced this pull request Sep 1, 2026
…hed job's own ceiling (#1532)

Owner-authorized admin bypass past opencode-review: this PR's own diff edits .github/workflows/opencode-review.yml, the required-check pipeline it depends on, and the pre-fix budget (90m poll vs 325m worker job) makes it structurally impossible for this exact fix to pass its own check under the old configuration. Chicken-and-egg bypass explicitly authorized by the repo owner on .github#1500/#1503/#1506/#1527/#1529 today. Full verification (2127 passed, 100% coverage, 100% docstrings) in the PR body.
@seonghobae

Copy link
Copy Markdown
Contributor Author

Contextual-Orchestrator와 관계한 것들을 같이 손보든 어쩌든 해결하세요. Bypass merge 필요하면 가능 (chicken and eggs 상황이라면) + NVIDIA NIM 만 쓰는 건 허용하지 않아요. Contextual-Orchestrator를 쓰세요. Timeout은 적어도 3시간으로 잡으세요. 120초 같은 건 당황스럽군요. Opencode와 Noema 는 Coderabbitai 및 Devin 수준으로 실제로 리뷰를 하게 하시오. Strix도 보안 리뷰를 꼼꼼하게 하도록 하시오. 특히 보안 리뷰는 전체 코드로 수행하는 것입니다. Contextual-Orchestrator는 실시간으로 빠르면서 능력이 좋은 모델에 요청을 보내어 시간을 당기시오.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants