Support local MLX orchestration and audited model judgment - #109
Support local MLX orchestration and audited model judgment#109seonghobae wants to merge 281 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughCLI 인증, 로컬 MLX provider 전송, 모델 기반 verifier 판정, SQL ledger 바인딩을 변경했습니다. 보안 경계, 평가 문서, ADR, 예시와 테스트도 갱신했습니다. Changes인증 및 평가 실행 경로
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant ModelClient
participant LocalEndpoint
participant Orchestrator
participant Ledger
CLI->>ModelClient: 로컬 provider 설정 전달
ModelClient->>LocalEndpoint: loopback HTTP 요청 전송
LocalEndpoint-->>ModelClient: 응답 및 usage 반환
ModelClient-->>Orchestrator: 모델 결과 전달
Orchestrator->>Ledger: usage 기록
Ledger-->>Orchestrator: 기록 완료
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Nitpick comments (2)
contextual_orchestrator/cost_ledger.py (1)
571-581: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRuff S608 결과를 해결했는지 확인하세요.
Ruff 0.16.1은
_USAGE_COLUMNS_SQL을 f-string에 삽입하는 Line 573-574, Line 577-578, Line 581을 S608 오류로 보고합니다._USAGE_COLUMNS가 내부 고정 상수이므로 사용자 입력 기반 SQL injection은 확인되지 않습니다. 그러나 Ruff가 CI 차단 조건이면 현재 코드는 SQL 하드닝 PR의 정적 검사를 통과하지 못할 수 있습니다. 전체 SQL을 리터럴 상수로 선언하거나, 고정 상수 조합에 대한 좁은 범위의 예외를 근거와 함께 적용한 뒤 실제 CI/SAST 명령으로 확인하세요.#!/bin/bash set -euo pipefail ruff check --select S608 contextual_orchestrator/cost_ledger.py🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@contextual_orchestrator/cost_ledger.py` around lines 571 - 581, Resolve Ruff S608 findings for _USAGE_INSERT_SQL and _USAGE_SELECT_SQL by either declaring the complete SQL statements as literal constants or applying a narrowly scoped, justified suppression for interpolation of the fixed _USAGE_COLUMNS_SQL constant. Preserve the qmark and pyformat queries, then validate with ruff check --select S608.tests/test_cost_ledger.py (1)
246-252: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win
pyformat실행 경로의 회귀 테스트를 추가하세요.이 추가 테스트는
paramstyle="named"거부만 확인합니다.SqlLedgerStore의 지원 계약인pyformat에서_seed_dimension_catalog,append,query(start, end)가 실제 DB-API 바인딩으로 실행되는지는 확인하지 않습니다.tests/test_cost_ledger.py의 다른 테스트가 이 경로를 이미 검증하지 않는다면, 기존psycopg연결 또는 동일한 바인딩을 재현하는 테스트 더블로 네 가지 시작/종료 조건을 검증하세요. 이 검사가 없으면 placeholder 수, 값 순서, 드라이버 바인딩 오류가 회귀해도 테스트가 통과합니다.As per coding guidelines: 실질적인 동작 변경에는 대응하는 실행 테스트와 일치하는 문서 계약을 함께 갱신해야 합니다.
#!/bin/bash set -euo pipefail rg -n -C 4 'paramstyle|pyformat|psycopg|SqlLedgerStore' tests/test_cost_ledger.py🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_cost_ledger.py` around lines 246 - 252, Add regression coverage in test_sql_ledger_rejects_unknown_parameter_style’s test module for SqlLedgerStore with paramstyle="pyformat", exercising _seed_dimension_catalog, append, and query(start, end) against a real psycopg connection or equivalent binding-faithful test double. Cover all four start/end boundary combinations and verify correct results, placeholder counts, value ordering, and successful driver binding; update the matching API documentation contract for this behavior if required by the repository guidelines.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@contextual_orchestrator/__main__.py`:
- Around line 83-91: Update the CLI token-mode detection and resolution around
split_requested so --admin-token-key and --inference-token-key both select
split-token mode and participate in completeness validation. Change these two
argument defaults to None, applying the standard credential names only during
resolution, and add an executable regression test covering use of only the two
key options; update the corresponding documentation contract for this behavior.
In `@contextual_orchestrator/orchestrator.py`:
- Around line 366-373: Update the provider connection flow around
_validate_provider and the HTTP/HTTPSConnection creation to connect using the
already validated sockaddr rather than resolving parsed.hostname again. Preserve
the original hostname for TLS SNI and the HTTP Host header, while ensuring both
HTTP and HTTPS paths enforce the validated public-IP destination.
- Around line 1755-1764: Update the judge response parsing in the surrounding
validation flow to parse reply.strip() as the complete JSON value, rejecting
wrapper text and any non-object response. Require the parsed object’s keys to be
exactly {"decision", "reason"} while preserving the existing decision enum and
non-empty reason validation, and add regression tests covering surrounding text
and extra fields.
In `@Dockerfile`:
- Line 29: Update the Dockerfile CMD to use --auth-token-key
CONTEXTUAL_ORCHESTRATOR_TOKEN instead of reading CONTEXTUAL_ORCHESTRATOR_TOKEN
into --auth-token. Keep token retrieval within the KV-backed authentication path
and rely on deployment-provided KV bootstrap environment variables rather than
exposing the token through the process environment or command line.
In `@docs/planning/adrs/0004-pr-review-merge-loop.md`:
- Around line 40-42: Update the ADR measurement target so unresolved security or
correctness findings cannot be satisfied by documentation alone and remain merge
blockers until fixed and revalidated. Preserve explicit acceptance only for
non-blocking risks, requiring an approver, rationale, tracking issue, and
expiration date, and align the wording with the existing merge requirements in
the ADR.
- Line 81: Update the merge procedure described at lines 81 and 129 to record
the SHA used for verification, then require that the current PR head and every
required check result match that SHA before merging. If any SHA differs, stop
the merge and re-verify rather than proceeding.
- Around line 35-38: Update the “PR verification” metric in the ADR to define
the exact merge-gate contract: identify each required repository-local and
central workflow/job or check-run by name, reconcile the documented checks with
the actual security workflow identifiers such as codeql_analysis and
python_supply_chain, and require every check to pass on the exact head SHA
before merge. Clarify that local reproducible commands are supplementary
evidence and do not replace GitHub required checks.
In `@tests/test_model_judge.py`:
- Around line 86-91: 구조화된 판정 파서의 신뢰할 수 없는 입력 처리를 퍼즈 테스트로 보강하십시오. `tests/fuzz/`에
Hypothesis 불변성 테스트를 추가해 임의 입력이 예외를 발생시키지 않고 정확히 허용된 스키마만 수락하는지 검증하고, `fuzz/`에
동일한 파싱 경계를 대상으로 하는 Atheris 타깃을 추가하십시오. 기존
`test_plain_keyword_reply_is_rejected`의 거부 동작은 유지하십시오.
In `@tests/test_security_hardening.py`:
- Line 39: Update TaskOrchestrator._security_posture_criterion() to explicitly
accept external_bearer_verifier as an authenticated security mode alongside
split_token, preventing the no-bearer-token failure. Add a regression test
covering readiness reporting for the external_bearer_verifier path.
---
Nitpick comments:
In `@contextual_orchestrator/cost_ledger.py`:
- Around line 571-581: Resolve Ruff S608 findings for _USAGE_INSERT_SQL and
_USAGE_SELECT_SQL by either declaring the complete SQL statements as literal
constants or applying a narrowly scoped, justified suppression for interpolation
of the fixed _USAGE_COLUMNS_SQL constant. Preserve the qmark and pyformat
queries, then validate with ruff check --select S608.
In `@tests/test_cost_ledger.py`:
- Around line 246-252: Add regression coverage in
test_sql_ledger_rejects_unknown_parameter_style’s test module for SqlLedgerStore
with paramstyle="pyformat", exercising _seed_dimension_catalog, append, and
query(start, end) against a real psycopg connection or equivalent
binding-faithful test double. Cover all four start/end boundary combinations and
verify correct results, placeholder counts, value ordering, and successful
driver binding; update the matching API documentation contract for this behavior
if required by the repository guidelines.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: fab96597-db98-4afe-8ade-ba659450f78b
📒 Files selected for processing (28)
.adr-config.ymlCLAUDE.mdDockerfileREADME.mdcontextual_orchestrator/__main__.pycontextual_orchestrator/cost_ledger.pycontextual_orchestrator/orchestrator.pycontextual_orchestrator/server.pydocs/benchmarks/2026-07-06-openai-optimizer.mddocs/benchmarks/2026-08-11-polytomous-llm-judge.mddocs/kv-credentials.mddocs/planning/adrs/0001-fail-closed-model-judgment.mddocs/planning/adrs/0002-explicit-local-mlx-evaluation.mddocs/planning/adrs/0003-keyverse-authentication-boundary.mddocs/planning/adrs/0004-pr-review-merge-loop.mddocs/planning/adrs/0005-irt-response-matrix-contract.mddocs/planning/adrs/0006-polytomous-llm-judge-bias-calibration.mddocs/planning/adrs/0007-sast-transport-and-sql-hardening.mddocs/planning/adrs/0008-fast-judge-review-hardening.mdexamples/agents.mlx.jsontests/test_batch_optimizer.pytests/test_cli_auth.pytests/test_cost_ledger.pytests/test_generated_workflow.pytests/test_local_mlx.pytests/test_model_judge.pytests/test_provider_tls.pytests/test_security_hardening.py
|
Remediation iteration pushed at |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
contextual_orchestrator/__main__.py (1)
106-113: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winCLI 입력값을 parser에서 검증하십시오.
ModelClient는local_concurrency < 1을 거부하지만, 예외가parser.error로 변환되지 않아 traceback으로 종료됩니다.chat_template_args는null과 키-값 배열을 허용합니다. CLI에서 양의 정수와 JSON object만 허용하십시오.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@contextual_orchestrator/__main__.py` around lines 106 - 113, Validate the --local-concurrency and --chat-template-args arguments at the argparse layer in the parser setup: require local_concurrency to be a positive integer and require chat_template_args to parse as a JSON object, rejecting null and non-object values through parser.error rather than allowing downstream ModelClient failures or tracebacks.
🧹 Nitpick comments (2)
docs/planning/adrs/0007-sast-transport-and-sql-hardening.md (2)
104-109: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win
nosemgrep예외의 범위를 재현 가능하게 기록하세요.
rule-specific라는 설명만으로는 적용한 Semgrep rule과 호출 위치를 확인할 수 없습니다. 정확한 rule ID, 파일 경로와 호출 라인, 그리고 인증서 검증·URL 검증을 확인하는 테스트 이름을 기록하세요. 이후 변경에서 예외 범위가 넓어지는 것을 방지할 수 있습니다.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/planning/adrs/0007-sast-transport-and-sql-hardening.md` around lines 104 - 109, Update the documented nosemgrep exception to record the exact Semgrep rule ID, the file path and call line for the HTTPSConnection site, and the names of the tests covering certificate and URL validation. Keep the exception limited to that single reviewed call and preserve the existing false-positive boundary.
148-155: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winADR의 affected components 목록을 실제 검증 범위와 맞추세요.
본문은 transport regression tests를 source of truth로 지정하지만, 목록에는
tests/test_provider_integration.py와tests/test_provider_reliability.py가 없습니다. 두 테스트를 추가하거나 의도적으로 제외한 이유를 명시하세요. 그래야 validated sockaddr pinning과 transport 실패 동작을 검증하는 위치를 유지보수자가 찾을 수 있습니다.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/planning/adrs/0007-sast-transport-and-sql-hardening.md` around lines 148 - 155, ADR의 Affected Components 목록을 transport regression tests의 실제 검증 범위와 일치시키세요. 본문에서 source of truth로 지정한 tests/test_provider_integration.py와 tests/test_provider_reliability.py를 목록에 추가하고, 제외해야 한다면 해당 의도를 ADR에 명시하세요.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@contextual_orchestrator/__main__.py`:
- Around line 152-155: Update the parser.error message in the split_requested
validation to state that both admin and inference roles must be provided using
either their token values or KV key options, including --admin-token-key and
--inference-token-key.
In `@docs/planning/adrs/0004-pr-review-merge-loop.md`:
- Line 155: Update the merge policy entries at the neutral/no-report Strix
result row and the structured-report requirement row so a missing structured
report always blocks merging; if an exception is retained, define the explicit
security-owner role, required approver, justification, tracking issue,
expiration date, and revalidation conditions in the ADR, consistent with the
risk-acceptance criteria.
---
Outside diff comments:
In `@contextual_orchestrator/__main__.py`:
- Around line 106-113: Validate the --local-concurrency and --chat-template-args
arguments at the argparse layer in the parser setup: require local_concurrency
to be a positive integer and require chat_template_args to parse as a JSON
object, rejecting null and non-object values through parser.error rather than
allowing downstream ModelClient failures or tracebacks.
---
Nitpick comments:
In `@docs/planning/adrs/0007-sast-transport-and-sql-hardening.md`:
- Around line 104-109: Update the documented nosemgrep exception to record the
exact Semgrep rule ID, the file path and call line for the HTTPSConnection site,
and the names of the tests covering certificate and URL validation. Keep the
exception limited to that single reviewed call and preserve the existing
false-positive boundary.
- Around line 148-155: ADR의 Affected Components 목록을 transport regression tests의
실제 검증 범위와 일치시키세요. 본문에서 source of truth로 지정한 tests/test_provider_integration.py와
tests/test_provider_reliability.py를 목록에 추가하고, 제외해야 한다면 해당 의도를 ADR에 명시하세요.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 78e3e78f-5261-4708-aa60-5749e4ff8c8d
📒 Files selected for processing (20)
.github/workflows/fuzz.ymlDockerfilecontextual_orchestrator/__main__.pycontextual_orchestrator/orchestrator.pydocs/planning/adrs/0001-fail-closed-model-judgment.mddocs/planning/adrs/0003-keyverse-authentication-boundary.mddocs/planning/adrs/0004-pr-review-merge-loop.mddocs/planning/adrs/0007-sast-transport-and-sql-hardening.mddocs/planning/adrs/0008-fast-judge-review-hardening.mdfuzz/corpus/judge/valid.jsonfuzz/corpus/judge/wrapped.txtfuzz/fuzz_model_judge.pyfuzz/targets.pytests/fuzz/test_fuzz_properties.pytests/test_cli_auth.pytests/test_local_mlx.pytests/test_model_judge.pytests/test_provider_integration.pytests/test_provider_reliability.pytests/test_sales_readiness.py
🚧 Files skipped from review as they are similar to previous changes (6)
- Dockerfile
- tests/test_model_judge.py
- docs/planning/adrs/0001-fail-closed-model-judgment.md
- docs/planning/adrs/0008-fast-judge-review-hardening.md
- tests/test_local_mlx.py
- docs/planning/adrs/0003-keyverse-authentication-boundary.md
|
Safety hold at current head |
|
Review remediation on current head
Validation: |
|
Security remediation on current head
The prior Strix failure is therefore treated as a real remediation trigger, not as a provider-only blocker. New required checks and a fresh structured Strix report are required on |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
contextual_orchestrator/__main__.py (2)
105-106: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win구성 이름을 두 개 이상의 의미 단어로 지정하십시오.
Line 105-106의
--temperature와 Line 125의args.temperature는 단일 의미 단어입니다. 내부 이름을sampling_temperature로 변경하고 전달 코드도 갱신하십시오.수정 예시
-parser.add_argument("--temperature", type=float, default=0.2, +parser.add_argument("--sampling-temperature", dest="sampling_temperature", type=float, default=0.2, help="Default provider sampling temperature (default: 0.2).") - temperature=args.temperature, + temperature=args.sampling_temperature,As per coding guidelines, “Use lower snake_case with at least two semantic words for configurable, API, and database object names.”
Also applies to: 125-125
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@contextual_orchestrator/__main__.py` around lines 105 - 106, Rename the configurable CLI option from temperature to sampling_temperature in the argument definition and update the corresponding args.temperature reference at the provider configuration call to use the new destination name.Source: Coding guidelines
111-112: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
chat-template-args를 JSON 객체로 제한하십시오.
json.loads는[],null, 문자열, 숫자도 성공적으로 파싱합니다. 이 값은ModelClient에 keyword arguments로 전달되므로 객체가 아닌 값은 이후 실행에서 오류를 만들 수 있습니다. 파서에서dict인지 검증하고 즉시argparse.ArgumentTypeError를 반환하십시오.수정 예시
+def _parse_chat_template_args(raw: str) -> dict[str, object]: + value = json.loads(raw) + if not isinstance(value, dict): + raise argparse.ArgumentTypeError("chat-template-args must be a JSON object") + return value + - parser.add_argument("--chat-template-args", type=json.loads, default={}, + parser.add_argument("--chat-template-args", type=_parse_chat_template_args, default={},🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@contextual_orchestrator/__main__.py` around lines 111 - 112, Restrict the --chat-template-args parser argument to JSON objects by replacing the direct json.loads converter with a validation helper that parses the value, verifies the result is a dict, and raises argparse.ArgumentTypeError for any other JSON type. Keep the existing empty-dict default and pass validated keyword arguments to ModelClient.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_repository_security_metadata.py`:
- Line 76: Update the test around the Dependabot configuration assertion to
parse each configured update entry separately and verify that every ecosystem’s
cooldown.default-days equals 7. Replace the aggregate dependabot_text.count
check with ecosystem-specific validation covering all configured entries,
including github-actions and pip.
---
Outside diff comments:
In `@contextual_orchestrator/__main__.py`:
- Around line 105-106: Rename the configurable CLI option from temperature to
sampling_temperature in the argument definition and update the corresponding
args.temperature reference at the provider configuration call to use the new
destination name.
- Around line 111-112: Restrict the --chat-template-args parser argument to JSON
objects by replacing the direct json.loads converter with a validation helper
that parses the value, verifies the result is a dict, and raises
argparse.ArgumentTypeError for any other JSON type. Keep the existing empty-dict
default and pass validated keyword arguments to ModelClient.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c62b2541-7a2b-4c8d-8675-a8f80a3e0791
📒 Files selected for processing (7)
.adr-config.yml.github/dependabot.ymlcontextual_orchestrator/__main__.pydocs/planning/adrs/0004-pr-review-merge-loop.mddocs/planning/adrs/0009-supply-chain-dependency-cooldown.mdtests/test_cli_auth.pytests/test_repository_security_metadata.py
🚧 Files skipped from review as they are similar to previous changes (2)
- .adr-config.yml
- docs/planning/adrs/0004-pr-review-merge-loop.md
|
Governance finding recorded in ADR 0004 at current head
The contextual PR remains on hold: auto-merge is disabled, |
|
@coderabbitai review Review exact unchanged contributor head Current deterministic PR-associated gates are successful: Tests Verify the test-first argparse repair, |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== revision identity =='
git rev-parse HEAD
git show -s --format='HEAD %H%nparents %P%nsubject %s' HEAD
git cat-file -e 9a701f200f6c25a56c31876f3421e1eb42183cb3^{commit}
git cat-file -e 6841b71935e0b7cb98fb52bcb4709cc5100c8d87^{commit}
echo '== relationship =='
git merge-base 9a701f200f6c25a56c31876f3421e1eb42183cb3 6841b71935e0b7cb98fb52bcb4709cc5100c8d87
echo '== changed files =='
git diff --name-status 6841b71935e0b7cb98fb52bcb4709cc5100c8d87...9a701f200f6c25a56c31876f3421e1eb42183cb3
echo '== source outlines =='
ast-grep outline contextual_orchestrator/__main__.py --items all --type function,class
ast-grep outline contextual_orchestrator/orchestrator.py --items all --type function,class --match 'ModelClient|parse|judge|provider|destination|chat|batch'
ast-grep outline contextual_orchestrator/server.py --items all --type function,class --match 'SecurityConfig|auth|bearer|readiness'
echo '== PR metadata =='
gh pr view 109 --repo ContextualWisdomLab/contextual-orchestrator --json headRefOid,baseRefOid,mergeStateStatus,reviewDecision,isDraft,statusCheckRollup,reviewsLength of output: 46454 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== CLI implementation and tests =='
nl -ba contextual_orchestrator/__main__.py | sed -n '1,210p'
nl -ba tests/test_cli_auth.py | sed -n '1,260p'
echo '== model judgment parser and call site =='
nl -ba contextual_orchestrator/orchestrator.py | sed -n '45,105p;210,450p;1700,1805p'
nl -ba tests/test_model_judge.py | sed -n '1,240p'
nl -ba fuzz/targets.py | sed -n '1,180p'
nl -ba tests/fuzz/test_fuzz_properties.py | sed -n '1,220p'
echo '== local MLX transport and tests =='
nl -ba contextual_orchestrator/orchestrator.py | sed -n '250,420p;650,900p'
nl -ba tests/test_local_mlx.py | sed -n '1,260p'
nl -ba tests/test_provider_integration.py | sed -n '1,220p'
nl -ba tests/test_provider_reliability.py | sed -n '1,210p'
echo '== authentication and configuration =='
nl -ba contextual_orchestrator/server.py | sed -n '60,220p'
nl -ba Dockerfile | sed -n '1,100p'
nl -ba examples/agents.mlx.json
nl -ba docs/kv-credentials.md | sed -n '1,220p'
echo '== SQL parameterization and current coverage =='
nl -ba contextual_orchestrator/cost_ledger.py | sed -n '1,210p;520,650p'
rg -n -C 5 'paramstyle|pyformat|SqlLedgerStore|query\(' tests/test_cost_ledger.py
echo '== merge policy and check-suite association =='
nl -ba docs/planning/adrs/0004-pr-review-merge-loop.md | sed -n '25,180p'
gh api "repos/ContextualWisdomLab/contextual-orchestrator/commits/9a701f200f6c25a56c31876f3421e1eb42183cb3/check-runs?per_page=100" \
--jq '.check_runs[] | [.name, .status, .conclusion, .head_sha, .details_url] | `@tsv`'Length of output: 1923 🏁 Script executed: #!/bin/bash
set -euo pipefail
show() {
file="$1"
ranges="$2"
echo "== $file:$ranges =="
for range in $ranges; do
sed -n "${range}p" "$file"
done
}
show contextual_orchestrator/__main__.py '1,210'
show tests/test_cli_auth.py '1,260'
show contextual_orchestrator/orchestrator.py '45,105 210,450 650,900 1700,1805'
show tests/test_model_judge.py '1,240'
show fuzz/targets.py '1,180'
show tests/fuzz/test_fuzz_properties.py '1,220'
show tests/test_local_mlx.py '1,260'
show tests/test_provider_integration.py '1,220'
show tests/test_provider_reliability.py '1,210'
show contextual_orchestrator/server.py '60,220'
show Dockerfile '1,100'
show examples/agents.mlx.json '1,120'
show docs/kv-credentials.md '1,220'
show contextual_orchestrator/cost_ledger.py '1,210 520,650'
echo '== ledger test references =='
rg -n -C 5 'paramstyle|pyformat|SqlLedgerStore|query\(' tests/test_cost_ledger.py
show docs/planning/adrs/0004-pr-review-merge-loop.md '25,180'
echo '== check runs directly associated with contributor SHA =='
gh api "repos/ContextualWisdomLab/contextual-orchestrator/commits/9a701f200f6c25a56c31876f3421e1eb42183cb3/check-runs?per_page=100" \
--jq '.check_runs[] | [.name, .status, .conclusion, .head_sha, .details_url] | `@tsv`'Length of output: 131134 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== provider transport call sites =='
rg -n -C 4 '_validate_provider\(|_open_provider\(|_send_with_retry\(|_send_raw_with_retry\(|_stream_send\(|proxy_send\(' \
contextual_orchestrator/orchestrator.py
echo '== complete provider networking implementation =='
sed -n '390,700p' contextual_orchestrator/orchestrator.py
echo '== final contributor-only commits and changed hunks =='
git log --oneline 6841b71935e0b7cb98fb52bcb4709cc5100c8d87..9a701f200f6c25a56c31876f3421e1eb42183cb3
git diff --check 6841b71935e0b7cb98fb52bcb4709cc5100c8d87...9a701f200f6c25a56c31876f3421e1eb42183cb3
git diff --unified=30 6841b71935e0b7cb98fb52bcb4709cc5100c8d87...9a701f200f6c25a56c31876f3421e1eb42183cb3 -- \
contextual_orchestrator/__main__.py \
contextual_orchestrator/orchestrator.py \
contextual_orchestrator/server.py \
tests/test_cli_auth.py \
tests/test_local_mlx.py \
tests/test_model_judge.py \
docs/planning/adrs/0004-pr-review-merge-loop.md
echo '== explicit source/docs contract references =='
rg -n -C 3 'sampling-temperature|--temperature|local-concurrency|chat-template-args|mlx://|bearer_verifier|fail.closed|structured.*Strix|verified_head_sha' \
README.md CLAUDE.md docs tests contextual_orchestrator --glob '!docs/planning/adrs/0004-pr-review-merge-loop.md'Length of output: 178142
I found no actionable code findings. Verified areas:
GitHub associates the inspected completed check runs with the contributor SHA. I did not treat the synthetic-merge unit-suite execution as direct contributor-head execution proof. The PR is not merge-ready.
|
|
@coderabbitai review Review exact unchanged contributor head Current PR-associated gates are successful: Tests Verify strict category-derived score validation, fail-closed JSON judgment, the argparse repairs and compatibility alias, MLX/provider/KV/egress/response-trust boundaries, coverage/docstrings, and merge-governance evidence. Treat predecessor, author-only, status-only, rate-limited, neutral/no-report Strix, and synthetic-merge evidence according to their actual identity. Submit only a current-head formal review; automated review does not substitute for qualifying independent non-author approval. |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="ContextualWisdomLab/contextual-orchestrator"
head="73969b28dce2773ec979a6bd811bf038ff9af8fd"
base="6841b71935e0b7cb98fb52bcb4709cc5100c8d87"
echo "== revision identity =="
git rev-parse HEAD
git cat-file -e "${head}^{commit}"
git cat-file -e "${base}^{commit}"
git show -s --format='HEAD %H%nparents %P%nsubject %s' "$head"
echo "merge-base: $(git merge-base "$head" "$base")"
echo "== changed files =="
git diff --name-status "${base}...${head}"
echo "== commits =="
git log --oneline "${base}..${head}"
echo "== focused outlines =="
ast-grep outline contextual_orchestrator/__main__.py --items all --type function,class
ast-grep outline contextual_orchestrator/orchestrator.py --items all --type function,class --match 'ModelClient|judge|parse|validate|provider|destination|chat|batch'
ast-grep outline contextual_orchestrator/server.py --items all --type function,class --match 'SecurityConfig|auth|bearer|readiness'
echo "== CLI and tests =="
sed -n '1,260p' contextual_orchestrator/__main__.py
sed -n '1,300p' tests/test_cli_auth.py
echo "== score and judge implementation/tests =="
rg -n -C 4 'category|score|_parse_model_judge_reply|JudgeFormatError|verifier_judge|judge' \
contextual_orchestrator tests fuzz
sed -n '1,280p' tests/test_model_judge.py
sed -n '1,200p' fuzz/targets.py
sed -n '1,220p' tests/fuzz/test_fuzz_properties.py
echo "== transport/authentication implementation/tests =="
sed -n '200,940p' contextual_orchestrator/orchestrator.py
sed -n '50,240p' contextual_orchestrator/server.py
sed -n '1,280p' tests/test_local_mlx.py
sed -n '1,260p' tests/test_provider_integration.py
sed -n '1,220p' tests/test_provider_reliability.py
echo "== documentation and coverage signals =="
rg -n -C 3 'sampling-temperature|--temperature|local-concurrency|chat-template-args|mlx://|bearer_verifier|fail.closed|Strix|verified_head_sha|category' \
README.md CLAUDE.md docs tests contextual_orchestrator
git diff --check "${base}...${head}"
echo "== PR metadata =="
gh pr view 109 --repo "$repo" \
--json headRefOid,baseRefOid,mergeStateStatus,reviewDecision,isDraft,statusCheckRollup,reviews
echo "== check runs directly associated with contributor SHA =="
gh api "repos/${repo}/commits/${head}/check-runs?per_page=100" \
--jq '.check_runs[] | [.name, .status, .conclusion, .head_sha, .details_url] | `@tsv`'Length of output: 200827 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="ContextualWisdomLab/contextual-orchestrator"
head="73969b28dce2773ec979a6bd811bf038ff9af8fd"
base="6841b71935e0b7cb98fb52bcb4709cc5100c8d87"
echo "== exact current-head delta =="
git diff --unified=20 "${base}...${head}" -- \
contextual_orchestrator/__main__.py \
contextual_orchestrator/orchestrator.py \
contextual_orchestrator/server.py \
tests/test_cli_auth.py \
tests/test_model_judge.py \
tests/test_local_mlx.py \
tests/test_provider_integration.py \
tests/test_provider_reliability.py \
tests/test_security_hardening.py \
tests/test_sales_readiness.py \
docs/planning/adrs/0004-pr-review-merge-loop.md \
docs/benchmarks/2026-08-11-polytomous-llm-judge.md
echo "== CLI validation definitions and construction call =="
rg -n -A25 -B10 '^def (_positive_int|_json_object|_resolve_auth|main)\b|sampling_temperature|local_concurrency|chat_template_args|split_requested' \
contextual_orchestrator/__main__.py
sed -n '1,180p' tests/test_cli_auth.py
echo "== strict model judgment parser and call site =="
rg -n -A80 -B15 '^def _parse_model_judge_reply\b|^def exercise_model_judge_reply\b|_model_judge_verification' \
contextual_orchestrator/orchestrator.py fuzz/targets.py
sed -n '1,260p' tests/test_model_judge.py
sed -n '1,180p' tests/fuzz/test_fuzz_properties.py
echo "== category-derived score implementation and tests =="
rg -n -A100 -B15 'criterion_categories|derived.*score|derive.*score|n_categories|category.*score|score.*category' \
contextual_orchestrator tests --glob '*.py'
echo "== tests workflow checkout identity =="
fd -a '.*\.ya?ml' .github/workflows -x sh -c \
'if rg -q "Full unit and contract suite|pytest|actions/checkout" "$1"; then echo "== $1 =="; rg -n -C3 "actions/checkout|ref:|Full unit and contract suite|pytest" "$1"; fi' sh {}
echo "== specific workflow-job metadata =="
gh api "repos/${repo}/actions/jobs/93856762720" \
--jq '{name,conclusion,head_sha,started_at,completed_at,steps:[.steps[]|{name,conclusion}]}'
echo "== unresolved review threads =="
gh api graphql -F owner="ContextualWisdomLab" -F name="contextual-orchestrator" -F number=109 -f query='
query($owner:String!, $name:String!, $number:Int!) {
repository(owner:$owner, name:$name) {
pullRequest(number:$number) {
reviewThreads(first:100) {
nodes { isResolved comments(first:1) { nodes { author { login } } } }
}
}
}
}' --jq '[.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false)] | length'Length of output: 185603 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="ContextualWisdomLab/contextual-orchestrator"
head="73969b28dce2773ec979a6bd811bf038ff9af8fd"
echo "== contributor-SHA test-job identity =="
gh api "repos/${repo}/actions/jobs/93856762720" \
--jq '{name,conclusion,head_sha,started_at,completed_at,html_url,steps:[.steps[]|{name,conclusion,number}]}'
echo "== tests workflow checkout configuration =="
fd -a 'tests.yml' .github/workflows -x sh -c \
'echo "== $1 =="; rg -n -C4 "actions/checkout|ref:|Full unit and contract suite|pytest" "$1"' sh {}
echo "== unresolved review-thread count =="
gh api graphql -F owner="ContextualWisdomLab" -F name="contextual-orchestrator" -F number=109 -f query='
query($owner:String!, $name:String!, $number:Int!) {
repository(owner:$owner, name:$name) {
pullRequest(number:$number) {
reviewThreads(first:100) { nodes { isResolved } }
}
}
}' --jq '[.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false)] | length'
echo "== category-derived score ownership and linked source =="
gh repo view ContextualWisdomLab/fast-mlsirm --json nameWithOwner,defaultBranchRef,url
gh api "repos/ContextualWisdomLab/fast-mlsirm/commits?per_page=30" \
--jq '.[] | [.sha, .commit.message] | `@tsv`' | rg -i 'judge|category|score|calibrat' || true
gh api "repos/ContextualWisdomLab/fast-mlsirm/git/trees/main?recursive=1" \
--jq '.tree[] | select(.path | test("judge|irt|score"; "i")) | .path' | head -100Length of output: 6775
I found no actionable code findings. Verified areas:
The strict category-derived score implementation is in This PR is not merge-ready:
|
|
Current-head ADR update pushed at |
|
ADR follow-up pushed as bfe8e2a: the trusted central review dispatch (run 31514989573) failed closed on the exact central head because coverage evidence failed, the model pool was empty, and the authorized OpenCode App review identity was unavailable. No formal approval will be substituted. This PR remains held until the exact-head review, aggregate APPROVED state, structured Strix evidence, and all required checks are available. |
|
Central scheduler follow-up advanced to exact head 12e3d1f5 (empty reviewDecision regression test; 114 tests and self-test pass). This invalidates any prior central-head review evidence, so contextual merge remains held until the central PR is independently reviewed at 12e3d1f5 and the linked exact-head gates are revalidated. |
|
ADR follow-up pushed as 088eeed: central scheduler evidence invalidation is now recorded because PR #937 advanced to exact head 12e3d1f5. Prior central review/security evidence is stale by policy; re-fetch and independently review 12e3d1f5 before any linked merge consideration. |
|
Exact-head update: ADR evidence was extended in commit b05c81d. The current PR head is b05c81d (full SHA is b05c81d; re-fetch required for the full SHA), so all earlier CI, security, and review evidence is stale. Queued, runner-unassigned, in-progress, missing, or unreported required checks are not passes. Keep this PR blocked until the current head has completed required checks, structured same-head Strix evidence, an authorized independent approval, aggregate reviewDecision APPROVED, and zero unresolved threads; do not merge on local tests alone. |
|
Exact-head update: documentation commit 414c759 corrected the ADR status for the already-tested provider DNS pinning. All earlier contextual CI, security, and review evidence is stale again. Re-fetch every required check and review against this full SHA; queued, runner-unassigned, in-progress, missing, or unreported states remain non-passing. Merge remains prohibited until the exact-head aggregate review, structured Strix result, required checks, and independent authorized approval are present. |
|
Exact-head update: ADR commit 1c6819d records the newly fixed scheduler acceptance gate: an approved head cannot merge or enable/retain auto-merge while any status context is queued/in progress or same-head Strix evidence is missing. All prior contextual CI/review/security evidence is stale; re-fetch this full SHA and require terminal checks, completed structured Strix, aggregate APPROVED review, independent authorization, and zero unresolved threads. |
|
Exact-head update: ADR commit 06e6ffb records the central Strix fail-open neutralization finding and its fix on central head 58561518. All prior contextual CI/review/security evidence is stale; this full SHA needs fresh required checks, completed structured same-head Strix evidence, aggregate APPROVED review, authorized independent approval, and a final re-fetch before merge. |
|
Exact-head review request for current commit d5236bbd5db6d02bd9dc2e8ba22e2e2ac8e1c1a2.\n\nThe ADR now records the latest predecessor Strix run 31845665320 / job 94911468155 / artifact 9236148265, including report SHA-256 19d522787ac31cd9e4e46fdd9b406e8d1255b6242b04c1c78b4fd934e0dec8e4 and run.json SHA-256 8cdc067aaabf5e233bbdbaca45e4df89702cff18e4cd193910d995f9257d5502. That report found no exploitable authentication bypass or other critical weakness, but it had no evidence-binding.json and its metadata did not bind repository, head, job, and report digest, so it is provider/content evidence only and not a clean protected gate.\n\nThe ADR-only push invalidates predecessor checks, Strix evidence, and review snapshots. Please produce a fresh independent review and trusted structured same-head Strix binding for d5236bbd5db6d02bd9dc2e8ba22e2e2ac8e1c1a2 after all required checks are terminal. Do not self-approve, use an administrator bypass, or convert a status-only/unbound result into approval. |
|
Correction: the exact current commit is d5236bb (the preceding request contained a mistyped SHA). All requested fresh-review and trusted-binding conditions apply to this exact SHA. |
|
Exact-head review request for current commit de4d95a.\n\nThe ADR now records the predecessor run 31847116067 / job 94915707782 / artifact 9236521645. Its report was zero-finding, but gate-console explicitly emitted provider/failure-marker messages saying the scan was failing closed; the Actions job was nevertheless green. The artifact had no evidence-binding.json, and run.json contained only a temporary target path with no repository/head/commit binding. This is non-clean provider/failure evidence, not a protected security pass.\n\nThe ADR-only push invalidates predecessor checks, Strix evidence, and review snapshots. Please produce fresh terminal checks, a trusted structured same-head Strix binding, and an independent review for de4d95a. The trusted workflow must preserve the failure-marker fail-closed behavior; do not convert a green status, unbound report, or manual/status-only result into approval. |
|
Exact-head review request for current commit 2acb3a4.\n\nThe ADR now records predecessor Strix run 31847996163 / job 94918215754 / artifact 9236803993. That run completed with zero findings and no failure markers, but the artifact had no evidence-binding.json and run.json had null repository/head/commit metadata, so it is bounded provider/content evidence only and not a clean protected gate.\n\nThis ADR-only push invalidates predecessor checks, Strix evidence, and review snapshots. Please produce fresh terminal checks, a trusted structured same-head binding, and an independent review for 2acb3a4. Do not convert an unbound zero-finding result or status-only/manual approval into Merge eligibility. |
|
Current-head review request for |
|
Exact-head update: ADR 0002 now records a live |
|
Correction: the current exact PR head is |
|
@opencode-agent Please perform a fresh review-only review of exact head |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head775ea133f45fdca50bd323dc95481cd5c6de6561. -
Head SHA:
775ea133f45fdca50bd323dc95481cd5c6de6561 -
Workflow run: 31852637612
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: fuzz.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: fuzz.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Docs (16 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (16 files)"]
R3 --> V3["docs review"]
Evidence --> S4["Test (19 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (19 files)"]
R4 --> V4["targeted test run"]
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headbadcf287ec0dc905b6ce839b59a99523430e35e7. -
Head SHA:
badcf287ec0dc905b6ce839b59a99523430e35e7 -
Workflow run: 31855666848
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (21 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (21 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: fuzz.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: fuzz.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Docs (16 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (16 files)"]
R3 --> V3["docs review"]
Evidence --> S4["Test (19 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (19 files)"]
R4 --> V4["targeted test run"]
|
@opencode-agent review Please perform a fresh review-only review of exact current head |
|
@cwl-noema-review review Review exact current head |
|
@opencode-agent review Review-only request for exact current head |
|
@cwl-noema-review review Review exact current head |
|
@opencode-agent review Review-only request for exact current head |
|
@cwl-noema-review review Review exact current head |
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current heada1d486a2c0a219df90672231ebed02e7e7000ebb. -
Head SHA:
a1d486a2c0a219df90672231ebed02e7e7000ebb -
Workflow run: 31861229760
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (21 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (21 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: fuzz.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: fuzz.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Docs (16 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (16 files)"]
R3 --> V3["docs review"]
Evidence --> S4["Test (19 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (19 files)"]
R4 --> V4["targeted test run"]
github-project-automation
Bot
moved this from In Progress
to Done
in naruon Platform Roadmap
2 participants
Status: Draft — valuable local-provider and Judge slice, not integration-ready
This PR is intentionally Draft. It carries a large, divergent local-MLX/Judge slice and must not outrun the accepted provider-security line, current-head review, terminal hosted checks, structured Strix provenance, or protected merge policy.
Exact identity
main@6841b71935e0b7cb98fb52bcb4709cc5100c8d87codex/local-llm-benchmark2e36f9ba773f8eae29bf8e4320905726650c1621c24614f1df4e0b0b6f7d1aaaf80006c409e19b9c: 236 ahead and 111 behind; histories diverge.MERGEABLEbutBLOCKED, auto-merge disabled, aggregate reviewREVIEW_REQUIRED.Every older SHA, review, workflow, local run, and synthetic merge is historical unless explicitly classified below.
Implemented scope
Local MLX execution
mlx://provider transport rather than treating a local service as arbitrary remote HTTPS.Audited model judgment
Authentication and operability
Current evidence
fast-mlsirm.ContextualOrchestratorJudge -> _FastMLSIJudgeAdapter -> TaskOrchestrator -> ModelClient -> mlx-lm, Gemma 4 e4b, temperature 0, thinking disabled, four boundary calls, two criteria, three ordered categories, and produced a valid row[2,2]. This is transport/shape evidence only.31800737624, Security31800737629, Tests31800737711, Semgrep31800737612, Strix31800737700, required-workflow bootstrap31800737668, Noema31800737761, and queue/close/security fan-out runs31800737737,31800737755,31800737576. Queued is not passing evidence..githubPR #1009 is Ready at2833d8a1c2f2cbb02387a2af752db51298cc64c4, but remainsBLOCKEDwith aggregateREVIEW_REQUIRED, queued hosted checks, and no qualifying independent current-head approval. Its aggregate-review scheduler gate and structured Strix artifact-hold fix still require normal protected integration; no pre-merge base-workflow result proves the changed Strix workflow.31797316247/ artifact9219034211is preserved as provider/content evidence only: noevidence-binding.json, and null repository/head/commit/report metadata. It is not a clean security gate.Review state
CHANGES_REQUESTEDreview was bound to predecessor216177f2c3524a145b24e6b9eafa3e8ca86306f5; it was dismissed as stale after verifying the current head. This dismissal is not an approval.2e36f9ba773f8eae29bf8e4320905726650c1621.Required integration order
main; do not rebase or merge destructively.No predecessor, synthetic-merge-only, status-only, no-op required-workflow, author-only, bot-comment, unbound-artifact, self-approval, Admin-bypass, or keyword-matching evidence is merge authority.
Fresh routed recheck — 2026-08-15
Latest exact-head update — 2026-08-15
The new exact head
2e36f9ba773f8eae29bf8e4320905726650c1621rejects mixed single/split token configurations and selects split tokens by requested scope; the focused security and CLI tests passed (28 passed), and the full contextual suite passed (393 passed).The predecessor Strix finding was a real CRITICAL auth bypass in the previous head, not a provider flake; its report and remediation are recorded in the Keyverse and merge-loop ADRs.
The hosted checks for this head are terminal with no failure conclusions. Strix run 31821615827 reported 0 findings, but its artifact lacks repository/head/job binding; keep that as provider/content evidence only. This Draft PR remains BLOCKED/REVIEW_REQUIRED until structured same-head provenance, an independent current-head approval, zero unresolved threads, and final refetch are present.
The follow-up review also covers mutable dataclass state: authorization re-evaluates scope-specific precedence on every call and rejects unknown scopes.
Terminal exact-head audit — 2026-08-15