Skip to content

feat(core): add fail-closed benchmark release decision contract - #218

Merged
seonghobae merged 73 commits into
mainfrom
feat/release-benchmark-decision-contract
Aug 27, 2026
Merged

feat(core): add fail-closed benchmark release decision contract#218
seonghobae merged 73 commits into
mainfrom
feat/release-benchmark-decision-contract

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Buyer-visible gap

Issue #203 requires a deterministic commercial-release decision that cannot promote missing, inconclusive, duplicated, failed, oversized, ambiguously represented, or non-canonical mandatory-suite evidence into acceptance. This PR owns only that pure fail-closed decision boundary; benchmark execution, thresholds, corpus licensing/statistics, signed artifact binding, and suite runners remain outside scope.

Implemented boundary

originweave_core::release_acceptance provides the five mandatory suite identities; deterministic accepted / accepted_with_declared_limitations / rejected / inconclusive aggregation; exact failed/inconclusive/missing evidence; fail-closed duplicate-suite and duplicate-limitation-claim rejection; bounded buyer-visible limitations; Unicode 17.0.0 presentation-safety validation; NFC and canonical surrounding-whitespace rejection; meaningful alphanumeric-content validation; and no model/status/author override path.

A known threshold failure remains rejected even when other evidence is incomplete. Missing or inconclusive mandatory evidence is never promoted to acceptance.

Current-head repair

A current-head coverage failure exposed a real evidence defect in the public generic DeclaredLimitation::new(impl Into<String>, impl Into<String>) boundary: LLVM emitted coverage mappings for multiple caller-specific monomorphizations of the validation-heavy constructor, leaving exact branch coverage below the repository's required 100% even though the behavioral regressions passed.

Exact repair commit 911ea33d8a5aca7673307bb6fdcad4b450f5c111 keeps the public API unchanged but converts both inputs to owned String and delegates all validation to one private monomorphic from_owned_text helper. The validation order and fail-closed semantics remain unchanged: empty/whitespace-only, surrounding whitespace, over-budget UTF-8 text, non-NFC Unicode, disallowed presentation/control characters, and punctuation-only content are still rejected with typed errors. No gate or authority was weakened.

Exact-current state and evidence

Protected main is exact b05d5acca82b9d916ada2c8e82f59f92a89817e1. This PR is open, non-draft, mergeable, and exact head 911ea33d8a5aca7673307bb6fdcad4b450f5c111 against that base.

On unchanged exact head 911ea33d8a5aca7673307bb6fdcad4b450f5c111:

  • Rust contracts: success on the exact head;
  • Production coverage: success on the exact head; scripts/ci/verify_coverage.py reports production functions, lines, regions, and branches are all 100% covered;
  • exact coverage artifact 9621936916, exact-coverage-911ea33d8a5aca7673307bb6fdcad4b450f5c111, SHA-256 2812d6958489fc7196fa56988efef6ac19a9ce8ecec40245f1c11b530a15c0e1;
  • exact release-acceptance tests cover generic success input shapes, borrowed-input validation exits, punctuation-only rejection, non-NFC/canonical-text rejection, resource limits, Unicode-17 default-ignorable rejection, and separate U+2028/U+2029 rejection;
  • opencode-review, noema-review, coverage evidence/source-tree, Semgrep OSS/SAST, Trivy filesystem scan, OSV scanner, dependency review, and Scorecard exact-head checks are success where bound to this PR/head;
  • strix remains in progress and therefore is not passing evidence.

Formal review inventory contains automated COMMENTED evidence only; GitHub records no qualifying counted independent APPROVED review. Fresh inline-thread inventory has no unresolved thread: the punctuation-only defect is fixed, and the remaining validation-order and Unicode-superset notes were verified as informational fail-closed behavior before resolution.

No predecessor-head check, review, scanner result, status, synthetic merge, queued/skipped/cancelled evidence, or model output is promoted as current-head proof or independent approval.

Scope and authority boundary

This PR does not claim release readiness and does not implement #201 signed-artifact authority, #202 enterprise operability, browser/security/recovery suite runners, benchmark persistence, model routing, or benchmark statistics. It introduces no database and no workflow mutation.

Protected-main AGENTS.md remains authoritative. Scheduled repository work does not merge, self-approve, tag, publish, mutate workflows, add secrets, force-push, destructively rebase, or weaken gates.

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

새로운 공개 release_acceptance 모듈이 필수 벤치마크 결과와 선언된 제한사항을 검증하고 릴리스 결정을 생성합니다. 테스트는 fail-closed 처리, 입력 경계, 유니코드 규칙 및 결과 보존을 검증합니다.

Changes

릴리스 수용

Layer / File(s) Summary
수용 계약과 입력 검증
crates/originweave-core/src/release_acceptance.rs, crates/originweave-core/src/root.rs
공개 벤치마크 스위트, 제한사항, 결정, 오류 및 보고서 형식을 추가합니다. 제한사항 입력의 텍스트와 자원 한도를 정의합니다.
결정 집계와 보고서 생성
crates/originweave-core/src/release_acceptance.rs, crates/originweave-core/tests/release_acceptance.rs
decide_release가 필수 스위트 증거와 제한사항을 집계합니다. 누락, 실패, 불충분, 중복 입력을 fail-closed로 처리하고 결정 보고서에 세부 증거를 보존합니다.
수용 동작 검증과 릴리스 기록
crates/originweave-core/tests/release_acceptance*.rs
빈 값, 주변 공백, 제어 문자, 모호한 유니코드 서식 문자, UTF-8 바이트 초과 및 제한사항 개수 초과를 검증합니다. 허용되는 국제 문자와 공백의 원문 보존도 검증합니다.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: 🟡 Moderate · up to 28cd9

The release-decision contract still accepts limitation text with surrounding spaces, which can let equivalent limitations evade duplicate detection and create ambiguous buyer-visible release reports. This is localized but affects fail-closed correctness; merge should wait for the input-validation fix and documentation follow-up.

Sequence Diagram(s)

sequenceDiagram
  participant BenchmarkEvidence
  participant decide_release
  participant DeclaredLimitation
  participant ReleaseDecisionReport
  BenchmarkEvidence->>decide_release: 결과 전달
  DeclaredLimitation->>decide_release: 제한사항 전달
  decide_release->>decide_release: 입력과 필수 스위트 검증
  decide_release->>ReleaseDecisionReport: 릴리스 결정 및 증거 반환
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 필수 벤치마크 스위트에 대한 fail-closed 릴리스 판정 계약을 추가하는 주요 변경을 정확하고 간결하게 요약합니다.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/release-benchmark-decision-contract

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 22, 2026 16:17

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread crates/originweave-core/src/release_acceptance.rs

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread crates/originweave-core/src/release_acceptance.rs
@seonghobae
seonghobae merged commit e61ef9d into main Aug 27, 2026
31 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant