Skip to content

Add gateReview to fail a pipeline on findings, matching ci review --fail-on - #41

Open
durang wants to merge 1 commit into
prooflinedev:mainfrom
durang:feat/gate-review
Open

durang wants to merge 1 commit into
prooflinedev:mainfrom
durang:feat/gate-review

Conversation

@durang

@durang durang commented Nov 11, 2026

Copy link
Copy Markdown

gateReview(outcome, { failOn }) measures a ReviewOutcome from waitForReview against a WARNING or SERIOUS threshold with the rules of proofline ci review --fail-on. It returns REACHED, NOT_REACHED, NOT_COMPLETE or UNAVAILABLE, which correspond to the CLI's exit codes 3, 0, 2 and 4.

import { gateReview } from "@proofline/sdk";

const outcome = await client.waitForReview(organizationId, reviewId);
const result = gateReview(outcome, { failOn: "SERIOUS" });
if (result !== "NOT_REACHED") {
  console.error(`Proofline: ${result}, ${outcome.reviewUrl}`);
  process.exitCode = { REACHED: 3, NOT_COMPLETE: 2, UNAVAILABLE: 4 }[result];
}

Who this helps

Node pipelines, deploy scripts and coding agents that already call waitForReview and want to stop on findings. Today they have to rebuild the rule from the CLI's Rust source, or spawn the CLI separately and read its exit code. Each part of the rule is easy to get subtly wrong: completeness decides first, missing counts must not read as a pass, an unknown severity reaches every threshold, and CODE findings count only from a complete record of the review's own commits.

Decisions

  • Port the CLI's rule, not a new one. The order and edge cases follow ci_verdict, code_reaches, code_assessment and Severity::reached_by in prooflinedev/cli, so a Node job and a CLI job agree on the same review. Measure a review against a severity threshold like ci review --fail-on sdk-python#39 proposes the same helper for Python; the names and the four results match it.
  • Four results, not a boolean. "Incomplete" and "can't be measured" are different failures with different exit codes; a boolean would let an incomplete review read as a pass.
  • The server's counts decide, not the findings on the page. open_items.open_findings / serious_findings already exclude dismissed findings and unexpired risk acceptances, so nothing is recounted from assessments[].findings.
  • Fail closed. A timed-out wait, or missing, non-integer or negative counts, give UNAVAILABLE, because the generated types don't validate them at runtime. An invalid failOn throws TypeError.
  • One definition of a usable CODE record. The commit-match and withheld-finding checks move from codeAssessment into recordedCode; codeAssessment keeps its rule that a CODE result never stands in for an environment result and calls recordedCode. waitForReview behaves exactly as before.
  • Separate from merge_gate. The threshold is the caller's, as with --fail-on; the README says so.
  • Hand-written parts only: src/index.ts, tests/gate.test.mjs, README. No generated code, contract or workflow changes.

Verification

  • tests/gate.test.mjs reads each case through waitForReview against a local HTTP fixture, so it gates real ReviewOutcomes. It ports the CLI's ci_verdict test table (counts × threshold, incomplete/failed/stopped, a gap deciding before missing counts) and adds: timed-out wait, malformed counts (string, negative, fractional, boolean, missing), CODE findings by severity and lifecycle (code-only and beside environments, unknown severities), partial or foreign-commit CODE records, and invalid thresholds.
  • The full CI sequence passes locally on Node 22+: pnpm generate && git diff --exit-code, pnpm lint, pnpm typecheck, node check-claims.mjs, pnpm build, pnpm test (63 tests, 0 failures), and npm pack.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant