Skip to content

security: fail closed when required Strix review is unavailable #197

Description

@seonghobae

Buyer-visible merge-assurance gap

Protected main is currently 7faf7a3b8a47980113982914000e724ab6a6cda5. The organization-required Strix path can still reach a nominal successful required-check conclusion without authoritative complete semantic scan evidence. Under EgressWeave's evidence contract, unavailable, skipped, neutral, no-report, no-scannable, stale, self-declared preliminary/inconclusive, or otherwise non-authoritative reviewer evidence is non-passing.

This is an assurance/control-plane defect, not evidence that a particular EgressWeave source change contains a Strix-detectable vulnerability. Protected-main release admission already rejects missing/non-semantic required evidence before publication (#191), so this issue remains specifically the merge-time organization-required Strix evidence boundary.

Fresh exact evidence (2026-08-13)

Three current evidence shapes define the boundary.

1. No semantic scan despite required-check success — PR #212

  • protected/live base: 7faf7a3b8a47980113982914000e724ab6a6cda5;
  • exact Ready head: 417ed4b066c50584b41942c61bc68bd82348d700;
  • required Strix run 31656787007 concluded wrapper success;
  • exact-head execution reported No scannable changed files in pull request; skipping Strix quick scan.;
  • no substantive scan report was produced;
  • result: non-passing semantic evidence despite the successful check wrapper.

2. Report exists, but the report itself says the scan is preliminary — PR #203

  • protected/live base: 7faf7a3b8a47980113982914000e724ab6a6cda5;
  • exact Draft head: 9ac4bc2de5bdba2e92be184e46a42ef55627bd77;
  • required Strix run 31663947448, job/check 94334450406, completed success;
  • artifact 9167636849 (strix-reports), SHA-256 ec665dbe9b99532fff3605bc4766282be2d4074daf62d3d15343d607b901347c;
  • workflow evidence says one scannable changed file was materialized for findings attribution;
  • the first provider path hit a typed NVIDIA NIM 429 Too Many Requests, then bounded fallback reached nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5;
  • the successful internal run records scan_completed=true, but its executive summary explicitly characterizes the result as a preliminary scan and recommends further action for thorough validation; its technical analysis likewise calls the work a preliminary phase.

Therefore job=success, artifact existence, and even scan_completed=true are insufficient by themselves. A report that declares its own result preliminary/incomplete is non-passing semantic acceptance evidence. The 429 is provider/infrastructure evidence, not a source vulnerability.

3. Positive control — substantive scoped semantic scan — PR #214

  • exact Draft head: 6ecc1b2e2caa1af5293a28a5d07a5eddda6f0327 on the same protected-main base;
  • required Strix run 31655407058, job/check 94308699374;
  • artifact 9164364412, SHA-256 ef1171b8f94cd6c71f8fa44f3979c84487134218bf18dff8d34b826b3e2822c3;
  • the report records a completed scoped semantic result rather than a no-scan or self-declared preliminary outcome.

This is the positive-control evidence shape the terminal gate must preserve. It does not by itself satisfy other independent merge gates.

Historical #192 backend-unavailable evidence remains RCA history only and is not transferred to current heads.

Read-only central owner

The authoritative organization-owned incident remains ContextualWisdomLab/.github#891, [Security] Fail closed when Strix produces no authoritative scan evidence, and is still open. Protected central main is currently 6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba. .github remains read-only to the EgressWeave writer.

RCA

  • First failing boundary: Strix provider/executor output -> organization-required terminal security decision -> GitHub merge evidence.
  • Immediate causes now observed: backend/provider unavailability, no-scannable/no-report execution, and success-shaped reports whose own semantics remain preliminary/incomplete.
  • Root cause: workflow transport completion and semantic-review completion/quality are not represented as one fail-closed, machine-verifiable terminal admission decision.
  • Systemic consequence: GitHub can see a successful required check while the exact source head lacks authoritative completed semantic review.
  • Owner: organization-owned central Strix required workflow/control plane.

Falsifiable hypothesis

If the central Strix path emits and validates a typed exact-head evidence envelope whose terminal semantic status distinguishes completed-authoritative from provider failure, skipped/no-scannable, malformed/indeterminate, and preliminary/incomplete outcomes, then only the positive-control shape can pass while all observed false-green shapes become terminal non-passing.

Distinct remedies and feasibility

  1. Preferred — typed central evidence envelope plus always-running terminal gate. Bind repository, PR/source event identity, exact head, relevant base context, trusted producer, scan scope, provider execution state, semantic terminal status, findings/result digest, and artifact identity. The gate passes only completed-authoritative evidence for the exact accepted source identity.
  2. Provider/executor emits an explicit completeness disposition. Do not make GitHub acceptance depend primarily on parsing free-form model prose such as “preliminary”. A machine-readable terminal semantic status must be produced by the trusted workflow/provider boundary and validated fail-closed.
  3. Bounded retry/fallback for classified transient provider failures. 429, transport failure, timeout, or equivalent transient errors may retry within attempt/wall-clock limits; exhausted or semantically incomplete outcomes remain non-passing.
  4. Deterministic negative controls. Cover backend unavailable, empty output, no-scannable, report-missing, malformed receipt, stale head, scan_completed=true with non-authoritative/preliminary terminal status, timeout, cancelled execution, and fallback exhaustion.
  5. Deterministic positive control. Preserve a security(tls): require exact trust scalar values #214-equivalent exact-head completed scoped semantic report as passing.
  6. Rejected: clean-head churn merely to retrigger a provider, treating wrapper/report existence as semantic success, substituting CodeRabbit/OpenCode/Noema/SAST/GHAS, parsing unconstrained prose as the sole gate, copying/forking the central Strix workflow locally, adding credentials, or weakening branch/ruleset gates.

Acceptance criteria

  • Central owner fixes the required Strix result-mapping boundary without weakening branch protection, independent review, least privilege, immutable pins, or other scanners.
  • Required-check success is emitted only for trusted, machine-verifiable, exact-head completed-authoritative semantic evidence in the applicable live-base context.
  • Backend unavailable, provider/rate-limit exhaustion, timeout, empty output, skipped/no-scannable/no-report, cancelled, malformed/indeterminate, predecessor/stale/synthetic/untrusted-producer, and self-declared preliminary/incomplete semantic outcomes produce terminal non-passing evidence.
  • scan_completed=true, wrapper success, or artifact existence alone cannot satisfy the gate.
  • A typed terminal completeness/status field is authoritative; free-form model prose is diagnostic, not the primary admission API.
  • Transport state, semantic Strix result, other model reviews, SAST/GHAS, Dependency Review and merge authorization remain separate evidence classes.
  • Provider retries/fallback remain finite and classify infrastructure failures honestly without synthesizing source vulnerabilities.
  • Diagnostics do not print credentials, secret material or unnecessary source/request data.
  • Negative tests cover all observed false-green shapes, including ci: audit orphaned Actions workflow identities #203-equivalent preliminary/incomplete output after a nominally completed scan.
  • Positive control preserves a security(tls): require exact trust scalar values #214-equivalent completed exact-head semantic report as eligible to satisfy the Strix gate.
  • After corrected central logic reaches protected central main, run an unchanged EgressWeave candidate through it and prove both fail-closed negative controls and authoritative recovery.
  • Revalidate EgressWeave live ruleset, exact head/live base, all other required workflows, Dependency Review, unresolved findings and review evidence before any affected merge.
  • Keep protected-main release verifier fail-closed behavior intact; this issue does not weaken or replace release: fail closed unless protected-main required workflow evidence is proven #191.
  • Close only after protected-central repair plus protected EgressWeave consumer evidence proves corrected merge-time behavior end to end.

Non-goals

No claim that Strix is the sole security reviewer or that a current source vulnerability exists. No EgressWeave-local required-workflow fork, substitute scanner, extra reviewer credential, autonomous merge authority, release bypass, or clean-head mutation solely to retrigger provider behavior.

Related boundaries

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions