Skip to content

Execute an independent post-remediation production-readiness audit #72

Description

@plx

Parent

#26

Purpose

Closing implementation tickets is evidence of activity, not proof that the resulting tool is suitable for production use. Before any 0.2.0 publication, a fresh reviewer must reassess the integrated release candidate, look for regressions and interactions across workstreams, and produce an evidence-backed PASS/FAIL/CONDITIONAL verdict.

The repository contains the required procedure at audits/production-readiness-reassessment-playbook.md. That playbook is normative for this audit; this ticket defines the gate and deliverables.

Preconditions

Do not start the formal audit until all native child epics #27#33 are closed and their release-blocking children are either fixed or have an explicit, reviewed waiver with owner, rationale, expiry, and compensating control. The audit must target one immutable candidate commit with no uncommitted changes.

The auditor should be independent of the remediation sequence where practical. At minimum, use a fresh session/context and do not rely on the original report’s conclusions as substitutes for reproducing evidence.

Required work

  1. Follow every applicable section of audits/production-readiness-reassessment-playbook.md; mark items PASS, FAIL, NOT APPLICABLE with rationale, or BLOCKED. No silent omissions.
  2. Reproduce the original release blockers first and prove their regression tests fail against the vulnerable baseline (or otherwise demonstrate the test detects the old defect) and pass on the candidate.
  3. Audit the integrated grammar/parser/dumper/verifier/CLI/library behavior, including adversarial paths, symlinks, concurrent mutation, resource bounds, platform semantics, and false-success cases.
  4. Run the full clean-build/toolchain/OS matrix, property tests, fuzzing budget, mutation/coverage checks, package/install smoke tests, docs examples, advisory/license checks, SemVer/version checks, and release-workflow dry run specified by the playbook.
  5. Inspect source and workflows manually; green tests alone are insufficient. Trace each public claim and security boundary to implementation and a test.
  6. Verify GitHub rules, protected tags/environment, trusted-publishing scope, maintainer recovery, and private security-report route without exposing secrets.
  7. Build the exact crate and binary artifacts from the candidate without publishing. Record hashes and prove smoke tests use those artifacts rather than a workspace build.
  8. Perform a defect/dependency inventory reconciliation: every finding in the 2026-07-25 audit and every child of [Epic] Production readiness and formal v0.2.0 release #26 must have an evidence-backed disposition.

Deliverables

Commit a dated reassessment report under audits/ that includes:

  • candidate commit, tool versions, OS/filesystem matrix, and environment limitations;
  • command/evidence ledger with results and artifact hashes;
  • original-finding traceability table;
  • new findings with severity, reproduction, impact, and proposed ticket links;
  • residual risks and accepted waivers with owner/expiry;
  • explicit verdict: PASS, CONDITIONAL, or FAIL for production use and for release publication separately.

Attach only non-sensitive logs/evidence. Redact local paths or credentials where the documented policy requires it.

Failure handling

  • Any false-success, containment escape, unsafe publish path, unsupported claimed platform, unreconciled P0/P1 finding, or untested release artifact makes the release verdict FAIL.
  • Open a self-contained issue for every new actionable finding, label it using this program’s taxonomy, add native dependencies, and block the publication ticket.
  • Do not repair substantive defects in the audit PR. A fix invalidates affected evidence; land it separately and rerun the impacted audit sections plus the final gate.

Acceptance criteria

  • Every playbook section has a recorded disposition and reproducible evidence.
  • All original findings and remediation issues are reconciled against the exact candidate commit.
  • The exact unpublished artifacts pass all required platform/install/smoke checks.
  • Repository, publishing, security, and recovery controls are verified without relying on assertions.
  • The committed report gives an unambiguous verdict and identifies every residual risk/waiver.
  • This issue closes only on an unconditional PASS for publication; CONDITIONAL or FAIL leaves Publish the audited v0.2.0 crate and GitHub release from one immutable commit #73 blocked.

Dependencies

Blocked by epics #27, #28, #29, #30, #31, #32, and #33. Publication is separately represented by #73 and is not authorized by beginning this audit.

Approved #35 filesystem trust handoff

The maintainer-approved filesystem trust decision is recorded in PR #85 and the normative docs/v0.2-contract.md. It is a binding input to this issue, but it does not make the current runtime conformant.

Audit the complete 65-row adversarial trust ledger against the immutable candidate. Reproduce every applicable negative guide, intermediate-containment, output no-clobber/cleanup, confidentiality-sentinel, root-alias/unresolved-parent, creator-race, Windows device/reparse, and platform case.

Prove sentinel content is absent from all stdout, stderr, logs, annotations, summaries, and retained evidence. Distinguish stable-filesystem guarantees and observed-mutation failures from the explicitly unsupported hostile-replacement model, and reject any sandbox claim not supported by implementation evidence.

Preserve the exact normative outcomes and focused ownership. If this issue changes runtime behavior, capture its own red-before-change evidence; coordinating work must not claim another issue's trust rows. Never claim sandbox or hostile-concurrent-replacement safety.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Critical, urgent issuescomponent: cross-cuttingWork spanning multiple product componentscomponent: governanceRepository controls, ownership, and maintenance policycomponent: test-infraFixtures, property tests, fuzzing, coverage, and benchmarksdiscovered-by-codexIssues discovered via review by codex.domain: correctnessBehavioral correctness and accurate validation resultsdomain: governanceProject policy, support, ownership, and release authoritydomain: reliabilityFail-closed, deterministic, and robust operational behaviordomain: securityFilesystem, data-boundary, workflow, or secret-handling securitydomain: supply-chainDependencies, workflows, provenance, and publishing trustproduction-readinessPart of the audited path from prototype to production-ready userelease-blockerMust close before the production-readiness release gate can passtarget: v0.2.0Must be resolved or explicitly waived before the v0.2 production auditworkflow:production-readinessIncluded in automatic production-readiness work selectionworkflow:production-readiness-gateEpic, audit, publication, or program gate requiring landed prerequisites

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions