Skip to content

feat: add DD-078 evidence report ledger - #179

Draft
larimonious wants to merge 7 commits into
feat/dd078-slice-1a-truth-modelfrom
feat/dd078-slice-1b-evidence-report
Draft

feat: add DD-078 evidence report ledger#179
larimonious wants to merge 7 commits into
feat/dd078-slice-1a-truth-modelfrom
feat/dd078-slice-1b-evidence-report

Conversation

@larimonious

Copy link
Copy Markdown
Contributor

Summary

Implements DD-078 Task/Slice 1B on top of Slice 1A (#178):

  • adds the versioned RunReport evidence ledger and committed schema v1;
  • makes human output, JSON, coverage, thresholds, and exit status projections of one ledger;
  • adds intent check --json and intent coverage --json plus implementation/executable/verified thresholds;
  • preserves concrete legacy technical tests, feature scenarios, expanded runs, preconditions, and component scenarios through fail-closed compatibility projection;
  • restores feature/scenario human output and failure diagnostics without reintroducing ad hoc summary arithmetic;
  • keeps component linkage authority honest and represents unsupported inherent behavior as selected unbound/no-result evidence;
  • rejects duplicate binding and obligation identities so generated/explicit collisions cannot produce false success.

Truth guarantees

  • Every selected required binding must be bound, executable, current, resolved, passed, and carry concrete evidence atoms.
  • Missing, extra, reordered, ambiguous, warning, pending, skipped, zero-assertion, or otherwise unmappable live results remain visible and fail closed.
  • Pass-after-failure remains flaky with attempt history.
  • Advisory and profile-excluded evidence remains visible but cannot satisfy required obligations.
  • @implements/linkage metadata cannot prove execution.
  • Fast/profile-qualified evidence cannot become a global/full claim.
  • Duplicate obligation or binding IDs cannot share evidence or manipulate coverage denominators.

Validation

Exact head: 296c0a6bd747e92830a9b4333d95393addb7bbe9

  • cargo fmt --check
  • cargo test --lib
  • RUST_MIN_STACK=8388608 cargo nextest run2,009/2,009 passed
  • RUST_MIN_STACK=8388608 cargo test --doc
  • cargo build --profile dev-release
  • schema JSON parse and integration validation
  • focused report tests — 27/27 passed
  • Intent Studio tests — 39/39 passed
  • git diff c3d179e...HEAD --check
  • exact-head Greptile — 5/5, safe to merge, no comments
  • exact-head immutable cold review — PASS, no blocker/major findings

Review notes

This is a stacked PR. Its base is feat/dd078-slice-1a-truth-model / #178, not main.

JUnit, project discovery, policy/grants, planning, execution providers, fixtures/resources, and later DD-078 slices remain out of scope.

@greptile-apps

greptile-apps Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR implements the RunReport versioned evidence ledger (DD-078 Slice 1B), making human output, JSON, coverage metrics, threshold enforcement, and exit status all projections of one central ledger rather than ad-hoc summary arithmetic. It also adds intent check --json and intent coverage --json with --min-implementation/executable/verified thresholds, and restores correct [LINKED]/[UNLINKED] rendering for the implementation profile.

  • New src/verification/ module (ids.rs, model.rs, report.rs): introduces StableId/SourceSpan, VerificationTruth/Obligation, and the RunReport with fail-closed evidence ledger, flaky detection, duplicate obligation/binding ID rejection, and two human renderers.
  • src/intent.rs parser extension: adds stable id:/verification:/rationale: declarations; backward-compatible via IdMode::Compatibility; enforces uniqueness and format at parse time.
  • src/main.rs wiring: replaces ~250 lines of per-command summary arithmetic with RunReport and propagates exit code in both check and coverage paths.

Confidence Score: 5/5

Safe to merge. All changes are additive and backward-compatible through IdMode::Compatibility, covered by 2,009 nextest cases plus focused report and schema-validation tests.

The ledger model, exit-code logic, coverage projection, and both renderers are thoroughly tested. The previously flagged issues are verifiably fixed at this head with regression tests covering both paths. No new correctness issues were identified.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
src/verification/report.rs New 3,940-line RunReport evidence ledger with coverage projection, threshold evaluation, exit-code logic, and both human renderers. Previously flagged issues resolved and covered by tests.
src/intent.rs Extends Feature/Scenario with StableId fields; wires id:/verification:/rationale: parsing with duplicate detection; backward-compatible via IdMode::Compatibility.
src/main.rs Replaces ad-hoc summary arithmetic with RunReport, adds --json/--min-* flags to intent coverage, removes ~250 lines of redundant output code.
tests/verification_truth_tests.rs New 394-line test file covering duplicate ID detection, malformed IDs, strict/compat mode, and truth model obligation linkage.
tests/intent_studio_tests.rs Replaces old JSON assertions with schema-validated RunReport, adds human-output LINKED/UNLINKED regression test, legacy feature ID coverage test, and threshold flag integration test.
tests/fixtures/verification/reports/schema-v1.json Committed JSON Schema (draft 2020-12) for RunReport v1; strict (additionalProperties: false) and used in integration tests.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[intent check / intent coverage] --> B{profile}
    B -->|live execution| C[run_compatibility_live]
    B -->|implementation| D[implementation_coverage]
    C --> C1[run_tests_against_server]
    C1 --> C2[from_live_results]
    C2 --> C3[assemble]
    D --> D1[generate_coverage_report]
    D1 --> D2[from_implementation_coverage]
    D2 --> D3[from_truth → assemble]
    C3 --> E[RunReport]
    D3 --> E
    E --> F{output}
    F -->|--json| G[serde_json stdout]
    F -->|human| H{implementation profile?}
    H -->|yes| I[render_implementation_human LINKED/UNLINKED]
    H -->|no| J[render_human PASS/FAIL/DOCS]
Loading

Reviews (3): Last reviewed commit: "fix: report implementation coverage trut..." | Re-trigger Greptile

Comment thread tests/intent_studio_tests.rs
Comment thread src/verification/report.rs
@larimonious
larimonious changed the base branch from feat/dd078-slice-1a-truth-model to main July 31, 2026 02:48
@larimonious
larimonious changed the base branch from main to feat/dd078-slice-1a-truth-model July 31, 2026 02:50
@larimonious
larimonious changed the base branch from feat/dd078-slice-1a-truth-model to main July 31, 2026 02:50
@larimonious larimonious reopened this Jul 31, 2026
@larimonious
larimonious changed the base branch from main to feat/dd078-slice-1a-truth-model July 31, 2026 02:50
Comment thread src/verification/report.rs
@larimonious
larimonious changed the base branch from feat/dd078-slice-1a-truth-model to main July 31, 2026 21:18
@larimonious larimonious reopened this Jul 31, 2026
@larimonious
larimonious changed the base branch from main to feat/dd078-slice-1a-truth-model July 31, 2026 21:18
@larimonious
larimonious marked this pull request as draft August 5, 2026 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant