Skip to content

Implement Proof Mode artifact validation with Zod schemas #2

@CryptoThaler

Description

@CryptoThaler

Context

CASM's Proof Mode is the transparency mechanism that makes every swarm decision auditable. The skills/proof-mode/SKILL.md defines the requirement: emit four deterministic artifacts when --proof-mode true is passed. The README.md contains full JSON Schema definitions for all four artifacts. This issue tracks converting those specs into working Zod validation and emission logic.

Objective

Implement the Proof Mode artifact pipeline: validate, emit, and snapshot the four transparency artifacts that make CASM's cybernetic control loop inspectable.

Artifacts to Implement

1. policy_decisions.json

  • Run context (run_id, template, repo, commit)
    • Ruleset applied (id, sha256 hash, source)
      • Risk score with components and weights
        • Signal evidence (path sensitivity, diff patterns, dependency changes)
          • HITL gate status and approval chain

2. routing_decisions.json

  • Routing strategy and constraints (budget, latency SLO, quality floor)
    • Per-task model selection with candidate ranking
      • Decision reason codes and fallback triggers

3. cost_forecast.json

  • Model price registry snapshot
    • Forecast vs. actual cost comparison (LLM, tools, infra)
      • Unit economics: cost per PR, cost per issue fixed, cost per 1k LOC audited
        • Budget enforcement status

4. run_replay.sh

  • POSIX-compatible deterministic re-run script
    • Pins: template, repo, commits, ruleset hash, routing strategy, budget
      • No embedded secrets — requires env vars

Requirements

  • Zod schemas in @casm/core matching the JSON Schema definitions in README
  • - [ ] Artifact emitter that writes to .casm/artifacts/ directory
  • - [ ] All artifacts use stable key ordering and deterministic IDs (ULID recommended)
  • - [ ] No raw code snippets or secrets in any artifact — use sha256 diff hunk hashes
  • - [ ] Golden fixture snapshots under examples/ for regression testing
  • - [ ] --proof-mode true flag in CLI triggers emission
  • - [ ] Unit tests validating schema compliance

Acceptance Criteria

casm run --template noop --repo ./examples/demo-repo --proof-mode true
# Produces:
#   .casm/artifacts/policy_decisions.json   (valid against Zod schema)
#   .casm/artifacts/routing_decisions.json  (valid against Zod schema)
#   .casm/artifacts/cost_forecast.json      (valid against Zod schema)
#   .casm/artifacts/run_replay.sh           (executable, no secrets)

Why This Matters

Proof Mode is what separates CASM from other agent orchestrators. Every routing decision, policy gate, and cost calculation is traceable. This is the foundation for HITL trust — operators approve changes because they can verify why the system made each decision, not because they blindly trust the output.

References

  • skills/proof-mode/SKILL.md — emission rules and constraints
    • packages/router/AGENTS.md — routing transparency requirement
      • packages/policy/AGENTS.md — policy artifact constraints
        • README.md — full JSON Schema definitions for all four artifacts

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions