[Feature] Hash-chained, verifiable audit log with retention and export controls

[dgenio]

Summary

Strengthen the audit trail from "recorded" to "cryptographically verifiable": each persisted ActionTrace carries an HMAC over its own content plus the previous record's signature (a hash chain), and a verify_chain() API detects any insertion, deletion, or mutation in the log.

Why this matters

The project's headline value is authorization plus auditable proof of what ran. A hash-chained log upgrades that proof from "we wrote it down" to "any tampering is detectable on replay" — the property compliance reviews increasingly look for (EU AI Act Article 12 guidance converges on append-only, hash-chained logs as the technical standard). It also differentiates the kernel from gateway-style tools whose audit logs are plain database rows.

Proposed scope

• Add record_hash and prev_hash fields to the persisted trace record envelope (not to ActionTrace semantics).
• Chain via HMAC-SHA256 keyed by WEAVER_KERNEL_SECRET (consistent with the existing token signing approach), with a documented genesis record.
• TraceStore.verify_chain() -> ChainVerificationResult reporting first divergent record on failure.
• Deterministic canonical serialization for hashing (sorted keys, explicit encoding) so verification is reproducible across processes.
• Retention helper: prune(before=...) that preserves verifiability of the remaining suffix (records a checkpoint of the pruned head).
• Document the integrity model and its limits honestly in docs/security.md (a host that controls the secret can forge a chain; this is tamper-evidence against post-hoc edits, not non-repudiation).

Implementation notes

• Builds on the persistent trace store work (pluggable backends); chain logic should live in one module, applied by any backend.
• Reuse the HMAC secret-loading path from tokens.py (WEAVER_KERNEL_SECRET, dev-secret fallback with warning) — never log key material (see docs/agent-context/invariants.md).
• Canonicalization: follow the existing deterministic-output rule (no randomness) — json.dumps(..., sort_keys=True, separators=(",", ":")).
• Tests must cover: valid chain verifies; mutated record detected; deleted record detected; reordered records detected; prune-then-verify succeeds; wrong secret fails verification with a clear error from errors.py (no bare ValueError).

Acceptance criteria

• Every persisted trace record carries prev_hash/record_hash; chain verifies end-to-end.
• verify_chain() detects mutation, insertion, deletion, and reordering, identifying the first bad record.
• Pruning preserves verifiability of retained records via a checkpoint.
• docs/security.md documents the integrity model, the threat it does and does not cover, and key-management caveats.
• Error cases raise typed exceptions with tested messages.
• CHANGELOG updated.

Out of scope

• External timestamping / transparency-log anchoring (e.g., publishing chain heads) — possible follow-up.
• Encryption of trace contents at rest.
• Export formats (covered by Interop: export action traces for downstream lesson extraction #94 / Export ActionTrace to OpenTelemetry / LangWatch-compatible trace format #125) — though exported records should include chain fields.

References

• EU AI Act Article 12 logging guidance (append-only, SHA-256 hash-chained logs as common practice).
• RFC 6962 (Certificate Transparency) for the verifiable-log mental model.
• Existing HMAC envelope signing in federation_discovery.py (sign_manifest/verify_manifest) as in-repo precedent.

---

Priority: P0 · Effort: M · Impact: Transformational

[dgenio]

Closing as completed — this is implemented and merged.

The hash-chained, verifiable audit log shipped in v0.11.0 via PR #148 (commit e593456, on main at e78a863).

Evidence against the acceptance criteria:

• src/weaver_kernel/stores/audit_chain.py — each persisted record carries prev_hash/record_hash (TraceRecord), chained via HMAC-SHA256 keyed by WEAVER_KERNEL_SECRET over a canonical JSON serialization (compute_record_hash, canonical_json), with a documented GENESIS_HASH.
• verify_chain() -> ChainVerificationResult detects mutation, broken links (insertion/reordering), and deletion, reporting the first divergent record; exposed on both backends (SQLiteTraceStore.verify_chain(), JsonlTraceStore.verify_chain()).
• Retention: SQLiteTraceStore.prune() drops old records while preserving verifiability of the retained suffix via a checkpoint hash (checkpoint_hash/checkpoint_next_seq).
• docs/security.md ("Audit-log integrity (hash chain)") documents the integrity model honestly as tamper-evidence, not non-repudiation, including the secret-holder caveat and the explicit out-of-scope of tail-truncation/whole-log deletion.
• Typed errors on corrupt/duplicate store data (no bare ValueError); covered by tests in tests/test_audit_chain.py.

The out-of-scope items you listed (external timestamping/transparency-log anchoring, at-rest encryption) remain available as follow-ups.

---

Generated by Claude Code