Skip to content

calibration: self-host Postgres parity — verify every calibration surface on the pg shim + pg-capable corpus/track-record/backfill CLIs #8171

Description

@JSONbored

Problem

The entire #8082 calibration/backtest system reads and writes through the review engine's env.DB. On the cloud deployment that is D1 (verified: the ledger of record, the corpus, the persisted backtest runs, and the trend/status endpoints all live there). The self-host stack backs the same code with Postgres via the cf-workers shim — which means calibration parity for self-hosters (and for our own server, if review execution consolidates onto it) is presumed but unverified, and the two maintainer CLIs are D1-only by construction (wrangler d1 execute): the corpus export, track record, and the #8157 backfill cannot run against a self-host Postgres at all.

Requirements

⚠️ Required pattern. The Worker-path surfaces must be verified through the existing selfhost Postgres integration harness (test/integration/selfhost-pg*, the .github/workflows/selfhost.yml PG service pattern) — real Postgres, not the in-memory double. The CLI variants must reuse each existing script's PURE CORE untouched (backtest-corpus-export-core, backfill-calibration-corpus-core) with a pg thin-IO wrapper — never a second transform implementation.

  1. Verification pass (Worker paths): capture writers (signal.rule_fired/human_override incl. raw context), backtest persistence (calibration.* events), /v1/internal/calibration-trend, /v1/internal/calibration/satisfaction-floor, and the loosening loop's system_flags read/write — each exercised against real Postgres through the shim, with a regression test where gaps appear.
  2. pg thin-IO CLI variants: corpus export, track record, and backfill runnable against a self-host Postgres (--pg <conn> or an env-driven driver switch mirroring how the selfhost stack already picks its DB), same dry-run/report/idempotency contracts.
  3. Docs: the backtest-calibration docs page gains a short self-host section (where the corpus lives, how to run the CLIs against your own Postgres).

Deliverables

  • PG integration coverage for every calibration surface listed above
  • pg-capable corpus-export / track-record / backfill CLIs (shared cores untouched)
  • Self-host section on content/docs/backtest-calibration.mdx

Test Coverage Requirements

Worker-path verification lands in the selfhost-pg integration suite (real-PG, per the existing pattern). CLI wrappers stay thin-IO (scripts exemption); any new pure driver-selection logic gets full unit coverage.

Links & Resources

#8082 (epic), #8157/#8170 (the backfill this ports), src/selfhost/** + test/integration/selfhost-pg* (the shim + harness), scripts/backtest-corpus-export.ts / scripts/backfill-calibration-corpus.ts (the D1 thin-IO wrappers being paralleled)

Boundaries

No changes to the pure cores or event shapes — parity means the same rows in a different store. No cross-store sync: each deployment's corpus is its own, private by design (the federated surface is separate).

maintainer-only — deployment-parity work across the product's storage drivers.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

Status
Done
Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions