Skip to content

Add a bounded Store-backed evidence correction pilot - #21

Merged
k2jac9 merged 6 commits into
mainfrom
feat/controlled-correction-pilot-20261116
Nov 16, 2026
Merged

k2jac9 merged 6 commits into
mainfrom
feat/controlled-correction-pilot-20261116

Conversation

@k2jac9

@k2jac9 k2jac9 commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Adds and evaluates an isolated evidence-correction pilot over existing scoped-model-reuse fixtures. The completed local run used 13 calls and 2,515 reported tokens with no unresolved holds. After the shared bootstrap, Store repair used 751 tokens/four calls versus 1,167 tokens/six calls for fresh recomputation: 416 fewer tokens (35.6%) in these particular comparisons. Including the same bootstrap in each hypothetical lifecycle gives a 23.6% difference.

All four conditions behaved as specified: corrected evidence changed deployment to hold, unrelated policy changes reused the report judgment, withdrawal refused before dispatch, and surviving occurrence B reused its own accepted origin. Three generated pairs matched exact requests and checked outputs; withdrawal issued no requests. Independent audit reconciled raw responses, claims, SQLite lineage/admission, all 13 CPU observations and owned-runtime shutdown.

The supervisor pins a cached local model/runtime, retains partial responses and unknown holds, and limits one run to 13 calls and one original 30-minute window. Independent review reproduced and fixed total-HTTP-deadline and delayed-container-cleanup defects. Configured token envelopes are explicitly not proven hard budgets.

This PR targets main after #20, #19 and #18 merged. Earlier exact-head hosted CI at 31f303bf3f5cbf732ee91699e1e110bcdf990e6d passed all five runtime lanes: each ran 365 Rust tests, 399 Python script tests, 18 reference and 8 oracle checks plus integration/example stages, with no warnings or skips. The previous-stable checkout job also passed. CI.

Observed results link the full recorded trace and protocol. Execution was frozen at f669411; the final commit adds observations only.

One correlated synthetic example per condition does not establish population accuracy, latency benefits or held-out incident quality. The submission report remains pinned to 20fab84. No dataset expansion, new AutoScientist job or remote-provider inference was performed. The recorded unintended host rustup installation did not change the accepted project runtime.

Final integration at e9e5e9602efb2c6475db6c73229820b72d03de5a passes both push and pull-request CI, with all five runtime lanes and the checkout compatibility job successful. The stable lane runs 365 Rust, 462 script, 18 reference and 8 oracle tests, with no warnings or skips, plus the fixture integration stages. The combined tree matches the independently reviewed composition with only the separately reviewed receipt-watchdog test/documentation correction inherited from #18. These integration checks made no new model or provider requests; the measured pilot remains frozen to its recorded source.

@k2jac9
k2jac9 marked this pull request as ready for review November 16, 2026 06:24
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T07:06:44.650917Z e9e5e96 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@k2jac9
k2jac9 changed the base branch from feat/evidence-reliability-20261115 to main November 16, 2026 06:51
@k2jac9
k2jac9 merged commit 109798d into main Nov 16, 2026
12 checks passed
@k2jac9
k2jac9 deleted the feat/controlled-correction-pilot-20261116 branch November 16, 2026 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant