Repository navigation
Add a bounded Store-backed evidence correction pilot - #21
Merged
Merged
Conversation
k2jac9
marked this pull request as ready for review
November 16, 2026 06:24
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
k2jac9
changed the base branch from
feat/evidence-reliability-20261115
to
main
November 16, 2026 06:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds and evaluates an isolated evidence-correction pilot over existing scoped-model-reuse fixtures. The completed local run used 13 calls and 2,515 reported tokens with no unresolved holds. After the shared bootstrap, Store repair used 751 tokens/four calls versus 1,167 tokens/six calls for fresh recomputation: 416 fewer tokens (35.6%) in these particular comparisons. Including the same bootstrap in each hypothetical lifecycle gives a 23.6% difference.
All four conditions behaved as specified: corrected evidence changed deployment to hold, unrelated policy changes reused the report judgment, withdrawal refused before dispatch, and surviving occurrence B reused its own accepted origin. Three generated pairs matched exact requests and checked outputs; withdrawal issued no requests. Independent audit reconciled raw responses, claims, SQLite lineage/admission, all 13 CPU observations and owned-runtime shutdown.
The supervisor pins a cached local model/runtime, retains partial responses and unknown holds, and limits one run to 13 calls and one original 30-minute window. Independent review reproduced and fixed total-HTTP-deadline and delayed-container-cleanup defects. Configured token envelopes are explicitly not proven hard budgets.
This PR targets main after #20, #19 and #18 merged. Earlier exact-head hosted CI at
31f303bf3f5cbf732ee91699e1e110bcdf990e6dpassed all five runtime lanes: each ran 365 Rust tests, 399 Python script tests, 18 reference and 8 oracle checks plus integration/example stages, with no warnings or skips. The previous-stable checkout job also passed. CI.Observed results link the full recorded trace and protocol. Execution was frozen at
f669411; the final commit adds observations only.One correlated synthetic example per condition does not establish population accuracy, latency benefits or held-out incident quality. The submission report remains pinned to
20fab84. No dataset expansion, new AutoScientist job or remote-provider inference was performed. The recorded unintended host rustup installation did not change the accepted project runtime.Final integration at
e9e5e9602efb2c6475db6c73229820b72d03de5apasses both push and pull-request CI, with all five runtime lanes and the checkout compatibility job successful. The stable lane runs 365 Rust, 462 script, 18 reference and 8 oracle tests, with no warnings or skips, plus the fixture integration stages. The combined tree matches the independently reviewed composition with only the separately reviewed receipt-watchdog test/documentation correction inherited from #18. These integration checks made no new model or provider requests; the measured pilot remains frozen to its recorded source.