diff --git a/README.md b/README.md index 1bd444a..328cdb5 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,23 @@ -# Kairotensor +# KairoTensor Kairotensor is a Rust framework for versioned evidence memory. Its first component is a local SQLite store that records which observations an agent used and checks whether a saved result is still eligible to read or resume after evidence changes. +## AI x Epistemics sprint submission + +Author: **Alejandro Castellanos**, **Independent Research**. + +The author confirms that no KairoTensor-specific ideas, code or experiments +predate the 13-15 November 2026 sprint. The implementation and reported experiments +were developed during the sprint. This is an author declaration, not a conclusion +inferred solely from Git timestamps. Third-party dependencies and cited prior work +remain identified in their manifests and the report. + +The [submission guide](docs/submission/README.md) maps the report's claims to raw +evidence and offline reproduction commands. It includes a real Qwen reuse example +and the failed frozen recovery evaluation; neither establishes general improvement +in agent accuracy. Measured source is pinned to +`20fab84a6baa525452191ecf80eeee339f0b5ac7`; submission documentation is additional. + ## Run the withdrawal example With stable Rust, Cargo, and a C compiler for bundled SQLite, run these commands from the repository root. diff --git a/docs/submission/DEMO.md b/docs/submission/DEMO.md new file mode 100644 index 0000000..e9d9f40 --- /dev/null +++ b/docs/submission/DEMO.md @@ -0,0 +1,61 @@ +# Optional four-minute presentation + +This is a recording script. No presentation recording or live demonstration is +claimed. It can be recorded from the report and archived evidence without model +downloads or new inference. + +## 0:00-0:40 - The problem + +Show the report title and author. + +"An agent reaches a conclusion from several observations. One observation is +later withdrawn. Can the application identify which saved results may still be +used? KairoTensor records the declared evidence for a computation and checks +whether its stored result remains eligible after a correction. It checks +eligibility, not truth." + +## 0:40-1:25 - The mechanism + +Show the README withdrawal example and selective-revision contract. + +"Dependency tracking has a long history in truth-maintenance systems. Here it is +integrated with durable records at the model execution boundary. Each evidence +occurrence has an identity. A computation captures its selected +inputs and current branch. Settlement and later reads check that this context is +still applicable. Deterministic rules can name alternative groups of witnesses: +withdrawing one report need not discard a conclusion if another declared witness +group remains complete. Usage stays recorded even when work becomes unusable." + +## 1:25-2:15 - What was measured + +Show the atomic benchmark table, including the scan baseline. + +"In one dense synthetic workload, atomic edits were 13.46 times faster than +per-branch edits by the median paired ratio. The lighter full scan was faster end +to end in every matched trial. The comparison supports batching an expensive +path; it does not establish that indexing beats a scan with equivalent history." + +## 2:15-3:05 - The real model example + +Show `docs/scoped-model-reuse.md` and its four-row result table. + +"A real Qwen run reused a report judgment when only deployment policy changed. +Repair used one new call and 193 tokens; a clean rebuild used two calls and 404 +tokens. Both returned the same checked plan. Replacing the report forced a new +judgment and a hold decision. This is one synthetic example, not an estimate of +typical savings. The earlier prompt failed its negative control and is retained." + +## 3:05-3:45 - The failure matters + +Show the recovery-v2 held-out results and audit receipt. + +"A separate recovery evaluation made 20 actual Qwen calls. All responses violated +the strict JSON contract, and all three replayed policies had zero held-out +successes. Every retry missed the soft deadline. The ledger retained 5,241 tokens +of usage. A successful collection or audit is not a successful model evaluation." + +## 3:45-4:00 - What comes next + +"The next tests need representative correction streams, equally durable +baselines, concurrency and independent model replication. The reviewer package +contains the frozen evidence, reproduction commands and the limitations." diff --git a/docs/submission/EVIDENCE.md b/docs/submission/EVIDENCE.md new file mode 100644 index 0000000..043fdb2 --- /dev/null +++ b/docs/submission/EVIDENCE.md @@ -0,0 +1,43 @@ +# Evidence index + +Paths below are relative to the measured source root. The accompanying source +archive is an exact snapshot of `20fab84a6baa525452191ecf80eeee339f0b5ac7`. +CI collection success and model-task success are separate outcomes. Later +reliability commits, unmerged PR #18 and active AutoScientist runs are outside +this measured snapshot and are excluded from the report's outcomes. + +| Report claim | Raw evidence and protocol | Result and limit | +| --- | --- | --- | +| Atomic revision reduces the expensive per-branch edit path | `docs/benchmarks/atomic-revision-2026-11-13.csv`; `docs/revision-benchmark.md` | 90 rows, 30 matched triplets. Dense 128-rule/32-edit cell: 1,972.508 vs 146.650 ms median edit time; median paired ratio 13.460, range 13.244-13.563, six trials. Full scan wins total time in all 30 trials but preserves less history. | +| Bulk setup reduces setup cost | `docs/benchmarks/bulk-setup-2026-11-13.csv`; `docs/revision-benchmark.md` | 1,024-rule fixture: 3,690.273 vs 214.389 ms median setup; paired ratio 16.617, range 14.886-34.549. Scan wins median total in all five cells and 29/30 individual trials. Single-writer synthetic traces. | +| Scoped reuse saves calls in one controlled correction | `docs/evaluations/scoped-reuse-qwen-2026-11-13.json`; `docs/scoped-model-reuse.md`; `docs/evaluation-prompts.md` | Repair: one call, 193 tokens. Fresh rebuild: two calls, 404 tokens. Identical checked canary plans. Report replacement forces new judgment and hold. Seven actual generations across all four routes; one observation per route. | +| Earlier scoped-reuse prompt failed | `docs/evaluations/scoped-reuse-first-qwen-2026-11-13.json`; `docs/scoped-model-reuse.md` | Model labeled an 11/12 report as passing; validator rejected it. The historical attempt remained unsettled. A later zero-call replay records rejection; it does not rewrite the original trace. | +| Frozen recovery did not succeed | `docs/evaluations/recovery-policy-v2-2026-11-14.zip`; matching `.md` | Ten synthetic cases, 20 recorded Qwen calls, 5,241 charged tokens. All replies fenced and invalid under strict JSON contract. Each policy scores 0/6 held-out successes; policies replay shared trajectories. All retry receipts miss the soft deadline. | +| The measured snapshot has automated contract coverage | `.github/workflows/check.yml`; `reference/test_memory.py`; `reference/belief_oracle.py`; `scripts/test_*.py`; Rust integration tests | Historical exact-source CI plus local Python receipts. New incident calibration is unexecuted; fixture tests are not live-model evidence. | + +## Source and execution identities + +| Evidence | Historical source / execution | +| --- | --- | +| Atomic revision | `5d699c44ebfec97bae016a2f8965cc1d9765ffa3`; [run 43476626433](https://github.com/k2jac9/kairotensor/actions/runs/43476626433) | +| Bulk setup | `10e3a57475246f77c8eaa5a05469f1fe79361cfb`; [run 43478880682](https://github.com/k2jac9/kairotensor/actions/runs/43478880682) | +| Revised scoped reuse | [run 43491028189](https://github.com/k2jac9/kairotensor/actions/runs/43491028189); exact requests and lineage retained in the JSON trace | +| Failed initial reuse | [run 43490325882](https://github.com/k2jac9/kairotensor/actions/runs/43490325882) | +| Recovery v2 | `11234573896b68699efa00df63e88553f8d8e99a`; [run 43571699931](https://github.com/k2jac9/kairotensor/actions/runs/43571699931) | +| Main contract CI | `20fab84a6baa525452191ecf80eeee339f0b5ac7`; [run 43622988515](https://github.com/k2jac9/kairotensor/actions/runs/43622988515) | + +Both cited live-model experiments use Qwen/Qwen2.5-1.5B-Instruct, model and +tokenizer revision `989aa7980e4cf806f80c7fef2b1adb7bc71aa306`, CPU with two +threads, PyTorch `2.6.0+cpu` and Transformers `4.48.3`. Consult each execution's +manifest for dtype and hardware. Changed hosts and prompts prevent treating +different experiments as a controlled latency comparison. + +The recovery ZIP SHA-256 is +`74e8895335f304be0db1d8c705b1e448cb0588dd1b1ccff7296e4a2ff6f8af0d`. +It contains raw responses, exact requests, environment records, policy summaries +and a SQLite ledger. The audit checks consistency between them; it does not prove +that the producer's hardware, model or timestamp attestations are authentic. + +The claimed evidence boundary is application-declared dependence and eligibility. +It is not proof of truth, complete causal attribution, source authenticity or +tamper resistance. All reported tasks and workloads are synthetic. diff --git a/docs/submission/README.md b/docs/submission/README.md new file mode 100644 index 0000000..5debb5c --- /dev/null +++ b/docs/submission/README.md @@ -0,0 +1,84 @@ +# KairoTensor submission guide + +**KairoTensor: Evidence-Grounded Revision for Agent Memory** + +Alejandro Castellanos, Independent Research + +Apart Research AI x Epistemics Research Sprint, 13-15 November 2026 + +Recommended track: **Track 2: Trust Infrastructure** + +KairoTensor records the evidence supplied to an agent computation and checks +whether a saved result remains eligible after a correction or withdrawal. The +report evaluates atomic revision, scoped reuse and a failed recovery experiment. +It does not claim source authentication, tamper evidence, or general improvement +in real-world decisions. + +## Provenance + +The author confirmed on 15 November 2026 that no KairoTensor-specific ideas, code +or experiments existed before the sprint. This is an author declaration. Git +timestamps alone are not proof of when work began. Dependencies and prior work +are separately attributed in manifests and the report. + +The measured main-branch snapshot is +`20fab84a6baa525452191ecf80eeee339f0b5ac7`. Historical experiments use the source +identities in [EVIDENCE.md](EVIDENCE.md). These submission documents were prepared +after that snapshot. Later reliability commits, unmerged PR #18 and active +AutoScientist runs are excluded from the reported results. + +## Review + +- [Submission report (PDF)](kairotensor-report.pdf) and [editable manuscript](report.md). +- [Evidence index](EVIDENCE.md): findings, raw artifacts, source identities and limits. +- [Reproduction guide](REPRODUCE.md): offline summaries and consistency checks. +- [Four-minute presentation script](DEMO.md): an optional presentation, not a recording. + +The accompanying reviewer ZIP contains the seven-page report, exact source archive, +these guides, verification receipts and SHA-256 manifest. The source archive has +162 entries and SHA-256 +`e5928a4daf5b9d68cfbec0f58afdfd53d65f4202e3f1c5d54a4284baa1c0b821`. +It preserves the measured snapshot rather than incorporating later documentation. + +The 16 November finalization adds Doyle's truth maintenance system and de Kleer's +assumption-based TMS to the related work. The contribution is the durable +integration of declared evidence dependence at the model execution boundary; +dependency tracking itself is established prior work. Measured outcomes are +unchanged. The report retains a 192-word abstract, five pages through code and +data, one reference page, and one page with the ethics appendix and LLM statement. + +New receipts are in [verification/finalization-20261116](verification/finalization-20261116/README.md). +The original 15 November receipts remain unchanged and describe the earlier +six-page export. They are historical records, not checks of the final PDF. + +The repository was private when the package was prepared. The owner plans to +make it public; this preparation does not change visibility or submit an entry. +Reviewers need the ZIP or repository access to inspect the evidence. + +## Verification scope + +On 15 November, local verification passed 18 reference tests and 8 belief-oracle +tests. The scripts suite reported 203 tests with 28 skips: 175 executed without +failure. It emitted resource warnings about unclosed SQLite connections; the +warnings are retained in the reviewer receipts. Skipped checks are not passes. +Rust execution is supported by historical exact-source CI +[43622988515](https://github.com/k2jac9/kairotensor/actions/runs/43622988515), +not a fresh local Rust run during packaging. + +Finalization on 16 November repeated the offline commands from a fresh reviewer +package extraction under Python 3.15.2: the same 201 executed tests passed and +28 script tests skipped. Two unclosed-SQLite-connection resource warnings remain +in the new stderr receipt. Both benchmark summaries matched the historical +outputs byte for byte, and the frozen recovery audit again passed with the same +recorded failures. This was an evidence/package check, not fresh model execution. + +Archived benchmark summaries were recomputed. The frozen recovery ZIP passed its +read-only internal consistency audit, including 20 recorded calls, 5,241 charged +tokens and zero unresolved reservations. Its bytes were unchanged after auditing. +No new inference was run in preparing this package. A successful audit is not +independent authentication of model or hardware execution. + +The newer incident-routing calibration remains unexecuted. The current CI does +not implement the complete stable/preview/previous-stable compatibility matrix; +no such matrix is claimed here. Historical model/runtime pins are retained for +reproduction, not presented as current release recommendations. diff --git a/docs/submission/REPRODUCE.md b/docs/submission/REPRODUCE.md new file mode 100644 index 0000000..37ddef6 --- /dev/null +++ b/docs/submission/REPRODUCE.md @@ -0,0 +1,85 @@ +# Reproduce the archived checks + +These steps inspect the submitted evidence without model inference. Use Python +3.11 or newer. The source snapshot has no third-party Python requirements for +these specific summary and audit commands. + +## 1. Open the pinned source + +From the extracted reviewer package, verify the manifest and unpack the source: + +```sh +sha256sum -c SHA256SUMS +python3 -m zipfile -e kairotensor-source-20fab84.zip source +cd source +``` + +The archive places repository files directly in `source/`. If reviewing through Git instead, +use a separate checkout of `20fab84a6baa525452191ecf80eeee339f0b5ac7`. +Do not reset a working checkout that contains local work. + +## 2. Recompute the benchmark tables + +```sh +python3 scripts/summarize_revision_bench.py docs/benchmarks/atomic-revision-2026-11-13.csv +python3 scripts/summarize_revision_bench.py docs/benchmarks/bulk-setup-2026-11-13.csv +``` + +Compare with the `verification/atomic-summary.md` and `bulk-summary.md` files in +the reviewer package. Ratios are medians of matched per-trial ratios, not ratios +of medians. A lighter full-scan contract is not an equally durable baseline. + +## 3. Audit the frozen recovery records + +Choose a new temporary directory so existing evidence cannot be overwritten: + +```sh +recovery_audit_dir=$(mktemp -d) +python3 -m zipfile -e docs/evaluations/recovery-policy-v2-2026-11-14.zip "$recovery_audit_dir" +python3 scripts/audit_recovery_bundle.py "$recovery_audit_dir" +``` + +Expected: `verified_complete`, mode `live_qwen`, 10 cases, 20 recorded backend +calls, 5,241 charged tokens and zero outstanding tokens. All held-out policy +success counts are zero. The audit field `new_model_calls: 20` describes the +historical collection recorded in the bundle. Running this audit creates **zero** +new model calls. The policy replay itself also generates no model calls. + +## 4. Run the lightweight contract tests + +```sh +python3 -m unittest discover -s reference -p test_memory.py +python3 reference/belief_oracle.py +python3 -m unittest discover -s scripts -p 'test_*.py' +``` + +Packaging verification on 15 November observed 18 reference passes, 8 oracle +passes, and 203 script tests with 28 skips (175 executed, no failures). Some +script tests require separately built Rust examples and are skipped without them. +Resource warnings about unclosed SQLite connections are retained in the receipts. + +The 16 November finalization reran these exact commands after unpacking the +reviewer package, using Python 3.15.2. It observed the same 18 reference passes, +8 oracle passes, and 203 script tests with 28 skips. Two unclosed-SQLite-connection +resource warnings were emitted. The new stdout/stderr and exact runtime identity +are retained under `verification/finalization-20261116/`; the original receipts +remain unchanged. Both benchmark summary outputs matched the earlier receipt +bytes, and the recovery archive remained unchanged after the read-only audit. + +The final ZIP adds those fresh receipts to the tested payload. Its manifest and +unpacked source bytes are checked again; the source archive is identical to the +one used for the fresh command run. No new source build or model call is implied. + +## Full Rust and model reproduction + +The repository README and historical workflows contain full Rust commands. +These require a Rust toolchain, a C compiler and build capacity. On the author's +workstation, storage admission is required before builds. The packaging checks +did not rerun Rust; [exact-main CI](https://github.com/k2jac9/kairotensor/actions/runs/43622988515) +is the recorded Rust evidence. + +Live Qwen reproduction is optional and separate: follow +`docs/scoped-model-reuse.md`, `docs/corrected-handoff.md` and the frozen recovery +protocol at its recorded commit, preserving model, tokenizer and runtime pins. +It requires downloading assets and running inference. There is no claim that +the author performed a fresh live replication while preparing this package. diff --git a/docs/submission/kairotensor-report.pdf b/docs/submission/kairotensor-report.pdf new file mode 100644 index 0000000..013bebe Binary files /dev/null and b/docs/submission/kairotensor-report.pdf differ diff --git a/docs/submission/report.md b/docs/submission/report.md new file mode 100644 index 0000000..0e8eb34 --- /dev/null +++ b/docs/submission/report.md @@ -0,0 +1,87 @@ +# KairoTensor: Evidence-Grounded Revision for Agent Memory + +Author: Alejandro Castellanos; affiliation: Independent Research + +## Abstract + +Agents often retain conclusions after the evidence underlying them changes. We study a local evidence store that records the occurrences supplied to an agent computation and checks whether a saved result remains eligible after a withdrawal or correction. KairoTensor uses SQLite transactions to capture evidence views, branch heads, exact dispatch bytes and usage. For deterministic support rules, it also tracks alternative sets of witnesses and atomically revises affected verdicts. In a synthetic, single-writer benchmark, atomic revision reduced median edit time by 13.46 times for 32 dense edits across 128 rules relative to a per-branch implementation; a lighter full-scan baseline was faster end to end in all 30 matched trials. In a single synthetic live-model handoff, reuse of an unchanged report judgment reduced correction-route model usage from 404 to 193 tokens; replacing the report triggered a fresh judgment. These observations are narrow: the task was synthetic, timings were not a latency distribution, and a separate frozen recovery evaluation yielded zero successful held-out cases under its strict contract. The prototype demonstrates a testable mechanism for managing evidence changes and accounting for work, not a general claim that agent decisions become more accurate or trustworthy. + +## 1. Introduction + +An agent may make a decision from a set of observations and later receive a correction. Saving the answer alone loses a practical question: which evidence did the computation actually receive, and is the result still eligible when a source is withdrawn? Reusing an obsolete continuation can carry a superseded claim into subsequent actions even when the new evidence is available. + +KairoTensor investigates a narrower, operational question: can a local store keep a durable record of an agent computation's declared evidence and refuse stale reads or continuations, while permitting explicitly bounded reuse? The prototype provides an occurrence-level evidence ledger, revision-checked branches, and a deterministic alternative-witness mechanism. It does not authenticate evidence or infer what a model internally attended to. The caller must authenticate actors, supply all relevant dependencies, validate outputs, and correctly report usage. + +Alejandro Castellanos confirms that no KairoTensor-specific ideas, code or experiments predate the sprint. The repository history begins on 13 November 2026, and the measured revision and model experiments were committed on 13-14 November. This provenance statement is the author's declaration; commit timestamps alone cannot establish when work began. This report covers the measured snapshot 20fab84a6baa525452191ecf80eeee339f0b5ac7. Later reliability work, the unmerged hosted-integration branch and active AutoScientist runs are excluded from the reported results. + +## 2. Related work + +W3C PROV models entities, activities and agents for exchanging provenance [1]. C2PA describes signed claims about media provenance and authenticity [2]. KairoTensor focuses on application-level read and continuation eligibility after a local evidence-view change. Its SQLite history is not signed or tamper-evident, and no standards compatibility or comparative evaluation is claimed. + +Doyle's truth maintenance system records reasons for program beliefs and revises those beliefs when assumptions change [3]. De Kleer's assumption-based TMS maintains supporting assumption sets so a problem solver can reason across alternative contexts [4]. KairoTensor's alternatives of conjunctive witness groups draw on this established tradition. They are caller-declared availability rules, not a new dependency-tracking formalism or a complete implementation of either TMS. An admitted witness is not verification that a real-world claim is true. + +The contribution studied here is a durable integration of declared evidence dependence with the model execution boundary: occurrence identities and admission state, frozen input and dispatch bytes, revision-checked settlement and reuse, and retained usage when work becomes stale. The experiments evaluate costs and one bounded reuse example of that integration. They do not establish algorithmic novelty in truth maintenance, equivalence to an ATMS, or improved semantic reliability. + +## 3. Methods + +### Store and revision + +The Rust store gives each observation occurrence an identity and tracks admission for an actor and scope. Beginning an attempt freezes the selected bytes, ordered occurrence IDs, view epoch and expected branch head. Dispatch can record exact adapter request bytes. Settlement checks the captured view and branch before advancing the head and records usage even when a delivery is stale. Current reads and resumes recheck eligibility. A withdrawal cannot revoke bytes already delivered to an external caller. + +For deterministic support, each rule contains alternatives of conjunctive evidence groups. SQLite maintains counts for affected clauses and publishes changed verdicts in the same transaction as a membership edit. An unchanged verdict can retain its head. An independent evaluator checks the incremental implementation across all admission subsets of four observations [5]. The implementation assumes the caller's rule and witness assignments are appropriate; an admitted witness is not proof of truth. + +### Experiments and baselines + +The revision benchmark uses file-backed SQLite, synthetic rules with two three-premise alternatives, and identical ordered edits. It compares per-branch indexed revision, atomic indexed revision with equivalent publication history, and a batched full scan with a lighter history contract. Each timed path is checked against an independent evaluator; separate replays check persisted verdicts after every edit. Five fixed workload cells use six matched repetitions each, with all execution order permutations represented. The full scan is a useful performance baseline but does not persist the same attempt, parent, binding, delivery and usage history [6]. + +The model reuse example separates a test-report judgment from a deployment planner. A policy correction changes the planner's input but not the report judgment's closed, versioned inputs. Reuse requires the same request bytes, candidate list, contract and relevant admissions; changing the report is a negative control. The live example uses Qwen2.5-1.5B-Instruct, model and tokenizer revision 989aa7980e4cf806f80c7fef2b1adb7bc71aa306, CPU, two threads, PyTorch 2.6.0+cpu and Transformers 4.48.3. Route timings begin after evidence-view preparation and model loading. One revised-prompt run compares repair and fresh rebuild; it is not a randomized population study [7]. + +## 4. Results + +In the atomic revision experiment, 128 rules with 32 dense edits took a median 1,972.5 ms for per-branch edits versus 146.7 ms for atomic edits. The median paired speedup was 13.46 times, range 13.24–13.56 across six trials. For 1,024 rules and 128 sparse edits, it was 1.40 times, range 1.34–1.68. Atomic edits improved edit time in all 30 matched comparisons. The lighter full scan had the lowest total time in every matched trial; for the dense cell its median total was 27.8 ms versus 369.1 ms for atomic indexed revision. The baseline's reduced historical contract prevents attributing the difference to indexing alone [6]. + +In a later bulk setup experiment, the 1,024-rule fixture's median setup time was 3,690 ms using individual production calls versus 214 ms using bulk calls. The median paired setup ratio was 16.62, range 14.89–34.55, across six trials. The scan retained the lowest median total in all five cells and finished first in 29 of 30 matched trials [6]. These are fixed synthetic, single-writer traces on hosted machines, not a general break-even result. + +In the revised-prompt model example, repair reused one report judgment, made one new call and used 145 input plus 48 output tokens. A clean rebuild made two calls and used 326 input plus 78 output tokens. Both produced the same checked canary plan; the observed savings were 211 of 404 tokens. Replacing the report forced a fresh judgment and a hold decision. The earlier prompt had produced an incorrect passing status for an 11/12 report, which the validator rejected; the prompt was revised before the reported successful run. Each route is one observation, not evidence of typical savings or semantic reliability [7]. + +A separate frozen recovery-policy evaluation illustrates the remaining model risk. Ten synthetic cases generated 20 Qwen calls. All responses included Markdown fences and failed the unchanged strict JSON contract. On the six held-out cases, the stop, affordable-retry and evidence-gated-retry policies each had zero successes. These policy outcomes are paired replays of shared trajectories, not independent policy arms. All retry receipts arrived after the 60-second operation deadline; the deadline was soft and did not interrupt dispatched calls [8]. + +## 5. Discussion and limitations + +The implementation makes evidence dependence explicit at an application boundary and preserves the cost of work that becomes unusable. The measurements show that atomic batching can improve a costly per-branch path without establishing that indexed revision beats a lighter scan. The model example shows that a carefully scoped cached judgment can be reused in one controlled correction, not that agent decisions improve in the wild. The strict recovery failures show how validation, model formatting and time budgets can dominate the usefulness of such a mechanism. + +Future work should evaluate real correction streams and user decisions, test concurrency and adversarial callers, compare equally durable baselines, and run preregistered, independently measured model evaluations. The frozen incident-routing workload and bounded controller on main are offline preparations; they do not establish model performance. A reviewer package contains the pinned source snapshot, evidence index and offline verification commands. Repository and workflow links require access while the repository remains private. + +## 6. Conclusion + +Versioned evidence and conservative checks can prevent this prototype from serving certain stale stored results. A small live-model example demonstrates a possible reuse benefit; benchmark results and failed recovery trials also show the costs and limits. Trustworthy deployment requires authenticated inputs, sound dependency declarations, independent validation, and evidence from representative settings. + +## Code and data + +Measured main-branch snapshot: 20fab84a6baa525452191ecf80eeee339f0b5ac7, https://github.com/k2jac9/kairotensor. Raw benchmark CSVs, prompts, model traces and evaluation protocols are in docs/benchmarks and docs/evaluations. Historical CI executions use their recorded source commits, not the current main commit. The accompanying reviewer package preserves the source archive and its checksum; exact experimental sources and commands are documented in [6-8]. + +## References + +[1] Moreau, L. and Missier, P. (eds.), 2013. PROV-DM: The PROV Data Model. W3C Recommendation. https://www.w3.org/TR/prov-dm/ + +[2] Coalition for Content Provenance and Authenticity, 2026. C2PA Technical Specification 2.4. https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html + +[3] Doyle, J., 1979. A truth maintenance system. Artificial Intelligence 12(3), 231-272. https://doi.org/10.1016/0004-3702(79)90008-0 + +[4] de Kleer, J., 1986. An assumption-based TMS. Artificial Intelligence 28(2), 127-162. https://doi.org/10.1016/0004-3702(86)90080-9 + +[5] KairoTensor, 2026. Selective revision implementation and tests. [Source snapshot 20fab84.](https://github.com/k2jac9/kairotensor/tree/20fab84a6baa525452191ecf80eeee339f0b5ac7) + +[6] KairoTensor, 2026. Benchmarking selective revision, raw samples and run identities. [Benchmark protocol and raw samples.](https://github.com/k2jac9/kairotensor/blob/20fab84a6baa525452191ecf80eeee339f0b5ac7/docs/revision-benchmark.md) + +[7] KairoTensor, 2026. Scoped model-result reuse, prompts and traces. [Scoped reuse protocol and traces.](https://github.com/k2jac9/kairotensor/blob/20fab84a6baa525452191ecf80eeee339f0b5ac7/docs/scoped-model-reuse.md) + +[8] KairoTensor, 2026. Recovery policy v2: measured contract and deadline failures. [Protocol and raw bundle.](https://github.com/k2jac9/kairotensor/blob/20fab84a6baa525452191ecf80eeee339f0b5ac7/docs/evaluations/recovery-policy-v2-2026-11-14.md) + +## Appendix: Limitations and Dual-Use / Ethical Considerations + +An operator could use admission controls to suppress inconvenient evidence, present a selective record as complete, or convey unwarranted authority through a provenance-like interface. The store does not establish source authenticity, prevent tampering by a privileged operator, detect omitted dependencies, guarantee semantic correctness, or erase copies already delivered. Deployment would need access control, independent auditing, transparent inclusion and exclusion policies, and protection for stored sensitive information. These are recommendations, not implemented protections. The reported examples use synthetic reports and deployment data; this statement does not establish the privacy properties of future applications. + +## LLM usage statement + +AI assistance was used to help develop code and prepare this report. The author is responsible for the claims and final submission. Automated checks recomputed benchmark summaries and reconciled the archived recovery records with their SQLite ledger; these checks establish internal consistency, not independent replication of model execution. diff --git a/docs/submission/verification/atomic-summary.md b/docs/submission/verification/atomic-summary.md new file mode 100644 index 0000000..5858e67 --- /dev/null +++ b/docs/submission/verification/atomic-summary.md @@ -0,0 +1,9 @@ +I = per-branch indexed, B = atomic batch, S = full scan. Times and database sizes are medians. Ratios are calculated within each trial and reported as median [min, max]. Values above 1 favor batch. An edit ratio is unavailable for zero edits or a zero-microsecond edit time in either indexed arm. + +| Scenario | Rules | Edits | Triplets | Setup ms (I / B / S) | Edit ms (I / B / S) | Total ms (I / B / S) | I / B edit | S / B total | Database KiB (I / B / S) | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| dense | 128 | 1 | 6 | 226.668 / 231.425 / 3.033 | 62.100 / 4.580 / 0.725 | 293.450 / 236.067 / 3.769 | 13.676 [12.080, 15.952] | 0.016 [0.015, 0.097] | 140.000 / 140.000 / 32.000 | +| dense | 128 | 32 | 6 | 226.686 / 221.019 / 2.959 | 1972.508 / 146.650 / 24.859 | 2198.191 / 369.123 / 27.755 | 13.460 [13.244, 13.563] | 0.075 [0.073, 0.102] | 280.000 / 280.000 / 56.000 | +| sparse | 128 | 1 | 6 | 238.065 / 242.812 / 2.949 | 1.718 / 1.055 / 0.684 | 239.931 / 243.948 / 3.738 | 1.731 [1.382, 1.918] | 0.016 [0.012, 0.017] | 132.000 / 132.000 / 32.000 | +| sparse | 128 | 32 | 6 | 231.900 / 233.962 / 3.084 | 57.136 / 30.514 / 22.049 | 292.346 / 263.330 / 25.122 | 1.898 [1.673, 2.598] | 0.094 [0.087, 0.106] | 140.000 / 140.000 / 32.000 | +| sparse | 1024 | 128 | 6 | 2473.183 / 2477.150 / 8.067 | 258.505 / 189.174 / 101.647 | 2736.232 / 2660.274 / 109.905 | 1.397 [1.344, 1.682] | 0.041 [0.040, 0.046] | 708.000 / 708.000 / 136.000 | diff --git a/docs/submission/verification/atomic-summary.stderr.txt b/docs/submission/verification/atomic-summary.stderr.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/belief-oracle-tests.stderr.txt b/docs/submission/verification/belief-oracle-tests.stderr.txt new file mode 100644 index 0000000..3284793 --- /dev/null +++ b/docs/submission/verification/belief-oracle-tests.stderr.txt @@ -0,0 +1,13 @@ +test_conditioning_can_amplify_compression_error (__main__.BeliefOracleTests.test_conditioning_can_amplify_compression_error) ... ok +test_copies_cannot_be_second_factor (__main__.BeliefOracleTests.test_copies_cannot_be_second_factor) ... ok +test_impossible_evidence_is_explicit (__main__.BeliefOracleTests.test_impossible_evidence_is_explicit) ... ok +test_independence_is_an_assumption (__main__.BeliefOracleTests.test_independence_is_an_assumption) ... ok +test_independent_four_statuses (__main__.BeliefOracleTests.test_independent_four_statuses) ... ok +test_one_step_regret_bound_is_tight (__main__.BeliefOracleTests.test_one_step_regret_bound_is_tight) ... ok +test_unversioned_empty_certificate_is_rejected (__main__.BeliefOracleTests.test_unversioned_empty_certificate_is_rejected) ... ok +test_zero_likelihood_retraction_restores_prior (__main__.BeliefOracleTests.test_zero_likelihood_retraction_restores_prior) ... ok + +---------------------------------------------------------------------- +Ran 8 tests in 0.000s + +OK diff --git a/docs/submission/verification/belief-oracle-tests.stdout.txt b/docs/submission/verification/belief-oracle-tests.stdout.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/bulk-summary.md b/docs/submission/verification/bulk-summary.md new file mode 100644 index 0000000..0085e5c --- /dev/null +++ b/docs/submission/verification/bulk-summary.md @@ -0,0 +1,29 @@ +Batch and bulk both use atomic edits. Batch seeds and registers one item at a time. Bulk uses the bulk setup APIs. Scan uses the full-scan baseline. Setup includes opening the database, seeding evidence, and registering rules. Phase durations and database sizes are medians. + +| Scenario | Rules | Edits | Arm | Seed ms | Register ms | Setup ms | Edit ms | Total ms | Database KiB | +| --- | ---: | ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | +| dense | 128 | 1 | batch | 103.423 | 162.210 | 271.289 | 4.208 | 275.575 | 140.000 | +| dense | 128 | 1 | bulk | 1.496 | 13.786 | 16.777 | 4.282 | 21.112 | 140.000 | +| dense | 128 | 1 | scan | 0.774 | 1.220 | 3.007 | 0.806 | 3.824 | 32.000 | +| dense | 128 | 32 | batch | 137.725 | 228.259 | 357.699 | 152.202 | 591.520 | 280.000 | +| dense | 128 | 32 | bulk | 1.592 | 13.992 | 17.050 | 182.391 | 200.160 | 280.000 | +| dense | 128 | 32 | scan | 0.859 | 1.259 | 3.313 | 33.962 | 42.969 | 56.000 | +| sparse | 128 | 1 | batch | 145.392 | 185.158 | 355.913 | 1.087 | 357.211 | 132.000 | +| sparse | 128 | 1 | bulk | 1.485 | 13.938 | 16.768 | 1.137 | 17.976 | 132.000 | +| sparse | 128 | 1 | scan | 0.796 | 1.303 | 3.297 | 0.835 | 4.144 | 32.000 | +| sparse | 128 | 32 | batch | 109.430 | 180.574 | 294.625 | 39.122 | 330.687 | 140.000 | +| sparse | 128 | 32 | bulk | 1.546 | 14.002 | 17.136 | 36.684 | 54.175 | 140.000 | +| sparse | 128 | 32 | scan | 0.790 | 1.258 | 3.150 | 25.212 | 28.747 | 32.000 | +| sparse | 1024 | 128 | batch | 1164.432 | 2565.256 | 3690.273 | 180.397 | 3870.816 | 708.000 | +| sparse | 1024 | 128 | bulk | 6.341 | 206.607 | 214.389 | 279.073 | 493.250 | 708.000 | +| sparse | 1024 | 128 | scan | 1.052 | 4.720 | 6.788 | 123.107 | 129.927 | 136.000 | + +Speed ratios are measured within each matched trial and reported as median [min, max]. Values above 1 favor bulk. A zero measured setup time makes the setup ratio unavailable. + +| Scenario | Rules | Edits | Triplets | Batch / bulk setup | Batch / bulk total | Scan / bulk total | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| dense | 128 | 1 | 6 | 16.195 [15.559, 22.877] | 12.607 [4.555, 13.174] | 0.183 [0.055, 0.336] | +| dense | 128 | 32 | 6 | 19.929 [7.603, 33.118] | 2.557 [1.767, 5.040] | 0.193 [0.140, 0.489] | +| sparse | 128 | 1 | 6 | 20.851 [16.816, 24.449] | 19.485 [15.801, 22.917] | 0.228 [0.213, 0.331] | +| sparse | 128 | 32 | 6 | 17.328 [16.089, 24.651] | 6.183 [5.045, 9.792] | 0.551 [0.435, 1.609] | +| sparse | 1024 | 128 | 6 | 16.617 [14.886, 34.549] | 9.298 [6.880, 14.753] | 0.258 [0.226, 0.354] | diff --git a/docs/submission/verification/bulk-summary.stderr.txt b/docs/submission/verification/bulk-summary.stderr.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/README.md b/docs/submission/verification/finalization-20261116/README.md new file mode 100644 index 0000000..f00512f --- /dev/null +++ b/docs/submission/verification/finalization-20261116/README.md @@ -0,0 +1,42 @@ +# Submission finalization receipts + +These new receipts cover the 16 November 2026 report and reviewer-package +finalization. The earlier files in the parent directory remain immutable and +describe the 15 November package, including its six-page PDF. + +- `report-check.json`: the seven-page final PDF, unchanged 192-word abstract, + preserved measured results, manuscript/native-document/PDF agreement, and + visual review of all seven pages. The main report including title ends on + page 5; references occupy page 6; the ethics appendix and LLM statement are on + page 7. Link annotations point to the cited primary sources and exact Git + snapshot; this does not establish anonymous access to the private repository. +- `source-archive-check.json`: fresh byte comparison of all 145 files (162 ZIP + entries including directories) against Git snapshot + `20fab84a6baa525452191ecf80eeee339f0b5ac7`. +- `related-work-check.md`: primary-source checks and the bounded contribution. +- `reproduction-check.json` and matching stdout/stderr files: the documented + commands run after a fresh candidate-package extraction under Python 3.15.2. + Reference tests: 18 passed. Belief oracle: 8 passed. Script tests: 203 total, + 28 skipped, 175 executed without failure. Two unclosed SQLite connection + warnings remain visible. Both benchmark summaries match the historical bytes. + The read-only recovery audit reconciles 20 recorded calls and 5,241 charged + tokens with zero outstanding reservations; held-out success remains zero. + +The final package includes these fresh receipts. Its manifest and fresh unpack +are verified separately after packaging, and the measured-source ZIP bytes are +required to equal the archive used for the command run. That outer archive +receipt is retained alongside the final ZIP and in the repository, rather than +inside the ZIP whose checksum it records. The nested recovery audit's field +`new_model_calls: 20` describes historical collection; this finalization made +zero new model calls. + +No new model inference, dataset creation or training occurred during this +finalization. Later reliability work, PR #18 and active AutoScientist runs remain +excluded from the measured results. No fresh Rust run or complete +stable/preview/previous-stable matrix is claimed. Visibility and sharing settings +were not changed, and no event entry was submitted. + +The official-template document was edited in place with revision guards. Before +and after snapshots, the earlier and final exports, rendered pages and packaging +records are retained in the separate finalization artifact directory. The +historical submission artifact directory was read only throughout this work. diff --git a/docs/submission/verification/finalization-20261116/atomic-summary.stderr.txt b/docs/submission/verification/finalization-20261116/atomic-summary.stderr.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/atomic-summary.stdout.txt b/docs/submission/verification/finalization-20261116/atomic-summary.stdout.txt new file mode 100644 index 0000000..5858e67 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/atomic-summary.stdout.txt @@ -0,0 +1,9 @@ +I = per-branch indexed, B = atomic batch, S = full scan. Times and database sizes are medians. Ratios are calculated within each trial and reported as median [min, max]. Values above 1 favor batch. An edit ratio is unavailable for zero edits or a zero-microsecond edit time in either indexed arm. + +| Scenario | Rules | Edits | Triplets | Setup ms (I / B / S) | Edit ms (I / B / S) | Total ms (I / B / S) | I / B edit | S / B total | Database KiB (I / B / S) | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| dense | 128 | 1 | 6 | 226.668 / 231.425 / 3.033 | 62.100 / 4.580 / 0.725 | 293.450 / 236.067 / 3.769 | 13.676 [12.080, 15.952] | 0.016 [0.015, 0.097] | 140.000 / 140.000 / 32.000 | +| dense | 128 | 32 | 6 | 226.686 / 221.019 / 2.959 | 1972.508 / 146.650 / 24.859 | 2198.191 / 369.123 / 27.755 | 13.460 [13.244, 13.563] | 0.075 [0.073, 0.102] | 280.000 / 280.000 / 56.000 | +| sparse | 128 | 1 | 6 | 238.065 / 242.812 / 2.949 | 1.718 / 1.055 / 0.684 | 239.931 / 243.948 / 3.738 | 1.731 [1.382, 1.918] | 0.016 [0.012, 0.017] | 132.000 / 132.000 / 32.000 | +| sparse | 128 | 32 | 6 | 231.900 / 233.962 / 3.084 | 57.136 / 30.514 / 22.049 | 292.346 / 263.330 / 25.122 | 1.898 [1.673, 2.598] | 0.094 [0.087, 0.106] | 140.000 / 140.000 / 32.000 | +| sparse | 1024 | 128 | 6 | 2473.183 / 2477.150 / 8.067 | 258.505 / 189.174 / 101.647 | 2736.232 / 2660.274 / 109.905 | 1.397 [1.344, 1.682] | 0.041 [0.040, 0.046] | 708.000 / 708.000 / 136.000 | diff --git a/docs/submission/verification/finalization-20261116/belief-oracle-tests.stderr.txt b/docs/submission/verification/finalization-20261116/belief-oracle-tests.stderr.txt new file mode 100644 index 0000000..3284793 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/belief-oracle-tests.stderr.txt @@ -0,0 +1,13 @@ +test_conditioning_can_amplify_compression_error (__main__.BeliefOracleTests.test_conditioning_can_amplify_compression_error) ... ok +test_copies_cannot_be_second_factor (__main__.BeliefOracleTests.test_copies_cannot_be_second_factor) ... ok +test_impossible_evidence_is_explicit (__main__.BeliefOracleTests.test_impossible_evidence_is_explicit) ... ok +test_independence_is_an_assumption (__main__.BeliefOracleTests.test_independence_is_an_assumption) ... ok +test_independent_four_statuses (__main__.BeliefOracleTests.test_independent_four_statuses) ... ok +test_one_step_regret_bound_is_tight (__main__.BeliefOracleTests.test_one_step_regret_bound_is_tight) ... ok +test_unversioned_empty_certificate_is_rejected (__main__.BeliefOracleTests.test_unversioned_empty_certificate_is_rejected) ... ok +test_zero_likelihood_retraction_restores_prior (__main__.BeliefOracleTests.test_zero_likelihood_retraction_restores_prior) ... ok + +---------------------------------------------------------------------- +Ran 8 tests in 0.000s + +OK diff --git a/docs/submission/verification/finalization-20261116/belief-oracle-tests.stdout.txt b/docs/submission/verification/finalization-20261116/belief-oracle-tests.stdout.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/bulk-summary.stderr.txt b/docs/submission/verification/finalization-20261116/bulk-summary.stderr.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/bulk-summary.stdout.txt b/docs/submission/verification/finalization-20261116/bulk-summary.stdout.txt new file mode 100644 index 0000000..0085e5c --- /dev/null +++ b/docs/submission/verification/finalization-20261116/bulk-summary.stdout.txt @@ -0,0 +1,29 @@ +Batch and bulk both use atomic edits. Batch seeds and registers one item at a time. Bulk uses the bulk setup APIs. Scan uses the full-scan baseline. Setup includes opening the database, seeding evidence, and registering rules. Phase durations and database sizes are medians. + +| Scenario | Rules | Edits | Arm | Seed ms | Register ms | Setup ms | Edit ms | Total ms | Database KiB | +| --- | ---: | ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | +| dense | 128 | 1 | batch | 103.423 | 162.210 | 271.289 | 4.208 | 275.575 | 140.000 | +| dense | 128 | 1 | bulk | 1.496 | 13.786 | 16.777 | 4.282 | 21.112 | 140.000 | +| dense | 128 | 1 | scan | 0.774 | 1.220 | 3.007 | 0.806 | 3.824 | 32.000 | +| dense | 128 | 32 | batch | 137.725 | 228.259 | 357.699 | 152.202 | 591.520 | 280.000 | +| dense | 128 | 32 | bulk | 1.592 | 13.992 | 17.050 | 182.391 | 200.160 | 280.000 | +| dense | 128 | 32 | scan | 0.859 | 1.259 | 3.313 | 33.962 | 42.969 | 56.000 | +| sparse | 128 | 1 | batch | 145.392 | 185.158 | 355.913 | 1.087 | 357.211 | 132.000 | +| sparse | 128 | 1 | bulk | 1.485 | 13.938 | 16.768 | 1.137 | 17.976 | 132.000 | +| sparse | 128 | 1 | scan | 0.796 | 1.303 | 3.297 | 0.835 | 4.144 | 32.000 | +| sparse | 128 | 32 | batch | 109.430 | 180.574 | 294.625 | 39.122 | 330.687 | 140.000 | +| sparse | 128 | 32 | bulk | 1.546 | 14.002 | 17.136 | 36.684 | 54.175 | 140.000 | +| sparse | 128 | 32 | scan | 0.790 | 1.258 | 3.150 | 25.212 | 28.747 | 32.000 | +| sparse | 1024 | 128 | batch | 1164.432 | 2565.256 | 3690.273 | 180.397 | 3870.816 | 708.000 | +| sparse | 1024 | 128 | bulk | 6.341 | 206.607 | 214.389 | 279.073 | 493.250 | 708.000 | +| sparse | 1024 | 128 | scan | 1.052 | 4.720 | 6.788 | 123.107 | 129.927 | 136.000 | + +Speed ratios are measured within each matched trial and reported as median [min, max]. Values above 1 favor bulk. A zero measured setup time makes the setup ratio unavailable. + +| Scenario | Rules | Edits | Triplets | Batch / bulk setup | Batch / bulk total | Scan / bulk total | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| dense | 128 | 1 | 6 | 16.195 [15.559, 22.877] | 12.607 [4.555, 13.174] | 0.183 [0.055, 0.336] | +| dense | 128 | 32 | 6 | 19.929 [7.603, 33.118] | 2.557 [1.767, 5.040] | 0.193 [0.140, 0.489] | +| sparse | 128 | 1 | 6 | 20.851 [16.816, 24.449] | 19.485 [15.801, 22.917] | 0.228 [0.213, 0.331] | +| sparse | 128 | 32 | 6 | 17.328 [16.089, 24.651] | 6.183 [5.045, 9.792] | 0.551 [0.435, 1.609] | +| sparse | 1024 | 128 | 6 | 16.617 [14.886, 34.549] | 9.298 [6.880, 14.753] | 0.258 [0.226, 0.354] | diff --git a/docs/submission/verification/finalization-20261116/final-package-check.json b/docs/submission/verification/finalization-20261116/final-package-check.json new file mode 100644 index 0000000..93688e0 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/final-package-check.json @@ -0,0 +1,21 @@ +{ + "verified_at_utc": "2026-11-16T05:44:11.991159+00:00", + "reviewer_zip_sha256": "3103cfbd5ecbdc8fec0eaa36f0712e435e8d76112293b642300297465e4c9405", + "reviewer_zip_bytes": 666641, + "manifest_payload_files": 42, + "manifest_exact_file_set_and_hashes_verified": true, + "fresh_unpack_verified": true, + "packaged_guides_and_pdf_match_repository": true, + "measured_source_commit": "20fab84a6baa525452191ecf80eeee339f0b5ac7", + "source_archive_sha256": "e5928a4daf5b9d68cfbec0f58afdfd53d65f4202e3f1c5d54a4284baa1c0b821", + "source_files_verified_against_git": 145, + "source_bytes_identical_to_freshly_tested_candidate": true, + "pdf_sha256": "a755007e453b633cae4f19d38fd1c1f557c365a04d07ddfffb52d5753bf10c5a", + "candidate_reproduction_receipt_included": "verification/finalization-20261116/reproduction-check.json", + "outer_receipt_excluded_from_zip_to_avoid_self_hashing": true, + "heavy_build_or_download_performed": false, + "expected_growth_limit_mib": 64, + "output_lifecycle": "Retain final artifacts, before snapshots, intermediate exports, fresh extraction and logs in the new finalization packet. Historical artifact directory unchanged.", + "event_submitted": false, + "visibility_or_sharing_changed": false +} diff --git a/docs/submission/verification/finalization-20261116/recovery-v2-audit.stderr.txt b/docs/submission/verification/finalization-20261116/recovery-v2-audit.stderr.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/recovery-v2-audit.stdout.txt b/docs/submission/verification/finalization-20261116/recovery-v2-audit.stdout.txt new file mode 100644 index 0000000..1e56018 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/recovery-v2-audit.stdout.txt @@ -0,0 +1,94 @@ +{ + "status": "verified_complete", + "cases": 10, + "mode": "live_qwen", + "actual_backend_calls": 20, + "new_model_calls": 20, + "charged_tokens": 5241, + "outstanding_tokens": 0, + "summary": { + "calibration": { + "always_stop": { + "attempts": 4, + "charged_tokens": 840, + "reserved_tokens": 0, + "correct": 0, + "on_time": 4, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "affordable_retry": { + "attempts": 8, + "charged_tokens": 2045, + "reserved_tokens": 0, + "correct": 0, + "on_time": 0, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "evidence_gated_retry": { + "attempts": 7, + "charged_tokens": 1736, + "reserved_tokens": 0, + "correct": 0, + "on_time": 1, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + } + }, + "heldout": { + "always_stop": { + "attempts": 6, + "charged_tokens": 1316, + "reserved_tokens": 0, + "correct": 0, + "on_time": 6, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "affordable_retry": { + "attempts": 12, + "charged_tokens": 3196, + "reserved_tokens": 0, + "correct": 0, + "on_time": 0, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "evidence_gated_retry": { + "attempts": 10, + "charged_tokens": 2548, + "reserved_tokens": 0, + "correct": 0, + "on_time": 2, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + } + } + }, + "limitations": "Internal consistency audit only; model, tokenizer, timing and source provenance are producer attestations, not authenticated execution evidence." +} diff --git a/docs/submission/verification/finalization-20261116/reference-tests.stderr.txt b/docs/submission/verification/finalization-20261116/reference-tests.stderr.txt new file mode 100644 index 0000000..a017007 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/reference-tests.stderr.txt @@ -0,0 +1,5 @@ +.................. +---------------------------------------------------------------------- +Ran 18 tests in 0.057s + +OK diff --git a/docs/submission/verification/finalization-20261116/reference-tests.stdout.txt b/docs/submission/verification/finalization-20261116/reference-tests.stdout.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/related-work-check.md b/docs/submission/verification/finalization-20261116/related-work-check.md new file mode 100644 index 0000000..488ed45 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/related-work-check.md @@ -0,0 +1,33 @@ +# Related work checked on 16 November 2026 + +The report cites the original publications and distinguishes established +dependency tracking from the integration evaluated by KairoTensor. + +1. Jon Doyle, *A truth maintenance system*, Artificial Intelligence 12(3), + 231-272 (1979), DOI `10.1016/0004-3702(79)90008-0`. + The [publisher record](https://www.sciencedirect.com/science/article/abs/pii/0004370279900080) + and [MIT primary archive](https://dspace.mit.edu/entities/publication/5377b306-4ecc-4687-b1f5-78cbb4a0543a) + describe recording reasons for beliefs and revising beliefs when assumptions + are contradicted. Publisher direct fetch returned HTTP 403; the indexed + publisher abstract and MIT archive were available. +2. Johan de Kleer, *An assumption-based TMS*, Artificial Intelligence 28(2), + 127-162 (1986), DOI `10.1016/0004-3702(86)90080-9`. + The [publisher record](https://www.sciencedirect.com/science/article/pii/0004370286900809) + and [author's paper](https://dekleer.org/Publications/An%20Assumption-Based%20TMS.pdf) + support the account of assumption sets and alternative contexts. The indexed + primary paper includes section 4.1 on the problem-solver/TMS boundary and + labels describing contexts. Direct PDF fetch timed out; indexed primary text + was available. No independent reproduction of either historical system is + claimed. +3. [PROV-DM](https://www.w3.org/TR/prov-dm/) was checked against the W3C + Recommendation dated 30 April 2013. +4. [C2PA 2.4](https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html) + was checked as the version cited in the report. It is a comparison of scope, + not a dependency upgrade or a claim that 2.4 is the latest specification. + +The contribution is durable integration of occurrence identities, admission, +frozen execution inputs, revision-checked settlement and reuse, and retained +usage at the model execution boundary. The report does not claim invention of +dependency tracking, a complete TMS/ATMS, equivalence to those systems, source +authenticity, or improved semantic reliability. All measured claims and failed +experiments remain unchanged; only citation numbers changed within Results. diff --git a/docs/submission/verification/finalization-20261116/report-check.json b/docs/submission/verification/finalization-20261116/report-check.json new file mode 100644 index 0000000..beec10d --- /dev/null +++ b/docs/submission/verification/finalization-20261116/report-check.json @@ -0,0 +1,32 @@ +{ + "verified_at_utc": "2026-11-16T05:42:15.484576+00:00", + "pdf_sha256": "a755007e453b633cae4f19d38fd1c1f557c365a04d07ddfffb52d5753bf10c5a", + "pdf_bytes": 139531, + "pdf_pages": 7, + "main_pages_including_title": 5, + "reference_pages": [ + 6 + ], + "appendix_and_llm_statement_pages": [ + 7 + ], + "abstract_words": 192, + "abstract_unchanged": true, + "results_unchanged_except_citation_numbers": true, + "manuscript_paragraphs_matching_native_doc_and_pdf": 43, + "all_seven_final_pages_visually_reviewed": true, + "visual_review_resolution_dpi": 120, + "no_clipping_overlap_or_missing_glyphs_observed": true, + "eight_reference_link_targets_checked_in_pdf_annotations": true, + "private_repository_links_require_reviewer_access": true, + "native_document_id": "1HR8MGOaxe7Yc5FRYpzYyGZMBqriIv6dqnvgQqmjYJwA", + "native_revision_id": "AHj4eMStYcmSsd6wgaa-5gvZO8UqoY25_uMrCifNUuHLbz8dywPIAsoOjXRFqLiFJeuKqgklysn97SD7p9uZDlmvigD5dheuqWCnhZIVKE4", + "measured_source_commit": "20fab84a6baa525452191ecf80eeee339f0b5ac7", + "documentation_base_commit": "99b6fe24177e56b450c543c95723961fed440d4d", + "historical_receipt_files_unchanged": 17, + "new_model_calls_during_finalization": 0, + "fresh_rust_execution": false, + "full_compatibility_matrix_executed": false, + "event_submitted": false, + "visibility_or_sharing_changed": false +} diff --git a/docs/submission/verification/finalization-20261116/reproduction-check.json b/docs/submission/verification/finalization-20261116/reproduction-check.json new file mode 100644 index 0000000..f935de1 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/reproduction-check.json @@ -0,0 +1,196 @@ +{ + "verified_at_utc": "2026-11-16T05:43:12.331296+00:00", + "measured_source_commit": "20fab84a6baa525452191ecf80eeee339f0b5ac7", + "source_archive_sha256": "e5928a4daf5b9d68cfbec0f58afdfd53d65f4202e3f1c5d54a4284baa1c0b821", + "tested_package": "candidate-reviewer-package.zip; final package retains identical measured source bytes", + "python_version": "3.15.2 (main, Oct 25 2026, 16:57:51) [GCC 16.2.0]", + "python_executable": "/home/neo/.nix-profile/bin/python3", + "python_resolved_executable": "/nix/store/w3ps4vf470m69d9l5bcbbipay9insx32-python3-3.15.2/bin/python3.15", + "commands": [ + { + "label": "atomic-summary", + "command": [ + "/home/neo/.nix-profile/bin/python3", + "scripts/summarize_revision_bench.py", + "docs/benchmarks/atomic-revision-2026-11-13.csv" + ], + "exit_code": 0, + "started_at_utc": "2026-11-16T05:42:53.993345+00:00", + "elapsed_seconds": 0.029359 + }, + { + "label": "bulk-summary", + "command": [ + "/home/neo/.nix-profile/bin/python3", + "scripts/summarize_revision_bench.py", + "docs/benchmarks/bulk-setup-2026-11-13.csv" + ], + "exit_code": 0, + "started_at_utc": "2026-11-16T05:42:54.022719+00:00", + "elapsed_seconds": 0.030028 + }, + { + "label": "recovery-v2-audit", + "command": [ + "/home/neo/.nix-profile/bin/python3", + "scripts/audit_recovery_bundle.py", + "/home/neo/Projects/kairotensor-artifacts/submission-finalization-mkpl39bc/report/final-b850dFB3/candidate-recovery-audit" + ], + "exit_code": 0, + "started_at_utc": "2026-11-16T05:42:54.052760+00:00", + "elapsed_seconds": 0.050662 + }, + { + "label": "reference-tests", + "command": [ + "/home/neo/.nix-profile/bin/python3", + "-m", + "unittest", + "discover", + "-s", + "reference", + "-p", + "test_memory.py" + ], + "exit_code": 0, + "started_at_utc": "2026-11-16T05:42:54.103436+00:00", + "elapsed_seconds": 0.110642 + }, + { + "label": "belief-oracle-tests", + "command": [ + "/home/neo/.nix-profile/bin/python3", + "reference/belief_oracle.py" + ], + "exit_code": 0, + "started_at_utc": "2026-11-16T05:42:54.214102+00:00", + "elapsed_seconds": 0.045485 + }, + { + "label": "scripts-tests", + "command": [ + "/home/neo/.nix-profile/bin/python3", + "-m", + "unittest", + "discover", + "-s", + "scripts", + "-p", + "test_*.py" + ], + "exit_code": 0, + "started_at_utc": "2026-11-16T05:42:54.259601+00:00", + "elapsed_seconds": 18.071512 + } + ], + "summary_bytes_match_historical_receipts": true, + "recovery_archive_unchanged": true, + "recovery_archive_sha256": "74e8895335f304be0db1d8c705b1e448cb0588dd1b1ccff7296e4a2ff6f8af0d", + "recovery_audit": { + "status": "verified_complete", + "cases": 10, + "mode": "live_qwen", + "actual_backend_calls": 20, + "new_model_calls": 20, + "charged_tokens": 5241, + "outstanding_tokens": 0, + "summary": { + "calibration": { + "always_stop": { + "attempts": 4, + "charged_tokens": 840, + "reserved_tokens": 0, + "correct": 0, + "on_time": 4, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "affordable_retry": { + "attempts": 8, + "charged_tokens": 2045, + "reserved_tokens": 0, + "correct": 0, + "on_time": 0, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "evidence_gated_retry": { + "attempts": 7, + "charged_tokens": 1736, + "reserved_tokens": 0, + "correct": 0, + "on_time": 1, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + } + }, + "heldout": { + "always_stop": { + "attempts": 6, + "charged_tokens": 1316, + "reserved_tokens": 0, + "correct": 0, + "on_time": 6, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "affordable_retry": { + "attempts": 12, + "charged_tokens": 3196, + "reserved_tokens": 0, + "correct": 0, + "on_time": 0, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "evidence_gated_retry": { + "attempts": 10, + "charged_tokens": 2548, + "reserved_tokens": 0, + "correct": 0, + "on_time": 2, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + } + } + }, + "limitations": "Internal consistency audit only; model, tokenizer, timing and source provenance are producer attestations, not authenticated execution evidence." + }, + "new_model_calls_during_reproduction": 0, + "fresh_rust_execution": false, + "complete_runtime_compatibility_matrix": false, + "tested_candidate_package_sha256": "35850b59423c8f38719895b51c260443c945798373799e25155e081b7c8d66ec", + "test_counts": { + "reference_passed": 18, + "belief_oracle_passed": 8, + "scripts_total": 203, + "scripts_skipped": 28, + "scripts_executed_without_failure": 175, + "total_executed_without_failure": 201 + }, + "unclosed_sqlite_connection_resource_warnings": 2 +} diff --git a/docs/submission/verification/finalization-20261116/scripts-tests.stderr.txt b/docs/submission/verification/finalization-20261116/scripts-tests.stderr.txt new file mode 100644 index 0000000..5ae9bb1 --- /dev/null +++ b/docs/submission/verification/finalization-20261116/scripts-tests.stderr.txt @@ -0,0 +1,9 @@ +.sssssssssssssss.........................:500: ResourceWarning: unclosed database in +ResourceWarning: Enable tracemalloc to get the object allocation traceback +:500: ResourceWarning: unclosed database in +ResourceWarning: Enable tracemalloc to get the object allocation traceback +........................................................................................................................................sssssssssssss............. +---------------------------------------------------------------------- +Ran 203 tests in 17.950s + +OK (skipped=28) diff --git a/docs/submission/verification/finalization-20261116/scripts-tests.stdout.txt b/docs/submission/verification/finalization-20261116/scripts-tests.stdout.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/finalization-20261116/source-archive-check.json b/docs/submission/verification/finalization-20261116/source-archive-check.json new file mode 100644 index 0000000..63af61d --- /dev/null +++ b/docs/submission/verification/finalization-20261116/source-archive-check.json @@ -0,0 +1,9 @@ +{ + "verified_at_utc": "2026-11-16T05:42:15.484576+00:00", + "source_commit": "20fab84a6baa525452191ecf80eeee339f0b5ac7", + "archive_sha256": "e5928a4daf5b9d68cfbec0f58afdfd53d65f4202e3f1c5d54a4284baa1c0b821", + "archive_entries": 162, + "git_blob_files_verified": 145, + "file_set_and_all_bytes_match_git": true, + "copied_historical_archive_without_modification": true +} diff --git a/docs/submission/verification/recovery-v2-audit.stderr.txt b/docs/submission/verification/recovery-v2-audit.stderr.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/recovery-v2-audit.stdout.json b/docs/submission/verification/recovery-v2-audit.stdout.json new file mode 100644 index 0000000..1e56018 --- /dev/null +++ b/docs/submission/verification/recovery-v2-audit.stdout.json @@ -0,0 +1,94 @@ +{ + "status": "verified_complete", + "cases": 10, + "mode": "live_qwen", + "actual_backend_calls": 20, + "new_model_calls": 20, + "charged_tokens": 5241, + "outstanding_tokens": 0, + "summary": { + "calibration": { + "always_stop": { + "attempts": 4, + "charged_tokens": 840, + "reserved_tokens": 0, + "correct": 0, + "on_time": 4, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "affordable_retry": { + "attempts": 8, + "charged_tokens": 2045, + "reserved_tokens": 0, + "correct": 0, + "on_time": 0, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "evidence_gated_retry": { + "attempts": 7, + "charged_tokens": 1736, + "reserved_tokens": 0, + "correct": 0, + "on_time": 1, + "success": 0, + "abstained": 4, + "cases": 4, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + } + }, + "heldout": { + "always_stop": { + "attempts": 6, + "charged_tokens": 1316, + "reserved_tokens": 0, + "correct": 0, + "on_time": 6, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "affordable_retry": { + "attempts": 12, + "charged_tokens": 3196, + "reserved_tokens": 0, + "correct": 0, + "on_time": 0, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + }, + "evidence_gated_retry": { + "attempts": 10, + "charged_tokens": 2548, + "reserved_tokens": 0, + "correct": 0, + "on_time": 2, + "success": 0, + "abstained": 6, + "cases": 6, + "success_rate": 0.0, + "correct_late": 0, + "successes_per_charged_token": 0.0 + } + } + }, + "limitations": "Internal consistency audit only; model, tokenizer, timing and source provenance are producer attestations, not authenticated execution evidence." +} diff --git a/docs/submission/verification/reference-check.txt b/docs/submission/verification/reference-check.txt new file mode 100644 index 0000000..825ba20 --- /dev/null +++ b/docs/submission/verification/reference-check.txt @@ -0,0 +1 @@ +Verified 2026-11-15 against primary sources: PROV-DM is a W3C Recommendation dated 30 April 2013, edited by Luc Moreau and Paolo Missier (https://www.w3.org/TR/prov-dm/). C2PA Technical Specification 2.4 version history section 5.3.1 dates the release to April 2026; corrected the earlier draft citation year 2025 to 2026 (https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html). diff --git a/docs/submission/verification/reference-tests.stderr.txt b/docs/submission/verification/reference-tests.stderr.txt new file mode 100644 index 0000000..5c89288 --- /dev/null +++ b/docs/submission/verification/reference-tests.stderr.txt @@ -0,0 +1,5 @@ +.................. +---------------------------------------------------------------------- +Ran 18 tests in 0.055s + +OK diff --git a/docs/submission/verification/reference-tests.stdout.txt b/docs/submission/verification/reference-tests.stdout.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/report-check.json b/docs/submission/verification/report-check.json new file mode 100644 index 0000000..5c91683 --- /dev/null +++ b/docs/submission/verification/report-check.json @@ -0,0 +1,13 @@ +{ + "verified_at_utc": "2026-11-15T17:47:25.750138+00:00", + "pdf_pages": 6, + "pdf_bytes": 127982, + "abstract_words": 192, + "all_six_pages_visually_reviewed": true, + "required_author_affiliation_provenance_ethics_and_ai_disclosure_present": true, + "contact_email_excluded_from_public_documents": true, + "source_code_unchanged": true, + "new_model_calls_during_packaging": 0, + "submission_sent": false, + "pdf_sha256": "5ac036b2d192446bdc61477b514bc69a786e4968a14c1555e5660493f2db5333" +} diff --git a/docs/submission/verification/scripts-tests.stderr.txt b/docs/submission/verification/scripts-tests.stderr.txt new file mode 100644 index 0000000..123e4f2 --- /dev/null +++ b/docs/submission/verification/scripts-tests.stderr.txt @@ -0,0 +1,9 @@ +.sssssssssssssss.........................:500: ResourceWarning: unclosed database in +ResourceWarning: Enable tracemalloc to get the object allocation traceback +:500: ResourceWarning: unclosed database in +ResourceWarning: Enable tracemalloc to get the object allocation traceback +........................................................................................................................................sssssssssssss............. +---------------------------------------------------------------------- +Ran 203 tests in 17.477s + +OK (skipped=28) diff --git a/docs/submission/verification/scripts-tests.stdout.txt b/docs/submission/verification/scripts-tests.stdout.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/submission/verification/source-archive-check.json b/docs/submission/verification/source-archive-check.json new file mode 100644 index 0000000..e341701 --- /dev/null +++ b/docs/submission/verification/source-archive-check.json @@ -0,0 +1,7 @@ +{ + "source": "20fab84a6baa525452191ecf80eeee339f0b5ac7", + "archive_sha256": "e5928a4daf5b9d68cfbec0f58afdfd53d65f4202e3f1c5d54a4284baa1c0b821", + "git_blob_files_verified": 145, + "archive_file_set_matches_git_tree": true, + "all_file_bytes_match_git_blobs": true +} diff --git a/docs/submission/verification/test-receipts.json b/docs/submission/verification/test-receipts.json new file mode 100644 index 0000000..683c78b --- /dev/null +++ b/docs/submission/verification/test-receipts.json @@ -0,0 +1,41 @@ +[ + { + "name": "reference", + "command": [ + "python3", + "-m", + "unittest", + "discover", + "-s", + "reference", + "-p", + "test_memory.py" + ], + "exit_code": 0, + "elapsed_seconds": 0.102 + }, + { + "name": "belief-oracle", + "command": [ + "python3", + "reference/belief_oracle.py" + ], + "exit_code": 0, + "elapsed_seconds": 0.046 + }, + { + "name": "scripts", + "command": [ + "python3", + "-m", + "unittest", + "discover", + "-s", + "scripts", + "-p", + "test_*.py" + ], + "exit_code": 0, + "elapsed_seconds": 17.547 + } +] diff --git a/docs/submission/verification/verification.json b/docs/submission/verification/verification.json new file mode 100644 index 0000000..697895c --- /dev/null +++ b/docs/submission/verification/verification.json @@ -0,0 +1,18 @@ +{ + "checked_at_utc": "2026-11-15T17:34:47.794575+00:00", + "source": "20fab84a6baa525452191ecf80eeee339f0b5ac7", + "recovery_archive_sha256": "74e8895335f304be0db1d8c705b1e448cb0588dd1b1ccff7296e4a2ff6f8af0d", + "recovery_archive_file_hashes": { + "environment.txt": "46646d8cbf109401698c940a8320bec2098faa641002b4205ff783d050366df6", + "stderr.log": "a6e90f4ceb7013597d89927e2df011300cbaa3d90ca8ac763fda3b11716bc26e", + "stdout.log": "c85c37745706becdd8cd40c8d102ab081209663538af3d4c0020d58b34149405", + "summary.json": "721432de295988eb1ad0464c64d7903c788d1f487ac246811274755eb34b7ead", + "trace.json": "9c26a6a443f2b048f6589ce4c7092468d6b688e3e9800f042a1d6da2a2641614", + "trace.json.raw.json": "d93c19bad64a6bfb6e97a3848923462e1c8221344ae137efc67d02e93830312a", + "trace.json.sqlite": "a8ac40fc7ac16ad80f957cf710bb32fd46f89f3e79f7156d0bd8c27125d71e17" + }, + "audit_exit_code": 0, + "archive_unchanged_after_audit": true, + "inference_calls_made_by_this_audit": 0, + "scope": "Recomputed archived summaries and internal recovery consistency; no fresh model replication." +}