Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,23 @@
# Kairotensor
# KairoTensor

Kairotensor is a Rust framework for versioned evidence memory. Its first component is a local SQLite store that records which observations an agent used and checks whether a saved result is still eligible to read or resume after evidence changes.

## AI x Epistemics sprint submission

Author: **Alejandro Castellanos**, **Independent Research**.

The author confirms that no KairoTensor-specific ideas, code or experiments
predate the 13-15 November 2026 sprint. The implementation and reported experiments
were developed during the sprint. This is an author declaration, not a conclusion
inferred solely from Git timestamps. Third-party dependencies and cited prior work
remain identified in their manifests and the report.

The [submission guide](docs/submission/README.md) maps the report's claims to raw
evidence and offline reproduction commands. It includes a real Qwen reuse example
and the failed frozen recovery evaluation; neither establishes general improvement
in agent accuracy. Measured source is pinned to
`20fab84a6baa525452191ecf80eeee339f0b5ac7`; submission documentation is additional.

## Run the withdrawal example

With stable Rust, Cargo, and a C compiler for bundled SQLite, run these commands from the repository root.
Expand Down
61 changes: 61 additions & 0 deletions docs/submission/DEMO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Optional four-minute presentation

This is a recording script. No presentation recording or live demonstration is
claimed. It can be recorded from the report and archived evidence without model
downloads or new inference.

## 0:00-0:40 - The problem

Show the report title and author.

"An agent reaches a conclusion from several observations. One observation is
later withdrawn. Can the application identify which saved results may still be
used? KairoTensor records the declared evidence for a computation and checks
whether its stored result remains eligible after a correction. It checks
eligibility, not truth."

## 0:40-1:25 - The mechanism

Show the README withdrawal example and selective-revision contract.

"Dependency tracking has a long history in truth-maintenance systems. Here it is
integrated with durable records at the model execution boundary. Each evidence
occurrence has an identity. A computation captures its selected
inputs and current branch. Settlement and later reads check that this context is
still applicable. Deterministic rules can name alternative groups of witnesses:
withdrawing one report need not discard a conclusion if another declared witness
group remains complete. Usage stays recorded even when work becomes unusable."

## 1:25-2:15 - What was measured

Show the atomic benchmark table, including the scan baseline.

"In one dense synthetic workload, atomic edits were 13.46 times faster than
per-branch edits by the median paired ratio. The lighter full scan was faster end
to end in every matched trial. The comparison supports batching an expensive
path; it does not establish that indexing beats a scan with equivalent history."

## 2:15-3:05 - The real model example

Show `docs/scoped-model-reuse.md` and its four-row result table.

"A real Qwen run reused a report judgment when only deployment policy changed.
Repair used one new call and 193 tokens; a clean rebuild used two calls and 404
tokens. Both returned the same checked plan. Replacing the report forced a new
judgment and a hold decision. This is one synthetic example, not an estimate of
typical savings. The earlier prompt failed its negative control and is retained."

## 3:05-3:45 - The failure matters

Show the recovery-v2 held-out results and audit receipt.

"A separate recovery evaluation made 20 actual Qwen calls. All responses violated
the strict JSON contract, and all three replayed policies had zero held-out
successes. Every retry missed the soft deadline. The ledger retained 5,241 tokens
of usage. A successful collection or audit is not a successful model evaluation."

## 3:45-4:00 - What comes next

"The next tests need representative correction streams, equally durable
baselines, concurrency and independent model replication. The reviewer package
contains the frozen evidence, reproduction commands and the limitations."
43 changes: 43 additions & 0 deletions docs/submission/EVIDENCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Evidence index

Paths below are relative to the measured source root. The accompanying source
archive is an exact snapshot of `20fab84a6baa525452191ecf80eeee339f0b5ac7`.
CI collection success and model-task success are separate outcomes. Later
reliability commits, unmerged PR #18 and active AutoScientist runs are outside
this measured snapshot and are excluded from the report's outcomes.

| Report claim | Raw evidence and protocol | Result and limit |
| --- | --- | --- |
| Atomic revision reduces the expensive per-branch edit path | `docs/benchmarks/atomic-revision-2026-11-13.csv`; `docs/revision-benchmark.md` | 90 rows, 30 matched triplets. Dense 128-rule/32-edit cell: 1,972.508 vs 146.650 ms median edit time; median paired ratio 13.460, range 13.244-13.563, six trials. Full scan wins total time in all 30 trials but preserves less history. |
| Bulk setup reduces setup cost | `docs/benchmarks/bulk-setup-2026-11-13.csv`; `docs/revision-benchmark.md` | 1,024-rule fixture: 3,690.273 vs 214.389 ms median setup; paired ratio 16.617, range 14.886-34.549. Scan wins median total in all five cells and 29/30 individual trials. Single-writer synthetic traces. |
| Scoped reuse saves calls in one controlled correction | `docs/evaluations/scoped-reuse-qwen-2026-11-13.json`; `docs/scoped-model-reuse.md`; `docs/evaluation-prompts.md` | Repair: one call, 193 tokens. Fresh rebuild: two calls, 404 tokens. Identical checked canary plans. Report replacement forces new judgment and hold. Seven actual generations across all four routes; one observation per route. |
| Earlier scoped-reuse prompt failed | `docs/evaluations/scoped-reuse-first-qwen-2026-11-13.json`; `docs/scoped-model-reuse.md` | Model labeled an 11/12 report as passing; validator rejected it. The historical attempt remained unsettled. A later zero-call replay records rejection; it does not rewrite the original trace. |
| Frozen recovery did not succeed | `docs/evaluations/recovery-policy-v2-2026-11-14.zip`; matching `.md` | Ten synthetic cases, 20 recorded Qwen calls, 5,241 charged tokens. All replies fenced and invalid under strict JSON contract. Each policy scores 0/6 held-out successes; policies replay shared trajectories. All retry receipts miss the soft deadline. |
| The measured snapshot has automated contract coverage | `.github/workflows/check.yml`; `reference/test_memory.py`; `reference/belief_oracle.py`; `scripts/test_*.py`; Rust integration tests | Historical exact-source CI plus local Python receipts. New incident calibration is unexecuted; fixture tests are not live-model evidence. |

## Source and execution identities

| Evidence | Historical source / execution |
| --- | --- |
| Atomic revision | `5d699c44ebfec97bae016a2f8965cc1d9765ffa3`; [run 43476626433](https://github.com/k2jac9/kairotensor/actions/runs/43476626433) |
| Bulk setup | `10e3a57475246f77c8eaa5a05469f1fe79361cfb`; [run 43478880682](https://github.com/k2jac9/kairotensor/actions/runs/43478880682) |
| Revised scoped reuse | [run 43491028189](https://github.com/k2jac9/kairotensor/actions/runs/43491028189); exact requests and lineage retained in the JSON trace |
| Failed initial reuse | [run 43490325882](https://github.com/k2jac9/kairotensor/actions/runs/43490325882) |
| Recovery v2 | `11234573896b68699efa00df63e88553f8d8e99a`; [run 43571699931](https://github.com/k2jac9/kairotensor/actions/runs/43571699931) |
| Main contract CI | `20fab84a6baa525452191ecf80eeee339f0b5ac7`; [run 43622988515](https://github.com/k2jac9/kairotensor/actions/runs/43622988515) |

Both cited live-model experiments use Qwen/Qwen2.5-1.5B-Instruct, model and
tokenizer revision `989aa7980e4cf806f80c7fef2b1adb7bc71aa306`, CPU with two
threads, PyTorch `2.6.0+cpu` and Transformers `4.48.3`. Consult each execution's
manifest for dtype and hardware. Changed hosts and prompts prevent treating
different experiments as a controlled latency comparison.

The recovery ZIP SHA-256 is
`74e8895335f304be0db1d8c705b1e448cb0588dd1b1ccff7296e4a2ff6f8af0d`.
It contains raw responses, exact requests, environment records, policy summaries
and a SQLite ledger. The audit checks consistency between them; it does not prove
that the producer's hardware, model or timestamp attestations are authentic.

The claimed evidence boundary is application-declared dependence and eligibility.
It is not proof of truth, complete causal attribution, source authenticity or
tamper resistance. All reported tasks and workloads are synthetic.
84 changes: 84 additions & 0 deletions docs/submission/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# KairoTensor submission guide

**KairoTensor: Evidence-Grounded Revision for Agent Memory**

Alejandro Castellanos, Independent Research

Apart Research AI x Epistemics Research Sprint, 13-15 November 2026

Recommended track: **Track 2: Trust Infrastructure**

KairoTensor records the evidence supplied to an agent computation and checks
whether a saved result remains eligible after a correction or withdrawal. The
report evaluates atomic revision, scoped reuse and a failed recovery experiment.
It does not claim source authentication, tamper evidence, or general improvement
in real-world decisions.

## Provenance

The author confirmed on 15 November 2026 that no KairoTensor-specific ideas, code
or experiments existed before the sprint. This is an author declaration. Git
timestamps alone are not proof of when work began. Dependencies and prior work
are separately attributed in manifests and the report.

The measured main-branch snapshot is
`20fab84a6baa525452191ecf80eeee339f0b5ac7`. Historical experiments use the source
identities in [EVIDENCE.md](EVIDENCE.md). These submission documents were prepared
after that snapshot. Later reliability commits, unmerged PR #18 and active
AutoScientist runs are excluded from the reported results.

## Review

- [Submission report (PDF)](kairotensor-report.pdf) and [editable manuscript](report.md).
- [Evidence index](EVIDENCE.md): findings, raw artifacts, source identities and limits.
- [Reproduction guide](REPRODUCE.md): offline summaries and consistency checks.
- [Four-minute presentation script](DEMO.md): an optional presentation, not a recording.

The accompanying reviewer ZIP contains the seven-page report, exact source archive,
these guides, verification receipts and SHA-256 manifest. The source archive has
162 entries and SHA-256
`e5928a4daf5b9d68cfbec0f58afdfd53d65f4202e3f1c5d54a4284baa1c0b821`.
It preserves the measured snapshot rather than incorporating later documentation.

The 16 November finalization adds Doyle's truth maintenance system and de Kleer's
assumption-based TMS to the related work. The contribution is the durable
integration of declared evidence dependence at the model execution boundary;
dependency tracking itself is established prior work. Measured outcomes are
unchanged. The report retains a 192-word abstract, five pages through code and
data, one reference page, and one page with the ethics appendix and LLM statement.

New receipts are in [verification/finalization-20261116](verification/finalization-20261116/README.md).
The original 15 November receipts remain unchanged and describe the earlier
six-page export. They are historical records, not checks of the final PDF.

The repository was private when the package was prepared. The owner plans to
make it public; this preparation does not change visibility or submit an entry.
Reviewers need the ZIP or repository access to inspect the evidence.

## Verification scope

On 15 November, local verification passed 18 reference tests and 8 belief-oracle
tests. The scripts suite reported 203 tests with 28 skips: 175 executed without
failure. It emitted resource warnings about unclosed SQLite connections; the
warnings are retained in the reviewer receipts. Skipped checks are not passes.
Rust execution is supported by historical exact-source CI
[43622988515](https://github.com/k2jac9/kairotensor/actions/runs/43622988515),
not a fresh local Rust run during packaging.

Finalization on 16 November repeated the offline commands from a fresh reviewer
package extraction under Python 3.15.2: the same 201 executed tests passed and
28 script tests skipped. Two unclosed-SQLite-connection resource warnings remain
in the new stderr receipt. Both benchmark summaries matched the historical
outputs byte for byte, and the frozen recovery audit again passed with the same
recorded failures. This was an evidence/package check, not fresh model execution.

Archived benchmark summaries were recomputed. The frozen recovery ZIP passed its
read-only internal consistency audit, including 20 recorded calls, 5,241 charged
tokens and zero unresolved reservations. Its bytes were unchanged after auditing.
No new inference was run in preparing this package. A successful audit is not
independent authentication of model or hardware execution.

The newer incident-routing calibration remains unexecuted. The current CI does
not implement the complete stable/preview/previous-stable compatibility matrix;
no such matrix is claimed here. Historical model/runtime pins are retained for
reproduction, not presented as current release recommendations.
85 changes: 85 additions & 0 deletions docs/submission/REPRODUCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Reproduce the archived checks

These steps inspect the submitted evidence without model inference. Use Python
3.11 or newer. The source snapshot has no third-party Python requirements for
these specific summary and audit commands.

## 1. Open the pinned source

From the extracted reviewer package, verify the manifest and unpack the source:

```sh
sha256sum -c SHA256SUMS
python3 -m zipfile -e kairotensor-source-20fab84.zip source
cd source
```

The archive places repository files directly in `source/`. If reviewing through Git instead,
use a separate checkout of `20fab84a6baa525452191ecf80eeee339f0b5ac7`.
Do not reset a working checkout that contains local work.

## 2. Recompute the benchmark tables

```sh
python3 scripts/summarize_revision_bench.py docs/benchmarks/atomic-revision-2026-11-13.csv
python3 scripts/summarize_revision_bench.py docs/benchmarks/bulk-setup-2026-11-13.csv
```

Compare with the `verification/atomic-summary.md` and `bulk-summary.md` files in
the reviewer package. Ratios are medians of matched per-trial ratios, not ratios
of medians. A lighter full-scan contract is not an equally durable baseline.

## 3. Audit the frozen recovery records

Choose a new temporary directory so existing evidence cannot be overwritten:

```sh
recovery_audit_dir=$(mktemp -d)
python3 -m zipfile -e docs/evaluations/recovery-policy-v2-2026-11-14.zip "$recovery_audit_dir"
python3 scripts/audit_recovery_bundle.py "$recovery_audit_dir"
```

Expected: `verified_complete`, mode `live_qwen`, 10 cases, 20 recorded backend
calls, 5,241 charged tokens and zero outstanding tokens. All held-out policy
success counts are zero. The audit field `new_model_calls: 20` describes the
historical collection recorded in the bundle. Running this audit creates **zero**
new model calls. The policy replay itself also generates no model calls.

## 4. Run the lightweight contract tests

```sh
python3 -m unittest discover -s reference -p test_memory.py
python3 reference/belief_oracle.py
python3 -m unittest discover -s scripts -p 'test_*.py'
```

Packaging verification on 15 November observed 18 reference passes, 8 oracle
passes, and 203 script tests with 28 skips (175 executed, no failures). Some
script tests require separately built Rust examples and are skipped without them.
Resource warnings about unclosed SQLite connections are retained in the receipts.

The 16 November finalization reran these exact commands after unpacking the
reviewer package, using Python 3.15.2. It observed the same 18 reference passes,
8 oracle passes, and 203 script tests with 28 skips. Two unclosed-SQLite-connection
resource warnings were emitted. The new stdout/stderr and exact runtime identity
are retained under `verification/finalization-20261116/`; the original receipts
remain unchanged. Both benchmark summary outputs matched the earlier receipt
bytes, and the recovery archive remained unchanged after the read-only audit.

The final ZIP adds those fresh receipts to the tested payload. Its manifest and
unpacked source bytes are checked again; the source archive is identical to the
one used for the fresh command run. No new source build or model call is implied.

## Full Rust and model reproduction

The repository README and historical workflows contain full Rust commands.
These require a Rust toolchain, a C compiler and build capacity. On the author's
workstation, storage admission is required before builds. The packaging checks
did not rerun Rust; [exact-main CI](https://github.com/k2jac9/kairotensor/actions/runs/43622988515)
is the recorded Rust evidence.

Live Qwen reproduction is optional and separate: follow
`docs/scoped-model-reuse.md`, `docs/corrected-handoff.md` and the frozen recovery
protocol at its recorded commit, preserving model, tokenizer and runtime pins.
It requires downloading assets and running inference. There is no claim that
the author performed a fresh live replication while preparing this package.
Binary file added docs/submission/kairotensor-report.pdf
Binary file not shown.
Loading
Loading