Skip to content

RECON: cold-start verification audit - 9/9 capsules verify, layer 1 adjudicated, first tamper test - #52

Merged
fathomlab merged 1 commit into
mainfrom
feat/cold-start-recon
Sep 1, 2026
Merged

RECON: cold-start verification audit - 9/9 capsules verify, layer 1 adjudicated, first tamper test#52
fathomlab merged 1 commit into
mainfrom
feat/cold-start-recon

Conversation

@fathomlab

Copy link
Copy Markdown
Contributor

RECON, not a result. No preregistration covers this work, the sample is one environment and
one pass, and it therefore carries no headline finding. It reports what a machine that had never
run this code did when pointed at the published artifacts. styxx was installed from PyPI into
an empty virtualenv before the repo was cloned and kept separate from the tree throughout — a
stranger has the package, not the tree.

Two new files. Nothing existing is modified, no paper, receipt, certificate, seal or capsule
is touched, and .github/ is untouched.


The capsule claim, adjudicated

A capsule opened by someone with no styxx install verifies something entirely offline, and it
is not the verdict.

The offline half holds. All nine capsules are genuinely self-contained: no src, no href,
no fetch, no CDN, no external host. Loaded in a real browser, the network log reads no network
requests recorded
. Hashing is done by the browser's own WebCrypto.

The self-verifying half is narrower than the phrase. The embedded script SHA-256s the
embedded document and receipts and compares them to hashes in the embedded certificate. That is
an internal consistency check. It does not re-derive the verdict — the verdict is read out of
the certificate (['verdict', C.verdict]) and displayed. The certificate carries no signature and
no anchor outside the file; searching all nine for any signature-shaped field returns nothing.

So the strong form — a capsule re-derives its own verdict in a browser with no install — is not
what these capsules do. The weak form — a capsule carries its bytes, receipts and certificate and
can show a reader offline that they are mutually consistent
— is exactly what they do. The
capsule's own footer states this correctly
("a portable binding, not a portable oath of
origin"). The artifact does not overclaim; prose that compresses this to "self-verifying" does.

The tamper test — first ever run against these capsules

Three copies, in a temp directory. The committed file was never touched; the tree was confirmed
clean afterwards.

tamper layer 2 (installed package) layer 1 (browser)
1 byte, 0.16 to 0.96 in the sealed doc, cert untouched caught, exit 1: document bytes != certificate.document_sha256 caught — red banner, INTEGRITY: FAILED
same edit plus certificate hash recomputed caught, exit 1: verdict not reproduced: live OATH-FAILED vs embedded OATH-HELD not caughtintegrity: all hashes match, no banner, OATH-HELD, accused 0, renders the forged 0.96
same file where crypto.subtle is unavailable n/a fails open — static green OATH-HELD, integrity stuck at checking integrity…, no banner, empty body

Layer 2 catching the coordinated forgery by re-derivation is the strongest observation here: the
forged number is unsupported by the sealed receipts, so re-running the instrument turns it into an
accusation and flips the verdict. That is the mechanism working as designed.

The fail-open is the worst behaviour found. The verdict badge is static HTML present before any
verification runs
; the script only recolours it. Where the crypto path does not complete, the
page still asserts a verdict, with no error and no banner, for a tampered file.


Everything else

All 9 capsules verify at layer 2, exit 0, from the installed package run outside the clone.
(Note VERIFIED means the capsule reproduces its own embedded verdict — two of the nine
reproduce an OATH-FAILED.)

REPLICATIONS.md's corpus audit reproduced — all nine exceptions and every count, from both the
clone and the installed package, 12 s on CPU. One undocumented extra line (epistemics: ...)
against a "character for character" bar, and CRLF/LF makes a literal diff show all ten lines
differing.

The worst blocker for a stranger: the CPU-only replication targets cannot be run from the
published instructions. REPLICATE_legibility.md says pip install numpy scipy; both run_b37.py
("one command, no GPU") and run_b45.py ("the single easiest check in this repo") die on
ModuleNotFoundError: No module named 'torch', then on transformers. None of the six CPU
scripts uses torch at all
— it arrives through a module-scope import in a file they borrow one
constant from. With a stand-in torch that raises on any real use, run_b45.py still completed in
3 s and reproduced every scientific field of the committed receipt exactly. The computation is
CPU-only and ~4 s as advertised; the import graph is what is broken.

A silent extraction gap, new. _NUM in styxx/certify.py ends every alternative with
(?![\w.]), so a number immediately followed by a period never matches — and is not abstained on,
it is absent from the ledger entirely:

document text verdict exit
This document reports a precision of 0.55. OATH-HELD, 0 tokens 0
This document reports a precision of 0.55 OATH-FAILED, 1 accused 1

Across the corpus, 90 of 211 certified documents contain at least one sentence-final decimal
absent from their own ledger — 170 tokens.
Several are outcome numbers whose thresholds on the
same line were examined (FINDING_p1_third_quarantine: "G2 demanded 0.85 and got 0.75" — 0.85
in the ledger, 0.75 not). This does not mean any number or verdict is wrong; it means coverage is
narrower than "every numeric claim" and the shortfall is invisible rather than declared. The lab
already named this class in the same file: "certified-by-omission, the inverse of the oath."

First-run honesty: clean. Verification makes no network calls — confirmed with black-hole
proxies and with socket.connect monkeypatched to raise. No API key, no model download, no
telemetry.

The quickstart works end to end from the installed package alone (styxx.certify to
styxx.capsule create to styxx.capsule verify), and correctly accused a number no receipt
supported. But README.md mentions styxx.certify once in prose with no command line, and
"capsule" zero times; capsule create --help documents --cert without saying how to obtain
one. OATH_CONTRACT.md is the exception and is good.

This document fails its own contract, and is published failing: oathready returns
OATH-FAILED, 22 accused — almost entirely on the tamper strings it quotes. That is a cold-start
reproduction of SYNTHESIS_mention_and_use_2026_08_26.md, arrived at without knowledge of it.

Deliberately not fixed

No defect is repaired here — a fix bundled with an audit obscures what the audit found. The one
exception is papers/ERRATA_replications_2026_09_01.md, recording the corrected install command
in a new file. REPLICATIONS.md and REPLICATE_legibility.md are left exactly as published.

UNCHECKABLE from this environment

The file:// double-click path — the harness rewrote file:// into data:, so layer 1 was only
verified over http://127.0.0.1, a secure context. Whether crypto.subtle is present for a
double-clicked capsule is UNCHECKED, and it decides whether the fail-open is an edge case or the
default experience.
Also unchecked: cold-network install time (pip cache was warm), all four GPU
targets, whether receipts truthfully record reality, and any platform but this one.

🤖 Generated with Claude Code

First verification pass run from a machine that had never run this code:
styxx installed from PyPI into a clean virtualenv before the repo was cloned,
and kept separate from the tree throughout.

All 9 capsules verify at layer 2 from the installed package (exit 0). The
corpus audit in REPLICATIONS.md reproduces, all nine exceptions and every
count matching, in 12s on CPU.

Adjudicates the capsule claim precisely: layer 1 runs genuinely offline with
no styxx install and no network requests, but checks internal byte-to-
certificate consistency only -- it does not re-derive the verdict, and the
certificate is unsigned. A tamper that also recomputes the certificate hash
passes layer 1 clean and is caught only by layer 2.

Records the first tamper test run against these capsules, including a
fail-open: where crypto.subtle is unavailable the script throws before the
tamper flag is set, and the page shows a static green OATH-HELD badge with
no banner for a tampered file.

Also records a silent extraction gap: styxx.certify's _NUM guard means a
number ending a sentence is never examined -- 90 of 211 certified documents
carry at least one such unexamined decimal.

No defects are repaired in this pass. Errata added as a new file for the one
published command that cannot work as written; REPLICATIONS.md and
REPLICATE_legibility.md are left exactly as published.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@fathomlab
fathomlab merged commit 384fc8e into main Sep 1, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants