Skip to content

fix(gateway): stored evidence and certificates tell the truth (N80) - #459

Merged
LamaSu merged 5 commits into
masterfrom
fix/evidence-n80-stored-truth
Oct 1, 2026
Merged

fix(gateway): stored evidence and certificates tell the truth (N80)#459
LamaSu merged 5 commits into
masterfrom
fix/evidence-n80-stored-truth

Conversation

@LamaSu

@LamaSu LamaSu commented Sep 29, 2026

Copy link
Copy Markdown
Owner

N80: stored evidence and certificates tell the truth

Board row N80 (owner: evidence), from the steward's rehearsal R0. refvertical ran the real onboarding flow against a local build of master ac86a404 (returns/pcc-refvertical-work/rehearsal/R0-findings.md, G3 and G4). Built from master, per the WIP rule: it depends on no unreviewed PR.

G3: the operator relay made up the evidence hash

POST /api/operator/evidence (operator-relay.ts) had four problems:

  • For any body that was not a device-signed bundle, it stored bundleHash = "sha256-" + bundleId (so sha256-ev-<uuid>).
  • It kept none of the pushed events.
  • It answered stored: true, and every read then showed eventCount: 0.
  • The rehearsal's device document (instrument, run, readings, log) was thrown away. What remained was a "hash" that committed to nothing received.

commitRelayEvidence (new, services/relay-evidence-commitment.ts) now decides what is committed, before anything is written:

The body carries Stored bundleHash Events stored
LO-EV events (type, timestamp, source object, payload object) hashBundle over the recomputed hashEvent hashes. A supplied event hash, or a supplied or device-signed bundleHash, must equal the recomputation, or 422 yes
a device-signed bundle without LO-EV events the digest the device signed, only as a canonical sha256: + 64 lowercase hex (else 422) no
anything else (pcc-node's pushes, whose events have no source; a client's own document) sha256 of the canonical JSON of the body received no (it has none in LO-EV form)

Also:

G4: certificates were minted with no checks

POST /api/certificates/mint (rewards.ts):

  • answered minted: true for any kernelDid, with a caller-chosen tier and a placeholder Merkle tree (TreeAAAA…);
  • set mintedAt to when the module loaded, about 27 minutes before the request and before the kernel existed;
  • checked, stored and minted nothing.

GET /api/certificates listed three certificates for kernels that do not exist, and the MCP server shows that list to agents.

A mint service exists: packages/contracts/ts/capability-certificates.ts mints Metaplex Core soulbound NFTs, in mock mode by default. But no gateway route uses it, and nothing checks a kernel's registration or jobs before a mint. Until that is wired, the honest state is none:

  • the list is empty;
  • a lookup returns 404;
  • mint returns 501 minted: false, saying nothing was minted.

The agent context pack no longer advertises a working mint. Real minting (wiring that service behind checks against the registration and jobs, with a truthful mintedAt) is a feature for whoever owns DePIN; I'll propose it as a follow-up row.

Not changed (other rows)

Evidence (run on the Spark)

  • New tests:
    • relay-evidence-commitment.test.ts: 10;
    • operator-relay.test.ts: 4 new N80 tests (24 in total);
    • certificates-n80.test.ts: 3.
    • 37/37 pass.
  • 9 mutants, each killed with the counts read: a made-up hash in the route, event hash unchecked, claimed bundle hash unchecked, kernel check removed, mint claiming success, events not stored, a non-canonical device digest accepted, source not required, and the list serving a certificate.
  • Gateway tsc clean. Full gateway suite, run twice. Run 1: 2 failed, 2996 passed, 6 skipped. The failures were carrier.test.ts (R5-2 purchase lock) and completion-real-tier.test.ts; both pass in isolation with and without this change (44/44), so they are load-dependent. Run 2: 189/189 files, 2998 passed, 6 skipped, 0 failed.

🤖 Generated with Claude Code

LamaSu and others added 2 commits September 29, 2026 16:17
@LamaSu @claude
fix(gateway): the operator relay stores a true evidence hash, never s…
a907c4c 
…ha256-<bundleId> (N80)

Rehearsal R0 finding G3, on master ac86a40: POST /api/operator/evidence
stored `sha256-<bundleId>` for every body that was not a device-signed
bundle, kept none of its events, and answered `stored: true`. The "hash"
committed to nothing that was received, and every read showed eventCount 0.

commitRelayEvidence now decides what is committed, before anything is
written:
- LO-EV events (a type, timestamp, source and payload each): every event
  hash is recomputed, and the bundle hash is hashBundle over them. A
  supplied event hash or bundleHash, or a device-signed one, must equal
  the recomputation or the body is refused (422). The events are stored.
- A device-signed bundle without LO-EV events: the digest the device
  signed, only in the canonical `sha256:` form (else 422).
- Anything else, including pcc-node's pushes (events without a source)
  and a client's own document: the sha256 of the canonical JSON received.

The stored bundle belongs to the job's kernel; a body naming another
kernel is refused (409). The response says what was committed (contentHash,
hashModel, eventsStored) and that no signature was verified. The
placeholder signature is unchanged, so isDeviceSignedSignature still
reads it as not device-signed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@LamaSu @claude
fix(gateway): no certificate is served or minted until minting is rea…
48c7db9 
…l (N80)

Rehearsal R0 finding G4, on master ac86a40: POST /api/certificates/mint
answered `minted: true` for any kernelDid, with a caller-chosen assurance
tier, a placeholder Merkle tree (`TreeAAAA...`) and a mintedAt taken when
the module loaded, before the kernel it certified existed. Nothing was
checked, stored or minted. GET /api/certificates listed three certificates
for kernels that do not exist, and the MCP server shows that list to agents.

Minting has no registration or job checks, no tree and no store, so the
honest state is none: the list is empty, a lookup is 404, and mint answers
501 `minted: false` saying nothing was minted. The agent context pack no
longer advertises a working mint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
LamaSu added a commit that referenced this pull request Sep 29, 2026
@LamaSu
fix(gateway): no release path pays out an escrow that was given back …
9ec8302 
…(N79)

The previous commit guarded the gateway's own settlement flow (PUT
/complete, resume-settlement, the keeper). An audit of every release
entry found two more that ignored a refund decision; both are now
refused. Each was reproduced first, with a test that failed on 830e4ff.

1. The raw chain routes: POST /api/escrow/chain/:address/{evidence,
   attestation,release}/:idx, on the EAS (V2/V3) and V1 paths. They are
   operator/admin-scoped, but they read no refund state, so an operator
   could push a refund-pending escrow to a release. 24 cases failed:
   "expected 200 to be 409". They now answer 409 escrow_refunded and send
   nothing. They also fail closed with 503 when the escrow registry cannot
   be read. Disputes stay open, because V2 refunds only through
   resolveDispute; approve-release only encodes calldata for the payer's
   own wallet, so it is unchanged.
2. SettlementService.releaseMilestone: POST /api/settlement/release and
   the automatic release after evidence. It released a refund-pending
   escrow and then set its job "settled", the N79 negative itself (4
   cases failed). It now refuses when the job's own escrow, or the escrow
   it is asked to release (the named address or the default, in any
   letter case), was given back. Its activity no longer retries that
   refusal.

The lookup is shared: givenBackEscrow and escrowByContractAddress in
escrow-refund.ts.

Also:
- The route tests assert the escrow rows, which are what this change
  writes. They no longer go through GET /api/jobs/:id/settlement: that
  read is readmodels', and #441 makes it authenticated.
- Two import lines move one line away from #459's and #441's insertion
  points, so both merge cleanly with this branch.

All ten new mutants are killed: each route guard alone, the status check,
fail-open on registry errors, the service guard, the job and address
halves of the lookup, the checksum form and the activity's retry. The
gateway suite passes: 3037 passed, 6 skipped. tsc is clean.
@LamaSu LamaSu mentioned this pull request Sep 29, 2026
Open
LamaSu added a commit that referenced this pull request Sep 30, 2026
@LamaSu
fix(gateway): a settlement owns its escrow, so no refund lands undern…
fa935e5 
…eath it (N79 round 2)

astra's pack 116b on #462 @38cf4116: DO-NOT-SHIP, with three HIGH
findings and one MEDIUM. Each was reproduced first with a failing test at
38cf411 (n79-refund-vs-settlement.test.ts):
- F1 HIGH: a repeated failure write while /complete is in flight
  refunded ("expected 'refunded' to be 'skipped'");
- F2 HIGH: cancelling one job refunded an escrow another job shares;
- F3 HIGH: a refund landed while SettlementService.releaseMilestone
  awaited the chain ("expected 'refund_pending' to be 'skipped'");
- F4 MEDIUM: a completion that failed after a failure write left the
  failed job's escrow funded.
Two more were reproduced the same way:
- the keeper, which sweeps a findAll() snapshot across awaits, drove an
  escrow given back mid-sweep (astra's Q2 note);
- the raw chain release route let a refund land while it awaited the
  chain (the F3 class, found by extending the audit).

The fix is one mechanism: DURABLE SETTLEMENT OWNERSHIP on the escrow row.
- A settlement takes its escrow with a compare-and-set from
  funded/active to `completing` (already in @pcc/spec's vocabulary),
  synchronously at its claim: /complete (same stretch as the job claim),
  resume-settlement, SettlementService.releaseMilestone (before the
  chain await) and the raw release route (by contract address).
- A refund never lands on a `completing` escrow, however often the
  job's mutable status is rewritten (F1, F3, raw route).
- A settlement that gives up BEFORE recording evidence hands the escrow
  back (prior status). If the job ended meanwhile, the escrow goes to the
  payer then, in one transaction (F4). After evidence exists, the
  settlement keeps it, since resume-settlement continues it.
- A release that went through records its milestone `released`, and the
  escrow `completed` once every milestone is.
- The refund gives back a whole escrow only if no other job references
  it; a shared escrow is skipped as `escrow_shared` (F2).
- The keeper re-reads the row right before each drive.

Tests: the six reproductions, plus the gaps astra named:
- a failure on the LAST write rolls back every milestone and the status;
- a failure after evidence keeps ownership;
- resume takes a legacy row (the write repeated, as in F1);
- a partial release hands the escrow back;
- failed releases (service and raw route) hand it back.
All 40 mutants (both rounds) are killed. The gateway suite passes: 3053
passed, 6 skipped (six threads; the Spark is at load 37, where two /complete
tests time out at 5 s and pass alone). tsc is clean. It merges cleanly with
#326, #385, #403, #424, #441, #445, #450, #451 and #459.
LamaSu and others added 3 commits September 29, 2026 19:40
@LamaSu @claude
fix(gateway): the relay stores only what it can reproduce, atomically…
103c747 
…, from one envelope (review E4)

Cross-family review E4 on 48c7db9: DO-NOT-SHIP. Each finding was
reproduced first, as a test failing at 48c7db9 (10 failed):

1. HIGH, the document model hashed a body and then dropped it: stored:true
   with a digest nothing in the store can reproduce. The gateway has no
   place to keep such a document (that would be a schema change, the
   operator's call), so the relay now refuses it (422 evidence_not_lo_ev)
   and stores nothing.
2. HIGH, the bundle and its events were two independent inserts. With
   insertEvents made to throw, the route answered stored:false and left
   the bundle behind (6 rows where 5 were expected). They now run in one
   getStore().db.transaction, the gateway's existing pattern
   (routes/artifacts.ts), so both commit or neither does.
3. MEDIUM, model selection downgraded silently. There is now ONE
   envelope: the body or its `bundle` wrapper, never both.
   - A non-object `bundle` is malformed_envelope.
   - A commitment field both at the root and in `bundle` is
     ambiguous_envelope.
   - `events`, when present, must be a non-empty list of LO-EV events
     (events_malformed, with the index), including under a device
     signature, which no longer falls back to device_signed_digest.

pcc-node's current pushes (events without a source) are now refused
rather than stored under an irreproducible hash. The device-bundle fixture
now carries LO-EV events and signs their real hashBundle, as kernel-sdk
does.

gateway: relay tests 42/42 (13 + 26 + 3 certificates); tsc clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@LamaSu @claude
fix(gateway): the relay binds a signed document to its job and kernel…
e415d95 
… (adk #4322)

adk's verdict 117 on #471 (HIGH, in this lane's files): the relay stored a
device-signed bundle's digest without recomputing it, and never compared
the bundle's own job or kernel with the request and the job row, so job
A's signed bundle could be filed under job B, or have its record edited
while keeping A's signed hash.

Reproduced at 103c747 first, as route tests that stored with 200:
- a signed document naming another job (expected 409);
- a signed document naming another kernel (expected 409);
- a document edited after signing (expected 422);
- lane-found: job A's signature stripped to a bare digest and filed under
  job B (expected 422).
adk's third claim did not reproduce: /complete already checks the session
scope against the SETTLED job (paid-job-flow.ts sets contractId: jobId);
a regression test now pins that.

The relay now:
- takes the job context from the job row (never the evidence);
- commits a device-signed DOCUMENT (pcc-node's job-port form, #471) only
  when it names this job and the job's kernel (else 409 job_mismatch /
  kernel_mismatch) and sha256(canonicalize(the document minus bundleHash
  and kernelSignature)) equals the signed digest (else 422);
  hashModel device_signed_document;
- refuses a bare device digest (evidence_not_bound): it names no job.
LO-EV event bundles are unchanged. Persisting the document itself needs a
schema change: operator decision #4578.

gateway 3012 passed, 6 skipped, 0 failed (189 files); tsc clean; 5 mutants
(each new check removed, and the 409 mapping) killed, 2 failures each.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qb6kQhDYRwUDd6AVes3Fwx
@LamaSu @claude
fix(gateway): an event bundle must commit the job and its kernel in i…
15ebb46 
…ts hashed events (E4b)

Cross-family review E4b on e415d95, DO-NOT-SHIP, one HIGH: the event
bundle branch returned before any job or kernel binding. hashBundle
commits only to event hashes, so a signed bundle for job A could be
filed under job B (an unhashed envelope jobId is no binding), and an
event naming another kernel, or none, was accepted.

Reproduced at e415d95 first (4 failing tests): the reviewer's own case
(job A's signed events committed under job B: ok:true), events naming
another kernel or no kernel, events committing no job or another job,
and the route storing a device-signed bundle for another job (200).

Every event is now hashed from ONE canonical snapshot (the bytes
hashEvent hashes, parsed back), and that snapshot must commit THIS job
and its kernel, LO-EV-9's rules 5 and 6: payload.jobId is the job,
source.kernelId and any payload.kernelId are the job's kernel (a job
without a kernel binds nothing). Otherwise 409 job_mismatch /
kernel_mismatch, with the event index, and nothing is stored. The
stored events are the snapshots. Fixtures now commit their job.

gateway 3017 passed, 6 skipped, 0 failed; tsc clean; 4 mutants killed
(the null-kernel one after a test pinned a null source kernel).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qb6kQhDYRwUDd6AVes3Fwx
@LamaSu
LamaSu marked this pull request as ready for review October 1, 2026 18:48
@LamaSu LamaSu closed this Oct 1, 2026
@LamaSu LamaSu reopened this Oct 1, 2026
@LamaSu
LamaSu merged commit 2c3d448 into master Oct 1, 2026
17 checks passed
LamaSu added a commit that referenced this pull request Oct 3, 2026
@LamaSu @claude
merge: master @753feb43 into #360 (merge-up after EC1c's SHIP)
8fe16b6 
#360 was SHIP at c73d708 (EC1c) with full CI green, but master moved on 9/30–10/01 (#424,
#459 N80, #436, #338, #477, #483 and others), and three files conflicted. Resolutions:

- packages/spec/src/index.ts: both sides added exports at the same place. Both are kept: #360's
  `economics` namespace, and master's money-status and readmodels exports.
- packages/gateway/src/routes/swf.ts: both sides made POST /api/swf/epochs/:epochId/distribute
  answer 501 not_available before anything is read. Master's hunk (#421's, reviewed and merged
  with #424) is kept. The behavior is identical.
- packages/gateway/src/routes/rewards.ts: #360 made the reward, certificate and treasury routes
  answer 501 and deleted their fixtures. Master's N80 (#459, reviewed and merged) changed only
  the certificates: an empty list, 404 for one, and a 501 mint that claims nothing. It kept the
  reward and treasury fixtures. Kept: master's N80 certificates, and #360's 501s for rewards and
  treasury, with no fixture import. economics-no-fabrication.test.ts drops its three certificate
  rows (certificates-n80.test.ts pins N80), and the reward and treasury rows still assert 501.

On the merged tree: spec 1286/1286; contracts 267/267; mcp-server 76/76; gateway 3717 passed,
13 skipped, 0 failed; dashboard 353/353; tsc clean for contracts, mcp-server, gateway and
dashboard.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
LamaSu added a commit that referenced this pull request Oct 3, 2026
@LamaSu @claude
merge: master (7d688ce) into #341, now that #338 merged and #341 targ…
4f70ebf 
…ets master

Brings in #459 (the relay binds a signed document to its job and kernel) and the
rest of master, so the subject binding is tested against current code.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGNHoFFhbAdeNBc4BWigst
LamaSu added a commit that referenced this pull request Oct 3, 2026
@LamaSu @claude
test(gateway): rule 5's negative isolates the delegation check under …
51dbabd 
…LO-EV-9

#459's test (adk #4322, rule 5) presented a session bundle signed for job A as
job B with no events or subject. Merged with #341 (LO-EV-9), the slot is refused
earlier as missing-subject, so the test no longer exercised rule 5 (found by
#341's first full CI on master: build-and-test, 1 failure).

The test now binds the events to job B, the job being settled, so the subject
binding passes, and signs them with a session key whose delegation names only
job A. The scope check refuses it: contract_not_allowed. The whole-bundle replay
(job A's bundle for job B) stays covered by the kernel-sdk subject test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGNHoFFhbAdeNBc4BWigst
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@LamaSu