Skip to content

test: consolidate the Fast Inbox end-to-end coverage - #189

Draft
spalladino wants to merge 3 commits into
spl/fi-n11-inbox-botfrom
spl/a-2019-consolidate-fast-inbox-e2e-coverage
Draft

spalladino wants to merge 3 commits into
spl/fi-n11-inbox-botfrom
spl/a-2019-consolidate-fast-inbox-e2e-coverage

Conversation

@spalladino

@spalladino spalladino commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Consolidate Fast Inbox e2e coverage

Implements A-2019 and the reviewed choreography in its attached plan document, "Fast Inbox e2e coverage
consolidation plan". It closes the remaining Fast Inbox confidence gaps inside the suites that already stand up the
environments those gaps need, instead of adding fixtures, and moves every boundary-value question out of e2e and
into fast unit tests.

Case accounting

The issue budgets the five affected files at fourteen it cases going to twelve. In this repository they start at
thirteen and end at twelve, and the one-case difference is not a deviation. A-2019 assumes the upstream case
consumes a message in the same block that inserts it, introduced by upstream #25323, which this port treats as
baseline and which three earlier rungs (N3b, N3c and FI-S06) independently confirmed is absent here; FI-S06 verified
by git merge-base --is-ancestor that the introducing commit is not an ancestor of this tree. Its assertion is
therefore written fresh into the expanded streaming case rather than folded in from an existing one, and only one
case is removed rather than two.

The five files now hold: streaming_inbox three (expanded mid-checkpoint, latency, message-only),
streaming_inbox_buckets two, streaming_inbox_backlog three, l1-reorgs/messages two, and
multi-node/.../cross_chain_messages two. No new CI environment or job is introduced, and both .parallel files
keep exactly two independently runnable cases.

The deterministic checkpoint gate

Sequencer.pause() drains the in-flight checkpoint and its pending L1 submissions, so it cannot leave the chain in
the state all of these scenarios need: after one block of a checkpoint is committed and before the next one freezes
its Inbox range. The change adds one optional in-process hook, CheckpointProposalJobTestHooks.onCheckpointPhase,
invoked at block-ready-to-broadcast immediately after the block has been signed and synced to the proposer's own
archiver and before it is gossiped or the next sub-slot selects its messages. The hooks are dependency-injected
through CreateAztecNodeDeps → SequencerClient.new → Sequencer → CheckpointProposalJob and are deliberately absent
from AztecNodeConfig and from every schema, so no serialized configuration or RPC surface can reach them; a node
that is passed none pays one undefined check per block.

CheckpointProposalJobTestGate, in end-to-end/src/fixtures, turns that hook into an ordering primitive: one-shot
predicate arming, a rejected second concurrent arm, an idempotent release() safe to call from a bare finally, a
watchdog that both fails the test through a failure channel separate from the matched-event promise and unblocks the
held job so a forgotten release cannot wedge teardown, and a remainingHoldBudgetMs a caller checks before letting
the proposer go. It orders real chain operations only: it never freezes protocol time and never sleeps.

What each suite now proves

streaming_inbox runs its own prover node (startProverNode: true, realProofs: false), so every proof claim in it
is a real wait on the production prover-node orchestration and TestCircuitProver's witness simulation of InboxParity
and the rollup circuits, rather than a hand-marked proven tip. This is a simulated proof accepted by the test
verifier, not cryptographic proof verification: TestCircuitProver does not simulate the AVM proof circuit. Its
mid-checkpoint case absorbs the former consume/double-spend case and becomes one scenario. The gate holds block zero;
while held the L1 message is sent and indexed, readiness is shown false at the tip and against a proven tip that is
demonstrably behind, a readiness wait against that proven tip is started, and a public consume is queued. After the
release the message enters a non-first block of the same checkpoint, that block also executes the consume against its
own post-bundle message root, the parent has no witness and the inserting block has one at L1's compact index,
readiness pinned to those two concrete blocks answers false and true, a second consume reverts, and the pending
proven-tip wait resolves on its own once the prover proves the covering checkpoint. The message-only case waits for
the same prover and demonstrates a non-empty bundle by the leaf-count delta against its parent rather than by a
witness alone.

streaming_inbox_buckets keeps its unique fresh-node replay and now proves the intermediate prefix explicitly: the
count where the first block stopped is not a live L1 bucket end, both halves keep one bucket sequence, and a later
block completes the bucket that is one.

streaming_inbox_backlog sends its first batch as 257 messages in a single Multicall3 transaction and asserts the
rollover on L1 from that receipt — dense monotonic indices across the boundary, messages 0..255 in one bucket
sequence and message 256 in the next, both buckets carrying the send's L1 block timestamp, and a cumulative Inbox
total advanced by exactly 257 — then, once the drain has carried them to L2, asserts the node resolves the same
compact indices and membership witnesses at positions 0, 255 and 256. Its endpoint oracle now groups receipt events
by bucketSeq and keeps one endpoint per group; the previous one-final-index-per-batch rule would have dropped the
first valid endpoint of a rolled-over batch and rejected a legal publication. The rollover batch replaces one uniform
batch, so the backlog still spans the same number of L1 blocks and still exceeds the per-checkpoint cap.

l1-reorgs/messages repurposes its changed-message case into a placement-only reorg. Two messages are sent in
separate L1 blocks and therefore separate buckets, a non-final block of the current checkpoint that consumed both is
held at the gate, every replacement call is prepared before L1 is touched, and one atomic same-height
reorgWithReplacement re-mines both in a single L1 block. No transient shorter prefix is ever exposed, because
pruning on one would be valid behaviour. The premise is asserted on both halves — the bucket boundary between the two
messages is gone, and the cumulative count and rolling hash are unchanged — and so is the absence of any rollup
transaction inside the reorg window, so a publication caught in the window fails the test with a clear reason rather
than silently changing what is under test. The assertions after release are about preservation of work in progress:
the same block hash at the same block number, never pruned; the same checkpoint number published for the same slot;
both messages at their original compact indices; and the checkpoint proven by the fixture's prover node.

Unit boundary matrix

Boundary coverage is completed in unit tests, with no e2e parameter sweep and no 4096-bucket e2e. InboxParity gains a
dedicated test file covering dispatch at every rung boundary from both sides (0, 1, 4, 5, 16, 17, 64, 65, 255, 256,
257, 1023, 1024) plus the 1025 rejection, and input construction and padding at each of those counts; the existing
native parity proving exercises whole rungs but never the dispatcher. Message selection gains a protocol-constant
table for 0, 1, 255, 256, 257, 1024 and 1025 — including the 257-message batch a single block may not take whole and
the hard stop at the checkpoint cap — and makes the 767/768/769 prospective-endpoint threshold explicit on both the
lookup decision and the safe local step. The validator's cap checks, which otherwise only run on small injected caps,
gain a protocol-value pass at 255/256/257 per block and 1024/1025 per checkpoint. The rolling hash gains a
split-invariance table over the same counts, which is the property that makes a signed prefix reference comparable
across different bucket groupings. SDK readiness gains message-index zero, a falsy index that must not read as
unseen, and waitForL1ToL2MessageReady gains the polling-transition, requested-tip forwarding and timeout cases it
had none of. Solidity ring-overwrite coverage is retained untouched; it lives in aztec-packages, not here.

What was verified, and how

yarn build, yarn format and yarn lint are clean across the monorepo. Unit suites, base to final:
sequencer-client 12 suites 306 passed/1 skipped to 12 suites 325 passed/1 skipped; validator-client 11 suites 317
passed/3 skipped to 11 suites 323 passed/3 skipped; stdlib 132 suites 1120 passed/1 skipped to 133 suites 1160
passed/1 skipped; aztec.js 12 suites 106 passed to 12 suites 113 passed; end-to-end's src/fixtures unit lane 0
tests to 1 suite 10 passed; aztec-node 7 suites 190 passed, unchanged.

The hook and the gate have real red/green. Moving the hook call to before the archiver sync reddens exactly the
placement test; moving it after the standalone gossip reddens six of the seven. Removing the gate's one-shot guard,
its duplicate-arm rejection and the watchdog's unblocking each redden exactly the test that names that property.
Removing the parity dispatcher's boundary inclusiveness reddens exactly the at-the-cap rows.

The e2e suites were not run. Earlier rungs measured the cost: a single-node cross-chain suite budgets 600s for
beforeAll alone plus 600s per case, against this environment's hard 600s per-command ceiling, so not even one it
fits. The e2e changes were authored and made to compile, and their correctness evidence here is static: every node,
wallet, cheat-code and contract call they make was read against its definition in this tree and matches the
signature used, and the TypeScript build type-checks all of them. What static reading cannot establish, and what CI
must confirm, is the timing: that the work done inside each hold fits the proposer's real remaining budget, that the
queued consume lands in the block that inserts its message, that the L1 reorg window stays free of publisher
transactions in practice, and that the two prover waits in streaming_inbox fit its per-test timeout.

Left undone, deliberately

E2, the multi-node content-changing reorg, is not implemented. The plan requires the test to observe an exact
named-validator result — a first Inbox-metadata-check outcome distinct from the final handleBlockProposal verdict,
each carrying proposal slot, block hash, proposer, reason and whether the reason is slashable — and quorum alone is
explicitly insufficient. No such observable exists: ProposalHandler returns { isValid, reason } values to its
caller and emits no events, and the disabled slasher API is ruled out. Building that production surface, plus the
four-validator choreography around it, and shipping it without exercising it even once would be inventing a fix
rather than implementing one, so the existing multi-node case is left as it stands apart from correcting its stale
inboxLag comment to the streaming behaviour. What it needs is a test-visible block-validation-result callback on
ProposalHandler carrying those five fields plus a separate observable for the first metadata comparison, both
unit-tested in proposal_handler.test.ts, before the e2e choreography is written.

No timing profile was adjusted. The issue permits adjusting one only if measured runs require it, and nothing
could be measured here, so the 36s/6s and 24s/5s cadences are untouched. The one timeout that did change is
streaming_inbox's per-test Jest budget, raised from 600s to 900s to match its sibling prover-enabled bucket suite,
because two of its cases now wait for a real prover where they previously marked a tip proven by hand.

bench/bench_build_block.test.ts remains knowingly red against the FI-S03 catch-up capacity floor. That is a
standing open item from that rung and is not touched here. The e2e profiles this change relies on were checked
against MIN_BLOCKS_FOR_INBOX_CATCHUP: both run at ethereumSlotDuration: 4, inside the fast-profile exemption.

@spalladino
spalladino added this pull request to stack #188 September 12, 2026 06:48
@spalladino spalladino changed the title spl/a 2019 consolidate fast inbox e2e coverage test: consolidate the Fast Inbox end-to-end coverage Sep 12, 2026
@spalladino spalladino added the ci-draft Run CI on this draft PR label Sep 12, 2026
… e2e gate

Checkpoint building has no barrier a test can stop at: Sequencer.pause() drains the
in-flight checkpoint and its pending L1 submissions, so it cannot leave the chain
after one block of a checkpoint and before the next one freezes its Inbox range.

Adds one optional in-process hook, invoked immediately after a block has been signed
and synced to the proposer's own archiver and before it is gossiped or the next
sub-slot selects its messages. The hooks are dependency-injected through
CreateAztecNodeDeps -> SequencerClient.new -> Sequencer -> CheckpointProposalJob and
are deliberately absent from the serialized node configuration, so no RPC or config
surface can reach them.

CheckpointProposalJobTestGate turns that hook into an ordering primitive for e2e
suites: one-shot predicate arming, idempotent release, a watchdog that both fails the
test and unblocks the held job, and a failure channel separate from the matched-event
promise. It orders real chain operations only; it never freezes protocol time and
never sleeps.
Pins the message-count boundaries the streaming Inbox turns on, in fast unit tests
rather than in an e2e parameter sweep.

InboxParity dispatch gains direct coverage at every rung boundary from both sides
(0/1/4/5/16/17/64/65/255/256/257/1023/1024) plus the 1025 rejection, along with input
construction and padding at each of those counts; existing native proving covers whole
rungs but never the dispatcher. Selection gains the protocol-constant table for
0/1/255/256/257/1024/1025 (including the 257-message rollover a single block may not
take whole and the hard stop at 1024) and makes the 767/768/769 prospective-endpoint
threshold explicit on both the lookup decision and the safe local step. The validator's
cap checks, which otherwise only run on small injected caps, gain a protocol-value pass
at 255/256/257 per block and 1024/1025 per checkpoint. The rolling hash gains a
split-invariance table over the same counts, which is the property that lets a signed
prefix reference be compared across different bucket groupings. SDK readiness gains
message-index zero (a falsy index that must not read as unseen), and
waitForL1ToL2MessageReady gains polling-transition, requested-tip forwarding and
timeout cases it had none of.
Folds the remaining streaming-Inbox confidence work into the suites that already
stand up the environments it needs, rather than into new fixtures. No new e2e case
and no new CI environment: the five affected files go from thirteen cases to twelve.

streaming_inbox now runs its own prover node, so every proof claim in it is a real
wait on the prover-node orchestration and the simulated protocol circuits instead of
a hand-marked proven tip. Its mid-checkpoint case absorbs the separate consume and
double-spend case and becomes one choreographed scenario: the checkpoint test gate
holds block zero after the proposer's archiver stored it, the L1 message is sent and
indexed while production is held, readiness is shown false at the tip and against a
proven tip that is demonstrably behind, a public consume is queued, and the release
lets one block both insert the message and spend it. Readiness is then asserted
against node views pinned to the parent and the inserting block, so neither answer
can come from the tip moving between two calls, and the pending proven-tip wait has
to resolve on its own once the checkpoint is proven. The message-only case waits for
the same prover and shows its bundle was non-empty by the leaf-count delta.

streaming_inbox_buckets proves the intermediate prefix explicitly: no live L1 bucket
ends where the first block stopped, and a later block completes the bucket that does.

streaming_inbox_backlog sends its first batch as 257 messages in one L1 transaction
and asserts the rollover on L1 (dense indices, two bucket sequences sharing the send's
block timestamp, a total advanced by exactly 257) and on the node once the messages
reach L2. Its endpoint oracle now groups receipt events by bucket sequence, so a
rolled-over batch contributes both of its endpoints instead of only the last.

l1-reorgs/messages repurposes its changed-message case into a placement-only reorg:
the same ordered messages are re-mined atomically at the same height under one bucket
instead of two, with count and rolling hash unchanged, while a non-final block of the
current checkpoint is held. No shorter prefix is ever exposed, since that would be a
legal reason to prune. The assertions are on preservation of work: the same block
hash, the same checkpoint number and slot, the original compact indices, and a proof.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-draft Run CI on this draft PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant