test: consolidate the Fast Inbox end-to-end coverage - #189
Draft
spalladino wants to merge 3 commits into
Draft
spalladino wants to merge 3 commits into
spalladino wants to merge 3 commits into
Conversation
spalladino
added this pull request to stack #188
September 12, 2026 06:48
… e2e gate Checkpoint building has no barrier a test can stop at: Sequencer.pause() drains the in-flight checkpoint and its pending L1 submissions, so it cannot leave the chain after one block of a checkpoint and before the next one freezes its Inbox range. Adds one optional in-process hook, invoked immediately after a block has been signed and synced to the proposer's own archiver and before it is gossiped or the next sub-slot selects its messages. The hooks are dependency-injected through CreateAztecNodeDeps -> SequencerClient.new -> Sequencer -> CheckpointProposalJob and are deliberately absent from the serialized node configuration, so no RPC or config surface can reach them. CheckpointProposalJobTestGate turns that hook into an ordering primitive for e2e suites: one-shot predicate arming, idempotent release, a watchdog that both fails the test and unblocks the held job, and a failure channel separate from the matched-event promise. It orders real chain operations only; it never freezes protocol time and never sleeps.
Pins the message-count boundaries the streaming Inbox turns on, in fast unit tests rather than in an e2e parameter sweep. InboxParity dispatch gains direct coverage at every rung boundary from both sides (0/1/4/5/16/17/64/65/255/256/257/1023/1024) plus the 1025 rejection, along with input construction and padding at each of those counts; existing native proving covers whole rungs but never the dispatcher. Selection gains the protocol-constant table for 0/1/255/256/257/1024/1025 (including the 257-message rollover a single block may not take whole and the hard stop at 1024) and makes the 767/768/769 prospective-endpoint threshold explicit on both the lookup decision and the safe local step. The validator's cap checks, which otherwise only run on small injected caps, gain a protocol-value pass at 255/256/257 per block and 1024/1025 per checkpoint. The rolling hash gains a split-invariance table over the same counts, which is the property that lets a signed prefix reference be compared across different bucket groupings. SDK readiness gains message-index zero (a falsy index that must not read as unseen), and waitForL1ToL2MessageReady gains polling-transition, requested-tip forwarding and timeout cases it had none of.
Folds the remaining streaming-Inbox confidence work into the suites that already stand up the environments it needs, rather than into new fixtures. No new e2e case and no new CI environment: the five affected files go from thirteen cases to twelve. streaming_inbox now runs its own prover node, so every proof claim in it is a real wait on the prover-node orchestration and the simulated protocol circuits instead of a hand-marked proven tip. Its mid-checkpoint case absorbs the separate consume and double-spend case and becomes one choreographed scenario: the checkpoint test gate holds block zero after the proposer's archiver stored it, the L1 message is sent and indexed while production is held, readiness is shown false at the tip and against a proven tip that is demonstrably behind, a public consume is queued, and the release lets one block both insert the message and spend it. Readiness is then asserted against node views pinned to the parent and the inserting block, so neither answer can come from the tip moving between two calls, and the pending proven-tip wait has to resolve on its own once the checkpoint is proven. The message-only case waits for the same prover and shows its bundle was non-empty by the leaf-count delta. streaming_inbox_buckets proves the intermediate prefix explicitly: no live L1 bucket ends where the first block stopped, and a later block completes the bucket that does. streaming_inbox_backlog sends its first batch as 257 messages in one L1 transaction and asserts the rollover on L1 (dense indices, two bucket sequences sharing the send's block timestamp, a total advanced by exactly 257) and on the node once the messages reach L2. Its endpoint oracle now groups receipt events by bucket sequence, so a rolled-over batch contributes both of its endpoints instead of only the last. l1-reorgs/messages repurposes its changed-message case into a placement-only reorg: the same ordered messages are re-mined atomically at the same height under one bucket instead of two, with count and rolling hash unchanged, while a non-final block of the current checkpoint is held. No shorter prefix is ever exposed, since that would be a legal reason to prune. The assertions are on preservation of work: the same block hash, the same checkpoint number and slot, the original compact indices, and a proof.
spalladino
force-pushed
the
spl/a-2019-consolidate-fast-inbox-e2e-coverage
branch
from
September 15, 2026 22:39
c9cd0d3 to
be1cf35
Compare
spalladino
removed this pull request from stack #188
September 15, 2026 22:42
spalladino
added this pull request to stack #217
September 15, 2026 22:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consolidate Fast Inbox e2e coverage
Implements A-2019 and the reviewed choreography in its attached plan document, "Fast Inbox e2e coverage
consolidation plan". It closes the remaining Fast Inbox confidence gaps inside the suites that already stand up the
environments those gaps need, instead of adding fixtures, and moves every boundary-value question out of e2e and
into fast unit tests.
Case accounting
The issue budgets the five affected files at fourteen
itcases going to twelve. In this repository they start atthirteen and end at twelve, and the one-case difference is not a deviation. A-2019 assumes the upstream case
consumes a message in the same block that inserts it, introduced by upstream #25323, which this port treats asbaseline and which three earlier rungs (N3b, N3c and FI-S06) independently confirmed is absent here; FI-S06 verified
by
git merge-base --is-ancestorthat the introducing commit is not an ancestor of this tree. Its assertion istherefore written fresh into the expanded streaming case rather than folded in from an existing one, and only one
case is removed rather than two.
The five files now hold:
streaming_inboxthree (expanded mid-checkpoint, latency, message-only),streaming_inbox_bucketstwo,streaming_inbox_backlogthree,l1-reorgs/messagestwo, andmulti-node/.../cross_chain_messagestwo. No new CI environment or job is introduced, and both.parallelfileskeep exactly two independently runnable cases.
The deterministic checkpoint gate
Sequencer.pause()drains the in-flight checkpoint and its pending L1 submissions, so it cannot leave the chain inthe state all of these scenarios need: after one block of a checkpoint is committed and before the next one freezes
its Inbox range. The change adds one optional in-process hook,
CheckpointProposalJobTestHooks.onCheckpointPhase,invoked at
block-ready-to-broadcastimmediately after the block has been signed and synced to the proposer's ownarchiver and before it is gossiped or the next sub-slot selects its messages. The hooks are dependency-injected
through
CreateAztecNodeDeps → SequencerClient.new → Sequencer → CheckpointProposalJoband are deliberately absentfrom
AztecNodeConfigand from every schema, so no serialized configuration or RPC surface can reach them; a nodethat is passed none pays one undefined check per block.
CheckpointProposalJobTestGate, inend-to-end/src/fixtures, turns that hook into an ordering primitive: one-shotpredicate arming, a rejected second concurrent arm, an idempotent
release()safe to call from a barefinally, awatchdog that both fails the test through a failure channel separate from the matched-event promise and unblocks the
held job so a forgotten release cannot wedge teardown, and a
remainingHoldBudgetMsa caller checks before lettingthe proposer go. It orders real chain operations only: it never freezes protocol time and never sleeps.
What each suite now proves
streaming_inboxruns its own prover node (startProverNode: true,realProofs: false), so every proof claim in itis a real wait on the production prover-node orchestration and
TestCircuitProver's witness simulation of InboxParityand the rollup circuits, rather than a hand-marked proven tip. This is a simulated proof accepted by the test
verifier, not cryptographic proof verification:
TestCircuitProverdoes not simulate the AVM proof circuit. Itsmid-checkpoint case absorbs the former consume/double-spend case and becomes one scenario. The gate holds block zero;
while held the L1 message is sent and indexed, readiness is shown false at the tip and against a proven tip that is
demonstrably behind, a readiness wait against that proven tip is started, and a public consume is queued. After the
release the message enters a non-first block of the same checkpoint, that block also executes the consume against its
own post-bundle message root, the parent has no witness and the inserting block has one at L1's compact index,
readiness pinned to those two concrete blocks answers false and true, a second consume reverts, and the pending
proven-tip wait resolves on its own once the prover proves the covering checkpoint. The message-only case waits for
the same prover and demonstrates a non-empty bundle by the leaf-count delta against its parent rather than by a
witness alone.
streaming_inbox_bucketskeeps its unique fresh-node replay and now proves the intermediate prefix explicitly: thecount where the first block stopped is not a live L1 bucket end, both halves keep one bucket sequence, and a later
block completes the bucket that is one.
streaming_inbox_backlogsends its first batch as 257 messages in a single Multicall3 transaction and asserts therollover on L1 from that receipt — dense monotonic indices across the boundary, messages 0..255 in one bucket
sequence and message 256 in the next, both buckets carrying the send's L1 block timestamp, and a cumulative Inbox
total advanced by exactly 257 — then, once the drain has carried them to L2, asserts the node resolves the same
compact indices and membership witnesses at positions 0, 255 and 256. Its endpoint oracle now groups receipt events
by
bucketSeqand keeps one endpoint per group; the previous one-final-index-per-batch rule would have dropped thefirst valid endpoint of a rolled-over batch and rejected a legal publication. The rollover batch replaces one uniform
batch, so the backlog still spans the same number of L1 blocks and still exceeds the per-checkpoint cap.
l1-reorgs/messagesrepurposes its changed-message case into a placement-only reorg. Two messages are sent inseparate L1 blocks and therefore separate buckets, a non-final block of the current checkpoint that consumed both is
held at the gate, every replacement call is prepared before L1 is touched, and one atomic same-height
reorgWithReplacementre-mines both in a single L1 block. No transient shorter prefix is ever exposed, becausepruning on one would be valid behaviour. The premise is asserted on both halves — the bucket boundary between the two
messages is gone, and the cumulative count and rolling hash are unchanged — and so is the absence of any rollup
transaction inside the reorg window, so a publication caught in the window fails the test with a clear reason rather
than silently changing what is under test. The assertions after release are about preservation of work in progress:
the same block hash at the same block number, never pruned; the same checkpoint number published for the same slot;
both messages at their original compact indices; and the checkpoint proven by the fixture's prover node.
Unit boundary matrix
Boundary coverage is completed in unit tests, with no e2e parameter sweep and no 4096-bucket e2e. InboxParity gains a
dedicated test file covering dispatch at every rung boundary from both sides (0, 1, 4, 5, 16, 17, 64, 65, 255, 256,
257, 1023, 1024) plus the 1025 rejection, and input construction and padding at each of those counts; the existing
native parity proving exercises whole rungs but never the dispatcher. Message selection gains a protocol-constant
table for 0, 1, 255, 256, 257, 1024 and 1025 — including the 257-message batch a single block may not take whole and
the hard stop at the checkpoint cap — and makes the 767/768/769 prospective-endpoint threshold explicit on both the
lookup decision and the safe local step. The validator's cap checks, which otherwise only run on small injected caps,
gain a protocol-value pass at 255/256/257 per block and 1024/1025 per checkpoint. The rolling hash gains a
split-invariance table over the same counts, which is the property that makes a signed prefix reference comparable
across different bucket groupings. SDK readiness gains message-index zero, a falsy index that must not read as
unseen, and
waitForL1ToL2MessageReadygains the polling-transition, requested-tip forwarding and timeout cases ithad none of. Solidity ring-overwrite coverage is retained untouched; it lives in
aztec-packages, not here.What was verified, and how
yarn build,yarn formatandyarn lintare clean across the monorepo. Unit suites, base to final:sequencer-client 12 suites 306 passed/1 skipped to 12 suites 325 passed/1 skipped; validator-client 11 suites 317
passed/3 skipped to 11 suites 323 passed/3 skipped; stdlib 132 suites 1120 passed/1 skipped to 133 suites 1160
passed/1 skipped; aztec.js 12 suites 106 passed to 12 suites 113 passed; end-to-end's
src/fixturesunit lane 0tests to 1 suite 10 passed; aztec-node 7 suites 190 passed, unchanged.
The hook and the gate have real red/green. Moving the hook call to before the archiver sync reddens exactly the
placement test; moving it after the standalone gossip reddens six of the seven. Removing the gate's one-shot guard,
its duplicate-arm rejection and the watchdog's unblocking each redden exactly the test that names that property.
Removing the parity dispatcher's boundary inclusiveness reddens exactly the at-the-cap rows.
The e2e suites were not run. Earlier rungs measured the cost: a single-node cross-chain suite budgets 600s for
beforeAllalone plus 600s per case, against this environment's hard 600s per-command ceiling, so not even oneitfits. The e2e changes were authored and made to compile, and their correctness evidence here is static: every node,
wallet, cheat-code and contract call they make was read against its definition in this tree and matches the
signature used, and the TypeScript build type-checks all of them. What static reading cannot establish, and what CI
must confirm, is the timing: that the work done inside each hold fits the proposer's real remaining budget, that the
queued consume lands in the block that inserts its message, that the L1 reorg window stays free of publisher
transactions in practice, and that the two prover waits in
streaming_inboxfit its per-test timeout.Left undone, deliberately
E2, the multi-node content-changing reorg, is not implemented. The plan requires the test to observe an exact
named-validator result — a first Inbox-metadata-check outcome distinct from the final
handleBlockProposalverdict,each carrying proposal slot, block hash, proposer, reason and whether the reason is slashable — and quorum alone is
explicitly insufficient. No such observable exists:
ProposalHandlerreturns{ isValid, reason }values to itscaller and emits no events, and the disabled slasher API is ruled out. Building that production surface, plus the
four-validator choreography around it, and shipping it without exercising it even once would be inventing a fix
rather than implementing one, so the existing multi-node case is left as it stands apart from correcting its stale
inboxLagcomment to the streaming behaviour. What it needs is a test-visible block-validation-result callback onProposalHandlercarrying those five fields plus a separate observable for the first metadata comparison, bothunit-tested in
proposal_handler.test.ts, before the e2e choreography is written.No timing profile was adjusted. The issue permits adjusting one only if measured runs require it, and nothing
could be measured here, so the 36s/6s and 24s/5s cadences are untouched. The one timeout that did change is
streaming_inbox's per-test Jest budget, raised from 600s to 900s to match its sibling prover-enabled bucket suite,because two of its cases now wait for a real prover where they previously marked a tip proven by hand.
bench/bench_build_block.test.tsremains knowingly red against the FI-S03 catch-up capacity floor. That is astanding open item from that rung and is not touched here. The e2e profiles this change relies on were checked
against
MIN_BLOCKS_FOR_INBOX_CATCHUP: both run atethereumSlotDuration: 4, inside the fast-profile exemption.