Skip to content

eth/protocols/wit, eth: WIT2 size oracle — accept non-deterministic witnesses within a signed size band - #2416

Merged
lucca30 merged 11 commits into
v2.10.2-candidatefrom
lmartins/wit2-size-oracle
Sep 22, 2026
Merged

lucca30 merged 11 commits into
v2.10.2-candidatefrom
lmartins/wit2-size-oracle

Conversation

@lucca30

@lucca30 lucca30 commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

What

WIT2 used the BP-signed witness hash as an exact-match identity: a serving peer whose witness bytes hash differently from the signed WitnessHash was rejected and struck, and two such servers fell the block back to WIT1. That assumes witnesses are byte-identical across nodes.

They are not. BlockSTM speculative reads make honest nodes collect different-but-valid trie-node sets, so a valid witness routinely hashes differently (observed ±5–9% node-set spread — e.g. one mainnet block seen at 12,551 / 13,194 / 13,688 nodes), and every node persists its own generated witness, so nearly every non-BP server's bytes differ from the BP's. The exact-hash gate therefore rejected valid witnesses and struck the honest peers serving them.

This PR replaces exact-hash equality with a size oracle.

How

  • Sign the size. SignedWitnessAnnouncement gains WitnessSize; the announce signing pre-image commits to it: keccak(domain ‖ blockHash ‖ blockNumber ‖ witnessHash ‖ witnessSize). Producers set it from their own witness length. At accept time a WitnessSize of zero or above the gas-derived absolute witness cap is refused and the sender struck (before deferral, so it never enters the deferred queue or the signed cache): a bad size is the signer's or forwarding relayer's doing, not the servers' it would later mis-judge. Because the size now decides the accept band, the signed-announce cache treats a same-hash announcement with another WitnessSize as a conflict like another hash (first commitment wins), so a second signature cannot move the band under an in-flight fetch.
  • Accept within a band for import. A witness whose encoded size is ≤ min(3×signedSize, absolute cap) is accepted for import regardless of hash — on the paged-fetch path (verifyAgainstSignedHash) and, with the same rule, on the broadcast paths (acceptSignedBroadcast, acceptDeferredBroadcast), so a peer pushing its own post-import witness to a waiter is no longer rejected downstream. Import-time state-root execution remains the content arbiter. The band math saturates and a zero signed size falls back to the absolute cap, so no input can collapse the ceiling to zero.
  • Re-serve only the BP's bytes. A within-band witness is cached for pre-import serving / relayed only when byte-identical to the BP's (hash match); a valid non-deterministic variant imports locally but is not re-served, so the pre-import fast path carries the producer's bytes exclusively and an upstream that pushed garbage cannot get a relay amplified or blamed for bytes it did not choose. Consequently the serve-only relay fetch (fetchAndVerifyWitness) keeps requiring the BP's bytes by design and is now documented as only succeeding against the producer or a node still holding the producer's bytes in its own pre-import cache; waiters otherwise fall back to the pull path.
  • Charge import failures to the server. With hash identity gone, the serving peer — not the BP — chose the bytes of a within-band divergent witness, so an import failure of such a witness is no longer silently forgotten: the witness manager records the serving peer, the divergence and the fetch closure on the import op (verifyAgainstSignedHash reports diverged; the fetcher now enqueues the manager's op instead of rebuilding it, which had dropped that provenance), and on insertChain failure importBlocks strikes the server, excludes it as a witness source for that block (SetWitnessSourceExcluder → handler witnessSourceExclusions, honoured by resolveWitnessFetchPeer at every tier and released on import), and hands the block back to the witness manager to re-fetch from another peer, bounded by maxWitnessImportRetries = 2 so a genuinely invalid block cannot cycle the peer set. A BP-identical witness that fails import is still the BP's fault and is handled as before (logged, forgotten). Strikes are thresholded (5 per sliding minute), so an honest server hit by a bad block is not disconnected.
  • Charge the pusher too. The same consequence applies to a within-band divergent body that arrives by NewWitness broadcast: acceptSignedBroadcast / acceptDeferredBroadcast report the divergence, InjectWitness carries it with the pusher into handleBroadcast and the before-the-block witness cache, and both attach sites record pusher, divergence and fetch closure on the op. An import failure of pushed bytes therefore strikes and excludes the pusher and re-fetches from another peer exactly like the fetch path, and a pusher excluded for a block has its further pushes for that block dropped (eth/wit2/serve/broadcast_excluded_source_drop) so it cannot beat every honest re-fetch with the same body.
  • Charge only witness-attributable failures. Because "diverged" is the normal case (every node persists its own generated witness), the charge is gated on a positive allowlist of failures the witness could have caused: a missing trie node (incomplete witness), or execution against the witness disagreeing with the header (ErrStatelessStateRootMismatch, ErrGasUsedMismatch, ErrReceiptRootMismatch, ErrBloomMismatch, ErrRequestsHashMismatch, and the fmt-built "invalid merkle root" / stateless self-validation mismatch messages). A contract bytecode missing from local disk — which core, eth: detect and self-heal missing contract code on stateless verification #2401 now surfaces as ErrStatelessIncompleteState wrapping *state.MissingCodeError, and which the downloader heals — is explicitly excluded (errors.As on the wrapped cause, since the same sentinel wraps a missing trie node), as are interrupted or stopped inserts, whitelist mismatches, header/DB errors and unknown errors. Without the gate, a missing bytecode would have struck and excluded two honest witness sources per block until the node had none left.
  • Bound the size. A witness beyond the band is rejected and the serving peer struck (first occurrence per (peer, block)); distinct servers exceeding the band for the same block fall back to WIT1. Only BP-identical bytes (hash match) clear a block's oversize count — a divergent in-band body proves nothing about the signed size, so a server alternating oversized and in-band bodies cannot keep a block out of quarantine. The retained gas-derived ceiling keeps the accepted size bounded even when the signed size is implausibly large.

Rollout

The signed-announce wire format changes in place, and this does not land on a clean slate: the WIT2 commit is in v2.10.2-beta, v2.10.2-preconf and v2.10.2-preconf2, and preconf builds are running on some Amoy nodes. Old and new builds both advertise wit/3, so an old-format SignedNewWitnessHashesMsg fails RLP decode on a new node (and vice versa once a new-format BP exists), the handler returns the error and the p2p layer drops the connection. No jail is involved; redial is gated only by the dial-history / inbound-throttle timers, so the churn is self-limiting and stops once every WIT2 node runs the same format.

  • All WIT2-capable hosts should be upgraded in one window; the earlier -beta / -preconf tags are superseded by this format.
  • A wit/4 bump was considered and not done: a relay cannot convert a BP-signed size-bearing announce into the legacy pre-image for a wit/3 peer, so a dual-format window would buy little beyond avoiding the self-limiting reconnect churn. If zero churn is required, that is a separate follow-up.

Scope

WIT2 signed-path only. The WIT1 page-count check was #2417 (merged). The branch has origin/v2.10.2-candidate merged in (for #2401's typed MissingCodeError), so it carries #2417 and #2401.

Metrics

New: eth/wit2/announce/implausible_size, eth/wit2/serve/broadcast_oversize, eth/wit2/serve/broadcast_hash_divergence, eth/wit2/serve/broadcast_excluded_source_drop, eth/fetcher/witness/oversized, eth/fetcher/witness/hash_divergence, eth/fetcher/witness/import_failure, eth/fetcher/witness/import_retry. Removed: eth/wit2/serve/broadcast_byte_mismatch, eth/fetcher/witness/byte_mismatch (no longer a rejection reason).

Tests

  • eth/protocols/wit: signing digest now covers witnessSize (stability, domain-separation, per-field sensitivity incl. a new size case).
  • eth/fetcher:
    • within-band hash divergence is accepted for import, not re-served (body=nil), and not struck (TestVerifyAgainstSignedHashAcceptsDivergentHashWithinBand, TestProcessWitnessResponseDoesNotDropOnByteMismatch); exact match is served (TestVerifyAgainstSignedHashServesOnExactMatch);
    • a witness beyond the band is struck and, across distinct servers, falls back to WIT1;
    • ceiling = min(3×S, absolute) incl. the implausibly-large clamp, plus the degenerate inputs: S=0 → absolute cap, S≈MaxUint64 saturates (TestAcceptableWitnessSizeCeiling*, TestSaturatingMulUint64);
    • import-failure consequence: TestImportFailureWithDivergedWitnessRefetchesFromAnotherPeer drives the real fetcher loop — first import fails with a state-root mismatch, server struck and excluded once, witness re-fetched from a second peer, block imports; TestChargeDivergedWitnessImportFailure (only fetched+diverged is charged, retry budget honoured), TestChargeDivergedWitnessImportFailureIgnoresNonWitnessErrors (a wrapped *state.MissingCodeError, interrupted/stopped inserts, whitelist.ErrMismatch, unknown errors: no strike, no exclusion, no re-fetch, no budget consumed), TestIsWitnessAttributableImportError (the allowlist, incl. ErrStatelessIncompleteState wrapping a missing node → attributable vs. wrapping missing code → not), TestRetryAfterImportFailureReRegistersPending; push path: TestImportFailureWithDivergedBroadcastWitnessChargesPusher (body cached before its block) and TestImportFailureWithDivergedBroadcastWitnessOnPendingBlockChargesPusher (body attached to a pending block) drive the real fetcher loop — the pusher, not a fetch server, is struck and excluded once, the witness re-fetched, the block imports; TestVerifyAgainstSignedHashDivergentInBandKeepsMismatchState (only an exact match clears the oversize count).
  • eth:
    • TestAcceptSignedAnnouncementRejectsImplausibleWitnessSize (0 and cap+1 refused and struck, not deferred; cap accepted);
    • broadcast gates: TestHandleWitnessBroadcastDivergentWithinBandImportsWithoutServing, TestHandleWitnessBroadcastOversizeDropped, and the deferred-path variant/oversize cases in TestHandleWitnessBroadcastAcceptedWhileAnnounceDeferred; TestAcceptSignedBroadcastReportsDivergence / TestAcceptDeferredBroadcastReportsDivergence (the provenance bit handed to the fetcher), TestHandleWitnessBroadcastDivergentBodyImportFailureChargesPusher (wire handler → push → cached body → block → attributable import failure → the pusher struck and excluded, witness re-fetched, block imported), TestHandleWitnessBroadcastDropsExcludedSource, TestSignedWitnessCacheRejectsConflictingWitnessSize (rejected before and after the relay window);
    • TestResolveWitnessFetchPeerSkipsExcludedSource (excluded at every tier, per block, released on import), TestWitnessSourceExclusionSetLifecycle, TestSizeOracleHelpersWithoutFetcher, TestDeferredAnnounceCacheHasWitnessSizeWithin (inclusive boundary, TTL).
  • Full eth witness/peer suites, eth/fetcher, eth/protocols/wit pass; golangci-lint run ./eth/... clean.

Test plan

  • go build ./...
  • go vet ./eth/...
  • golangci-lint run ./eth/... (0 issues)
  • go test ./eth/fetcher/ ./eth/protocols/wit/
  • go test -race ./eth/fetcher/ -run 'ImportFailureWithDiverged|ChargeDiverged|RetryAfterImportFailure' -count=3
  • go test ./eth/ -run 'Wit2|Witness|SignedWitness|Announce|Broadcast|Relay|Deferred|PeerSet|Peer'
  • Local hand-mutant pass over the Diffguard survivor sites from the last two CI runs (charge gate and budget, allowlist predicate, size band and saturation, broadcast/deferred gates, announce size sanity, source exclusion, deferred size-band TTL, diverged flag on every verifyAgainstSignedHash return, striker/excluder wiring): 37/37 mutants killed. importBlocks split into runBlockImport + logTrackedImport to stay under the complexity threshold. New functions are 100% line-covered in their own packages except the pre-existing encode-failure and shutdown branches.
  • Local hand-mutant pass over the push-path follow-up (14/14 killed): provenance at both attach sites incl. the fetch closure, divergence bit from both accept functions and at the inject call, excluded-source drop, quarantine clear placement, size-conflict key
  • devnet WIT2 propagation re-validation (kurtosis): 9-node mixed old/new WIT2 devnet run (v2.10.2-preconf2 ↔ this PR, full mesh) followed by an in-place upgrade of the old nodes — self-limiting redial churn, 0 jails / strikes / charges, churn stops after the upgrade; results in eth/protocols/wit, eth: WIT2 size oracle — accept non-deterministic witnesses within a signed size band #2416 (comment)

@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.17094% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.53%. Comparing base (68a4609) to head (a4d7a6a).
⚠️ Report is 24 commits behind head on v2.10.2-candidate.

Files with missing lines Patch % Lines
eth/fetcher/block_fetcher.go 66.34% 33 Missing and 2 partials ⚠️
eth/handler_wit.go 92.98% 3 Missing and 1 partial ⚠️
eth/handler_wit2.go 89.28% 3 Missing ⚠️
eth/fetcher/witness_manager.go 97.50% 2 Missing ⚠️
eth/fetcher/witness_manager_wit2.go 97.33% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                  Coverage Diff                  @@
##           v2.10.2-candidate    #2416      +/-   ##
=====================================================
- Coverage              57.93%   57.53%   -0.40%     
=====================================================
  Files                    957      959       +2     
  Lines                 177763   175858    -1905     
=====================================================
- Hits                  102984   101187    -1797     
+ Misses                 69118    69006     -112     
- Partials                5661     5665       +4     
Files with missing lines Coverage Δ
eth/fetcher/witness_import_errors.go 100.00% <100.00%> (ø)
eth/handler.go 69.22% <100.00%> (-6.84%) ⬇️
eth/handler_eth.go 74.79% <100.00%> (-0.21%) ⬇️
eth/handler_wit2_announces.go 98.02% <100.00%> (+0.08%) ⬆️
eth/handler_wit2_exclusions.go 100.00% <100.00%> (ø)
eth/handler_wit2_peer.go 76.11% <100.00%> (+1.11%) ⬆️
eth/handler_wit_relay_fetch.go 91.66% <100.00%> (ø)
eth/peerset.go 92.11% <ø> (ø)
eth/protocols/wit/protocol.go 68.00% <100.00%> (+1.33%) ⬆️
eth/fetcher/witness_manager.go 91.79% <97.50%> (+2.77%) ⬆️
... and 4 more

... and 39 files with indirect coverage changes

Files with missing lines Coverage Δ
eth/fetcher/witness_import_errors.go 100.00% <100.00%> (ø)
eth/handler.go 69.22% <100.00%> (-6.84%) ⬇️
eth/handler_eth.go 74.79% <100.00%> (-0.21%) ⬇️
eth/handler_wit2_announces.go 98.02% <100.00%> (+0.08%) ⬆️
eth/handler_wit2_exclusions.go 100.00% <100.00%> (ø)
eth/handler_wit2_peer.go 76.11% <100.00%> (+1.11%) ⬆️
eth/handler_wit_relay_fetch.go 91.66% <100.00%> (ø)
eth/peerset.go 92.11% <ø> (ø)
eth/protocols/wit/protocol.go 68.00% <100.00%> (+1.33%) ⬆️
eth/fetcher/witness_manager.go 91.79% <97.50%> (+2.77%) ⬆️
... and 4 more

... and 39 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lucca30
lucca30 force-pushed the lmartins/wit2-size-oracle branch from 0df3826 to cb11efb Compare September 16, 2026 12:17
…itnesses within a signed size band

Witnesses are not deterministic across nodes: BlockSTM speculative reads make
honest nodes collect different-but-valid trie-node sets, so a valid witness
routinely hashes differently from the BP-signed WitnessHash. WIT2's fetch-time
verifyAgainstSignedHash treated any hash divergence as a fault (reject the
bytes, strike the serving peer, fall back to WIT1 after two distinct servers),
so a valid witness that hashes differently is rejected and its serving peer
struck.

Replace exact-hash equality with a size oracle:

- Sign the witness size. SignedWitnessAnnouncement gains a WitnessSize field and
  the announce signing pre-image commits to it
  (keccak(domain || blockHash || blockNumber || witnessHash || witnessSize)).
  Producers set it from their own witness length.

- Accept within a band for import. verifyAgainstSignedHash accepts for import
  any witness whose encoded size is <= min(3*signedSize, gas-derived absolute
  ceiling), regardless of hash; content-correctness is still arbitrated by
  import-time state-root execution. A within-band witness is re-served/relayed
  only when byte-identical to the BP's (hash match); a valid non-deterministic
  variant imports locally but is not re-served, so the signed hash stays a
  faithful identifier of the bytes on the serving/relay fast-path. Blame for
  content rests with the producer that signed the announcement, not a relaying
  or serving peer — preserving WIT2's property of relaying a trusted witness
  before self-validating.

- Bound the size. A witness beyond the band is rejected and the serving peer
  struck (first occurrence per (peer, block), reusing the distinct-server
  bookkeeping); distinct servers exceeding the band for the same block fall back
  to WIT1. The retained gas-derived ceiling keeps the accepted size bounded even
  when the signed size is implausibly large.

No WIT2 is deployed yet, so the signed-announce format changes in place.

Scope: WIT2 signed-path only. The WIT1 page-count cross-peer verification is a
separate change, handled in a follow-up.
@lucca30
lucca30 force-pushed the lmartins/wit2-size-oracle branch from cb11efb to 0ed9568 Compare September 16, 2026 12:50
@lucca30
lucca30 marked this pull request as ready for review September 16, 2026 13:24

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@lucca30

lucca30 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 eth/fetcher/witness_manager.go — A maintainer reading this comment believes WIT2 still rejects a witness whose bytes don't hash to the BP-signed WitnessHash, which is no longer true after this PR. The comment at witness_manager.go:704-707 says the encoded witness "must hash to the signed witnessHash", but verifyAgainstSignedHash (witness_manager_wit2.go) now accepts any within-band size regardless of hash match. Fix: update this comment to describe the size-oracle acceptance (only oversized witnesses are rejected; hash divergence is observability-only), so the security invariant documented here matches the code a reader is about to call.

    Extended reasoning...

    processWitnessResponse at witness_manager.go:708 calls verifyAgainstSignedHash immediately below this comment. Before this PR, verifyAgainstSignedHash rejected on hash mismatch, so the comment was accurate. This PR changed verifyAgainstSignedHash so a differing hash within the size band is accepted (ok=true) and only an oversized witness is rejected/struck. The comment block at lines 704-707 was not updated and still states the old byte-correctness invariant. A future reviewer or on-call engineer investigating a witness-serving incident would read this comment, conclude byte-correctness is still enforced pre-import, and misjudge the trust boundary between BP-signed hash and actually-served bytes — exactly the kind of stale security-invariant documentation the repo's own commenting guidance (AGENTS.md 'constraints and assumptions') warns against.

    Verification: Severity: nit. The comment at eth/fetcher/witness_manager.go:704-707 states the encoded witness bytes "must hash to the signed witnessHash", and the diff did not touch it (git diff shows the only change to this file is the signedWitnessHashFn type signature adding witnessSize). This comment now contradicts the reworked callee verifyAgainstSignedHash in eth/fetcher/witness_manager_wit2.go,…

The comment above verifyAgainstSignedHash still described the pre-oracle
byte-correctness invariant (hash must match). This PR's size-oracle rework
now accepts hash divergence within the signed-size band, so the comment
misdescribed the actual trust boundary an incident responder would rely on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@cffls

cffls commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Reviewed the diff and traced the paths around it. The change of direction is right: with each node persisting its own generated witness (core/blockchain.go writes statedb.Witness() at all three write sites), nearly every non-BP server's bytes hash differently from the BP's, so the exact-hash gate was rejecting valid witnesses as routine behavior. A few things to address or explicitly accept before merge.

1. Rollout note: the format change is not landing on a clean slate (temporary)

The description says no WIT2 is deployed, but the WIT2 commit is in v2.10.2-beta, v2.10.2-preconf, and v2.10.2-preconf2, and preconf builds are already running on a few Amoy nodes, including a producing one. Both old and new builds advertise wit/3, so they always negotiate wit/3; an old-format SignedNewWitnessHashesMsg then fails RLP decode on a new node (input list has too few elements), the handler returns the error, and the p2p layer drops the whole connection (eth included). Same in reverse once a new-format BP exists.

This is self-limiting: no jail is involved, redial is gated only by the 35 s dial-history / 30 s inbound-throttle timers, and once every WIT2 node runs the same format the churn stops on its own. So it's a mixed-version window issue, not a design blocker. Please:

  • correct the description (it currently states WIT2 isn't deployed),
  • call out in the rollout notes that all WIT2-capable hosts should be upgraded in one window and that the earlier -beta/-preconf tags are superseded,
  • if zero churn is wanted instead, bump to wit/4 and gate the new struct/preimage on Version() >= WIT3, keeping wit/3 on the legacy shape.

2. The import-time arbiter has no consequence

The PR leans on "import-time state-root execution remains the content-correctness arbiter", but in BlockFetcher.importBlocks an insertChain failure is logged at Debug and the hash is forgotten via f.done: no re-fetch from another peer, no strike on the server. Under the old exact-hash gate a server feeding wrong bytes paid a strike and two distinct servers pushed the block to WIT1. Now a peer that relays the valid BP announce becomes announce-known, wins the fetch, and can serve up to 3 × signedSize of bytes that pass the band and fail import, per block, at no cost. This is the WIT1-class weakness WIT2 was closing, reopened on the WIT2 path.

Suggestion: when import fails for a block whose fetched witness diverged from the signed hash, strike the serving peer and re-request from another announce-known peer instead of forgetting the hash. If that's deferred to a follow-up, say so in the description so the trade-off is on record.

3. Only one of three exact-hash gates was relaxed

acceptSignedBroadcast (eth/handler_wit.go) still drops a divergent NewWitness push without injecting it for import, and fetchAndVerifyWitness (relay fetch) still rejects divergent bytes. Given nodes serve their own witness, the waiter-push path (flushWitnessWaitersForImported pushes the node's own bytes) will be rejected downstream, and relay-fetch will fail against any non-BP upstream. The pull path recovers, so it's a latency regression rather than a break, but those two WIT2 features become effectively dead under non-determinism. Either apply the same accept-for-import (not serve) rule there, or document that they only work for BP-identical bytes.

4. Small things

  • signedSize * wit2SizeBandMultiplier can wrap for a hostile signed size, and WitnessSize == 0 yields a zero ceiling; both strike honest servers. Saturating math, and rejecting announces with WitnessSize == 0 or above the absolute cap at accept time so the announcer is the one struck.
  • Stale byte-correctness comments: eth/fetcher/witness_manager.go:63-66, :71, :96; eth/handler_wit2.go:198-200; eth/handler.go:193.
  • Description says a within-band divergent witness is "accepted, served, and not struck"; code and TestVerifyAgainstSignedHashAcceptsDivergentHashWithinBand assert it is not served (body=nil).
  • The devnet re-validation box is unchecked; given (1), a mixed old/new WIT2 devnet run is worth adding.

Verified locally: go build ./..., go vet, and go test ./eth/fetcher/ ./eth/protocols/wit/ ./eth/ (witness subset) pass; merges cleanly with #2417 and the merged tree builds and passes the same suites.

lucca30 and others added 2 commits September 18, 2026 08:06
…d size, relax the broadcast gates

Review follow-ups for the WIT2 size oracle.

Import-time consequence. A witness accepted on the size oracle alone (hash
differs from the BP-signed one) that then fails import was logged at debug and
forgotten, so a peer relaying the valid BP announce could serve up to the size
band in unusable bytes per block at no cost — the WIT1-class weakness WIT2 was
closing. Now the witness manager records the serving peer, the divergence and
the fetch closure on the import op (verifyAgainstSignedHash returns diverged;
enqueueOp keeps the op instead of rebuilding it from parts), and on insertChain
failure importBlocks strikes the server, excludes it as a witness source for
that block (new SetWitnessSourceExcluder hook → handler witnessSourceExclusions,
honoured by resolveWitnessFetchPeer at every tier and released on import), and
hands the block back to the witness manager for a re-fetch from another peer
(new witnessRetry loop case → retryAfterImportFailure), bounded by
maxWitnessImportRetries. A BP-identical witness that fails import is still the
BP's fault and is handled as before.

Signed size sanity. signedSize*3 could wrap for a hostile size and a zero
WitnessSize yielded a zero ceiling; both struck honest servers. The band now
saturates and a zero size falls back to the absolute cap, and
acceptSignedAnnouncement refuses (and strikes the sender for) a WitnessSize of
zero or above the gas-derived absolute cap before deferral, so the announcer is
the one charged.

Broadcast gates. acceptSignedBroadcast and acceptDeferredBroadcast now apply the
same size oracle as the fetch path: a within-band body is accepted for import
(sender marked as body-holder) regardless of hash, so a pusher's own
post-import witness (flushWitnessWaitersForImported) is no longer rejected
downstream; only byte-identical bytes are cached for pre-import serving, and an
oversized body is dropped. fetchAndVerifyWitness (relay fetch, serve-only) keeps
requiring the BP's bytes by design and now documents that limitation.

Also refreshes every remaining "byte-correctness" comment to the size-oracle
semantics (fetcher, handler, peerset, wit protocol), and renames the broadcast
byte-mismatch meter to broadcast_oversize with new hash_divergence,
implausible_size, import_failure and import_retry meters.

Tests: end-to-end import-failure re-fetch through the real fetcher loop
(TestImportFailureWithDivergedWitnessRefetchesFromAnotherPeer — caught the
provenance loss in enqueue), charge/retry unit tests, degenerate-size and
saturating-multiply cases, implausible announce size (0 and cap+1 struck, cap
accepted), divergent/oversized broadcast on both the signed and deferred paths,
source exclusion in resolveWitnessFetchPeer, and the exclusion set lifecycle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Brme9KQBd7fZBMnMVEhAZU
#2417 changes the NewBlockFetcher call directly above the WIT2 striker wiring;
editing the adjacent comment here made the two branches conflict when merged
together even though each merges cleanly into the candidate. Leave the striker
comment as it was and document the size-oracle extension (import-failure strike
+ source exclusion) in its own block below.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Brme9KQBd7fZBMnMVEhAZU
@cffls

cffls commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Re-reviewed cd3ed0678 + 80e3f34f9. Everything from the first pass is addressed: import failures are now charged and re-fetched, the broadcast and deferred gates apply the same size band, announce-time WitnessSize plausibility, saturating band math, the stale comments, and the rollout section. Builds, vets, and the eth, eth/fetcher, eth/protocols/wit suites pass locally (including -race on the witness subset), and it still merges cleanly with #2417 with the merged tree passing the same suites.

One blocking issue remains, in the new import-failure charge.

chargeDivergedWitnessImportFailure charges the serving peer on any insertChain error

It checks op.witnessDiverged and op.witnessPeer, but never inspects importErr. The reasoning is "the server chose these bytes, so if the block won't import the server is at fault", which only holds when the failure is something the witness bytes could have caused.

Two things make this bite in practice:

  1. Diverged is the normal case. Every node persists its own generated witness, and BlockSTM makes those differ between honest nodes, so on a stateless node nearly every witness fetched from anyone but the BP is flagged diverged. The charge therefore applies to nearly every import failure.
  2. core, eth: detect and self-heal missing contract code on stateless verification #2401 is now on v2.10.2-candidate. A contract bytecode missing from the local disk now surfaces from ExecuteStateless as ErrStatelessIncompleteState wrapping *state.MissingCodeError, and that propagates unwrapped through ProcessBlockWithWitnesses and InsertChainStateless straight into importBlocks. core, eth: detect and self-heal missing contract code on stateless verification #2401's self-heal fetch lives only in the downloader's batch import, not on the fetcher's tip path, so at the tip it is just an import error.

Sequence on a stateless node with a missing bytecode: block touches the contract, the (valid) witness imports fail on the local gap, the honest server is struck and excluded, the re-fetch reaches a second honest peer whose (valid) witness fails the same way and it is struck too, the retry budget is spent, the block is forgotten. The node falls behind and the downloader eventually heals the code, but the strikes stay on the tracker. At 5 strikes per sliding minute, a contract that appears in most blocks costs two honest peers a strike per block; a node's announce-known WIT2 peers are a small set, so it disconnects and jails its own witness sources within a minute and then has none to fetch from.

The same branch also fires on errors that have nothing to do with the witness: chain shutdown mid-import (errInsertionInterrupted), a whitelist mismatch against a milestone, a header-verification failure, a writeBlockAndSetHead error.

To be clear, #2401 does not cause this; before it, the same local gap surfaced as a gas-used or state-root mismatch and would have been charged just the same, only indistinguishably. #2401 is what makes the case cheap to exclude.

Suggested fix: gate the charge on witness-attributable failures. Return false when errors.As(importErr, &mce) finds a *state.MissingCodeError, and for interrupted insertion, whitelist mismatch, header verification, and DB write errors. Charge on state-root mismatch, receipt-root mismatch, gas-used mismatch, and ErrStatelessIncompleteState when the wrapped error is a missing trie node rather than missing code (an incomplete witness is something the server did produce). The predicate can live in eth/fetcher or be injected from the handler the way the striker and excluder are. Add a test case with a MissingCodeError asserting no strike, no exclusion, no re-fetch; the current tests only use a generic errors.New(...), and the branch predates the #2401 merge so its CI cannot produce this error.

CI gates still red on this head

  • Diffguard: mutation score 65.1%, 15 survivors in the new code (block_fetcher.go:1346-1354, witness_manager_wit2.go:119-180, handler_wit.go:174-337, handler_wit2.go:250, handler_wit2_announces.go:415).
  • Codecov patch: 89.2% vs the 90% target; eth/handler_wit2.go is at 57% on the new lines.

On the e2e failure (now green on retry): it was Test 5 (load test with producer rotation), where heimdall rotated the span on 1 of 3 netem windows instead of 3 of 3. No witness, strike, or exclusion warnings appeared in the collected container logs, and the same commit passed on re-run, so I read it as the test's timing sensitivity rather than this change. The panic: close of closed channel in the heimdall bridge is separate shutdown noise from the machinery worker.

@pratikspatil024

Copy link
Copy Markdown
Member

Reviewed the two new commits. Everything from the earlier rounds looks addressed — I re-checked the saturating band math, the announce-time WitnessSize plausibility gate, the exclusion set's TTL/gc/drop wiring and its application at every tier of resolveWitnessFetchPeer, and the retry machinery (fresh op rebuilt without the stale witness so it doesn't trip the state.op.witness != nil short-circuit, witnessImportFailures carried through to bound the cycle, forget-then-re-register sequenced on the fetcher loop, no queue/announce accounting leak, no f.mu/m.mu inversion). That part holds up.

One gap in the new code, in the half of cd3ed0678 that relaxed the broadcast gates.

The relaxed broadcast path accepts divergent bodies for import but cannot charge them

acceptSignedBroadcast (eth/handler_wit.go:156) now returns true for a within-band body whose hash differs from the BP's, and acceptDeferredBroadcast does the same via hasWitnessSizeWithin. Both then flow into InjectWitness → witnessManager.handleBroadcast.

handleBroadcast (eth/fetcher/witness_manager.go:410) sets state.op.witness = msg.witness and never sets witnessPeer or witnessDiverged. handleWitnessFetchSuccess (eth/fetcher/witness_manager.go:767-768) is the only writer of either field.

So for a broadcast-delivered witness, chargeDivergedWitnessImportFailure returns on its first guard:

if op.witness == nil || !op.witnessDiverged || op.witnessPeer == "" {
    return false
}

A within-band, non-BP-identical witness that arrives by NewWitness and then fails import costs the sender nothing: no strike, no exclusion, no re-fetch — the block is simply forgotten, which is the pre-cd3ed0678 behaviour the charge was added to remove. The fetch path is now accountable; the push path is not.

Two things make the asymmetry worth closing rather than accepting:

  1. Before this commit the case wasn't reachable — the broadcast gate rejected divergent bytes outright — so relaxing the gate opened a path the charge doesn't cover.
  2. The push path is the easier one to land on: the sender chooses when to push, with no need to win resolveWitnessFetchPeer, and handleBroadcast takes the first witness to arrive for a pending block (if state.op.witness == nil), so a push can beat an honest in-flight fetch.

If the importErr predicate from the previous round lands without this, the gap widens — the fetch path gets more conservative about striking while the push path stays free.

Suggested fix. The provenance is already at hand: handleBroadcast has msg.peer, and both accept functions have already computed bodyHash against the signed commitment. Plumb the divergence bit from acceptSignedBroadcast / acceptDeferredBroadcast through InjectWitness into handleBroadcast, and set witnessPeer / witnessDiverged there the way handleWitnessFetchSuccess does. The witnessCache path needs the same for a body that arrives before its block — cachedWitness already carries peer, it just isn't propagated when the witness is later attached. Worth a test asserting that a divergent broadcast body whose import fails strikes and excludes the pusher, mirroring TestImportFailureWithDivergedWitnessRefetchesFromAnotherPeer.

Two smaller ones, neither blocking

  • clearSignedHashMismatch (eth/fetcher/witness_manager_wit2.go:136) runs on every in-band acceptance, including divergent ones. A server alternating oversized and in-band responses resets the distinct-server counter each time and can keep a block out of quarantine indefinitely. Previously only an exact hash match cleared it.
  • signedWitnessCache.putIfNewer (eth/handler_wit2_announces.go:516) still keys conflict detection on WitnessHash alone. Now that WitnessSize is signed and decides the accept band, two producer-signed announcements that agree on hash but differ in size aren't treated as conflicting, and the later one replaces the band once wit2RelayWindow lapses.

…s could have caused it

chargeDivergedWitnessImportFailure struck and excluded the serving peer on any
insertChain error. "Diverged" is the normal case on a stateless node (every
node persists its own generated witness), so every import failure — including a
contract bytecode missing from local disk, which the downloader heals and the
witness never carried, or an interrupted insert — would have struck two honest
witness sources per block until the node had none left.

Gate the charge on a positive allowlist of witness-attributable failures: a
missing trie node (incomplete witness), and execution against the witness
disagreeing with the header (ErrStatelessStateRootMismatch, ErrGasUsedMismatch,
ErrReceiptRootMismatch, ErrBloomMismatch, ErrRequestsHashMismatch, and the
fmt-built "invalid merkle root" / stateless self-validation mismatch messages).
Everything else — missing code, interrupted or stopped chain, whitelist
mismatch, header or DB errors, unknown errors — is not charged.

Also: drop the unreachable absBytes == 0 guard in acceptableWitnessSizeCeiling
(the page threshold floors at one page); make the deferred-broadcast hash-match
test use a size band that would reject the body, so only the hash match admits
it; cover the handler size-oracle helpers without a fetcher and the deferred
size-band binding's boundary and TTL.

Tests: TestChargeDivergedWitnessImportFailureIgnoresNonWitnessErrors,
TestIsWitnessAttributableImportError, TestSizeOracleHelpersWithoutFetcher,
TestDeferredAnnounceCacheHasWitnessSizeWithin; existing charge/re-fetch tests
now use attributable errors.
…ssing from local disk

#2401 surfaces a missing contract bytecode on the stateless path as
core.ErrStatelessIncompleteState wrapping a *state.MissingCodeError, the same
sentinel that wraps a missing trie node. Witnesses carry no code, so the server
could not have supplied it and the downloader's self-heal fetches the blob;
charging the server would strike and exclude honest witness sources on every
block that touches the contract until the node had none left.

Check for *state.MissingCodeError first in isWitnessAttributableImportError
and return false explicitly, so the wrapped cause — not the shared sentinel —
decides attribution: missing node → incomplete witness → charged; missing
code → local gap → not charged; the bare sentinel → cause unknown → not charged.

Tests: the wrapped MissingCodeError case in
TestChargeDivergedWitnessImportFailureIgnoresNonWitnessErrors (no strike, no
exclusion, no re-fetch, no budget consumed) and both wrappings of
ErrStatelessIncompleteState in TestIsWitnessAttributableImportError.
…ranches

Exercise AcceptableWitnessSizeCeiling from its own package (the handler is its
only production caller) and the two retryAfterImportFailure early exits — block
already known locally, witness marked unavailable — so the new code is fully
covered where it lives.
…he sampled mutation survivors

Diffguard flagged importBlocks at complexity 21 (threshold 15) after the
witness-retry plumbing. Move the goroutine body into runBlockImport, which
returns whether the failed import should be retried with a witness from another
peer, and the block-tracker log into logTrackedImport; importBlocks now only
routes the result to witnessRetry or done. Behaviour is unchanged.

Pin the five sampled survivors: assert the diverged flag on both the
exact-match and divergent returns of verifyAgainstSignedHash and on the
WIT1-only (nil lookup) return; add the exact-boundary case to the saturating
multiply (MaxUint64/2 * 2 must multiply, not saturate); and pin that the
fetcher's striker and source-excluder callbacks are wired into the handler
(StrikeWitnessServer / ExcludeWitnessSource are exported for that test).
@lucca30

lucca30 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in b0f0dff, 95e3f58 and 62954f9 (head 62954f9; origin/v2.10.2-candidate merged in at 8376f38 for #2401's types).

Import-failure charge is now gated on witness-attributable errors. chargeDivergedWitnessImportFailure calls a new isWitnessAttributableImportError (eth/fetcher/witness_import_errors.go), a positive allowlist:

  • *trie.MissingNodeError (incomplete witness) → charged;
  • the execution-mismatch sentinels ErrStatelessStateRootMismatch, ErrGasUsedMismatch, ErrReceiptRootMismatch, ErrBloomMismatch, ErrRequestsHashMismatch, plus the fmt-built "invalid merkle root" / stateless self-validation mismatch messages → charged;
  • *state.MissingCodeError is checked first and returns false explicitly — since ErrStatelessIncompleteState wraps both a missing node and missing code, the wrapped cause decides, not the sentinel;
  • interrupted/stopped insert, whitelist.ErrMismatch, header/DB errors, unknown errors → not charged, no retry budget consumed.

Tests: TestChargeDivergedWitnessImportFailureIgnoresNonWitnessErrors uses a real ErrStatelessIncompleteState-wrapped *state.MissingCodeError and asserts no strike, no exclusion, no re-fetch; TestIsWitnessAttributableImportError covers both wrappings of the sentinel. The existing charge/re-fetch tests now fail with attributable errors.

CI gates.

  • Quality metrics on 95e3f58: mutation score passed (89.4%, all tiers); the FAIL was importBlocks complexity 21 > 15 from the retry plumbing. Split into runBlockImport + logTrackedImport, behaviour unchanged. The five sampled survivors are pinned: diverged asserted on every verifyAgainstSignedHash return (exact match, divergent, WIT1-only), the saturating-multiply boundary (MaxUint64/2 * 2 must multiply), and TestFetcherWitnessPenaltiesAreWiredToHandler for the striker/excluder wiring. Local hand-mutant pass over all survivor sites from both runs: 37/37 killed.
  • codecov/patch is at 97%. codecov/project shows −0.43% vs 68a4609; that delta comes from merging the candidate into the branch, not from this diff.

Also from the first pass: unreachable absBytes == 0 guard removed; the deferred hash-match test now uses a size band that would reject the body, so only the hash match admits it; handler helpers covered without a fetcher; deferred size-band boundary and TTL covered.

Devnet: the mixed old/new WIT2 run you suggested is in progress; results will follow in a separate comment.

@lucca30

lucca30 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Mixed old/new WIT2 devnet — results

Ran the mixed-version window on a 9-node kurtosis devnet, then upgraded the old nodes in place and kept sampling.

Setup (full p2p mesh, so every old↔new pair exists): 4 validators + 2 full-sync relays + 3 stateless (sync-with-witness) nodes, 2 s blocks. Nodes 3 (a producing validator), 6 (relay) and 8 (stateless) ran v2.10.2-preconf2 — the 4-field announce that is on the Amoy preconf nodes; the other six ran this PR at 62954f9a8. Both builds negotiate wit/3. Phase 1 sampled heads, peer counts and the p2p + WIT2 meters every 15 s over 96 blocks in the mixed state; phase 2 kurtosis service updated nodes 3/6/8 to the PR image (≈50 s, they resumed from their own data) and sampled another 96 blocks. Debug logs and Prometheus scrapes dumped per node per phase; log counts below are restricted to each sample window.

Phase 1 — mixed window (blocks 205→300)

  • Decode fails in both directions, exactly where expected: old←new rlp: input list has too many elements for wit.SignedWitnessAnnouncement (195 / 213 / 210 lines on the three old nodes in the window); new←old rlp: input string too long for uint64, decoding into (*wit.SignedNewWitnessHashesPacket).Announcements[0].WitnessSize (dominant during bootstrap when the old producer's announces reached new nodes; 33–45 per new node over the run). The wit handler returns the error, p2p drops the connection (err="subprotocol error"), the peer redials.
  • Each old↔new connection lives ~1–3 s (median; dies on the first relayed announce). Redial cadence per pair: median 10–18 s (≈17 s over the whole run), p90 26–32 s — bounded by the 35 s dial-history / 30 s inbound-throttle timers, as you described. Per node over the 3-minute window: 65–71 connection drops on each old node, 32–35 on each new node; old nodes made ~2× the dials of new ones.
  • Jails: 0 on all nine nodes. WIT2 strikes: 0. Oversize / implausible-size refusals: 0. Import-failure charges: 0. The failure is at RLP decode, before any WIT2 handler or the strike tracker runs.
  • Liveness: heads stayed in lockstep on all nine nodes (max spread at any 15 s sample: 3 blocks, in 3 of 12 samples; 0 at the end), stateless nodes included. Old nodes held 2–3 peers (their new-format neighbours keep churning), new nodes 5 (up to 8 with transient old inbound).

Phase 2 — after upgrading nodes 3/6/8 in place (blocks 332→428)

  • RLP shape mismatches: 0 on all nine nodes. Connection drops: 0. Dials/serves during the window: 0. Peer counts: 8/8 on every node for the whole window (full mesh). The churn stops once every WIT2 node runs the same format.
  • WIT2 traffic healthy on the upgraded nodes: announce relay in/out over the window 995/533 (node 3), 1087/533 (node 6), 1171/416 (node 8); the three stateless nodes fetched one witness per block (96–98 each). Heads in lockstep, max spread 1 block. Still 0 jails / strikes / charges. The only non-zero unrelated signature in either window was a single block in the future propagated-import retry (block 418), i.e. sub-second clock skew.

Size oracle under load — from a first attempt of the same run (that attempt ran polycli loadtest at 120 rps and was aborted after the load OOM-killed nodes on my 14 GiB Docker host, so it has log evidence only, no counters): with 100–150-tx blocks the non-determinism showed up on real witnesses — the stateless nodes logged 6 × wit2: broadcast witness within the size band but not the BP's bytes; importing without re-serving plus 1 fetched witness with diverged=true. All imported; Import failed with a witness accepted on the WIT2 size oracle: 0 on every node, so no honest server was charged.

Artifacts (params, two-phase runner, per-phase CSV samples, analyzer, generated summary, first-attempt evidence) are in my agent-zero repo under investigations/witness-propagation/devnet-2026-09-21-mixed-rollout-2416/. Ticking the devnet box in the description.

… quarantine clear and announce conflict

Review follow-ups (pratikspatil024, 2026-09-21).

The broadcast path accepted a within-band divergent body for import but
could not charge it: handleBroadcast set only op.witness, and both
cached-witness attach sites rebuilt the op without provenance, so
chargeDivergedWitnessImportFailure returned on its first guard and a
NewWitness push was the free way to deliver unusable bytes — and the
easier one, since the first witness to arrive is the one attached.
acceptSignedBroadcast/acceptDeferredBroadcast now report the divergence,
InjectWitness carries it with the pusher, handleBroadcast records
witnessPeer/witnessDiverged and the announce's fetch closure, and the
before-the-block cache keeps peer+diverged so both attach sites build the
op via cachedWitness.injectFor with full provenance. An import failure of
pushed bytes now strikes and excludes the pusher and re-fetches from
another peer, as on the fetch path; a pusher excluded for a block has its
further pushes for that block dropped (broadcast_excluded_source_drop)
so it cannot beat every honest re-fetch with the same body.

verifyAgainstSignedHash cleared the oversize/quarantine state on every
in-band acceptance; a divergent in-band body proves nothing about the
signed size, so a server alternating oversized and in-band bodies could
keep a block out of quarantine indefinitely. Only BP-identical bytes
clear it now (the pending-removal exits and TTL sweep still bound the
maps).

signedWitnessCache.putIfNewer keyed conflicts on WitnessHash alone; the
signed WitnessSize decides the accept band, so a same-hash announce with
another size is now a conflict too (first commitment wins) instead of
replacing the band once the relay window lapses.

Tests: push-path import-failure e2e for both attach sites through the
real fetcher loop, provenance bit from both accept functions, excluded
pusher dropped, quarantine kept across a divergent acceptance, size
conflict rejected before and after the relay window.
@lucca30

lucca30 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 6b6f9405e. All three verified against the code before changing anything.

1. Push path could not charge a divergent body — fixed. Confirmed: handleBroadcast set only state.op.witness, and both cached-witness attach sites (handleInjectedBlock, handleFilterResult) rebuilt the op with just the witness, so chargeDivergedWitnessImportFailure returned on its first guard for anything that arrived by NewWitness. Now:

  • acceptSignedBroadcast / acceptDeferredBroadcast return (accepted, diverged); InjectWitness(peer, witness, diverged) carries it, handleBroadcast records witnessPeer / witnessDiverged and takes fetchWitness from the pending announce (mirroring handleWitnessFetchSuccess), and cachedWitness keeps peer + diverged so both attach sites build the op via cachedWitness.injectFor(origin, block, fetchWitness) with full provenance. A pushed within-band body that fails an attributable import now strikes and excludes the pusher and re-fetches from another peer, same as the fetch path.
  • One step further than suggested, because of your point (2) about the push winning the race: handleWitnessBroadcast drops a push from a peer already excluded as a source for that block (witnessSourceExclusions.excluded, new meter eth/wit2/serve/broadcast_excluded_source_drop). Without it the same pusher could re-push the same bytes into the retried pending state and burn the retry budget.
  • Tests: TestImportFailureWithDivergedBroadcastWitnessChargesPusher (body arrives before its block → cache path) and ...OnPendingBlockChargesPusher (body attached to a pending block, fetch responses held until the strike lands) drive the real fetcher loop: pusher struck and excluded exactly once, witness re-fetched, block imports. TestAcceptSignedBroadcastReportsDivergence, TestAcceptDeferredBroadcastReportsDivergence, TestHandleWitnessBroadcastDropsExcludedSource cover the handler side.

2. clearSignedHashMismatch on every in-band acceptance — fixed. Moved into the exact-match branch: only BP-identical bytes clear a block's oversize count; a divergent in-band body proves nothing about the signed size. The four pending-removal exits and the TTL sweep still bound the maps. TestVerifyAgainstSignedHashDivergentInBandKeepsMismatchState (divergent acceptance between two distinct oversizers still quarantines; an exact match resets the count).

3. putIfNewer conflict key — fixed. A same-hash announcement with another WitnessSize is now a conflict (first commitment wins, same meter), so a second signature cannot move the band after wit2RelayWindow. TestSignedWitnessCacheRejectsConflictingWitnessSize.

Build, vet, golangci-lint clean; eth/fetcher + eth/protocols/wit full packages and the eth witness/peer suites pass, the two new e2e tests also under -race ×3. Hand-mutant pass over the new lines: 14/14 killed (provenance at both attach sites incl. the fetch closure, divergence bit from both accept functions and at the inject call, excluded-source drop, quarantine clear placement, size-conflict key before and after the relay window). PR description updated.

Diffguard's file-size gate allows a file already over 800 lines to grow by
10% of its base size; block_fetcher.go had reached +138 (10.4%) on the
candidate base. chargeDivergedWitnessImportFailure and
maxWitnessImportRetries move to witness_import_errors.go, which already
holds isWitnessAttributableImportError, the predicate the charge is gated
on. No behaviour change.
@lucca30

lucca30 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@pratikspatil024 pratikspatil024 left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving.

Re-checked the three items from my earlier comment against a4d7a6aeb:

  • Push-path provenance: handleBroadcast records witnessPeer / witnessDiverged, cachedWitness carries both through to the attach sites, and InjectWitness takes the bit from both accept functions — including the handleInjectedBlock / handleFilterResult sites I had missed. A divergent pushed body that fails import is now charged to the pusher.
  • The quarantine clear is reached only on a byte-identical match.
  • putIfNewer treats a same-hash / different-size announce as a conflict.

Also read through isWitnessAttributableImportError. Checking *state.MissingCodeError ahead of the ErrStatelessIncompleteState sentinel is the right ordering given that sentinel wraps both a missing node and missing code, and defaulting unknown errors to "not charged" fails on the safe side.

go test ./eth/fetcher/ ./eth/protocols/wit/ and the eth witness subset pass locally on this head.

One rollout condition, not a code change. The devnet run puts a number on the mixed-version window: old nodes held 2–3 peers instead of 8 while both formats were live on wit/3, returning to 8/8 only after the in-place upgrade. On Amoy that lands on the v2.10.2-preconf* nodes, one of which produces. Please coordinate with whoever owns those hosts so every WIT2-capable Amoy node is upgraded in a single window rather than piecemeal. The description already says this — I'm attaching it to the approval so it doesn't get lost between merge and deploy.

@lucca30
lucca30 merged commit b50f0e8 into v2.10.2-candidate Sep 22, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants