fix(cicd,cmd,consensus/XDPoS,eth): fix fast sync by wgr523 · Pull Request #2272 · XinFinOrg/XDPoSChain

wgr523 · 2026-04-03T12:39:42Z

Proposed changes

Fix fast sync. I'll add a doc about how to do fast sync. Tested on private net and devnet.

Types of changes

What types of changes does your code introduce to XDC network?
Put an ✅ in the boxes that apply

Impacted Components

Which parts of the codebase does this PR touch?
Put an ✅ in the boxes that apply

Checklist

Put an ✅ in the boxes once you have confirmed below actions (or provide reasons on not doing so) that

This PR has sufficient test coverage (unit/integration test) OR I have provided reason in the PR description for not having test coverage
Tested on a private network from the genesis block and monitored the chain operating correctly for multiple epochs.
Provide an end-to-end test plan in the PR description on how to manually test it on the devnet/testnet.
Tested the backwards compatibility.
Tested with XDC nodes running this version co-exist with those running the previous version.
Relevant documentation has been updated as part of this PR
N/A

Add new CLI flags to configure a fixed pivot block for fast sync: - --fastsyncpivotnumber: Pivot block number (0 = use default calculation) - --fastsyncpivothash: Pivot block hash for verification Changes: - Add FastSyncPivotNumber and FastSyncPivotHash to ethconfig.Config - Add pivotNumber and pivotHash fields to Downloader struct - Add SetPivotBlock() method to configure pivot before sync - Use configured pivot in syncWithPeer instead of dynamic calculation - Prevent pivot block from moving during sync when configured - Verify pivot block header hash after state sync completes - Add state root verification after state sync completes This allows operators to sync from a specific trusted checkpoint block instead of relying on the dynamic pivot calculation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Changes: - Add pivotGapNumbers field to Downloader struct - Calculate gap pivot numbers in SetPivotBlock() - Sync gap pivot states in processFastSyncContent() before primary pivot

coderabbitai · 2026-04-03T12:39:54Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 471953ef-18f1-475a-93a7-f9a0ecaeca54

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fixfastsync-new

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Copilot

Pull request overview

This PR aims to fix fast sync on XDPoSChain by making fast sync pivot selection configurable (number/hash/root), adding “gap pivot” state sync + snapshot generation, and adjusting consensus header verification behavior to better support fast-sync header insertion.

Changes:

Add fast-sync pivot configuration to ethconfig (TOML + CLI flags) and wire it into the downloader at node startup.
Extend the downloader to support a fixed pivot, optional pivot hash verification, gap-pivot state syncs, and snapshot generation.
Adjust XDPoS v1/v2 header verification and epoch-switch info lookup to support reduced verification modes used during header-chain validation.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
eth/ethconfig/config.go	Adds FastSyncPivot* fields to the eth configuration struct.
eth/ethconfig/gen_config.go	Updates TOML marshal/unmarshal for the new fast-sync pivot fields.
eth/downloader/statesync.go	Removes an outdated comment about fast sync usage.
eth/downloader/downloader.go	Implements configurable pivot/gap pivot logic and snapshot generation; changes fast-sync header verification constants/behavior.
eth/backend.go	Wires configured pivot settings into the downloader during backend initialization.
consensus/XDPoS/engines/engine_v2/verifyHeader.go	Adjusts verification flow to use header-provided validators/penalties when not full-verifying.
consensus/XDPoS/engines/engine_v2/utils.go	Updates getEpochSwitchInfo call sites for new signature.
consensus/XDPoS/engines/engine_v2/timeout.go	Updates getEpochSwitchInfo call sites for new signature.
consensus/XDPoS/engines/engine_v2/snapshot.go	Exports snapshot constructor/storage helpers (NewSnapshot/StoreSnapshot).
consensus/XDPoS/engines/engine_v2/snapshot_test.go	Updates tests to use exported snapshot helpers.
consensus/XDPoS/engines/engine_v2/epochSwitch.go	Refactors getEpochSwitchInfo to accept parent-header slices for VerifyHeaders optimization and softens snapshot failure handling.
consensus/XDPoS/engines/engine_v2/engine.go	Updates snapshot usage and verifyQC/getEpochSwitchInfo interactions for new signatures/exports.
consensus/XDPoS/engines/engine_v1/engine.go	Skips checkpoint signer checks when not doing full verification.
cmd/XDC/main.go	Registers new CLI flags for fast-sync pivot configuration.
cmd/utils/flags.go	Defines and applies new fast-sync pivot flags into ethconfig.
cicd/testnet/start.sh	Adds environment-driven fast-sync pivot args.
cicd/mainnet/start.sh	Adds environment-driven fast-sync pivot args.
cicd/local/start.sh	Adds environment-driven fast-sync pivot args.
cicd/devnet/start.sh	Adds environment-driven fast-sync pivot args.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 9 comments.

Comments suppressed due to low confidence (1)

eth/downloader/downloader.go:523

When a fixed pivot is configured, syncWithPeer does not validate that pivotNumber is <= the remote height. If pivotNumber is greater than the chain head, header fetching can terminate while d.committed stays 0, potentially stalling fast sync. Consider validating d.pivotNumber <= height (and also d.pivotNumber > 0) up front and returning a configuration error if it's out of range.

		if d.pivotNumber != 0 {
			// Use configured pivot block
			log.Info("Using configured pivot block", "number", d.pivotNumber)
			pivot = d.pivotNumber
			if pivot <= origin {
				origin = pivot - 1
			}
		} else if height <= uint64(fsMinFullBlocks) {
			origin = 0
		} else {
			pivot = height - uint64(fsMinFullBlocks)
			if pivot <= origin {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-19T15:02:13Z

+		if !pivotHashSet || pivotHash == "" {
+			Fatalf("--%s must be set if --%s is set", FastSyncPivotHashFlag.Name, FastSyncPivotNumberFlag.Name)
+		}
+		if !pivotRootSet || pivotRoot == "" {
+			Fatalf("--%s must be set if --%s is set", FastSyncPivotRootFlag.Name, FastSyncPivotNumberFlag.Name)
+		}
+		cfg.FastSyncPivotNumber = ctx.Uint64(FastSyncPivotNumberFlag.Name)
+		cfg.FastSyncPivotHash = common.HexToHash(pivotHash)
+		cfg.FastSyncPivotRoot = common.HexToHash(pivotRoot)


The pivot hash/root flags are parsed with common.HexToHash, which silently converts invalid hex strings into the zero hash (hex decoding errors are ignored in common.FromHex/Hex2Bytes). This can accidentally bypass pivot verification or state-root selection. Please validate the input as a 32-byte hex value (e.g., via hexutil.Decode/UnmarshalFixedText) and fail fast on parse/length errors; also reconcile the behavior/usage text that says the root can be zero with the current requirement that the flag must be non-empty.

Suggested change

if !pivotHashSet || pivotHash == "" {

Fatalf("--%s must be set if --%s is set", FastSyncPivotHashFlag.Name, FastSyncPivotNumberFlag.Name)

}

if !pivotRootSet || pivotRoot == "" {

Fatalf("--%s must be set if --%s is set", FastSyncPivotRootFlag.Name, FastSyncPivotNumberFlag.Name)

}

cfg.FastSyncPivotNumber = ctx.Uint64(FastSyncPivotNumberFlag.Name)

cfg.FastSyncPivotHash = common.HexToHash(pivotHash)

cfg.FastSyncPivotRoot = common.HexToHash(pivotRoot)

if !pivotHashSet {

Fatalf("--%s must be set if --%s is set", FastSyncPivotHashFlag.Name, FastSyncPivotNumberFlag.Name)

}

if !pivotRootSet {

Fatalf("--%s must be set if --%s is set", FastSyncPivotRootFlag.Name, FastSyncPivotNumberFlag.Name)

}

var parsedPivotHash common.Hash

if err = parsedPivotHash.UnmarshalText([]byte(pivotHash)); err != nil {

Fatalf("invalid --%s flag: %v", FastSyncPivotHashFlag.Name, err)

}

var parsedPivotRoot common.Hash

if err = parsedPivotRoot.UnmarshalText([]byte(pivotRoot)); err != nil {

Fatalf("invalid --%s flag: %v", FastSyncPivotRootFlag.Name, err)

}

cfg.FastSyncPivotNumber = ctx.Uint64(FastSyncPivotNumberFlag.Name)

cfg.FastSyncPivotHash = parsedPivotHash

cfg.FastSyncPivotRoot = parsedPivotRoot

Copilot · 2026-04-19T15:02:14Z

+		if !pivotHashSet || pivotHash == "" {
+			Fatalf("--%s must be set if --%s is set", FastSyncPivotHashFlag.Name, FastSyncPivotNumberFlag.Name)
+		}
+		if !pivotRootSet || pivotRoot == "" {
+			Fatalf("--%s must be set if --%s is set", FastSyncPivotRootFlag.Name, FastSyncPivotNumberFlag.Name)
+		}
+		cfg.FastSyncPivotNumber = ctx.Uint64(FastSyncPivotNumberFlag.Name)


If the user passes --fastsyncpivotnumber 0, ctx.IsSet will be true and the code will require hash/root, but cfg.FastSyncPivotNumber stays 0 (meaning "use default"), and backend.go will not call SetPivotBlock at all. Consider treating a value of 0 as "not configured" (ignore the other flags) or rejecting it with a clear error when the flag is explicitly set.

Suggested change

if !pivotHashSet || pivotHash == "" {

Fatalf("--%s must be set if --%s is set", FastSyncPivotHashFlag.Name, FastSyncPivotNumberFlag.Name)

}

if !pivotRootSet || pivotRoot == "" {

Fatalf("--%s must be set if --%s is set", FastSyncPivotRootFlag.Name, FastSyncPivotNumberFlag.Name)

}

cfg.FastSyncPivotNumber = ctx.Uint64(FastSyncPivotNumberFlag.Name)

pivotNumber := ctx.Uint64(FastSyncPivotNumberFlag.Name)

if pivotNumber == 0 {

Fatalf("--%s must be greater than 0 when explicitly set", FastSyncPivotNumberFlag.Name)

}

if !pivotHashSet || pivotHash == "" {

Fatalf("--%s must be set if --%s is set", FastSyncPivotHashFlag.Name, FastSyncPivotNumberFlag.Name)

}

if !pivotRootSet || pivotRoot == "" {

Fatalf("--%s must be set if --%s is set", FastSyncPivotRootFlag.Name, FastSyncPivotNumberFlag.Name)

}

cfg.FastSyncPivotNumber = pivotNumber

Copilot · 2026-04-19T15:02:14Z

+// Chain layout (Epoch=900, Gap=450, pivot=536, gap pivot=[450]):
+//
+//	blocks 1-535  → fast-sync (receipts)
+//	block  450    → gap pivot: state synced + snapshot generated
+//	block  536    → primary pivot: state synced + committed
+//	blocks 537-600 → full-sync
+func TestFastSyncGapPivotSync(t *testing.T) {
+	t.Parallel()
+
+	tester := newTester()
+	// XDPoS config is required so SetPivotBlock can compute gap numbers.
+	// TestXDPoSMockChainConfig has Epoch=900, Gap=450.
+	tester.configOverride = params.TestXDPoSMockChainConfig
+	defer tester.terminate()
+
+	// 600 blocks: natural pivot = 600-64 = 536, gap pivot = 450.
+	chainLen := 600


The comment describing the chain layout/natural pivot is off by one: pivotNum is computed as (chainLen-1-fsMinFullBlocks) = 535 for chainLen=600, but the comment says pivot=536 and blocks 537-600 full-sync. Please update the comment so it matches the actual pivot calculation to avoid confusion when maintaining the test.

wgr523 and others added 18 commits March 31, 2026 21:51

feat: try to make full sync work. add log in syncState

c0f15e9

fix: when error getting snapshot, skip the error

23fcba6

chore: add fast sync args into docker entry start.sh

3724c25

style: more log about pivot

e6bd9b1

feat: add FASTSYNC_PIVOT_ROOT, remove unnecessary verify

144522d

feat: re-add verify header for block after pivot

0283f08

feat(downloader): sync multiple gap pivots during fast sync

7c90905

Changes: - Add pivotGapNumbers field to Downloader struct - Calculate gap pivot numbers in SetPivotBlock() - Sync gap pivot states in processFastSyncContent() before primary pivot

feat: move gap sync state after pivot sync state

3493c0d

feat: let processHeader wiat for state sync

1d56bf3

feat: add generateSnapshot in downloader for state sync

fbd8f9f

feat: generate snapshot after state sync

ba6eb2d

skip all header full verify during fast sync

e55af5f

chore: add fastsync args to devnet start.sh

1845dab

fix: statedb function, and better root choice in fast sync

f499084

feat: when fullVerify=false, skip checkSignersOnCheckpoint

5b3204b

fix: gen_config.go

d403c2b

style: clean code comment

8f0705a

wgr523 requested review from benjamin202410, gzliudan and wanwiset25 April 3, 2026 12:41

gzliudan requested review from AnilChinchawale, anunay-xin, Copilot and liam-lai April 3, 2026 14:01

Copilot started reviewing on behalf of gzliudan April 3, 2026 14:02 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes

Comment thread eth/downloader/downloader.go

Comment thread eth/downloader/downloader.go

Comment thread eth/downloader/downloader.go

Comment thread cmd/utils/flags.go Outdated

Comment thread eth/downloader/downloader.go

Comment thread eth/downloader/downloader.go

gzliudan changed the title ~~Fix fast sync~~ fix(cicd,cmd,consensus/XDPoS,eth): fix fast sync Apr 3, 2026

wgr523 force-pushed the fixfastsync-new branch from 62ad130 to b04ac82 Compare April 6, 2026 14:51

wgr523 and others added 4 commits April 12, 2026 23:17

feat: add fastsync args in start.sh

4025516

Update cmd/utils/flags.go or better check on flags

5349447

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

fix: use xdc_sort; better flags

868149d

test: add fast sync pivot gap tests

7b85d50

wgr523 force-pushed the fixfastsync-new branch from b04ac82 to 7b85d50 Compare April 12, 2026 15:18

benjamin202410 reviewed Apr 13, 2026

View reviewed changes

Comment thread cicd/devnet/start.sh

benjamin202410 reviewed Apr 13, 2026

View reviewed changes

Comment thread eth/downloader/downloader.go Outdated

chore: require all three fastsync pivot params together in cicd start.sh

39d33bf

gzliudan requested a review from Copilot April 19, 2026 14:53

Copilot started reviewing on behalf of gzliudan April 19, 2026 14:53 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

wgr523 added 2 commits May 27, 2026 22:48

feat: in fastsync make multiple headers feasible rather than one-by-one

1bad9b2

fix: if configured pivot <= origin, start at origin. fix comment

b496cd2

wgr523 force-pushed the fixfastsync-new branch from 1ba23e0 to b496cd2 Compare May 27, 2026 14:58

Conversation

wgr523 commented Apr 3, 2026

Proposed changes

Types of changes

Impacted Components

Checklist

Uh oh!

coderabbitai Bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderabbitai Bot commented Apr 3, 2026 •

edited

Loading