flatkv migration testings test

[blindchaser]

Describe your changes and provide context

Testing performed to validate your change

[cursor]

PR Summary

High Risk
High risk because it refactors core state-commit configuration and the CompositeCommitStore routing/commit paths while adding new migration modes that affect AppHash/consensus behavior; bugs could cause chain halts or state divergence.

Overview
Adds a new FlatKV 0→1 (MigrateEVM) migration workflow and test coverage. CI now includes a dedicated FlatKV EVM Migration 0->1 matrix job that boots a cluster in memiavl_only (GIGA_MIGRATE_FROM_MEMIAVL=true), performs a coordinated stop/flip to migrate_evm, polls seidb migrate-evm-status, and verifies cross-validator FlatKV digest agreement; the older offline import script is removed.

Refactors state-commit configuration around write modes and migration batching. Introduces state-commit.sc-keys-to-migrate-per-block (default 1024) and plumbs it through app/server config parsing; removes sc-read-mode and sc-enable-lattice-hash configuration and updates templates/tests accordingly.

Reworks CompositeCommitStore to use a migration Router and support migration lifecycle modes. NewCompositeCommitStore and Initialize now return errors, router construction happens on load (incl. migration-store seeding and FlatKV version seeding/reconciliation), reads/proofs/iterators route through the router, and commit info conditionally appends the FlatKV lattice hash; adds extensive new composite/rootmulti migration tests and updates existing FlatKV tests to target EVMMigrated/test_only_dual_write paths.

Updates devnet/docker tooling and diagnostics. Docker/Makefile forward GIGA_MIGRATE_FROM_MEMIAVL, localnode overrides support v0 boot, RPC node config aligns with test_only_dual_write, FlatKV verification scripts add better failure dumps, and upgrade log matching is made more tolerant.

^{Reviewed by Cursor Bugbot for commit 2364ea4. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	May 19, 2026, 5:09 PM

[cursor]

GetLatestVersion errors during first migration mode transition

High Severity

GetLatestVersion now strictly requires both backends to agree when both are present, but the version-seeding logic that brings a fresh flatkv into lockstep only runs inside LoadVersion. When first transitioning from MemiavlOnly to MigrateEVM (or TestOnlyDualWrite), memiavl reports version N while a newly-created flatkv reports version 0, causing GetLatestVersion to return an error. In rootmulti.NewStore (line 122), this error is silently discarded (scVersion, _ :=), making scVersion zero and bypassing the SS safety check. In rootmulti.LastCommitID (line 225), the same error triggers a panic if lastCommitInfo is nil.

Additional Locations (1)

• sei-cosmos/storev2/rootmulti/store.go#L120-L122

 

^{Reviewed by Cursor Bugbot for commit 75ffaf1. Configure here.}

[codecov]

Codecov Report

❌ Patch coverage is 59.18762% with 211 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.36%. Comparing base (adf03c1) to head (2364ea4).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
sei-db/state_db/sc/composite/store.go	70.60%	50 Missing and 37 partials ⚠️
...b/tools/cmd/seidb/operations/migrate_evm_status.go	0.00%	55 Missing ⚠️
scripts/evm_stress/main.go	0.00%	28 Missing ⚠️
sei-db/state_db/sc/flatkv/store_meta.go	61.90%	14 Missing and 10 partials ⚠️
sei-db/state_db/sc/migration/migration_manager.go	81.25%	3 Missing and 3 partials ⚠️
sei-cosmos/storev2/rootmulti/store.go	20.00%	2 Missing and 2 partials ⚠️
app/seidb.go	0.00%	2 Missing and 1 partial ⚠️
sei-db/config/sc_config.go	50.00%	1 Missing and 1 partial ⚠️
sei-db/tools/cmd/seidb/main.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3462      +/-   ##
==========================================
+ Coverage   59.24%   59.36%   +0.11%     
==========================================
  Files        2126     2119       -7     
  Lines      175715   175133     -582     
==========================================
- Hits       104097   103961     -136     
+ Misses      62538    62110     -428     
+ Partials     9080     9062      -18     

Flag	Coverage Δ
sei-chain-pr	64.85% <59.18%> (?)
sei-db	70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-cosmos/server/config/config.go	94.53% <100.00%> (+0.12%)	⬆️
sei-db/config/write_mode.go	100.00% <100.00%> (ø)
sei-db/state_db/sc/memiavl/store.go	92.59% <100.00%> (-0.06%)	⬇️
sei-db/state_db/sc/migration/router_builder.go	56.72% <100.00%> (+4.40%)	⬆️
sei-db/config/sc_config.go	87.50% <50.00%> (-12.50%)	⬇️
sei-db/tools/cmd/seidb/main.go	0.00% <0.00%> (ø)
app/seidb.go	73.61% <0.00%> (+1.61%)	⬆️
sei-cosmos/storev2/rootmulti/store.go	64.30% <20.00%> (-0.50%)	⬇️
sei-db/state_db/sc/migration/migration_manager.go	92.78% <81.25%> (-2.70%)	⬇️
sei-db/state_db/sc/flatkv/store_meta.go	67.71% <61.90%> (-5.73%)	⬇️
... and 3 more

... and 146 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75ffaf1428

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Wire the migration batch size into app config

Adding this field does not make it configurable at runtime: the generated app.toml template has no matching key and app/seidb.go only reads the existing FlagSC* options, while the new migration script writes keys-to-migrate-per-block expecting it to take effect. As a result, migrate_* runs configured from app.toml keep using the hard-coded default (and server/config.GetConfig leaves the field at zero in its literal), so operators/tests cannot throttle batches and the 0->1 test silently skips the intended small-batch resume coverage. Please add the TOML key/flag and parse it into StateCommitConfig.

Useful? React with 👍 / 👎.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ad6ce4a. Configure here.}

[cursor]

Migration script batch size not expanded

Medium Severity

When flipping validators to migrate_evm, the docker exec bash -c block puts the sed replacement for sc-keys-to-migrate-per-block inside single quotes on the host. Host-side single quotes prevent expanding KEYS_TO_MIGRATE_PER_BLOCK, so the container likely writes the literal string $KEYS_TO_MIGRATE_PER_BLOCK instead of 256, defeating the intended per-block migration batch for this test.

 

^{Reviewed by Cursor Bugbot for commit ad6ce4a. Configure here.}