refactor(test-benchmark): split stateful setup transactions across blocks - #3282
refactor(test-benchmark): split stateful setup transactions across blocks#3282LouisTsai-Csie wants to merge 5 commits into
Conversation
LouisTsai-Csie
added
A-test-benchmark
LouisTsai-Csie
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3282 +/- ##
================================================
Coverage 93.49% 93.49%
================================================
Files 625 625
Lines 37032 37039 +7
Branches 3385 3392 +7
================================================
+ Hits 34623 34630 +7
Misses 1653 1653
Partials 756 756
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Please carefully review this refactor, or it would slow down the entire benchmark process a lot! |
spencer-tb
left a comment
spencer-tb
left a comment
There was a problem hiding this comment.
Couple of comments, I think they should make sense :)
c68e82b to
3af71ea
Compare
| # between query timing and tx submission (basefee can climb a few blocks | ||
| # between the two; the bump keeps txs landing without per-tx requeries). | ||
| FEE_BUMP_MULTIPLIER = 1.5 | ||
| FEE_BUMP_MULTIPLIER = 3.0 |
There was a problem hiding this comment.
The max_fee_per_gas and other gas configuration values adjust based on network status. However, the FEE_MULTIPLIER is only 1.5x. For benchmarks with many blocks, the cumulative gas limit can exceed the value calculated at the start of the test, leading to lower base fee issues.
| responses: List[JSONRPCResponse] = [] | ||
| batch_size = self.max_transactions_per_batch | ||
| for i in range(0, len(all_calls), batch_size): | ||
| responses.extend( | ||
| self.post_batch_request(calls=all_calls[i : i + batch_size]) | ||
| ) |
There was a problem hiding this comment.
Instead of sending RPC requests all at once, this PR batches them to reduce server load.
| Transaction( | ||
| to=op_address, | ||
| data=calldata, | ||
| gas_limit=gas_limit, | ||
| sender=pre.fund_eoa(), | ||
| sender=execution_sender, | ||
| ) |
There was a problem hiding this comment.
It is better than the original version:
execution_txs.append(
Transaction(
to=op_address,
data=calldata,
gas_limit=gas_limit,
sender=pre.fund_eoa(), # This requires pre-funding for EVERY account
)
)Pre-funding a sender per transaction is unnecessary and slows setup. Empty account creation is now much more expensive after repricing.
| state_cost = fork.transaction_top_frame_state_gas( | ||
| sends_value=sends_value, | ||
| recipient_type=recipient_type, | ||
| ) | ||
| iteration_cost = regular_cost + state_cost | ||
| iteration_count = gas_benchmark_value // max(regular_cost, state_cost) |
There was a problem hiding this comment.
This is the actual worst case for ether transfer, the iteration cost is sum(R, S), while the gas usage in the header is max(R, S), not sum(R, S).
The original approach used ~90% of the block gas limit, but now it reaches 99%.
2 participants
Description
Problem 1: Gas Limit Mismatch Between EELS and synthetic state
EELS configures the block gas limit to 1 Trillion, while pre-state generated via state-actor uses 1 Gigagas. This mismatch prevents early detection of any exceeding block gas limit issues.
Example: In
test_auth_transactionwithempty_authority_Falseparametrization, the setup phase requires many authorization transactions. The total gas exceeds 1 Gigagas but stays under 1 Trillion. EELS construction succeeds, but the snapshot rejects it by default.Root Cause
Currently,
blockchain.pypacks all setup transactions (fund_eoa,deploy_contract, etc.) into a single block. After state repricing, this easily exceeds 1 Gigagas (some test would create hundreds of thousands of slots, accounts, delegation). Since setup gas isn't verified, a setup phase can exceed the benchmark's own gas limit and still pass infillmode. However, when filling against a live client withfill-stateful, the client's block gas limit rejects the block.Solution
This PR splits setup transactions across multiple blocks, ensuring each strictly respects the block gas limit. This could fix the following broken cases:
test_ether_transfers_to_precompiletest_ether_transferstest_auth_transactiontest_storage_access_coldtest_ext_account_query_coldtest_mixed_dependency_graphtest_state_root_computation(initcode prefix too long)test_deploy_then_interactProblem 2: RPC request exceeds size limit
Certain RPC calls have request body size limits. The
test_selfdestructing_existingbenchmark exceeds this limit because it callsget_allocwith a large batch of data at once. This PR splits the requests into smaller batches to stay within the limit.Additional Fixes
test_block_full_datafunds recipients with an explicit amount (no deferred balances)Related Issues or PRs
issue #3281
Checklist
just static<type>(<area>): <title>, where<type>and<area>come from an appropriateC-<type>, respectivelyA-<area>, label. The title should match the target squash commit message.Cute Animal Picture