fix(tests): enhance & un-skip Amsterdam ported static tests (Pt. 2)

.claude/commands/implement-eip.md

@@ -34,11 +34,11 @@ Each fork lives at `src/ethereum/forks/<fork_name>/`. Explore the latest fork di
 
 ## Gas Handling
 
-Recent forks meter two gas dimensions: regular gas and state gas (for durable state growth). Key rules:
+Recent forks meter two gas dimensions: execution gas and state gas (for durable state growth). Key rules:
 
 1. Gas constants and calculations go in `vm/gas.py`; a frame's mutable gas state lives on `Evm.gas_meter`.
 2. Extend the named helper vocabulary (`charge_*`, `credit_*`, `restore_*`, `withhold_*`, ...) instead of doing gas arithmetic by hand at call sites; encode each helper's invariant as an assert.
-3. State gas is charged by the frame whose opcode causes the creation, before the child's regular-gas share is withheld; the whole reservoir passes to the child.
+3. State gas is charged by the frame whose opcode causes the creation, before the child's execution-gas share is withheld; the whole reservoir passes to the child.
 4. A failing frame settles its own meter before returning, so parents incorporate children unconditionally.
 5. Opcodes that touch state use labeled stages, with all charging before the operation: `GAS (STATE-INDEPENDENT)` → `STATE ACCESS (STATE-DEPENDENT GAS)` → `STATE GAS` → `CHILD GRANT` → `OPERATION`. Simple opcodes keep the bare `GAS` marker. `generic_call`/`generic_create` contain no pricing; they run the child lifecycle: `PREFLIGHT` → `DESTINATION ACCESS` → `CHILD GRANT` → `DISPATCH` → `OUTCOME`.
 6. Avoid "frame" in gas identifiers (a future EIP claims the term); when a name diverges from the spec's variable name, cross-reference the spec name in the docstring.

.claude/commands/write-test.md

@@ -43,8 +43,20 @@ Conventions and patterns for writing consensus tests. Run this skill before writ
 ## Fork-Aware Logic
 
 - `fork >= Cancun` for conditional behavior based on fork
-- `fork.gas_costs()` returns `GasCosts` dataclass with constants like `G_WARM_SLOAD`, `G_COLD_ACCOUNT_ACCESS`, `G_BASE`, etc.
-- `fork.transaction_intrinsic_cost_calculator()` for computing tx intrinsic gas
+- `fork.fork_at(timestamp=...)` gives the fork active before/after a transition boundary
+- For gas amounts, see **Gas Cost Expectations** below — prefer framework cost constructs over reading `fork.gas_costs()` constants directly
+
+## Gas Cost Expectations
+
+Never hand-reconstruct a gas amount by summing `fork.gas_costs()` constants (`NEW_ACCOUNT`, `CALL_VALUE`, `COLD_STORAGE_WRITE`, `VERY_LOW`, ...). Re-deriving the schedule duplicates the framework's own calculation and silently breaks when a future fork reprices. Instead:
+
+- **Read the cost off the bytecode under test.** Set the relevant opcode metadata (`account_new`, `value_transfer`, `address_warm`, `key_warm`/`original_value`/`current_value`/`new_value`, `init_code_size`, `code_deposit_size`, `new_memory_size`, ...) and use `bytecode.gas_cost(fork)` (execution + state), `.execution_cost(fork)`, `.state_cost(fork)`, or `.refund(fork)`. Link the exact opcode to the behavior — e.g. `Op.SELFDESTRUCT(account_new=True).state_cost(fork)`.
+- **Transaction-level costs:** `fork.transaction_intrinsic_cost_calculator()`; `fork.transaction_top_frame_state_gas(contract_creation=True)` for the created account's `NEW_ACCOUNT` (under EIP-2780 it is NOT part of the intrinsic — never subtract it from the intrinsic); `fork.transaction_data_floor_cost_calculator()`; `fork.call_value_stipend()`.
+- **A single bare opcode/schedule cost** (e.g. an account-access constant) comes from a metadata-only opcode: `Op.BALANCE.with_metadata(address_warm=False).gas_cost(fork)`.
+- **Fork-transition / cross-fork comparisons:** evaluate the same bytecode or intrinsic at each fork (`before = fork.fork_at(timestamp=...)`, `after = ...`) and compare `before` vs `after` costs — do not compare raw schedule constants.
+- **Do not add "self-check" asserts** that compare a framework-computed value against a `fork.gas_costs()` decomposition of the same fork; they add no coverage over the runtime behavior the test already exercises and only break on repricing.
+- **If the framework cannot express a cost, fix the framework** (wire the opcode into its gas/state map, add an accessor) rather than reconstructing it in the test. If the use case does not support the framework, the framework needs an update.
+- **Exception:** a test whose *subject* is a specific schedule value (e.g. a regression that an opcode's cost is unchanged) may compare a runtime measurement (`CodeGasMeasure`) against `fork.gas_costs().OPCODE_*`. Even then, never hardcode the literal value.
 
 ## Transactions
 

.github/workflows/benchmark.yaml

@@ -3,54 +3,28 @@ name: Benchmarking
 on:
   push:
     branches:
-      - mainnet
-      - "forks/**"
-    paths:
-      - "tests/benchmark/**"
-      - "packages/testing/src/execution_testing/benchmark/**"
-      - "packages/testing/src/execution_testing/cli/pytest_commands/plugins/**"
-      - ".github/workflows/benchmark.yaml"
-  pull_request:
-    paths-ignore:
+      - "benchmarks/**"
+    paths-ignore: &non_benchmark_paths
       - "**.md"
       - "LICENSE*"
       - ".gitignore"
       - ".vscode/**"
       - "whitelist.txt"
-      - "docs/**"                                                                                                                     
+      - "docs/**"
       - "mkdocs.yml"
+  pull_request:
+    branches:
+      - "benchmarks/**"
+    paths-ignore: *non_benchmark_paths
   workflow_dispatch:
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
   cancel-in-progress: ${{ github.ref_name != github.event.repository.default_branch }}
 
 jobs:
-  unit-tests:
-    name: Benchmark Unit Tests
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
-        with:
-          submodules: true
-
-      - uses: ./.github/actions/setup-uv
-        with:
-          enable-cache: "false"
-
-      - uses: ./.github/actions/build-evm-base
-        id: evm-builder
-        with:
-          type: benchmark
-
-      - name: Run benchmark unit tests
-        run: just test-tests-bench
-        env:
-          EVM_BIN: ${{ steps.evm-builder.outputs.evm-bin }}
-
   sanity-checks:
     name: ${{ matrix.name }}
-    needs: [unit-tests]
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false

.github/workflows/test.yaml

@@ -6,6 +6,7 @@ on:
       - master
       - mainnet
       - "forks/**"
+      - "benchmarks/**"
     paths-ignore:
       - "**.md"
       - "LICENSE*"
@@ -163,10 +164,20 @@ jobs:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
       - uses: ./.github/actions/setup-uv
       - uses: ./.github/actions/build-evmone
+      # On benchmarks/** branches, build the pinned geth so the benchmark
+      # plugin tests fill against a real client (via EVM_BIN). On every other
+      # branch this step is skipped, EVM_BIN stays empty, and those fills fall
+      # through to fill's in-repo EELS t8n.
+      - uses: ./.github/actions/build-evm-base
+        id: evm-builder
+        if: ${{ startsWith(github.ref, 'refs/heads/benchmarks/') || startsWith(github.base_ref, 'benchmarks/') }}
+        with:
+          type: benchmark
       - name: Run test-tests
         run: just test-tests
         env:
           PYTEST_XDIST_AUTO_NUM_WORKERS: auto
+          EVM_BIN: ${{ steps.evm-builder.outputs.evm-bin }}
 
   test-tests-pypy:
     runs-on: [self-hosted-ghr, size-l-x64]

CLAUDE.md

@@ -42,6 +42,7 @@ When reviewing PRs that implement or test EIPs:
 ## When to Use Skills
 
 - Writing or modifying tests → run `/write-test` first
+- Cleaning up or future-proofing a `tests/ported_static/` test → run `/enhance-ported-test` first
 - Writing or modifying pytester-based plugin tests → run `/pytester` first
 - Filling test fixtures → run `/fill-tests` first
 - Implementing an EIP or modifying fork code in `src/` → run `/implement-eip` first
@@ -55,6 +56,7 @@ When reviewing PRs that implement or test EIPs:
 ## Available Skills
 
 - `/write-test` — test writing patterns, fixtures, markers, bytecode helpers
+- `/enhance-ported-test` — ordered methodology to clean up & future-proof `tests/ported_static/` tests
 - `/pytester` — pytester execution modes, isolation, output handling for plugin tests
 - `/fill-tests` — `fill` CLI reference, flags, debugging, benchmark tests
 - `/implement-eip` — fork structure, import rules, adding opcodes/precompiles/tx types

Justfile

@@ -220,7 +220,6 @@ test-tests *args:
     cd packages/testing && uv run pytest \
         -n {{ xdist_workers }} \
         --basetemp="{{ output_dir }}/test-tests/tmp" \
-        --ignore=src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_benchmarking.py \
         "$@" \
         src
 
@@ -235,16 +234,6 @@ test-tests-pypy *args:
         "$@" \
         src
 
-# Run benchmark framework unit tests (with Python)
-[group('unit tests')]
-[group('benchmark tests')]
-test-tests-bench *args:
-    @mkdir -p "{{ output_dir }}/test-tests-bench/tmp"
-    uv run pytest \
-        --basetemp="{{ output_dir }}/test-tests-bench/tmp" \
-        "$@" \
-        packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_benchmarking.py
-
 # Run CI release script integration tests
 [group('unit tests')]
 test-ci-scripts *args:

docs/getting_started/verifying_changes.md

@@ -13,7 +13,7 @@ Some CI jobs are slow. Only run the checks relevant to your change.
 | Any PR (baseline)                                 | `just static`                                                          | Lint, format, mypy, spellcheck, import isolation, workflow lint.                                       |
 | Added or modified tests                           | `just fill tests/path/to/new/tests`                                    | See [Filling Tests](../filling_tests/index.md).                                                        |
 | Framework changes (`packages/testing/`)           | `just test-tests`                                                      | Framework unit tests. Mirrors the `test-tests` CI job.                                                 |
-| Benchmark framework changes                       | `just test-tests-bench`, `just bench-gas`, `just bench-opcode`, `just bench-opcode-config` | Benchmark unit tests and sanity checks. Mirrors the benchmark CI workflow.                             |
+| Benchmark framework changes                       | `just test-tests`, `just bench-gas`, `just bench-opcode`, `just bench-opcode-config` | Benchmark plugin unit tests now run within `test-tests`; the `bench-*` recipes fill/verify the suite (geth-backed on `benchmarks/**`).                  |
 | Markdown touched                                  | `just lint-md`                                                         | Requires `markdownlint-cli2`; see [Linting Markdown](#linting-markdown).                               |
 | Docs touched                                      | `just docs` or `just docs-fast`                                        | `docs-fast` skips the Test Case Reference section for faster iteration.                                |
 

docs/writing_tests/fork_methods.md

@@ -38,11 +38,13 @@ def test_some_feature(fork):
 
 ```python
 def test_transaction_gas(fork, state_test):
-    gas_cost = fork.gas_costs().GAS_TX_BASE
+    # Derive the fork's intrinsic gas from the calculator rather than
+    # summing raw `gas_costs()` constants (see the Gas Parameters warning).
+    intrinsic_gas = fork.transaction_intrinsic_cost_calculator()()
 
     # Create a transaction with the correct gas parameters for this fork
     tx = Transaction(
-        gas_limit=gas_cost + 10000,
+        gas_limit=intrinsic_gas + 10000,
         # ...
     )
 
@@ -114,6 +116,9 @@ fork.memory_expansion_gas_calculator()  # Returns a callable
 fork.transaction_intrinsic_cost_calculator()  # Returns a callable
 ```
 
+!!! warning "Do not reconstruct expected gas from `gas_costs()` constants"
+    `fork.gas_costs()` exposes the raw schedule for framework internals. When a test needs an *expected* gas amount, derive it from a cost construct that tracks the live schedule (`bytecode.gas_cost(fork)` / `.execution_cost(fork)` / `.state_cost(fork)` / `.refund(fork)`, opcode metadata, the intrinsic/top-frame/data-floor calculators, `fork.call_value_stipend()`) rather than hand-summing constants — hand-built expectations silently break when a fork reprices. See [Opcode Metadata and Gas Calculations](opcode_metadata.md#do-not-hand-reconstruct-gas-from-constants).
+
 ### Transaction Types
 
 Methods for determining valid transaction types:

docs/writing_tests/opcode_metadata.md

@@ -9,6 +9,21 @@ The execution testing package provides capabilities to calculate gas costs and r
 - Validating gas cost calculations for specific opcode scenarios
 - Future-proofing tests against breaking in upcoming forks that change gas rules
 
+## Do Not Hand-Reconstruct Gas From Constants
+
+Never build an expected gas amount by summing `fork.gas_costs()` constants (`NEW_ACCOUNT`, `CALL_VALUE`, `COLD_STORAGE_WRITE`, `VERY_LOW`, ...). Re-deriving the schedule by hand duplicates the framework's own calculation and silently breaks when a future fork reprices or restructures a cost. Always derive the expectation from a framework construct that tracks the live schedule:
+
+- **The bytecode/opcode under test:** set the relevant metadata (see below) and read `bytecode.gas_cost(fork)` (execution + state), `.execution_cost(fork)`, `.state_cost(fork)`, or `.refund(fork)`. Link the exact opcode to the behavior, e.g. `Op.SELFDESTRUCT(account_new=True).state_cost(fork)`.
+- **A single bare opcode/schedule cost** comes from a metadata-only opcode: `Op.BALANCE.with_metadata(address_warm=False).gas_cost(fork)` yields the cold account-access cost with no operand pushes.
+- **Transaction-level costs:** `fork.transaction_intrinsic_cost_calculator()`, `fork.transaction_top_frame_state_gas(contract_creation=True)` (the created account's new-account state gas — on recent forks it is charged at the top frame, *not* in the intrinsic, so never subtract it from the intrinsic), `fork.transaction_data_floor_cost_calculator()`, and `fork.call_value_stipend()`.
+- **Cross-fork / fork-transition comparisons:** evaluate the *same* bytecode or intrinsic at each fork (`before = fork.fork_at(timestamp=...)`, `after = ...`) and compare the resulting costs — do not compare raw schedule constants.
+
+Additional rules:
+
+- **Do not add "self-check" assertions** that compare a framework-computed value against a `fork.gas_costs()` decomposition of the same fork. They add no coverage over the runtime behavior the test already exercises and only break on repricing.
+- **If the framework cannot express a cost, extend the framework** (wire the opcode into its gas/state map, add an accessor) rather than working around it in the test.
+- **Exception:** a test whose *subject* is a specific schedule value — for example a regression asserting that an opcode's cost is unchanged across a fork — may compare a runtime measurement (`CodeGasMeasure`) against the fork's declared `fork.gas_costs().OPCODE_*` value. Even then, never hardcode the literal number.
+
 ## Opcode Metadata
 
 Many opcodes accept metadata parameters that affect their gas cost calculations. Metadata represents runtime state information that influences gas consumption.

packages/testing/pyproject.toml

@@ -54,6 +54,7 @@ dependencies = [
     "tenacity>=9.0.0,<10",
     "Jinja2>=3,<4",
     "ijson>=3.3,<4",
+    "eth-remerkleable==0.1.31",
 ]
 
 [project.urls]