fix(spec-tools,testing): move evm_tools into the testing package #3307
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -69,6 +69,66 @@ jobs: | |
| EOF | ||
| uvx --from actionlint-py actionlint | ||
|
|
||
| packaging: | ||
|
Member
There was a problem hiding this comment. Thanks a lot for adding these additional checks, Guru! Can we plz also add easy local verification of these steps by adding these checks to the Every other test job here is a one-liner delegating to a just recipe; this is the only one with inline logic, including ~18 lines of Python in a heredoc that no linter or type checker ever sees. Moving the whole job into a Suggested shape: # --- Packaging ---
# Build every workspace wheel into .just/dist
[group('packaging')]
build-wheels:
uv build --wheel --all-packages --out-dir "{{ output_dir }}/dist"
# Smoke-test the built wheels: clean-venv install, real t8n run, spec-wheel-alone import check
[group('packaging')]
test-packaging: build-wheels
# 1. install both wheels by explicit path into a fresh venv
# 2. run the Frontier t8n transition, assert result.json
# 3. install the spec wheel alone, run .github/scripts/import_check.pyThe The job then reduces to checkout, setup-uv, |
||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| - uses: ./.github/actions/setup-uv | ||
| with: | ||
| python-version: "3.14" | ||
| # Build every workspace member, so a dependency on a sibling package | ||
| # resolves against the wheel built here rather than against an index. | ||
| - name: Build the workspace wheels | ||
| run: uv build --wheel --all-packages --out-dir dist | ||
| # Install into a bare venv, deliberately outside the uv workspace. | ||
| # Both wheels are passed by explicit path: resolving either through an | ||
| # index could silently substitute a published PyPI version for the | ||
| # branch's own build. | ||
| - name: Install the wheels into a clean environment | ||
| run: | | ||
| uv venv "$RUNNER_TEMP/wheel-venv" | ||
| uv pip install --python "$RUNNER_TEMP/wheel-venv/bin/python" \ | ||
| dist/ethereum_execution_testing-*.whl \ | ||
| dist/ethereum_execution-*.whl | ||
| # Run a real transition rather than `--help`, which returns inside | ||
| # argparse without ever reaching the imports that t8n needs. The output | ||
| # basedir is emptied before the run, so keep it out of the source tree. | ||
| - name: Smoke-test ethereum-spec-evm t8n | ||
| run: | | ||
| mkdir -p "$RUNNER_TEMP/t8n-out" | ||
| "$RUNNER_TEMP/wheel-venv/bin/ethereum-spec-evm" t8n \ | ||
| --state.fork=Frontier \ | ||
| --input.alloc=tests/evm_tools/t8n_build/alloc.json \ | ||
| --input.env=tests/evm_tools/t8n_build/env.json \ | ||
| --input.txs=tests/evm_tools/t8n_build/txs.json \ | ||
|
Comment on lines
+101
to
+103
Member
There was a problem hiding this comment. Now that t8n lives in the testing package, I think these new fixtures should be there too? Suggested home: These three lines are the only references repo-wide. The It would also leave (One caution: not the top-level
Member
There was a problem hiding this comment. Justification about the move/rename of |
||
| --output.basedir="$RUNNER_TEMP/t8n-out" | ||
| test -s "$RUNNER_TEMP/t8n-out/result.json" | ||
|
spencer-tb marked this conversation as resolved.
|
||
| # Install the spec wheel on its own: the spec package must never | ||
| # import the testing package, or standalone spec installs break | ||
| # again (#3236). Every module is imported, rather than a fixed | ||
| # list, so additions are covered automatically. The exceptions: | ||
| # `docc` plugins only load once the `doc` group installs docc, | ||
| # and importing a `__main__` would run it. | ||
| - name: Import-check the spec wheel alone | ||
|
Member
There was a problem hiding this comment. My friend and I tested this step against the pre-fix wheels: built the The reason: the old The step is still worth keeping. It catches future module-level leaks (now the likely form, since the cycle that motivated function-scoping is gone) and general wheel breakage. To ensure the invariant " |
||
| run: | | ||
| uv venv "$RUNNER_TEMP/spec-venv" | ||
| uv pip install --python "$RUNNER_TEMP/spec-venv/bin/python" \ | ||
| dist/ethereum_execution-*.whl | ||
| "$RUNNER_TEMP/spec-venv/bin/python" - <<'EOF' | ||
| import importlib | ||
| import pkgutil | ||
|
|
||
| import ethereum | ||
| import ethereum_spec_tools | ||
|
|
||
| SKIP = {"ethereum_spec_tools.docc"} | ||
| for pkg in (ethereum, ethereum_spec_tools): | ||
| for mod in pkgutil.walk_packages(pkg.__path__, f"{pkg.__name__}."): | ||
| if mod.name in SKIP or mod.name.endswith(".__main__"): | ||
| continue | ||
| importlib.import_module(mod.name) | ||
| EOF | ||
|
|
||
| fill: | ||
| name: fill (${{ matrix.label }}) | ||
| runs-on: [self-hosted-ghr, size-xl-x64] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -215,7 +215,6 @@ spec-tools *args: (_tmp "spec-tools") | |
| uv run pytest \ | ||
| -n {{ xdist_workers }} \ | ||
| --basetemp="{{ output_dir }}/spec-tools/tmp" \ | ||
| "$@" \ | ||
| tests/evm_tools | ||
|
Contributor
There was a problem hiding this comment. Should we move Definitely follow ups!
Contributor
Author
There was a problem hiding this comment. Yes. The folder name is a bit misleading. Happy to fix in a follow up
Member
There was a problem hiding this comment. A quick win for clarity, potentially in this PR: rename The rename touches three places: the recipe at |
||
|
|
||
|
|
@@ -227,6 +226,7 @@ test-tests *args: (_tmp "test-tests") | |
| cd packages/testing && uv run pytest \ | ||
| -n {{ xdist_workers }} \ | ||
| --basetemp="{{ output_dir }}/test-tests/tmp" \ | ||
| --ignore=src/execution_testing/evm_tools/tests/test_count_opcodes.py \ | ||
| "$@" \ | ||
| src | ||
|
|
||
|
|
@@ -237,6 +237,7 @@ test-tests-pypy *args: (_tmp "test-tests-pypy") | |
| -n auto --maxprocesses 6 \ | ||
| --basetemp="{{ output_dir }}/test-tests-pypy/tmp" \ | ||
| --ignore=src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_benchmarking.py \ | ||
| --ignore=src/execution_testing/evm_tools/tests/test_count_opcodes.py \ | ||
| "$@" \ | ||
| src | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # EVM Tools Package | ||
|
|
||
| ::: execution_testing.evm_tools |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,37 @@ | ||||||
| # The `ethereum-execution-testing` Package | ||||||
|
|
||||||
| Test generation and execution framework for the [Ethereum Execution Layer Specifications (EELS)](https://github.com/ethereum/execution-specs), derived from [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests). | ||||||
|
Member
There was a problem hiding this comment. I don't think we need a reference to the archived EEST repo here.
Suggested change
|
||||||
|
|
||||||
| The package provides: | ||||||
|
|
||||||
| - The `execution_testing` library: base types, fork definitions, and test-spec primitives used to write consensus test cases. | ||||||
| - The pytest-based commands that generate and run test fixtures against execution clients: `fill`, `execute`, `consume`, and friends. | ||||||
| - `ethereum-spec-evm` — the reference EVM CLI that executes the spec directly: a `t8n` transition tool (also available as a daemon), a `b11r` block builder, and a state-test runner. | ||||||
|
|
||||||
| ## Installing `ethereum-spec-evm` standalone | ||||||
|
|
||||||
| This package depends on `ethereum-execution` (the spec itself), and the two are developed in lockstep: the spec releases published on PyPI only carry forks that are live on mainnet and generally cannot satisfy this package's dependency pins. Install both packages from the same clone. | ||||||
|
|
||||||
| With `uv` (resolves the sibling spec package from the checkout automatically): | ||||||
|
|
||||||
| ```console | ||||||
| git clone https://github.com/ethereum/execution-specs | ||||||
| uv tool install ./execution-specs/packages/testing | ||||||
| ``` | ||||||
|
|
||||||
| With `pip`, in a virtual environment: | ||||||
|
|
||||||
| ```console | ||||||
| pip install ./execution-specs ./execution-specs/packages/testing | ||||||
| ``` | ||||||
|
|
||||||
| With `pipx`: | ||||||
|
|
||||||
| ```console | ||||||
| pipx install ./execution-specs | ||||||
| pipx inject --include-apps ethereum-execution ./execution-specs/packages/testing | ||||||
| ``` | ||||||
|
|
||||||
| ## Documentation | ||||||
|
|
||||||
| Repository documentation, including this framework's reference documentation: <https://steel.ethereum.foundation/docs/execution-specs/> | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| """ | ||
| EVM Trace Implementations. | ||
|
|
||
| See the spec's `ethereum.trace` module for the trace event definitions. | ||
| """ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All the sibling jobs gate on
needs: static; this is the only one that does not. I think it's worth gating here, too.