Skip to content

fix(test-execute): register per-test hive test cases with the shared client - #3317

Merged
spencer-tb merged 2 commits into
ethereum:forks/amsterdamfrom
danceratopz:execute-hive-per-test-client-registration
Aug 6, 2026
Merged

fix(test-execute): register per-test hive test cases with the shared client#3317
spencer-tb merged 2 commits into
ethereum:forks/amsterdamfrom
danceratopz:execute-hive-per-test-client-registration

Conversation

@danceratopz

Copy link
Copy Markdown
Member

Description

Follow-up to #3287, which made execute hive report each pytest test as an individual hive test case, but left out the client-registration step that consume enginex performs. This PR:

  • Adds client association: each per-test entry now shows the shared client it ran against (previously no client was attached).
  • Adds per-test client log segments: each entry links to only its own byte range of the shared client's log (previously no client log link at all).
  • Fixes double-reported failures: the "Base Hive Test" entry no longer duplicates every failure as an extra failing entry at the end of the run.

Implementation, mirroring the enginex pattern (_per_test_reporting / resolved_client.multi_test = True):

  • Set multi_test = True on the session-scoped shared client. It is set unconditionally on every xdist worker, so the flag also holds on workers that deserialize the client from the session temp folder rather than starting it.
  • Register the client with every per-test hive test case via hive_test.register_multi_test_client(client). Hive then copies the client info into each test case with its own log byte range (logOffsets) and marks the base test as the multi-test lifecycle owner (multiTestContext), so hiveview and hive-ui show the client and a scoped client-log link on each individual test entry.
  • End the base test pass=True ("Multi-test client context completed."), matching enginex's multi_test_hive_test, unless the client failed to start (detected via the client fixture's error file), in which case it ends pass=False. This removes the double-reported failure; the previous request.session.testsfailed check was also unreliable under xdist, since only the last worker to finish ended the test, using its own per-worker failure count.

Notes:

  • Hive version: this breaks execute hive against hive versions from before libhive,hiveview: Track per-test client log ranges for better reporting hive#1402 (on master since 2026-03-17; CI checks out master) — registration fails loudly during every test's setup. This was deemed reasonable: enginex has the same requirement with no fallback, and the failure clearly points at the missing /register/ endpoint ("update your hive").
  • Log segments are approximate under xdist: execute hive runs all tests concurrently against one shared client across -n workers, so a test's log byte range contains interleaved output from tests running in parallel on other workers (unlike enginex, where a pre-alloc group's tests run serially on one worker). Client association in the UI is the main win.
  • Non-hive execute modes are unaffected: only pytest-execute-hive.ini loads the rpc.hive plugin; rpc.remote imports nothing from it, and fill_stateful imports only pure builder functions from this module, none of which are touched.

Verified against local ./hive --dev (hive master) with go-ethereum and nethermind, including with -n 2: every per-test case in the resulting suite JSON carries clientInfo with its own logOffsets range, the base test is multiTestContext: true and passes while failing tests are counted exactly once, and the multi_test flag survives the client-sharing round trip between xdist workers.

Related Issues or PRs

Follow-up to #3287. Related: ethereum/hive#1402 (multi-test client registration), ethpandaops/hive-ui#76 (per-test client log sections in the UI).

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <title>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

A cat guarding its log segments

Set `multi_test` on the session-scoped shared client and register it
with every per-test hive test case, mirroring `consume enginex`. Hive
then attaches the client and a per-test log segment to each individual
test entry and marks the base test as the multi-test lifecycle owner,
enabling per-test client logs in hiveview.

This breaks `execute hive` against hive versions from before
ethereum/hive#1402 (on master since 2026-03-17): registration fails
loudly during every test's setup. This was deemed reasonable (enginex
has the same requirement and no fallback) and preferable to carrying a
defensive fallback path.
Since per-test hive test cases now report individual results (and hive
marks the base test as the multi-test lifecycle owner, which hiveview
and hive-ui exclude from test stats), ending the base test with the
run's failure status double-reported every failure as an extra failing
entry. End it `pass=True` as the multi-test context, matching
enginex's `multi_test_hive_test`, unless the client failed to start.

The previous `request.session.testsfailed` check was also unreliable
under xdist: Only the last worker to finish ended the test, using its
own per-worker failure count.
@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.30%. Comparing base (ee7a677) to head (e960b76).
⚠️ Report is 4 commits behind head on forks/amsterdam.

Additional details and impacted files
@@                 Coverage Diff                 @@
##           forks/amsterdam    #3317      +/-   ##
===================================================
- Coverage            93.49%   91.30%   -2.20%     
===================================================
  Files                  624      624              
  Lines                37056    37070      +14     
  Branches              3394     3394              
===================================================
- Hits                 34647    33846     -801     
- Misses                1653     2765    +1112     
+ Partials               756      459     -297     
Flag Coverage Δ
unittests 91.30% <ø> (-2.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@spencer-tb spencer-tb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@spencer-tb
spencer-tb merged commit b802df1 into ethereum:forks/amsterdam Aug 6, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants