fix(test-execute): register per-test hive test cases with the shared client - #3317
Merged
spencer-tb merged 2 commits intoAug 6, 2026
Conversation
Set `multi_test` on the session-scoped shared client and register it with every per-test hive test case, mirroring `consume enginex`. Hive then attaches the client and a per-test log segment to each individual test entry and marks the base test as the multi-test lifecycle owner, enabling per-test client logs in hiveview. This breaks `execute hive` against hive versions from before ethereum/hive#1402 (on master since 2026-03-17): registration fails loudly during every test's setup. This was deemed reasonable (enginex has the same requirement and no fallback) and preferable to carrying a defensive fallback path.
Since per-test hive test cases now report individual results (and hive marks the base test as the multi-test lifecycle owner, which hiveview and hive-ui exclude from test stats), ending the base test with the run's failure status double-reported every failure as an extra failing entry. End it `pass=True` as the multi-test context, matching enginex's `multi_test_hive_test`, unless the client failed to start. The previous `request.session.testsfailed` check was also unreliable under xdist: Only the last worker to finish ended the test, using its own per-worker failure count.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3317 +/- ##
===================================================
- Coverage 93.49% 91.30% -2.20%
===================================================
Files 624 624
Lines 37056 37070 +14
Branches 3394 3394
===================================================
- Hits 34647 33846 -801
- Misses 1653 2765 +1112
+ Partials 756 459 -297
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #3287, which made
execute hivereport each pytest test as an individual hive test case, but left out the client-registration step thatconsume enginexperforms. This PR:Implementation, mirroring the enginex pattern (
_per_test_reporting/resolved_client.multi_test = True):multi_test = Trueon the session-scoped shared client. It is set unconditionally on every xdist worker, so the flag also holds on workers that deserialize the client from the session temp folder rather than starting it.hive_test.register_multi_test_client(client). Hive then copies the client info into each test case with its own log byte range (logOffsets) and marks the base test as the multi-test lifecycle owner (multiTestContext), so hiveview and hive-ui show the client and a scoped client-log link on each individual test entry.pass=True("Multi-test client context completed."), matching enginex'smulti_test_hive_test, unless the client failed to start (detected via the client fixture's error file), in which case it endspass=False. This removes the double-reported failure; the previousrequest.session.testsfailedcheck was also unreliable under xdist, since only the last worker to finish ended the test, using its own per-worker failure count.Notes:
execute hiveagainst hive versions from before libhive,hiveview: Track per-test client log ranges for better reporting hive#1402 (onmastersince 2026-03-17; CI checks outmaster) — registration fails loudly during every test's setup. This was deemed reasonable: enginex has the same requirement with no fallback, and the failure clearly points at the missing/register/endpoint ("update your hive").execute hiveruns all tests concurrently against one shared client across-nworkers, so a test's log byte range contains interleaved output from tests running in parallel on other workers (unlike enginex, where a pre-alloc group's tests run serially on one worker). Client association in the UI is the main win.pytest-execute-hive.iniloads therpc.hiveplugin;rpc.remoteimports nothing from it, andfill_statefulimports only pure builder functions from this module, none of which are touched.Verified against local
./hive --dev(hive master) with go-ethereum and nethermind, including with-n 2: every per-test case in the resulting suite JSON carriesclientInfowith its ownlogOffsetsrange, the base test ismultiTestContext: trueand passes while failing tests are counted exactly once, and themulti_testflag survives the client-sharing round trip between xdist workers.Related Issues or PRs
Follow-up to #3287. Related: ethereum/hive#1402 (multi-test client registration), ethpandaops/hive-ui#76 (per-test client log sections in the UI).
Checklist
just static<type>(<area>): <title>, where<type>and<area>come from an appropriateC-<type>, respectivelyA-<area>, label. The title should match the target squash commit message.Cute Animal Picture