Repository navigation
ci(workbench): reland parallel smoke with one user per worker - #842
Conversation
xdist dispatches whole groups, queued by each group's first test, so sign-out sharing workbench_auth with the early login scenario ran at the start of a shared-auth run and ended the session other workers were using. Give it a group of its own so its order(95) queues it last.
There was a problem hiding this comment.
🟡 Changes recommended
The sign-out group can run before other workers finish, invalidating their shared session mid-test; it needs a separate serial phase and a multi-worker execution test.
1 open issue
What changed in this PR
Restores parallel Workbench smoke testing in VIP, relanding #840 after its revert and adding a dedicated sign-out group.
Changes:
- Creates one Workbench account per worker, isolates Git checkout directories, and increases the login retry delay.
- Ensures xdist sees VIP’s groups and assigns sign-out its own group.
- Pins pytest for xdist compatibility and adds regression coverage for worker isolation and grouping.
| File | Description |
|---|---|
uv.lock |
Locks pytest to 9.1.1. |
src/vip/plugin/selection.py |
Assigns groups before xdist reads them; leaves non-product tests ungrouped. |
src/vip_tests/workbench/test_git_ops.py |
Uses worker-specific Git checkout directories. |
src/vip_tests/workbench/naming.py |
Adds opt-in per-worker usernames. |
src/vip_tests/workbench/login.py |
Extends login retries beyond the default throttle window. |
src/vip_tests/workbench/conftest.py |
Enables per-worker grouping and usernames; separates sign-out’s group. |
src/vip_tests/workbench/cleanup.py |
Targets the worker’s account during token-based cleanup. |
selftests/test_workbench_parallel.py |
Covers usernames, retry timing, and applied xdist groups. |
selftests/test_workbench_ordering.py |
Checks that sign-out is collected in its own final group. |
selftests/test_workbench_git_ops.py |
Tests worker-specific checkout paths and directory creation. |
selftests/test_workbench_cleanup.py |
Checks cleanup uses the worker-specific account. |
pyproject.toml |
Pins pytest to the xdist-compatible version. |
AGENTS.md |
Documents per-worker accounts and grouping requirements. |
.github/workflows/workbench-smoke.yml |
Provisions worker accounts and enables parallel smoke execution. |
.github/dependabot.yml |
Holds pytest updates below 9.2 pending xdist compatibility. |
Review effort: Balanced
Workbench sign-out revokes every cookie the account holds, and xdist runs the last group while other workers finish theirs. Each worker now drops a marker at session end; under shared auth the sign-out step waits for them (unproven on timeout). A real -n 2 selftest fails without the wait.
There was a problem hiding this comment.
🟡 Changes recommended
On shared hosts that permit following other users’ /tmp symlinks, the predictable temporary path enables an attack that can make rm -rf delete the test account’s existing checkout outside VIP.
1 open issue
1 resolved since last review
Review effort: Balanced
The per-worker /tmp/vip_git_<worker> path is predictable, so another user could pre-create it as a symlink and aim the clone's rm -rf at the test account's files. The command now stops unless the directory is a real one the test user owns, and makes it 0700 first.
There was a problem hiding this comment.
🔵 Needs a closer look
Shared-account sign-out depends on xdist teardown timing, but the separate-user smoke run and synthetic barrier selftest do not validate a full parallel --headless-auth run against real Workbench, including cleanup and cache restoration.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
|

Relands #840 (reverted in #841 after an early merge), plus the fix for its last open review thread. Closes #644.
tryfirst), and pytest held at 9.1.1 until pytest-xdist supports 9.2 (Missing new release tag blocking upgrade to pytest 9.2.0 breaking loadgroup working pytest-dev/pytest-xdist#1407).--interactive-auth/--headless-authit also waits for every other worker to finish first, because Workbench sign-out revokes every cookie the account holds. A real-n 2selftest checks the wait.