test: stop every suite reading and writing the machine it runs on

[blarghmatey]

What are the relevant tickets?

Sweeps the defect class behind #282 and #285, and subsumes tk-two-agent-config-kit-cli-tests-assert-a-terminal-2bec4f.

Stacked on #285 (base is witan-code-identity-tests) — that PR fixed this class in one package; this generalises it. Review #285 first, or read this one's diff alone, which is self-contained.

Description (What does it do?)

Three separate discoveries of the same defect in four days — a test reading or writing the machine it runs on — is a pattern, not a coincidence. It survives because a GitHub runner is the empty case of every ambient input there is: no witan login, no agent configs, no ~/.gitconfig, and a terminal wide enough that nothing wraps. A test asserting any of those absences passes in CI and only in CI.

Rather than fix it a sixth time when it next trips someone, I measured it. A hostile-environment run — HOME seeded to look like a working developer's machine, narrow terminal, witan identity present — found considerably more than the two known cases:

package	finding
witan	19 of 49 test files create a real graph in ~/.local/share/witan
witan-code	test_ingest.py creates a real code store in the same tree
agent-config-kit	8 tests assert an unwrapped Rich message and fail on a narrow terminal
witan-core, agent-kit	clean

The witan and witan-code leaks are silent — every test passes. They have been running for a long time, and the evidence is sitting in my real store directory right now:

~/.local/share/witan/code/https_github.com_test_cg.omni

That is test_branches.py's fixture repo (REPO = "https://github.com/test/cg"), materialised as a real per-repo store next to genuine ones.

The fix

testsupport/hermetic.py redirects HOME, the XDG dirs, the witan state files and both graph stores into a throwaway directory, clears the ambient WITAN_* selectors, and pins the terminal width. Each package gets a rootdir conftest.py that loads it.

It runs at import, not in a fixture, and that is the load-bearing detail. Importing witan.server IS a write — _ensure_graph creates the store at module scope, deliberately, because the CLI's local-dispatch guard depends on it (#261, and #286 documents why it stays). That import happens during collection, so an autouse fixture body runs long after the store exists. Only something imported by a rootdir conftest is early enough — which is also the only place pytest accepts pytest_plugins at all ("Defining 'pytest_plugins' in a non-top-level conftest is no longer supported", _pytest/config/__init__.py).

PATH deliberately keeps its entry for the real ~/.local/bin. That is where CI installs the omnigraph binary and where _find_binary looks when PATH misses. A binary is a tool, not state; relocating it would break every test that needs a graph, to prevent a leak that cannot happen.

The guard

Redirection fixes today's leaks, not tomorrow's — a new package without the conftest, or anything resolving a real path itself, walks straight past it, silently. So pytest_sessionfinish reports anything that reached the real home anyway.

Strictness is inferred from CI rather than set per-workflow: every runner exports it, so this lands in all four test workflows, all five publish workflows, and whatever gets added next, with no list to keep in sync. Ten env: blocks would have been ten chances to forget the tenth. Warn-only locally, because your machine may legitimately be writing to those directories from another agent session, and a false failure people learn to re-run past is worth less than no check. AGENT_KIT_STRICT_HERMETICITY=1|0 overrides.

How can this be tested?

just test-all

2394 tests pass across the five suites (1 / 308 / 572 / 529 / 984). Under the hostile harness every suite is green and the seeded HOME comes back byte-identical.

The guard was verified by inducing real leaks — throwaway tests writing a store inside the pre-existing ~/.local/share/witan/code/ and appending to an existing merge-watermarks.json, i.e. the two shapes the actual defects had:

condition	exit	leak reported
CI=true (inferred strict)	1	yes
AGENT_KIT_STRICT_HERMETICITY=1	1	yes
AGENT_KIT_STRICT_HERMETICITY=0	0	yes
no CI, no override	0	yes

The sentinel was removed afterwards. The leak path is deliberately not a permanent test — proving the detector fires means writing to the real home, which is the thing the mechanism exists to prevent — so test_hermetic_guard.py asserts the logic that decision rests on, plus that the redirection is actually in effect.

The agent-config-kit failures were attributed by axis rather than guessed, at the base commit with the guard absent:

condition	failures
real HOME, my terminal	2
real HOME, COLUMNS=40	8
real HOME, COLUMNS=200	0

So all 8 are terminal width; none needed a populated HOME. (An earlier draft of the commit message said otherwise; corrected before this PR.)

just check-versions, just check-core-floor, ./bin/gen_docs.py --check and prek all pass. I also confirmed the new conftest.py and testsupport/ do not land in a built wheel, by building witan-code and listing the archive — no version bump or CHANGELOG entry is needed because nothing published changes.

Additional Context

Pinning the width is not a UX answer. It makes the suite assert message content, which is what those tests are about. Whether the CLI renders readably in a narrow terminal is a real question about the product that a green suite does not answer; the code says so where someone would otherwise be misled, and I can file it separately.

A new package needs its own rootdir conftest.py — nothing supplies one automatically, and without it that suite runs against your real home. Documented in AGENTS.md next to the isolation guidance.

testsupport/ is deliberately not a workspace package: a sixth package would need a version, a CHANGELOG, a bumpversion block and a publish workflow for thirty lines of test support, and would land in just check-versions. It is imported off the repo root via a sys.path insert in each rootdir conftest. (An earlier revision said this worked under a bare pytest from the repo root too — it does not, and never did: that fails on 46 pre-existing collection errors because one package's venv cannot satisfy five, the same count as main at deedb68.)

Not cleaned up: the historical debris already in my real ~/.local/share/witan/code (https_github.com_test_cg.omni and a number of other test-shaped stores). That is real user data on a real machine and not something a PR should delete.

---

Review round 1 (/code-review high, six findings — all addressed in 8b66899)

Two were real regressions this PR introduced, and the first is worth reading:

WITAN_REQUIRE_OMNIGRAPH was in _CLEARED. witan-core-tests.yml sets it to "1" so a missing omnigraph binary is a hard failure rather than a skip, and test_binary_contract.py reads it at module scope — after this plugin runs. Clearing it forced _REQUIRED to False. Confirmed with a probe reading the var at module scope: '<absent>' before the fix, '1' after.

Correcting the impact, which both the finding and my first reply overstated. This did not make the suite skip. The mark is skipif(_BINARY is None and not _REQUIRED) — both conditions — and CI installs the binary, so the 20 contract tests ran on the buggy commit too (run 32857593245 at 67e460e shows them executing). What clearing the var destroyed was the tripwire: with _REQUIRED false, deleting the install step or a failed fetch would have left the suite skipping green instead of failing loudly, which is precisely the scenario that workflow comment exists to describe. A disabled safety net, not an already-retired suite. Still a real regression — the fix stands — but worth stating accurately, since "the suite was silently retired" is a stronger claim than the evidence supports.

The leak detector was blind to the leak it was written for. It compared only immediate children, and the motivating defect lands at ~/.local/share/witan/code/<slug>.omni — inside a code directory that already exists on any machine that has run the indexer. It fired on CI, where the tree is absent, and nowhere else. My own verification missed this because the sentinel I induced was a top-level child — a leak of a shape the real one does not have. Now a depth-3 snapshot keyed on absolute path with size+mtime markers, so the #282 append-to-an-existing-file shape is detectable at all.

The rest: WITAN_TEST_OMNIGRAPH_SERVER/_GRAPH were being cleared, making the live-server opt-in tests unreachable (both now in an _EXEMPT tuple asserted disjoint from _CLEARED); six more ambient selectors added; CI=false/CI=0 no longer read as strict; pytest_plugins dropped for a direct import.

Deepening the watch surfaced a problem the shallow version hid: four of five suites then warned, every line of it ambient — the indexer touching .omni.repo sidecars, the OIDC client refreshing its token, Claude Code writing .claude.json. Measured with a 95-second idle probe (no tests running) and a parallel test-all. Modifications of those paths are now filtered; creations never are, so on CI — where none of them exist — a real write still reports. Without that filter the check would have been re-run past, which this file's own docstring calls worse than no check.

2399 tests pass, no warnings.