Test hygiene: dataset-loader timeouts, live test tiers, perf tests that can fail (#508, #510, #511)

[kgdunn]

Summary

• Fixes Remote dataset loaders have no timeout: distillateflow() / oildoe() hang forever against a black-holing host #508: _read_remote_csv now downloads with an explicit timeout via urllib.request.urlopen(url, timeout=...) into a buffer handed to pandas, so distillateflow() / oildoe() raise the documented RuntimeError (naming the URL) against a black-holing host instead of hanging forever. The default of 30 s is a new dataset_fetch_timeout knob on the config settings singleton, overridable via PROCESS_IMPROVE_DATASET_FETCH_TIMEOUT or a caller argument. An on-disk cache was deliberately not added (out of scope; can be a follow-up).
• Fixes Test tiers are documented but unused: -m "not slow" skips nothing and -m "not dataset" still hits the network #510: the ENG-29 test tiers are now real. --strict-markers is on in pytest.ini (a typo'd marker is a collection error), the one network-fetching test (test_pca_foods in tests/test_multivariate.py) carries @pytest.mark.dataset so -m "not dataset" performs no network access (a tests/ sweep for urlopen / openmv / http found no other unmarked fetches), and 42 tests measured at >= 5 s via --durations now carry @pytest.mark.slow (threshold is 2 s per CONTRIBUTING.md; the margin keeps the marks honest on faster machines). Two end-to-end tests (sensory pipeline, sklearn halving grid search) also carry @pytest.mark.integration. Wall clock on this runner: full suite 6:15, -m "not slow and not dataset" 1:32. Both CI test jobs run plain pytest with no -m filter, so every tier still executes in CI.
• Fixes Performance "baselines" in tests/perf/ cannot fail: no assertions, no stored baseline, and disabled under the default -n auto #511: the tests in tests/perf/ had zero assertions, no stored baseline, and pytest-benchmark disables itself under the default -n auto, so nothing could ever fail. Implemented the issue's option 2: they now assert the deterministic cost-shape properties the ENG-18 _LazyFrame work needs protected: the public DataFrame views are built exactly once and cached (identity plus a counted single build), pickling excludes _frame_cache and the views rebuild correctly after unpickling, and repeated transform / predict / diagnose calls cause zero lazy-frame rebuilds; check_random_state passes a Generator through by identity and resolves an int seed to a fresh generator with bit-identical draws. Verified the new tests fail when the _LazyFrame cache write is temporarily removed. CONTRIBUTING.md's performance-regression policy now describes this honestly; the ENG-15 wall-clock benchmark CI job stays planned, and no CI workflow was added.

Also ports the mcp 2.x migration from #528 (mcp_server.py plus the mcp>=2.0 pin) in its own commit, so this PR's typecheck gate can be green; it becomes a no-op once #528 merges.

Version is bumped to 1.73.4 (with CITATION.cff and CHANGELOG.md in sync); this assumes the earlier queued 1.73.x PRs merge first.

Test plan

• uv run pytest tests/test_experiments_datasets.py tests/test_sec10_path_and_fetch.py tests/perf/ --no-cov -q: 30 passed, 2 skipped (the two live openmv.net tests; that host is blocked by this environment's proxy, so they skip through the new RuntimeError path, which is itself the Remote dataset loaders have no timeout: distillateflow() / oildoe() hang forever against a black-holing host #508 behaviour under test). All new timeout tests are monkeypatched, no real network.
• uv run pytest -m "not slow and not dataset" --no-cov -q: 2611 passed, 5 skipped in 1:32 with --strict-markers on.
• Full suite (--no-cov, with all extras): 2701 passed, 8 skipped, 0 failed in 6:15. Local verification otherwise used targeted runs because this sandbox's proxy blocks openmv.net (the dataset host); CI runs the full matrix with the coverage gate.
• Perf tests demonstrated to fail when the _LazyFrame cache write is temporarily removed.
• .github/workflows/run-tests.yml verified to run every tier (no -m filter in either test job).
• uv run ruff check ., uv run ruff format --check ., and uv run mypy src/process_improve all pass.

Checklist

• Version bumped in pyproject.toml (PATCH for fixes/docs/config, MINOR for new features)
• Tests added or updated where relevant
• ruff check . passes
• CHANGELOG.md updated

---

Generated by Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/process_improve/config.py	83.33%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[kgdunn]

The macOS failure on the previous head was test_budget_clamped_to_minimum_model_size (assert 3 >= 4), which is not this PR's: it is an intermittent failure that has now hit two unrelated PRs. Root cause (fixed in #531): the documented max(budget, k + 1) floor exists only in the point-exchange fallback, so with pyoptex installed an infeasible budget goes through unclamped and pyoptex's handling of it is unpredictable. The #531 fix is ported verbatim onto this branch so the test is deterministic here in any merge order; the ported hunk no-ops once #531 merges. The CodeQL identical-values finding on the perf test was also addressed (two named reads instead of a self-comparison).

---

Generated by Claude Code