Run the examples/ unit test tier in CI

.github/workflows/ci-example.yml

@@ -0,0 +1,46 @@
+name: CI (examples)
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "examples/vlm/**"
+  pull_request:
+    branches: [main]
+    paths:
+      - "examples/vlm/**"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pull-requests: read
+
+jobs:
+  # Per-example gate, so touching one example does not run another's tests. Each
+  # new example adds one output + filter here and one job below.
+  changes:
+    runs-on: ubuntu-latest
+    outputs:
+      vlm: ${{ steps.filter.outputs.vlm }}
+    steps:
+      - uses: actions/checkout@v5
+      - uses: dorny/paths-filter@v3
+        id: filter
+        with:
+          filters: |
+            vlm:
+              - "examples/vlm/**"
+
+  vlm-tests:
+    needs: changes
+    if: needs.changes.outputs.vlm == 'true' || github.event_name == 'workflow_dispatch'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+      - uses: astral-sh/setup-uv@v6
+      - run: uv sync --group video
+      # Example tests are outside the root `testpaths`, so each tier is named here
+      # explicitly. Each needs its own pytest session: the VLM-eval unit conftest
+      # injects a fake `lmms_eval` that would leak into a tier sharing the session.
+      # The matching integration tier needs the real, undeclared lmms-eval; manual only.
+      - run: uv run pytest examples/vlm/eval/tests/unit -v --timeout=60

CHANGELOG.md

@@ -108,6 +108,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **BREAKING: the COCO-Karpathy prep helper moved to `examples/vlm/data/prep_vlm_coco.py`** (was `scripts/prep_vlm_coco.py`). It is VLM-experiment data prep, not general tooling, so it belongs with the example rather than in `scripts/`. No compat shim — invoke it at the new path. `--caption-json` and `--image-root` are now required: their old defaults were cluster-specific absolute paths, which a shipped example must not carry.
 - **Training entry point is now a library.** `scripts/train.py:main()` (~1000 lines) is decomposed into `kempnerforge/training/` modules: `runtime.py` (`RuntimeContext`, `PipelineBundle`, `setup_distributed`), `data_pipeline.py` (`DataPipeline`, `PhaseState`, data/eval/phase builders), `loop.py` (`BatchStream`, `StepResult`, `TrainingSession`, `run_training_loop`, and the `text_step` / `vlm_step` / `pipeline_step` bodies picked by `select_step_fn`), and `entry.py` (`run_training` plus the model/checkpoint/resume builders). `scripts/train.py` is now a thin CLI wrapper — same CLI, same log lines, same metric keys, same checkpoint format. Behavior-preserving; the loop is unit-testable with a fake `CheckpointManager` (`tests/unit/test_train_entry.py`). `run_training(config, *, step_fn=None, hooks=None)` lets an experiment own the step body or register hooks without copying the build phases, and both `run_training` and `run_training_loop` tear down in `finally` now that they are library calls rather than a script about to exit.
 - **BREAKING — VLM evaluation moved out of core into `examples/vlm/eval/`.** `kempnerforge/eval/` and `scripts/vlm_eval_harness.py` are gone; the lmms-eval adapter, CLI, tests and how-to now live in the example, so core carries no lmms-eval-facing code. The `[project.entry-points."lmms_eval.models"]` declaration is removed with no shim — `lmms_eval --model kempnerforge_vlm` no longer resolves, and the harness builds the adapter itself. See `examples/vlm/eval/README.md` for usage.
+- New `ci-example.yml` workflow runs the hermetic example test tiers, which sit outside the root `testpaths` and so were collected by no CI step. One job per example behind a per-example path filter, so touching one example does not run another's tests; one pytest session per tier. The VLM-eval integration tier stays manual (it needs the undeclared `lmms-eval`).
 - `docs/getting-started/install.md` Prerequisites: documents `.python-version` and uv's auto-fetch behavior.
 - `README.md` and `kempnerforge/README.md` Prerequisites: clarify that uv auto-fetches Python 3.12 via `.python-version`.
 - `docs/claude-ready.md` first-run flow: `/kempnerforge:install-and-verify` runs before `/kempnerforge:cluster-config`.

CONTRIBUTING.md

@@ -202,6 +202,7 @@ CI runs on every push to `main` and every PR. All jobs must pass before merge.
 | `lint` | `ruff check` + `ruff format --check` + `pyright` | Every push/PR |
 | `unit-tests` | `pytest tests/unit/ -v --timeout=60` | Every push/PR |
 | `gpu-tests` | `pytest tests/integration/` | Manual dispatch |
+| `<example>-tests` | that example's hermetic test tiers, one pytest session per tier | Pushes/PRs touching `examples/<example>/` |
 
 The most common CI failure is `ruff format --check`. Run `uv run ruff format --check kempnerforge/ tests/ scripts/` locally before pushing.
 

examples/vlm/eval/README.md

@@ -202,6 +202,9 @@ uv run pytest examples/vlm/eval/tests/unit
 uv run pytest examples/vlm/eval/tests/integration
 ```
 
+CI runs the unit tier on pushes and PRs that touch `examples/vlm/`; the
+integration tier stays manual, since real lmms-eval is not installed there.
+
 Keep the two directories in separate pytest sessions: in a combined run the unit
 conftest's injected fake replaces the `adapter` module in `sys.modules` after the
 integration modules have bound the real one, so the monkeypatch-based integration