Move VLM training configs into examples/vlm/ - #193
Merged
Conversation
The 13 configs/train/vlm_*.toml presets move to examples/vlm/configs/ under the same filenames, so the core configs/train/ tree ships only general presets. BREAKING for any path naming the old location; no shim. The example now owns its own entry point (examples/vlm/train.py) and a README. The script is a thin argv -> load_config -> run_training wrapper: the VLM step body still lives in core, where run_training selects it from config.is_vlm, so no seam work was needed here. The two tests that loaded the shipped VLM configs move to examples/vlm/tests/ — they assert the example's contents, and a core test must not depend on examples/. The loader contract they also happened to cover (list[FreezeSpec] plus the parallel [vision_encoder]/[adapter]/[vlm] tables) stays in tests/unit/test_config.py against an inline TOML. [video].data_root in vlm_video_webvid.toml becomes a placeholder; shipped example configs must be machine-independent, and the example's tests now enforce that for every config in the directory.
The vlm_debug preset has no [adapter] table and no freeze key, so the freeze assertions there restated a dataclass default and the docstring described coverage the test did not have. vlm_video_webvid is the only preset with both, so the list[FreezeSpec] check moves there. Also repoints the eval how-to and harness usage strings at the moved config paths; vlm_jd.toml has never existed, so those examples name vlm_7b.toml.
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
…to-examples Two conflicts from #190, which moved the COCO prep helper out of scripts/. CHANGELOG: both BREAKING entries kept in the same Changed block. configs/train/vlm_7b_siglip2.toml was a rename/modify — this branch moves the file to examples/vlm/configs/, #190 edited its prep-helper comment. Resolved by keeping both: the file lands at the new path carrying the examples/vlm/data/prep_vlm_coco.py reference. The example README gains a line for the data/ helper that #190 placed alongside configs/, train.py, and eval/.
The core loader test declared freeze = [{vision_encoder, true}], which is
exactly VLMConfig.freeze's default_factory, so it passed whether or not the
loader read the TOML. It now declares two entries with non-default modules
and flags. FreezeSpec has no __post_init__, so the docstring no longer
claims one runs; what the test proves is that the entries become FreezeSpec
instances rather than staying raw dicts.
10 tasks
Conflict in the freeze-schedule config: main pinned the FreezeStage hook to training/loop.py while this branch had made the reference location-neutral. Kept main's wording and this branch's moved paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
configs/train/vlm_*.tomlpresets toexamples/vlm/configs/(same filenames), soconfigs/train/ships only general presets.examples/vlm/train.pyandexamples/vlm/README.md. The script is deliberately thin (load_config→run_training): the VLM step body still lives in core, whererun_trainingpicks it fromconfig.is_vlm, so no new seam was needed.examples/vlm/tests/. The loader contract they touched (list[FreezeSpec]instantiation) was previously unasserted anywhere, so it is now pinned in core against an inline TOML.[video].data_rootinvlm_video_webvid.tomlbecomes a placeholder; the example's tests enforce that every config in the directory is machine-independent.configs/train/vlm_jd.tomlhas never existed, so those examples now name a real config.configs/train/vlm_*.toml.Testing
uv run ruff check kempnerforge/ tests/ scripts/passesuv run ruff format --check kempnerforge/ tests/ scripts/passes — 167 filesuv run pyright kempnerforge/passes (0 errors)uv run pytest tests/unit/ -v --timeout=120— 1725 passed, 3 skippeduv run pytest examples/vlm/tests/ -v— 33 passed (all 13 configs load + validate; no machine-specific paths; entry point plumbs CLI overrides)uv run pytest examples/vlm/eval/tests/unit -q— 104 passedgit grep -n "configs/train/vlm_"returns only CHANGELOG historylist[Dataclass]coercion disabled,test_vlm_freeze_list_instantiates_specsis the only core failure; the example'stest_vlm_video_webvid_tomlfails at its own assertion lineexamples/vlm/train.py examples/vlm/configs/vlm_debug.toml --train.max_steps=20→ loss 11.00 → 8.56,step_20committed. Also 2-GPU torchrun,dp_shard=2.step_352→ resume logsResumed DataLoader: epoch=0, skip_batches=352→ 1648 post-resume steps identical to an uninterrupted 2000-step reference at logged precision (4 dp,num_workers=0), with a 352-step pre-interrupt control also identicalCloses #188