Add CalMS21 Tasks 2 and 3 dataset support - #96
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds first-class support for fetching and converting CalMS21 Task 2 and Task 3 datasets into LISBET’s existing Movement record format, extending the existing CalMS21 ingestion path without introducing a new in-memory annotation model.
Changes:
- Register
CalMS21_Task2andCalMS21_Task3in thebetman fetch_datasetCLI andhub.fetch_dataset. - Download official Caltech archives with checksum validation and extract only the train/test keypoint JSON members, then reuse
calms21.load_taskxfor conversion. - Add targeted tests and user-guide documentation covering layout, filtering, and Task 3’s independent binary behavior groups.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_calms21.py |
Adds conversion, round-trip filtering, fetch configuration, and CLI parsing tests for Task 2/3. |
src/lisbet/io/ext_sources/calms21.py |
Adds a TODO note about potential future streaming conversion (no functional change in this diff hunk). |
src/lisbet/hub.py |
Extends fetch_dataset to download/validate/unzip Task 2/3 and convert via existing CalMS21 loader. |
src/lisbet/cli/commands/fetch.py |
Adds Task 2/3 dataset IDs to the CLI choices list. |
docs/user_guide/fine_tuning.rst |
Documents Task 2/3 dataset layout, filtering examples, and resource/benchmark limitations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CalMS21_Task2andCalMS21_Task3tobetman fetch_datasetData representation
Task 2 records retain the CalMS21 annotator hierarchy as
task2/<annotator>/<split>/<video-id>.Task 3 records retain the behavior hierarchy as
task3/<behavior>/<split>/<video-id>. Every behavior group remains an independentother-versus-target binary problem; the groups are not combined into a seven-class or fully observed multilabel dataset.No new in-memory annotation hierarchy, training orchestration, or streaming JSON dependency is introduced.
Validation
.venv/bin/pytest -q tests/test_calms21.py tests/test_io_core.py— 23 passedbetman fetch_dataset --help— lists both new dataset IDsgit diff --check— passedThe broader non-integration suite currently reports 159 passed, 17 failed, and 7 deselected. The failures are in unchanged augmentation configuration, dataset, inference, and training-helper areas and are outside this PR scope.
Closes #95