Skip to content

Add CalMS21 Tasks 2 and 3 dataset support - #96

Merged
gchindemi merged 2 commits into
mainfrom
feature/calms21-tasks-2-3
Jul 23, 2026
Merged

Add CalMS21 Tasks 2 and 3 dataset support#96
gchindemi merged 2 commits into
mainfrom
feature/calms21-tasks-2-3

Conversation

@gchindemi

Copy link
Copy Markdown
Member

Summary

  • add CalMS21_Task2 and CalMS21_Task3 to betman fetch_dataset
  • fetch the official archives with checksum validation and extract only the train and test keypoint JSON files
  • reuse the existing CalMS21-to-movement conversion path
  • document dataset layout, filtering, resource requirements, and benchmark limitations
  • add focused conversion, round-trip, fetch configuration, and CLI tests

Data representation

Task 2 records retain the CalMS21 annotator hierarchy as task2/<annotator>/<split>/<video-id>.

Task 3 records retain the behavior hierarchy as task3/<behavior>/<split>/<video-id>. Every behavior group remains an independent other-versus-target binary problem; the groups are not combined into a seven-class or fully observed multilabel dataset.

No new in-memory annotation hierarchy, training orchestration, or streaming JSON dependency is introduced.

Validation

  • .venv/bin/pytest -q tests/test_calms21.py tests/test_io_core.py — 23 passed
  • Ruff checks and formatting — passed
  • betman fetch_dataset --help — lists both new dataset IDs
  • git diff --check — passed

The broader non-integration suite currently reports 159 passed, 17 failed, and 7 deselected. The failures are in unchanged augmentation configuration, dataset, inference, and training-helper areas and are outside this PR scope.

Closes #95

Copilot AI review requested due to automatic review settings July 20, 2026 18:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class support for fetching and converting CalMS21 Task 2 and Task 3 datasets into LISBET’s existing Movement record format, extending the existing CalMS21 ingestion path without introducing a new in-memory annotation model.

Changes:

  • Register CalMS21_Task2 and CalMS21_Task3 in the betman fetch_dataset CLI and hub.fetch_dataset.
  • Download official Caltech archives with checksum validation and extract only the train/test keypoint JSON members, then reuse calms21.load_taskx for conversion.
  • Add targeted tests and user-guide documentation covering layout, filtering, and Task 3’s independent binary behavior groups.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/test_calms21.py Adds conversion, round-trip filtering, fetch configuration, and CLI parsing tests for Task 2/3.
src/lisbet/io/ext_sources/calms21.py Adds a TODO note about potential future streaming conversion (no functional change in this diff hunk).
src/lisbet/hub.py Extends fetch_dataset to download/validate/unzip Task 2/3 and convert via existing CalMS21 loader.
src/lisbet/cli/commands/fetch.py Adds Task 2/3 dataset IDs to the CLI choices list.
docs/user_guide/fine_tuning.rst Documents Task 2/3 dataset layout, filtering examples, and resource/benchmark limitations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@gchindemi
gchindemi merged commit b0f7eab into main Jul 23, 2026
1 check passed
@gchindemi
gchindemi deleted the feature/calms21-tasks-2-3 branch July 23, 2026 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add CalMS21 Tasks 2 and 3 dataset support

2 participants