Skip to content

feat(py): download datasets as files with optional unzip#32

Merged
ethanwharris merged 2 commits into
mainfrom
pr/dataset-download-archives
Jul 23, 2026
Merged

feat(py): download datasets as files with optional unzip#32
ethanwharris merged 2 commits into
mainfrom
pr/dataset-download-archives

Conversation

@ravivooda

@ravivooda ravivooda commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • download dataset files directly into a versioned directory with no post-download ZIP creation path
  • add explicit unzip=True / --unzip support for versions stored as exactly one ZIP artifact
  • reject unsafe remote and archive paths before writing outside the destination

Test plan

  • python -m pytest -q python/tests/test_dataset_download.py python/tests/core/test_teamspace.py python/tests/cli/dataset/test_download.py (89 passed)
  • pre-commit checks for all changed files
  • live 10-file dataset download verified as raw files with no generated ZIP
  • live single-ZIP dataset extracted with unzip=True and verified byte-for-byte
  • temporary live dataset deleted after validation

Base automatically changed from ravivooda/dataset-download-parallel to main July 21, 2026 19:01
ravivooda and others added 2 commits July 22, 2026 17:25
Avoid mandatory post-download archives while preserving opt-in ZIP output and adding safe explicit extraction for ZIP-backed datasets.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep directory downloads as the only output mode and retain explicit safe extraction for datasets stored as one ZIP artifact.

Co-authored-by: Cursor <cursoragent@cursor.com>
@ravivooda
ravivooda force-pushed the pr/dataset-download-archives branch from 982ad02 to 1ac5adf Compare July 22, 2026 17:26
@ravivooda ravivooda changed the title feat(py): download datasets without mandatory zipping feat(py): download datasets as files with optional unzip Jul 22, 2026
@ravivooda
ravivooda marked this pull request as ready for review July 22, 2026 21:40
@ethanwharris
ethanwharris merged commit 1389de2 into main Jul 23, 2026
37 checks passed
@ethanwharris
ethanwharris deleted the pr/dataset-download-archives branch July 23, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants