Add opt-in on_error hook for unreadable images during folder scan - #88
Merged
JonasWurst merged 5 commits intoJul 16, 2026
Conversation
Folder-scanning formats open every image to read dimensions, so one broken/undecodable file killed get_images() and get_labels() with a raw PIL error mid-iteration. - Normalize the failure: get_image_dimensions now wraps the PIL fallback and raises a typed ImageDimensionError carrying the offending path. - Add an opt-in on_error hook to get_images_from_folder. Default None re-raises (no behavior change); when set, the file is skipped so the generator survives. - Expose on_error on the folder-scanning readers (yolov8, kitti, lightly as a settable attribute; pascalvoc.from_dirs as a param; maskpair as a settable attribute, with get_labels kept aligned to readable images). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JonasWurst
commented
Jul 16, 2026
Move the on_error test-dataset builders out of the test classes into module-level helpers at the bottom of each file (no self), and hoist the repeated local imports to the top of the files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
/review |
…ror-hook-for-unreadable-images
MalteEbner
approved these changes
Jul 16, 2026
MalteEbner
left a comment
MalteEbner
left a comment
Contributor
There was a problem hiding this comment.
LGTM, but test naming does not follow our coding standards.
Rename the on_error tests in yolov8 and maskpair to the test_<method>__<scenario> convention used by neighboring tests (e.g. test_get_images__reraises_by_default). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- One test class per source class: dissolve the feature-grouping on_error test classes. yolov8's get_images tests move into Test_YOLOv8BaseInput after get_categories; maskpair's on_error tests merge into TestMaskPairIntegration. - Order tests by source order: move the get_image_dimensions corrupt test above the get_images_from_folder tests in test_utils. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JonasWurst
deleted the
jonas-lig-10211-labelformat-add-opt-in-on_error-hook-for-unreadable-images
branch
July 16, 2026 10:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
This is the upstream
labelformatchange that unblocks the studio-side work.Problem
Folder-scanning formats (
yolov8,kitti,pascalvoc,lightly,maskpair) open every image viaget_image_dimensions→PIL.Image.opento read dimensions. A broken/undecodable file raised mid-iteration, and since a generator cannot resume after raising, one bad file killedget_images()andget_labels()entirely, propagating a rawPIL.UnidentifiedImageError/OSError.Changes
get_image_dimensions(utils.py) now wraps the fallbackPIL.Image.openand raises a typedImageDimensionErrorcarrying the offending path instead of a raw PIL/OS error.get_images_from_foldergainson_error: Callable[[Path, ImageDimensionError], None] | None = None. DefaultNone→ re-raise (no behavior change). When set: invokeon_error(path, error)and skip the file so the generator survives.on_errorattribute onyolov8,kitti,lightly,maskpair;on_errorparameter onpascalvoc.from_dirs(its scan happens at construction).maskpair.get_labelswas kept aligned so dropped images don't break label lookup.Default behavior stays
raise; tolerance is strictly opt-in.Testing
utils, and per-format tests for yolov8, pascalvoc, and maskpair (including get_labels alignment).mypyclean,blackclean.🤖 Generated with Claude Code