Skip to content

fix(inference): replace image layout heuristics with explicit configuration - #285

Open
johyeongseob wants to merge 3 commits into
openvinotoolkit:mainfrom
johyeongseob:fix/220-explicit-image-layout
Open

johyeongseob wants to merge 3 commits into
openvinotoolkit:mainfrom
johyeongseob:fix/220-explicit-image-layout

Conversation

@johyeongseob

Copy link
Copy Markdown

Summary

  • Add a required keyword-only image_layout argument to ResizePreprocessor and ResizeSmolVLA, accepting "BCHW", "BHWC", or the corresponding ImageLayout enum values.
  • Replace shape-based layout detection with explicit layout handling while preserving output layouts.
  • Add tests for ambiguous shapes, equivalent pixels across layouts, camera masks, invalid or missing layouts, and configuration through both type and class_path.
  • Update fuzz harnesses, manifest seeds, and configuration documentation.

Why

Image dimensions alone cannot reliably determine the channel axis. For example, (1, 3, 3, 3) can represent either BCHW or BHWC input.

Passing the layout explicitly lets the caller specify the axis order without relying on channel-count heuristics.

Validation

Environment:

  • Ubuntu 24.04 on WSL
  • Python 3.12.3
  • Dependencies managed with uv

Results:

  • uv run --no-sync pytest tests/unit/inference/ -q
    • 549 passed, 2 skipped, 2 warnings.
    • Skips: missing optional torch dependency and restricted PaliGemma tokenizer access.
    • Warnings: pytest collection of TestAdapter and multiprocessing fork() in the OpenVINO tokenizer test.
  • Ruff 0.15.12 lint and format checks passed for the four changed preprocessor source files.
  • Pyrefly 1.2.0 reported 112 errors on both the baseline (de49e1f) and this branch using the same Python environment. Sorted error messages and their occurrence counts matched.
    • Three targeted suppressions were added in tests: two intentional missing-argument calls and one nested-camera input supported at runtime but absent from the existing input annotation.

Breaking changes

  • Existing Python calls and configurations for these two preprocessors must now provide image_layout.
  • For type configurations, add image_layout alongside the other arguments. For class_path configurations, add it under init_args.
  • All images passed to a preprocessor instance must use the configured layout.
  • Automatic layout detection is removed. Output layouts remain unchanged.

Related issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant