Skip to content

perf: faster JPEG streaming (zero-copy VTK capture, 4:2:0 turbo-jpeg) - #65

Open
user27182 wants to merge 2 commits into
Kitware:masterfrom
user27182:perf/rgbx-capture
Open

user27182 wants to merge 2 commits into
Kitware:masterfrom
user27182:perf/rgbx-capture

Conversation

@user27182

Copy link
Copy Markdown

Two speedups for JPEG image streaming, one commit each:

  1. Zero-copy VTK frames: VtkRemoteControlledArea captures RGBA and returns the RGB channels as a view over that buffer. The Pillow and PyTurboJPEG encoders detect the view and read the pixels in place, instead of Pillow copying every frame through Image.fromarray. img_cols_rows still returns a top-down RGB array, and output is byte-identical to master for all formats.
  2. 4:2:0 for turbo-jpeg: PyTurboJPEG defaults to 4:2:2, Pillow to 4:2:0. Matching Pillow makes turbo-jpeg frames about 16% smaller and faster to encode.

Benchmark

Capture + encode per 1920x1080 frame (Apple Silicon, rendering excluded):

jpeg (Pillow) turbo-jpeg turbo-jpeg frame
master 4.80 ms 4.46 ms 48.6 KB
this PR 3.96 ms (-18%) 3.72 ms (-17%) 40.8 KB

To reproduce (from a trame-rca checkout):

bench_rca.py

pip install -e . vtk
python bench_rca.py   # on master
git fetch https://github.com/user27182/trame-rca perf/rgbx-capture && git checkout FETCH_HEAD
python bench_rca.py   # on this PR

For real turbo-jpeg numbers, also pip install PyTurboJPEG with libjpeg-turbo >= 3.0 on the system; otherwise it falls back to Pillow (the script prints which one it used).

🤖 Generated with Claude Code

Capture RGBA from vtkWindowToImageFilter and flip it into a new buffer,
returning its RGB channels as a view. Pillow and PyTurboJPEG can then
read the 4-byte pixels in place (Image.frombuffer "RGBX" /
TJPF_RGBX) instead of Pillow copying the frame via Image.fromarray.

The returned array is still a top-down RGB array, so other RCAs, the
non-JPEG formats and the multiprocessing path are unchanged: VTK's alpha
(0 on the background) never reaches the encoders, and output is
byte-identical to before for jpeg, turbo-jpeg, png, webp and avif.

The flip no longer writes back into the VTK-owned buffer.
PyTurboJPEG defaults to 4:2:2 while Pillow uses 4:2:0, so turbo-jpeg
frames were larger and slower to encode than jpeg ones. With 4:2:0 both
encoders produce frames of the same size.

The test is skipped unless PyTurboJPEG can load libjpeg-turbo >= 3.0.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant