Repository navigation
Speed up stats computation for the whole video. - #60
Open
copybara-service[bot] wants to merge 1 commit into
Open
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
E.g. on the motion_floor_to_sky_av1_hdr10p.mp4 it went down from ~1 minute to ~7 seconds.
This is achieved in two ways:
- speed up frame decoding by using the WebCodecs API (if available) to decode frame by frame, rather than repeatedly seeking to the requested time which gets slower and slower the farther away you are from a keyframe
- speed up the stats computation itself by precomputing LUTs for math functions called many times, and a bunch of other optimizations
Detailed changes:
1. WebCodecs Hardware Video Decoding (video_decoder.ts):
- Added sequential hardware decoding of demuxed MP4 samples via WebCodecs
VideoDecoder, eliminating O(N^2) GOP seek times in HTML5 video.
- Added backpressure handling via decodeQueueSize and ondequeue.
- Added Annex E.3 HEVC and AV1 codec string builders (with O(1) bitwise
compatibility flag reversal) and box serializers.
- Preserved fallback to HTML5 video seek if WebCodecs is unsupported or fails.
2. Color Transfer Lookup Tables (image_stats.ts):
- Precomputed 64K-entry Float32Array LUTs for transfer functions (HLG, PQ,
sRGB) and HLG OOTF, replacing millions of Math.pow calls with fast table
lookups.
3. VideoFrame & WebGL Pipeline Optimizations (image_stats.ts, app.ts):
- Allowed ImageStats to accept VideoFrame directly, avoiding createImageBitmap
allocations.
- Consolidated and simplified WebGL2 resource management, reusing a shared
context, texture, and framebuffer across frames to eliminate GPU resource
recreation overhead.
- Added keepEncoded: false and shared scratch buffer reuse to avoid allocating
tens of gigabytes of temporary Float32Arrays during batch processing.
- Optimized getInverseDistribution by reusing bin boundaries, guarded
getPercentile against division-by-zero, and optimized averageStats math.
4. UI:
- Throttled panel re-rendering during dynamic metadata batch calculation.
5. Tests:
- Added unit tests for video_decoder (codec strings, bit reversal).
- Added unit tests for image_stats (ImageBitmap, scratch buffer, averaging).
PiperOrigin-RevId: 995801615
copybara-service
Bot
force-pushed
the
test_995801615
branch
from
October 8, 2026 16:20
774c6fa to
df99dd3
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Speed up stats computation for the whole video.
E.g. on the motion_floor_to_sky_av1_hdr10p.mp4 it went down from ~1 minute to ~7 seconds.
This is achieved in two ways:
Detailed changes:
WebCodecs Hardware Video Decoding (video_decoder.ts):
VideoDecoder, eliminating O(N^2) GOP seek times in HTML5 video.
compatibility flag reversal) and box serializers.
Color Transfer Lookup Tables (image_stats.ts):
sRGB) and HLG OOTF, replacing millions of Math.pow calls with fast table
lookups.
VideoFrame & WebGL Pipeline Optimizations (image_stats.ts, app.ts):
allocations.
context, texture, and framebuffer across frames to eliminate GPU resource
recreation overhead.
tens of gigabytes of temporary Float32Arrays during batch processing.
getPercentile against division-by-zero, and optimized averageStats math.
UI:
Tests: