⚡ export: chunked parallel timeline rendering - #5
Conversation
Split long exports into frame-exact chunks rendered by parallel FFmpeg processes, then join them with a stream-copying concat mux. A single composition pass is bottlenecked on serial filter stages (zoompan, the overlay composites), which is what starves hardware encoders — N chunk pipelines multiply throughput without changing emitted frames. - render_timeline_composition splits into a dispatcher plus render_composition_window(window, pass): a shifted pass re-anchors the concatenated stream to absolute plan time so between(t,..), camera setpts offsets, zoompan 'it' and libass cues all keep working unmodified - zoompan re-emits frames on a 0-based pts grid, so shifted passes re-shift its output back to absolute time for downstream pairing - mid-segment cuts keep the fps resampler on the same absolute frame grid via a +cut*speed/TB phase on the chain anchor, and gap/trim clamps tile the window exactly - camera overlays, masks and caption passes skip building chains that can never fire inside the window - audio renders once as a single continuous AAC track and is spliced in at the mux — no per-chunk AAC priming at seams; embedded chapters and faststart move to the mux stage too - workers = cores/2 (clamped 2..4), ~2 chunks per worker, chunk >= 2s, enabled only for non-gif/webp exports >= 20s on >=4 cores; a shared halt flag kills sibling passes on the first failure, and the overlay producer's worker/memory budgets divide by the worker count - verified on FFmpeg 8.1: chunked vs single-pass output is bit-identical (360/360 frames, matching framemd5) including a mid-segment seam with zoom and a boundary-spanning camera overlay Co-Authored-By: Jean de Dieu HAGENIMANA <jdhagena77@gmail.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
eof_action=repeat overlays fed by infinite looped secondaries keep emitting clones of the last main frame past the stream's end; -t only trims them approximately, so a repeat landing on an exact-millisecond boundary leaks through as a duplicated seam frame (+1 frame, e.g. 901 vs 900 at 30s@30fps). Add trim=end_frame on the windowed pass's final stream to bound output by frame count instead of timestamp rounding. Co-Authored-By: Jean de Dieu HAGENIMANA <jdhagena77@gmail.com>
E2E test: chunked parallel export — found & fixed a frame-exactness bug at camera-overlay seamsTested on the real render path (temporary in-crate harness → Confirmed working: Bug found & fixed (5c6be54): with a camera/overlay element spanning a chunk boundary, that chunk emitted 114 frames instead of 113 — output 901 frames / 30.029s vs expected 900 / 30.000s, with the duplicated frame at the seam (source frame 338 appears twice consecutively, ~33ms visible freeze): Root cause (bisected on the captured chunk graph): the camera shadow/border layers are Also noted (not blockers for this PR)
|
Re-verified: boundary-dup fix confirmed (commit 5c6be54)Re-ran the overlay-spanning-boundary repro through the real chunked path on the fix commit:
Fix verified end-to-end. |
📝 Description
Splits timeline exports into frame-exact chunks rendered by parallel FFmpeg worker processes, then joins them with a stream-copying concat mux. Today a single composition pass serializes every filter stage —
zoompan, the plate/camera/mask composites all run on one thread of one pipeline — which is exactly what starves a hardware encoder (NVENC sat at ~25% on an RTX 3050 in testing this pipeline). N chunk pipelines multiply compositing throughput up to the worker count and keep the encoder fed.How it works
render_timeline_compositionbecomes a dispatcher:should_render_chunked(non-gif/webp, ≥20 s, ≥4 cores) routes torender_timeline_chunked, everything else takes the old single pass verbatim viarender_composition_window(CompositionWindow::full, CompositionPass::standalone).[first_frame, end_frame)on the output frame grid. A shifted pass appliessetpts=PTS-STARTPTS+{win_start}/TBto the concatenated stream, so every absolute-time construct keeps working unmodified:between(t,…)enables, camerasetpts=PTS+{offset}/TB, zoompan'sit, and libass cue times.setpts=PTS-STARTPTS+{cut*speed}/TBand the source window becomessrc_in + cut*speed.zoompanre-emits frames on a 0-based pts grid, so shifted passes apply a secondsetpts=PTS+{win_start}/TBafter it to restore absolute timestamps for downstream pairing.+faststartlikewise move to the mux stage.workers = cores/2clamped to 2–4, ~2 chunks per worker, minimum 2 s per chunk. A sharedhaltflag kills sibling FFmpeg processes on first failure; cancel propagates through the existing flag. The overlay producer's worker/memory budgets divide by the worker count (parallel_divisor), keeping total RAM flat.first_frame_index + localplate frames — the cursor/items rasterizers are pure functions of absolute plan time, so plate content needs no changes.chunk_*.mkv,audio.m4a, concat list) live in a per-export temp dir removed on success and failure.Verified end-to-end on FFmpeg 8.1 with the exact generated graph shape (concat → tpad → zoompan + abs enables + camera overlay): single-pass vs 2 chunks at a mid-segment seam is bit-identical — 360/360 frames with matching framemd5, including a camera overlay spanning the boundary.
Closes #(issue)
🎯 Type of Change
🧪 Validation & Testing
Describe how you tested these changes:
bun run typecheckpasses with zero errors — n/a for Rust changes;cargo clippy -- -D warningscleanbun run testpasses (all unit & integration tests) —cargo testcannot link under GNU ld on this Windows box (ordinal-limit error, pre-existing); added unit tests forchunk_boundaries+CompositionWindow::from_frames, CI covers the suite. All existing render tests use ≤2 s plans so they exercise the standalone path unchanged.bun run format:checkpasses —cargo fmt --checkclean📋 Checklist
CONTRIBUTING.mdNotes for reviewers:
-sswould risk non-frame-exact landing). Decode is the cheap part; the serial filters were the bottleneck.Link to Devin session: https://app.devin.ai/sessions/52c7ba7f829f4aecbefa2f889efc8dc5
Open in Devin Desktop: https://app.devin.ai/desktop/session/52c7ba7f829f4aecbefa2f889efc8dc5?variant=devin
Requested by: @jeandedieuH