Skip to content

⚡ export: screen-rect cursor plate, bounded unpremultiply, hwaccel decode - #3

Merged
jeandedieuH merged 1 commit into
mainfrom
devin/1790025735-export-hw-decode-cursor-plate
Sep 22, 2026
Merged

jeandedieuH merged 1 commit into
mainfrom
devin/1790025735-export-hw-decode-cursor-plate

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

📝 Description

Export profiling showed NVENC idle (~25% on RTX 3050) because every stage upstream is CPU-bound: software decode, a single-threaded zoompan, and a full-canvas RGBA overlay plate re-rasterized per frame in Rust and piped to FFmpeg's stdin as rawvideo (8–16 MB/frame). This PR does the cheap, no-restructure wins; the bigger reworks (pre-rendered overlay inputs, cursor-sprite sendcmd, chunked parallel export) stay as follow-ups.

Cursor plate shrinks to the fitted screen rect (cursor-only exports): the renderer still rasterizes into a canvas-sized pixmap (every draw is already clipped to video_screen), then crop_rgba_region extracts that rect and the graph composites it back via overlay=x={rx}:y={ry}. Per-frame rasterize→unpremultiply→pipe→composite work drops by the padding fraction (~10% at default padding, more for letterboxed/aspect-fit layouts). Dual-plane and items plates are unchanged — they draw in canvas coordinates.

unpremultiply_rgba_bounded restricts the premultiplied→straight-alpha pass to the bounding box of non-zero-alpha pixels. Pixels with alpha 0 are untouched by the conversion either way, so output is bit-identical while skipping ~90%+ of pixels on typical frames (a cursor sprite on a large canvas). Applied to both the cursor plane and the cached items plane.

-hwaccel auto on media inputs when a hardware encoder is selected: moves decode off the CPU where a backend exists (D3D11VA/DXVA2 on Windows, VideoToolbox on macOS, VAAPI on Linux), auto-downloads frames for the software filter graph, and falls back to software decode when no backend applies (verified on a GPU-less machine). Software encodes — including the existing hardware-failure retry, which re-enters with ExportEncoder::Software — still get a clean software command.

🎯 Type of Change

  • ⚡ Performance optimization

🧪 Validation & Testing

  • cargo fmt --check — clean

  • cargo clippy -- -D warnings — clean

  • cargo check — clean (stable-x86_64-pc-windows-gnu; this VM has no MSVC toolchain, so cargo test linking hits the GNU ld ordinal limit — left to CI)

  • Standalone equivalence harness: unpremultiply_rgba_bounded produces byte-identical output to unpremultiply_rgba across sparse / dense / fully-transparent / single-edge-pixel / mixed-alpha inputs; crop_rgba_region verified pixel-exact

  • -hwaccel auto verified to fall back to software decode cleanly on a GPU-less box (frame=60 ... rc=0)

  • bun run typecheck — unchanged files are Rust-only; covered by CI

  • bun run test — covered by CI

  • cargo fmt --check / cargo clippy -D warnings / cargo check pass

  • Tested on target operating systems:

    • Windows (Server 2022 VM; compile + unit-level verification — no GPU on this machine, so real NVENC utilization needs a run on actual hardware)
    • macOS
    • Linux

📋 Checklist

  • Style follows existing conventions; cargo fmt clean
  • Non-obvious decisions commented (rect-mirroring of ClipRect rounding, hwaccel gating rationale)
  • No raw frames/secrets across Tauri boundaries
  • No telemetry added

Expected impact: removes a chunk of the CPU work feeding NVENC (smaller plate + dirty-bbox unpremultiply + hw decode). GPU utilization should rise but likely stays well under 100% — zoompan remains single-threaded and the dual-plane path still ships full-canvas plates; raising GPU util substantially needs the chunked/sprite follow-ups.

Link to Devin session: https://app.devin.ai/sessions/52c7ba7f829f4aecbefa2f889efc8dc5
Open in Devin Desktop: https://app.devin.ai/desktop/session/52c7ba7f829f4aecbefa2f889efc8dc5?variant=devin
Requested by: @jeandedieuH

…cel decode

- Cursor-only exports stream a screen-rect-sized RGBA plate over stdin
  instead of the full canvas: the renderer still draws into a canvas
  pixmap clipped to the fitted screen rect, then the rect is cropped
  out and composited back via overlay=x:y. Cuts per-frame rasterize,
  unpremultiply, pipe, and overlay work by the canvas padding fraction.
- unpremultiply_rgba_bounded restricts the straight-alpha pass to the
  drawn bounding box (alpha-0 pixels are untouched by the conversion
  anyway, so output is identical).
- Media inputs get -hwaccel auto when a hardware encoder is selected;
  auto falls back to software decode transparently, and the existing
  software-encoder retry still produces a clean software command.

Co-Authored-By: Jean de Dieu HAGENIMANA <jdhagena77@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@jeandedieuH
jeandedieuH merged commit fc518ad into main Sep 22, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant