Skip to content

fix(recording): heal Studio display fps that finalization replaced with the average rate - #2689

Merged
richiemcilroy merged 5 commits into
mainfrom
fix/heal-average-display-fps
Oct 18, 2026
Merged

richiemcilroy merged 5 commits into
mainfrom
fix/heal-average-display-fps

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Problem

Every fragmented Studio recording is finalized through RecoveryManager::remux_if_needed, which until #2687 stored the display track's average frame rate as display.fps. Decoders place frames on that rate. A mostly static screen averages far below its capture rate, so the editor and exports dropped frames, worst on idle screens (a 60 fps capture stored as 9 fps). #2687 stops new recordings getting the average. This PR repairs the recordings already on disk.

Fix

  • cap_enc_ffmpeg::remux::get_video_capture_fps estimates the rate frames were captured at from the first 20,000 packet timestamps:
    • take the shortest gap between frames (5% bins) with at least 10 gaps that sit beside another gap of the same length, or between two holds (gaps of at least 100 ms and 3 times its length). Content that changes at 30 fps for most of a 60 fps recording still has its faster stretches at the full rate. A change captured as a pair of frames sits between holds. A late frame's short gap always has one neighbour one capture interval long, under 100 ms at 10 fps and up;
    • recentre on the median of the gaps within 20% of it;
    • snap to a standard capture rate (24 to 240) when within 5%.
  • Neither existing probe works here. avg_frame_rate is the bug. r_frame_rate is the finest grid every timestamp lies on, which can be the display refresh: a 30 fps window capture on a 120 Hz screen reads 120.
  • track_heal::heal_average_display_fps acts only on segments whose stored fps equals the file's rounded average and whose capture rate is higher. It rewrites only recording-meta.json; the media is untouched. A healed fps no longer equals the average, so it runs once.
  • It runs first in heal_stretched_tracks, which the Tauri editor already calls on open. The GPUI editor never called heal_stretched_tracks; its preflight now does, non-fatally, the same way the Tauri path does.

Evidence

Scan of every Studio recording on a dev Mac (162 recordings, 177 display segments in 5 app data dirs), using a Python copy of the estimator:

  • 65 segments match the signature: stored fps equals the rounded average, and the capture rate is higher. They heal to 60 (56 segments, from 9 to 59), 120 (6, from 33 to 118), 48 (1, from 43), 40 (1, from 37) and 30 (1, from 26).
  • One of the 120s is stored at 33. Its gaps include runs of up to 63 consecutive 8 ms gaps, so it is a 120 fps capture of mostly slower content. A most-common-gap estimate reads 30 for it.
  • The other 112 segments all estimate exactly their stored rate.
  • A window capture with r_frame_rate 120 and 30 fps frames estimates 30, so it is left alone.

Exports of APFS clones of four affected recordings (cursor hidden, zooms off, at most 60 s each), cap export built from this branch, before and after the heal. "Distinct" counts output frames kept by mpdecimate. "Ideal" is the source display track resampled to the export rate, then mpdecimate.

Recording Stored fps Healed fps Export fps Distinct before Distinct after Ideal
Static screen, 60 s 9 60 60 96 251 268
Display, 13 s 50 60 60 135 148 162
Display, 17 s 50 60 60 402 456 506
Window, 41 s 81 120 120 619 619 653

The window recording shows no measurable change at mpdecimate's defaults. With stricter thresholds, encoder noise outweighs the signal (both exports count above the ideal), so that row is inconclusive, not a regression.

Tests

  • capture_fps_* (enc-ffmpeg) cover:
    • bursts on a 60 fps grid averaging under 12 fps (expects 60);
    • 30 fps content with holds and one 12-frame 60 fps scroll (expects 60);
    • 30 fps with every seventh frame 12, 25 or 30 ms late (expects 30);
    • one change a second captured as two frames 1/60 s apart (expects 60);
    • uniform jitter of ±3, ±2 and ±1 ms at 30, 60 and 120 fps;
    • jittered bursts;
    • 30 fps on a 120 Hz grid with holds (expects 30);
    • a non-standard 37 fps;
    • too few frames;
    • an encoded variable-rate mp4.
  • track_heal tests:
    • an average fps heals to 60, and a second run is a no-op;
    • a recorded fps of 60 on the same bursty track is left alone;
    • a steady 30 fps track at its average is left alone.
  • Mutations:
    • with the estimator replaced by the average rate, 4 of the original 6 capture_fps_* tests and the heal test fail;
    • starting from the most common gap fails the 12-frame scroll test;
    • counting gaps without the run requirement fails the late-frame test;
    • dropping the between-holds rule fails the paired-frames test;
    • dropping the 100 ms hold minimum fails the 25 ms late-frame test (reads 120).
  • cargo test -p cap-enc-ffmpeg --lib (166 passed), cargo test -p cap-recording --lib (668 passed), clippy clean for the touched crates and cap-desktop-gpui.

RetriggerView in GreptileConfidence Score: 5/5

No new independent blocking finding was identified in the changes since the previous review.

Summary

Repairs saved Studio display frame rates without changing the media.

  • Video timestamps reveal the rate at which a display was captured.
  • Editors restore the captured rate in older Studio recordings.

All supplied previous threads are unnumbered, so they receive no numbered previous-finding entries.

Reviews (4) · Last reviewed commit: "fix(enc-ffmpeg): a hold beside a short f..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 18, 2026 12:28am UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

Comment thread crates/enc-ffmpeg/src/remux.rs Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

Comment thread crates/enc-ffmpeg/src/remux.rs
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/enc-ffmpeg/src/remux.rs

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

Comment thread crates/enc-ffmpeg/src/remux.rs
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy
richiemcilroy merged commit dc35416 into main Oct 18, 2026
57 checks passed

This branch was successfully deployed

2 active deployments
Preview – cap-web — 763b344e Deployed Oct 18, 2026 by vercel[bot]
Preview – cap-status — 763b344e Deployed Oct 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant