Skip to content

Add dynamic letterbox crop detection - #1579

Merged
RadicalMuffinMan merged 8 commits into
Moonfin-Client:mainfrom
selmant:feat/dynamic-letterbox-crop
Oct 6, 2026
Merged

RadicalMuffinMan merged 8 commits into
Moonfin-Client:mainfrom
selmant:feat/dynamic-letterbox-crop

Conversation

@selmant

@selmant selmant commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request

Summary

Adds configurable dynamic re-cropping to the encoded-black-bar cropper introduced in #1524. This includes the presentation and subtitle fixes required for crop mode to work correctly across fullscreen and windowed playback.

Related Issues

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Performance improvement
  • UI/UX update
  • Documentation update
  • Build/CI change
  • Other (describe):

Changes Made

  • Adds recrop intervals: once at start, every second, 5 seconds, and 10 seconds.
  • Uses crop-fill only in desktop fullscreen; windowed playback retains the saved fit mode so side content is not clipped.
  • Remaps native mpv subtitle placement into the accepted crop rectangle and restores the prior position when crop is cleared.
  • Makes the player zoom control a Re-crop action while Crop Black Bars is enabled, preventing incompatible live zoom changes.

Platform

  • Android
  • Android TV
  • iOS
  • tvOS
  • Web
  • macOS
  • Windows
  • Linux
  • All / Shared code

Testing

  • Tested on emulator / simulator
  • Tested on physical device
  • Manual testing completed
  • Not tested (explain why):

Test Steps

  1. Enable Crop black bars in Video Playback Preferences.
  2. Test once-at-start and every-second re-cropping on letterboxed content.
  3. Verify fullscreen fills the cropped picture, windowed playback retains the complete picture, and subtitles remain visible.
  4. Turn crop off and verify the subtitle position returns to its original placement.

Screenshots (if applicable)

Not applicable; this changes playback behavior without adding a new UI surface.

Checklist

  • Code builds successfully
  • Code follows project style and conventions
  • No unnecessary commented-out code
  • No new warnings introduced

@selmant

selmant commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@mattsigal This follows up on the dynamic re-scan idea discussed on #1524. It adds configurable re-cropping and includes the fullscreen/windowed zoom and subtitle-placement fixes needed to make crop mode usable. Interested in your thoughts, especially on the crop/zoom interaction.

@github-actions github-actions Bot added the Missing Template Issue opened without one of the issue forms label Sep 17, 2026
@selmant

selmant commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

More specifically on subtitles: on desktop libmpv, native bottom-aligned subtitles can still use the original uncropped frame coordinates. After the lower letterbox bar is removed, that can place them outside the retained picture. This PR remaps the subtitle position into the accepted crop rectangle while crop mode is active, then restores the exact previous position when it is cleared.

@github-actions github-actions Bot added Android Android TV Bug Something isn't working Feature Request New feature or request Linux Windows and removed Missing Template Issue opened without one of the issue forms labels Sep 17, 2026
@selmant
selmant marked this pull request as draft September 17, 2026 19:35
@selmant
selmant marked this pull request as ready for review September 17, 2026 19:38
@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

✅ Build Successful

All platform builds and tests passed. You can download the test artifacts below.

Tests ✅ 4008 passed, analyzer clean

Platform Status Artifact
Android ✅ Passed Moonfin_Android_v* + Moonfin_AndroidTV_v*
iOS ✅ Passed Moonfin_iOS_v*_unsigned.ipa
macOS ✅ Passed Moonfin_macOS_v*.dmg
tvOS ✅ Passed Moonfin_tvOS_v*_unsigned.ipa
Windows x64 ✅ Passed Moonfin_Windows_v*.exe
Windows ARM64 ✅ Passed Moonfin_WindowsARM64_v*.exe
Linux x64 ✅ Passed Moonfin_Linux_v* (deb/rpm/AppImage/snap/flatpak/tar.gz)
Linux ARM64 ✅ Passed Moonfin_LinuxARM64_v* (deb/rpm/AppImage/snap/flatpak/tar.gz)
Property Value
Commit 9eabda1
Workflow run Build #1632

@selmant
selmant marked this pull request as draft September 19, 2026 20:18
@selmant
selmant marked this pull request as ready for review September 24, 2026 22:18
@selmant

selmant commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Optimized the scan so it no longer restarts hardware decode.

The every-second check downloads a few frames with hwdownload and runs cropdetect on the decoder that is already running (VAAPI, Vulkan, D3D11, VideoToolbox, or nvdec), then takes the filter back out. 8-bit frames download as nv12 and 10-bit frames as p010le, so a 10-bit file does not abort the graph. Copy-back is only the fallback, and it stays on the same accelerator instead of jumping to a different one.

After the crop is applied, the picture fills the source-sized frame. Fit was painting the removed bars back into that frame, which made a 16:9 screen look unchanged. The debug scan logs are gone.

A short hwdownload runs cropdetect on VAAPI, Vulkan, D3D11, VideoToolbox, and nvdec, then leaves the decoder alone. Copy-back is only the fallback, and a wider display keeps the cropped picture in fit instead of covering it again.
A wider crop now sets panscan so the bars are not painted back into the source-sized frame, and the player no longer prints the letterbox scan trace.
@selmant
selmant force-pushed the feat/dynamic-letterbox-crop branch from f446fc9 to e5ed717 Compare September 24, 2026 22:21
A centred picture with bars on every side is now cropped, confirmed over later reads in one-shot mode so a title card is not. Known ratios read the crop's own shape, seeks account for playback speed, Media3 keeps scanning through long pauses and retries a scan whose captures all failed, mpv serializes crop apply and clear, stops caching panscan and hands it to the Android TV surface, one-shot frame sampling waits out a pending crop, and the zoom mode is honoured when nothing is cropped. Unreachable screenshot-window code and unused constants are gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…op so the cropped picture fits any window and only covers in fullscreen, pace Media3 rescans by capture time, scope the libswscale error filter to a frame sample, and drop the format-only reflows and the panscan workaround
@RadicalMuffinMan
RadicalMuffinMan merged commit c2625a5 into Moonfin-Client:main Oct 6, 2026
1 check was pending
@coderabbitai

coderabbitai Bot commented Oct 6, 2026

Copy link
Copy Markdown

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Repository: Moonfin-Client/Moonfin-Core/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 219c3100-bd1c-467c-babb-01f6aecb5006
📥 Commits

Reviewing files that changed from the base of the PR and between d1244c9 and 9eabda1.

⛔ Files ignored due to path filters (59)
  • lib/l10n/app_localizations.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_af.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ar.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_be.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_bg.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_bn.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ca.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_cs.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_cy.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_da.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_de.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_el.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_en.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_eo.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_es.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_et.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_fa.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_fi.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_fr.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_gl.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_he.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_hi.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_hr.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_hu.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_id.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_it.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ja.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_kk.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_kn.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ko.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_lt.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_lv.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_mk.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ml.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_mn.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_nb.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_nl.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_pa.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_pl.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_pt.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ro.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ru.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_si.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_sk.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_sl.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_sq.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_sr.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_sv.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_sw.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ta.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_te.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_th.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_tl.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_tr.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_ug.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_uk.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_vi.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_yue.dart is excluded by !lib/l10n/app_localizations*.dart
  • lib/l10n/app_localizations_zh.dart is excluded by !lib/l10n/app_localizations*.dart
📒 Files selected for processing (22)
  • lib/l10n/app_en.arb
  • lib/playback/media3_letterbox_crop.dart
  • lib/playback/media3_player_backend.dart
  • lib/playback/media_kit_player_backend.dart
  • lib/playback/mpv_frame_sample.dart
  • lib/playback/mpv_frame_sampler.dart
  • lib/playback/mpv_frame_sampler_native.dart
  • lib/playback/mpv_frame_sampler_stub.dart
  • lib/playback/mpv_letterbox_crop.dart
  • lib/playback/player_key_bindings.dart
  • lib/preference/user_preferences.dart
  • lib/ui/screens/playback/video_player_screen.dart
  • lib/ui/screens/settings/panel/settings_search_index.dart
  • lib/ui/screens/settings/panel/video_playback_screen.dart
  • lib/ui/widgets/keyboard_shortcuts/keyboard_shortcut_reference.dart
  • packages/playback_core/lib/src/letterbox_crop.dart
  • packages/playback_core/lib/src/player_backend.dart
  • pubspec.yaml
  • test/playback/mpv_frame_sample_test.dart
  • test/playback/mpv_letterbox_crop_test.dart
  • test/playback/player_key_bindings_test.dart
  • test/ui/widgets/keyboard_shortcuts/keyboard_shortcut_reference_test.dart
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Android TV Android Bug Something isn't working Feature Request New feature or request Linux Windows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants