Skip to content

feat: Capture a screen region with a global hotkey and copy the recog…#4701

Open
adfastltda wants to merge 4 commits into
ramensoftware:mainfrom
adfastltda:mods/selection-text-ocr
Open

feat: Capture a screen region with a global hotkey and copy the recog…#4701
adfastltda wants to merge 4 commits into
ramensoftware:mainfrom
adfastltda:mods/selection-text-ocr

Conversation

@adfastltda

@adfastltda adfastltda commented Jul 6, 2026

Copy link
Copy Markdown

…nized text to the clipboard.

Changelog

If this pull request updates an existing mod, describe the changes below:

  • Changelog item 1...
  • Changelog item 2...

Mod authorship

If this pull request introduces a new mod, please complete the section below.

This mod was created by:

    • The submitter, without AI assistance
    • The submitter, with AI assistance
    • Claude
    • ChatGPT
    • Gemini
    • Another AI (please specify):
    • Other (please specify):

Please select the options that best apply. Your selection does not affect the acceptance criteria, but it helps reviewers understand the context of the code and provide relevant feedback.

Adfast Ltda added 4 commits July 6, 2026 14:01
esc to cancel and global hotkey
…h "no language pack" from "no text found" and avoid copying error output to the clipboard.
@m417z

m417z commented Jul 7, 2026

Copy link
Copy Markdown
Member

Submission review

Note: This review was done by Claude, and then refined manually. Due to the amount of submissions, doing a fully manual review for each pull request is no longer feasible. Thank you for understanding.

Please address the following issues. The items in the collapsed sections are optional, so it's your call whether to address them.


1. Use the exact copy of the code from the wiki for easier review. https://github.com/ramensoftware/windhawk/wiki/Mods-as-tools:-Running-mods-in-a-dedicated-process

The established pattern for a hotkey tool mod is window-center-resize.wh.cpp: spawn one worker thread in WhTool_ModInit, and have that thread call RegisterHotKey(nullptr, …) and run the GetMessage loop; keep EntryPoint_Hook as the wiki's ExitThread(0); and route settings reloads via PostThreadMessageW(tid, WM_APP_RELOAD, …) (and teardown via PostThreadMessageW(tid, WM_APP_EXIT, …)) rather than touching the window/hotkey from the uninit/settings thread.

2. OCR is run by spawning powershell.exe with a generated script — prefer calling the WinRT OCR API directly. For every capture the mod writes a .bmp and a .ps1 to %TEMP%, launches Windows PowerShell 5.1, Add-Types several WinRT assemblies, and parses stdout. That's slow (PowerShell + assembly load is on the order of seconds per capture), fragile (depends on Windows PowerShell 5.1 being present and on the WindowsRuntime interop assembly, execution-policy behavior, etc.), and leaves temp files on disk. The same Windows.Media.Ocr API is reachable directly from the mod via C++/WinRT, which the toolchain supports — see taskbar-clock-customization.wh.cpp (#include <winrt/Windows.Media.Control.h>, -lruntimeobject). Calling OcrEngine::TryCreateFromLanguage / RecognizeAsync on a SoftwareBitmap built from the captured pixels removes the subprocess, the temp files, and the PowerShell dependency entirely. This is a larger change, but it's the right architecture for the mod and avoids the workaround.

3. Add a screenshot/GIF to the README. This is a visual mod (drag-to-select overlay). A short GIF of the selection + result would make the catalog entry much clearer. Only i.imgur.com and raw.githubusercontent.com are allowed image hosts.

4. Default ocrLanguage to en-US rather than pt-BR. The catalog convention is English-first defaults; pt-BR reads as a personalization. Functionally it already falls back to the user's profile languages when the requested engine is unavailable, so switching the default to en-US costs nothing and matches convention.

Optional improvements

Minor polish — none of this affects users, so it's your call.

  • -lole32 is unused. The mod does no COM (clipboard/GDI are user32/gdi32, and OCR goes through PowerShell), so drop -lole32 from @compilerOptions. -lgdi32/-lmsimg32 (AlphaBlend) are used and should stay.
  • Redundant NULL checks on Wh_GetStringSetting. It never returns NULL (it returns L"" on unset/error), so in if (language && language[0]) the language && half is dead; language[0] alone suffices. The ParseModifiers/ParseSingleCharKey NULL guards are likewise unreachable.
  • Uninit can block while a selection is open. WhTool_ModUninitDestroyHotkeyWindow() does WaitForSingleObject(g_captureThread, INFINITE); if the mod is disabled while the drag-select overlay is up, uninit waits until the user finishes or presses Esc. Signal the in-progress capture to cancel first (e.g. post WM_CLOSE to g_captureState.hwnd) or use a finite timeout, so disabling the mod is always prompt.

Functionality notes

Non-critical observations about the feature behavior itself.

  • Per-monitor DPI. The capture and overlay use SM_*VIRTUALSCREEN and BitBlt with no DPI handling. If the dedicated windhawk.exe process isn't per-monitor-DPI-aware, on high-DPI/mixed-DPI setups the screenshot is grabbed at the scaled (lower) resolution — blurry input that hurts OCR accuracy — and the overlay geometry can be off. Consider SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) early so the capture is at native resolution.
  • The screen is captured twice. CaptureRegion grabs a full screenshot for the overlay (g_captureState.screenshot, a clean desktop image — the selection rectangle is drawn only into the temporary frame bitmap, not into it), and then ProcessCapturedRegion calls CaptureScreenBitmap again after the overlay is destroyed. Cropping the region out of the already-captured g_captureState.screenshot (before WM_DESTROY frees it) would be faster, would capture exactly what the user saw when selecting, and avoids a small race where the desktop may not have repainted after the overlay is dismissed (risking the dim/border pixels leaking into the re-capture).
  • Small selections. Windows.Media.Ocr accuracy drops on small/low-DPI text; upscaling the cropped bitmap (e.g. 2×) before OCR often improves recognition noticeably.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants