RetroLens Hand Portal

Real-time webcam hand tracking (OpenCV + MediaPipe) that turns your fingers into a portal: a window you hold open in mid-air, through which the live camera feed is re-rendered with a retro visual filter.

How the portal works

The app mirrors your webcam feed and tracks up to two hands. From every hand it sees, it takes exactly two landmarks — the thumb tip (landmark 4) and the index fingertip (landmark 8) — and draws a small cyan dot on each. Those points are the corners of the portal, so the portal's shape depends entirely on how many hands are in frame:

Hands in frame	Corner points	Portal shape
1	2 (thumb + index)	Axis-aligned rectangle spanning the two tips
2	4 (thumb + index on each hand)	Free-form quadrilateral with animated edge particles

Only the pixels inside the portal are filtered — everything outside stays as the raw camera feed. Move your fingers apart to open the portal wider, bring them together to close it.

Two-point box portal (one hand). The thumb and index tips define opposite corners of a rectangle. The region is filtered in place, outlined in yellow, and labelled PORTAL: <FILTER> above the top edge. The box is ignored until it is larger than 15×15 px, so a closed pinch simply turns the portal off.

Four-point quad portal (two hands). With four points available, the app sorts them into a top pair and a bottom pair (by y, then by x within each pair) and builds a proper quadrilateral. That polygon can be tilted and skewed — rotating or offsetting your hands warps the portal's shape live, rather than snapping it back to an upright rectangle. The filter is applied to the polygon's bounding box, then masked with fillPoly so only the quad itself shows the effect. The outline is drawn in white and each of the four edges is sprinkled with randomly jittered cyan particles every frame, giving the portal rim a shimmering, energized look.

Visual filters

Ten filters cycle in this fixed order:

#	Filter	Effect
1	MONO	Grayscale.
2	DUAL-TONE	Hard luminance threshold mapped to two flat colors — orange highlights, pink shadows.
3	PIXELATE	Downscales the region 10× and scales it back with nearest-neighbour, for chunky retro pixels.
4	INVERT	Photographic negative.
5	SEPIA	Classic warm sepia color matrix.
6	BLUR	25×25 Gaussian blur.
7	THERMAL	COLORMAP_JET false-color, like a thermal camera.
8	SKETCH	Pencil-sketch: inverted, blurred, and color-dodged into line art.
9	GLITCH	Horizontal RGB channel-shift (red and blue displaced in opposite directions).
10	NEON	Canny edge detection, recolored cyan and dilated into a glowing neon outline.

The active filter name is shown in the HUD and repeated as a label on the portal itself.

Gesture controls

Gesture / key	Action
Pinch: thumb + index (one hand)	Opens the 2-point box portal.
Frame with both hands: thumb + index on each	Opens the dynamic 4-point quad portal with edge particles.
Tap: thumb + pinkie (either hand)	Cycles to the next filter.
Touch both index fingertips together	Also cycles to the next filter.
q	Quit.

Both filter-cycling gestures fire when the two fingertips come within 40 px of each other, and both are edge-triggered: holding the tap closed advances exactly one filter, and you must separate the fingers before the next tap registers. Because filter cycling and the portal both read from the same hands, a thumb-to-pinkie tap changes the filter without disturbing the portal your thumb and index are holding open.

Setup

Requires Python 3.9–3.12 and a working webcam. MediaPipe is pinned to 0.10.14 (see Notes), which has no wheel for Python 3.13+ — so the system interpreter on a current macOS install will not work; use an explicitly versioned one.

python3.11 -m venv venv        # any interpreter in the 3.9-3.12 range
source venv/bin/activate       # Windows: venv\Scripts\activate
pip install -r requirements.txt

Running

python main.py

The app opens a window titled RETROLENS Portal Tracker on webcam device 0. There are no command-line flags — camera index and settings are set in main.py. Press q in the window to exit; the capture device is released and the window closed on the way out.

Notes

• Classic MediaPipe Solutions API. This app uses the legacy mp.solutions.hands API, pinned via mediapipe==0.10.14 in requirements.txt, specifically for native stability on macOS. Later MediaPipe releases (0.10.30+, 1.0.x) drop Solutions in favour of a Tasks API (HandLandmarker) whose macOS build dies with a hard, unrecoverable process abort — Check failed: service_ Service is unavailable inside DrishtiMetalHelper — regardless of BaseOptions(delegate=CPU). The Solutions API has no such GPU/Metal dependency and needs no separately downloaded .task model file. Hand detection runs with max_num_hands=2 and 0.7 detection/tracking confidence.
• main.py is fully self-contained — capture loop, filters, and gesture handling all live in that one file. The core/ and filters/ packages are left over from an earlier modular version and are not imported by the current app.
• main.py also generates a starfield "galaxy" background image at startup. It is not composited into the output in the current version.