Skip to content

feat: add PocketStation application audio capture - #82

Open
Raphjacksun7 wants to merge 1 commit into
harshitboots:mainfrom
Raphjacksun7:feat/pocketstation-audio
Open

feat: add PocketStation application audio capture#82
Raphjacksun7 wants to merge 1 commit into
harshitboots:mainfrom
Raphjacksun7:feat/pocketstation-audio

Conversation

@Raphjacksun7

Copy link
Copy Markdown

What does this PR do?

Adds PocketStation as an optional recorder for one desktop application or the default microphone. ScreenSight's existing soundcard system-output recorder remains the default.

Users can now run:

pip install 'screensight[pocketstation]'
export SCREENSIGHT_ENABLE_AUDIO=1
screensight on
screensight capture-audio --source application --application Zoom

The same choices are available through screen_capture_audio. Application selection accepts an exact display name, bundle:<id>, or pid:<number>.

This extends the audio work from #4 without changing screenshot capture, CaptureOutcome, watch mode, MCP registration, or the existing system-audio implementation.

Type of change

  • Bug fix
  • New feature
  • Capture backend change (macOS / Linux / Windows)
  • Docs / examples
  • Tests / tooling

Testing

  • pytest tests/ -v passes locally
  • I added or updated tests for this change

Platform(s) I actually ran this on:

  • macOS 26.6.2 arm64, Python 3.12
  • Published pocketstation==0.1.3 installed from PyPI
  • Selected afplay by process ID: 48 kHz stereo float WAV, 2.005 s, audible signal, SHA-256 5525a0634a2a7587a18b5463178f2d92f8dfb6eb6eecfd99a886854efb69ab29
  • Physical default microphone: 48 kHz mono float WAV, 2.000 s, audible signal, SHA-256 5e6fd81bbba753155bf49e8ca53a72e9afc8985c3d15bb1195811cb56390d5cd
  • Windows and Linux are implemented through the same published PocketStation API but were not rerun inside ScreenSight for this PR.
  • ScreenSight's existing macOS system-output recorder still requires a virtual loopback device and was not physically retested.

Checks run:

  • 41 passed, 22 WSL-only tests skipped on macOS
  • Ruff lint and format
  • Repository MyPy hook
  • strict MkDocs build
  • wheel and sdist build
  • clean install from the built wheel with the pocketstation extra

The repository's existing generic check-yaml hook cannot parse the Python tags used by mkdocs.yml; the strict MkDocs build passes.

Privacy invariants

  • Every screenshot still routes through core.capture_once().
  • The master switch remains in core.py; audio still requires SCREENSIGHT_ENABLE_AUDIO=1 as a second explicit opt-in.
  • Blocklist and redaction logic are unchanged.
  • No new screenshot or audio history accumulates. PocketStation records in a temporary directory, copies the selected stem to the existing audio.wav, and removes the temporary recording.
  • System capture remains available when PocketStation is not installed. App and mic capture return an installation hint instead of changing the existing recorder.
  • PocketStation is an optional dependency and is loaded only when app or mic capture is requested.

Notes for reviewers

One --source option keeps the behavior explicit:

  • system: existing soundcard recorder
  • application: PocketStation, with --application
  • microphone: PocketStation's default input selection

screensight off continues to delete the single audio.wav file.

@harshitboots

Copy link
Copy Markdown
Owner

Thanks! this looks close, but I’d like one more round of validation before merging:

  1. Please verify the PocketStation output contract in a real run, especially the expected WAV path/stem naming for application and microphone capture. The current implementation assumes stems/{source}.wav, which may be too brittle if the library layout differs.
  2. If PocketStation doesn’t guarantee that path, please switch to using the library’s returned recording path or add a safer fallback/validation when the file is missing.
  3. It would also help to add one integration-style test that exercises the real PocketStation output shape, not just mocked API behavior.
  4. Finally, please double-check the docs/CLI/MCP wording stays consistent around --source, --application, and the optional extras (screensight[audio] vs screensight[pocketstation]).

Repository owner deleted a comment from hkbritcore Sep 9, 2026
@himanshu231204

himanshu231204 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Review: PocketStation application audio capture

Reviewed across code quality, error handling, tests, comments, and types. Checked out the branch and confirmed all 11 audio tests pass locally. Solid, well-scoped feature — privacy invariants are respected (dual opt-in gate unchanged, temp dir cleaned via context manager, single reused audio.wav, optional dep imported lazily), error handling follows the fail-closed convention, and the existing system path is genuinely untouched. No critical issues. A few things worth addressing before merge.

Important

1. Exit code 3 now conflates the privacy gate with user/dependency errors__main__.py, docs/reference/cli.md
Previously exit 3 meant "opt-in gate not satisfied." The new failure modes — invalid --source, missing --application, PocketStation not installed, ambiguous app — all funnel through the same outcome.ok is False → sys.exit(3), but the CLI docs still describe 3 as the gate code only. A scripting consumer can't distinguish "blocked for privacy" from "you typo'd the app name." Either document 3 as the generic capture-failure code, or split user-input errors (e.g. 2) from the privacy gate.

2. Implicit contract between the source string and the stem filename is undocumentedcapture/audio.py

stem.record(source)
...
captured = Path(recording.session_directory) / "stems" / f"{source}.wav"

The code passes the source category ("application"/"microphone") as the PocketStation stem label, then relies on PocketStation writing that label to stems/<label>.wav to read it back. This coupling to an external API isn't explained by a comment. If PocketStation ever sanitizes/renames stem files, captured.is_file() silently fails with a misleading "did not create the recording." Add a one-line comment stating the invariant you depend on.

Suggestions

3. Module docstring is stalecapture/audio.py:1-17
The header still says "System-audio (loopback) capture" and documents only soundcard, but a public record_pocketstation_audio now lives in the same module. Add a sentence covering both recorders.

4. channels=1 request vs. observed stereo outputcapture/audio.py
Session(..., channels=1) is hardcoded, but the PR testing notes show application capture produced a 2-channel WAV. _read_wave_format correctly reports the actual channels, so output is accurate — but the hardcoded channels=1 is misleading since it isn't honored for app capture. A comment noting it's a hint the backend may override (or dropping it) would help.

5. Test coverage gapstests/test_audio.py
Happy paths and selector parsing are well covered. Missing behavioral tests for the new failure branches:

  • ImportError_POCKETSTATION_HINT
  • recording is None / not recording.complete → "could not finish"
  • captured.is_file() false → "did not create the recording"
  • invalid selectors: empty bundle:, non-decimal/negative pid:, pid:0
  • microphone real recording path (only the mocked core route is tested, not the stems/microphone.wav copy)

These are cheap pure-logic branches and exactly the paths a user hits when something goes wrong.

6. _read_wave_format only tested against the extensible header — the plain 16-bit PCM fmt path isn't directly asserted. Worth one small test for the standard PCM header the reader also supports.

Strengths

  • Lazy import pocketstation keeps the optional dep off the core screenshot path.
  • tempfile.TemporaryDirectory guarantees no stray recordings accumulate.
  • _read_wave_format handles both PCM and WAVE_FORMAT_EXTENSIBLE, including odd-chunk padding.
  • pid: validation is careful (isascii() before isdecimal() guards Unicode-digit spoofing; rejects <= 0).
  • Consistent fail-closed error handling matching record_system_audio.

Recommended action

#1 and #2 are the two I'd want addressed before shipping (both small). #5/#6 give the most confidence per effort. Nothing here blocks on correctness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants