Stress test and correctness check at real scale

[fastermadman]

Model: Sonnet 5 · Effort: high
Mechanical thoroughness across several bounded scenarios, not architectural judgment — wide surface, not deep reasoning. Every individual check is procedural: generate data, run it, measure or assert.

Role

You are a stress-tester and correctness checker, not the implementer. Every audio file used to test this codebase so far has been 1-second ffmpeg-generated silence, in fixtures of 1–3 tracks. Nothing has been tested at the scale or realism a real DJ library implies. Assume nothing works until you've made it fail or confirmed it holds.

Context (read first — assume no prior knowledge of this project)

beetsGUI is a local web app for beets, the command-line music library manager, running as a Safari Web App on macOS. No build step, no npm, no framework:

• server.py — Flask on http://localhost:1312. Key endpoints for this issue: GET /library (SQLite-backed search, read-only), GET /unimported (scans a folder, opens every audio file with mediafile.MediaFile() to compare artist/title against the library — see _known_track_keys()), POST /import/start + SSE /import/<id>/events + POST /import/<id>/decide (the in-process importer).
• importsession.py — WebImportSession, quality_rank() (lossless-vs-lossy duplicate ranking), ImportJob (one import at a time, queue.Queue-based decision handshake).
• beetsgui.html — single-file UI. The Inbox tab's "Scan for unimported music" calls /unimported; the decision queue is keyboard-driven.
• test_importsession.py — the existing end-to-end suite. Read it before writing new tests; it already has the harness pattern you'll want to copy (start_server()/stop_server() with BEETSDIR pointed at a temp dir, a stub MusicBrainz-alike plugin so imports don't hit the real network).

Absolutely do not touch

~/.config/beets or anything under /Volumes — the user has a real, live beets setup on external drives. Every test in this issue must run against a throwaway BEETSDIR in a temp directory, exactly like test_importsession.py already does. If you need a large test library, generate it — do not point any test at real user data.

Scope

1. Stress test at realistic scale

• Populate a throwaway library.db with 5,000+ album rows (you can insert directly via sqlite3 matching the schema — no need to actually import 5,000 real files) and measure GET /library?q=... latency for a few representative queries (empty query, artist match, title match via the EXISTS subquery).
• Create a scan folder with 2,000+ real (see §2) audio files and time GET /unimported end to end. It currently opens every file synchronously with no progress reporting — find out whether this is "a few seconds" or "long enough to look hung." If it's the latter, that's a finding, not something to quietly work around.
• Run a real import of 10+ albums back to back through the actual /import/* flow (not just test_importsession.py's existing single/few-album cases) and confirm no decision is lost, no double-application happens, and the final done event's decided count matches reality.

2. Real audio, not silence

Generate test fixtures with actual encoded variation — ffmpeg can encode real tones/noise at different bitrates and bit depths, which is enough to exercise the encoder paths that pure silence may special-case. Specifically verify quality_rank() in importsession.py against:

• Real MP3 at a few different bitrates (128/192/320kbps)
• Real FLAC and WAV at 16-bit and 24-bit
• A real ALAC (m4a) file
  Confirm the lossless/lossy ranking and the bitdepth-cap-at-24 logic (_MAX_RANKED_BITDEPTH in importsession.py) behave correctly against real encoder output, not just the format string.

3. Unicode and edge-case metadata

Artist/album/title containing Danish characters (æ/ø/å), other non-ASCII (emoji, CJK), embedded quotes, and very long strings — through the full pipeline: file tags → /unimported matching → import decision → /library search → the generated config.yaml. Also test paths with spaces, unicode directory names, and directory names containing shell metacharacters ($, backticks, ;) landing in any build*Cmd()-generated string in beetsgui.html.

4. Missing-dependency behavior

Temporarily rename or PATH-hide fd, ffmpeg, and xld (or run in an environment without them) and exercise every feature that depends on each. Document what actually happens — a clear error is fine, a silent no-op or a confusing partial failure is a finding.

5. Interrupted server recovery

Start an import, let it reach a decision, then kill -9 the server process mid-decision. Restart it against the same BEETSDIR. Confirm: state.pickle's resume behavior still works via the native resume-decision (see WebImportSession.should_resume and test_importsession.py's bullet 7), the library isn't left in a half-written state, and GET /import/current correctly reports no running job.

Acceptance

• /library search stays under 1s against a 5,000-album database, for the query patterns above — report actual numbers, not "seemed fine."
• /unimported against 2,000+ files either completes in a reasonable time or the issue explicitly documents how long, with a recommendation (progress reporting? background job?) if it's too slow to ship as-is.
• A 10-album real import completes with zero unhandled exceptions and a decided-count that matches what was actually decided.
• quality_rank() correctly ranks real encoded files at each of the tested bitrates/bitdepths/formats — show the actual ranks computed, not just pass/fail.
• No unicode corruption anywhere in the pipeline (file tags → scan → import → library search → generated config).
• Every build*Cmd() string correctly escapes or rejects shell-metacharacter-bearing paths, or the issue documents which ones don't and why that is/isn't a problem given /run's existing whitelist (cross-reference the parallel security-review issue if that lands first).
• Missing-dependency behavior is documented per dependency (fd, ffmpeg, xld) with a clear-error/silent-failure verdict for each.
• Server survives a hard kill mid-decision and recovers cleanly on restart, confirmed by re-running the exact sequence from test_importsession.py's resume test (bullet 7) after the kill.
• New coverage lands in test_importsession.py following its existing harness pattern, not a disposable one-off script — the whole point is these checks survive to catch regressions later.

Traps

• mediafile.MediaFile() per file in /unimported is almost certainly the bottleneck at scale — profile before assuming, don't guess.
• Real ffmpeg-encoded MP3 at very low bitrates can produce unusual bitrate/bitdepth values from mediafile — verify actual values before asserting expected ranks.
• import:write: no is used throughout test_importsession.py's fixtures to avoid tag-writing side effects during testing — keep that convention so your new tests don't silently start writing tags to fixture files in ways that mask bugs.
• The existing test suite spins up server.py as a subprocess and polls /status — reuse start_server()/stop_server() rather than writing a new harness.