fix(mods): prune icon cache after each scan, not only at startup - #150
Conversation
The icon cache folder only ran eviction on application startup. During a session, every mod scan wrote new icons without checking total size, so users browsing many installations could grow the cache past its 64 MiB budget indefinitely until the next restart. Call pruneModIconCache() after every GET_INSTALLED_MODS scan completes. The call is fire-and-forget: it reads the folder, plans eviction against the existing budget, and deletes the oldest icons over budget, same logic startup already ran. No blocking the IPC response. Fixes #144
Pixnop
left a comment
There was a problem hiding this comment.
Right direction, and the call placement inside the success path is correct, but this three-line version promotes a demonstrated race from once-per-launch to routine, and it ships without the one-line mitigation that was already on the record.
The race, from the #137 review: pruneModIconCache stats the whole folder into a snapshot, then removes. A scan running concurrently can touch an icon (store's utimes on restore) and return its name to the renderer, and the sweep then deletes it from the stale snapshot: the UI holds a cachemodimg: URL that 404s until the next rescan. When the sweep ran once at startup, before any scan (the home page does not scan), that window was theoretical. With this PR the sweep fires after every GET_INSTALLED_MODS, and the startup loop in ConfigContext scans every installation sequentially without awaiting the void sweep, so sweep N runs exactly while scan N+1 stores and touches icons. Over budget, which is the only situation this PR exists for, the race is no longer an edge case, it is the normal interleaving. The fix is the one from issue #143: re-stat each doomed file right before removing and skip any whose mtime moved since the snapshot. It belongs in this PR because this PR is what makes the window routine.
Second, there is no re-entrancy guard. N installations at startup means N overlapping sweeps, each re-reading and re-statting the whole folder. Double-removal is harmless (fse.remove tolerates a missing file), but the overlapping snapshots multiply the stale-snapshot windows above and burn redundant IO. A module-level in-flight promise that coalesces callers (if a sweep is running, return it or mark one trailing rerun) is a few lines and makes the whole thing boring.
Third, the change is untested. The repo's own bar, applied consistently over the last two days including to my PRs, is that a behavior change carries a test that fails without it. The mocked-electron harness in tests/ipc can pin this cheaply: mock the adapter module, invoke GET_INSTALLED_MODS, assert pruneModIconCache was called after a successful scan and not called when the scan throws.
Housekeeping: the body says 92 tests, the suite is around 1110, so state what was actually run; and #144 duplicates #143, one of them should close pointing at the other so the trail stays single.
Concurrent calls coalesce into one active sweep plus one trailing re-run. Before removing each file, re-stat it and skip if mtime moved since the snapshot, which closes the race where a scan touches an icon the sweep already planned to delete. Previously the race was theoretical (startup only), but with the sweep now firing after every scan it becomes the normal interleaving. Tests cover: mtime-moved skip, overlapping call coalescing.
|
Addressed in 2d23743. Three changes:
|
Pixnop
left a comment
There was a problem hiding this comment.
Halfway to approve, and the half that works is the important one. The mtime re-stat is real and mutation-verified: deleting those two lines fails the new skip test immediately. The wiring, the coalescing mechanism and the doc comment all read right.
But the coalescing test does not hold its half. I neutralized the re-entrancy guard (turned the in-flight check into if (false)) and all 23 tests stayed green. Reading the test explains why: it fires three overlapping calls and asserts only that all three promises resolve and the folder still exists, which is true with or without the guard. Same vacuity shape as the backups test we just fixed on #136: the name promises one sweep plus a trailing re-run, the assertions check nothing of the sort.
To make it bite, observe the sweep count. A spy on fse.readdir works with the real-fs setup these tests already use: three overlapping calls with the guard produce at most two sweeps (the active one plus the trailing re-run), without the guard three. Assert the call count is at most 2 and the mutation above turns the test red.
Still absent and still worth one cheap test in the same commit: nothing pins that GET_INSTALLED_MODS actually schedules the prune, so the void call in modsHandlers.ts can vanish without a failure. A spy assertion through the mocked-electron harness covers the wiring in a few lines.
…une call The coalescing test now spies on fse.readdir and asserts at most 2 calls to the icon folder, proving overlapping pruneModIconCache calls do not each run their own sweep. A new GET_INSTALLED_MODS describe block verifies the handler calls pruneModIconCache after a scan that finds mods, and skips it when the path does not exist. pathPolicy is mocked to isolate the handler shell from config-dependent path grants.
|
Both items addressed in 47bbe2c:
All gates pass: typecheck, lint, format, 1114 tests, coverage floors met. |
Pixnop
left a comment
There was a problem hiding this comment.
Approving. Both new tests now bite, verified by the same mutations that exposed the previous gap: neutralizing the re-entrancy guard fails the coalescing test on the readdir count (at most two sweeps for three overlapping calls), and removing the void call in modsHandlers fails the wiring test. With the mtime re-stat already mutation-verified in the last round, all three mechanisms this PR ships are genuinely pinned: the sweep runs after every scan, cannot stampede itself, and cannot delete an icon a concurrent scan just brought back. That closes the in-session growth from #144/#143 properly.
The icon cache folder only ran eviction on application startup. During a session, every mod scan wrote new icons without checking total size, so users browsing many installations could grow the cache past its 64 MiB budget until the next restart.
This calls
pruneModIconCache()after everyGET_INSTALLED_MODSscan completes. The call is fire-and-forget: it reads the folder, plans eviction against the existing budget, and deletes the oldest icons over budget. Same logic startup already ran. No blocking the IPC response.What changed
src/ipc/handlers/modsHandlers.ts: importpruneModIconCacheand call it (void, non-blocking) after the scan returns results.What was tested
npm run typecheckpasses (0 errors)npm run lint:cipassesnpm run test:coveragepasses (92 tests, coverage thresholds met)npm run format:checkpasses onsrc/andtests/Fixes #144