Repository navigation
Stop scheduled library probing: reconcile the index from watcher signals - #58
Merged
Merged
Conversation
…probe The reconciliation tick validated every indexed path every five minutes — a stat pass over the whole index 288 times a day, on a NAS or a slow spinning disk the most expensive thing an idle server does every five minutes. It only ever existed as a backstop for event handling that needed fixing sooner, which it now has: deletions, moves out of the library, empty libraries, restart revisions and dropped-event signals all apply to the index directly and publish the change to TVs. The tick now reconciles the index only when the watcher itself reports a problem — a root marked dirty by a queue overflow or a rescan signal — or when the configured full-rescan interval is due. That interval is now disabled by default: the watcher is authoritative, and a mount whose changes happen on another machine is exactly the case the setting documents and leaves up to the operator. OS events, wildcards and startup scans behave as they already did.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The reconciliation tick ran
validate_and_cleanup_deleted_files— a stat pass over every indexed path — unconditionally every five minutes. 288 times a day, and on a NAS or a slow spinning drive each pass opens every file again. This PR removes the schedule: the index is now reconciled only when the watcher itself reports a problem.Why the probe was there, and why it is not needed anymore
The probe was a backstop for event handling that had gaps at the time. Those gaps have since been closed by the preceding work:
publish_content_changebumps the revision, clears cached Browse responses, and sends GENA notifications. What a TV shows changes the moment the OS tells us.a_watcher_signal_reconciles_missing_files_and_notifies_tvs).With all three in place, a scheduled probe only paid the cost without carrying correctness:
full_rescan_interval_hourswas introduced to avoid in Make a large library affordable: flat idle CPU, and a third of the memory #39.Changes
monitoring.rs:validate_and_cleanup_deleted_filesnow runs only underfull_sweep_due || !dirty_roots.is_empty()— the roots the backend reported or the user-configured backstop. Availability checks (read_dir per root) and dirty-root sweeps are unchanged; no per-file stats happen without cause.config/model.rs:full_rescan_interval_hoursdefault changed 24 → 0. The watcher is authoritative; the sweep is the operator opt-in for mounts that cannot deliver events (a network share changing on another machine never will). Explicit configs are untouched,VUIO_FULL_RESCAN_INTERVAL_HOURSstill overrides, and the admin UI description already documents the 0 behavior.cleanup_deleted_filessemantics are unchanged (honored per tick, live).watcher/mod.rs: test-onlymark_roots_dirty()to simulate a backend reporting lost events.events.rs: replaces the test that codified the five-minute probe with two tests —ticks_without_a_watcher_signal_do_not_scan_the_library(interval 0: rows stay, revision stays, cached listings stay warm) anda_watcher_signal_reconciles_missing_files_and_notifies_tvs(dirty root: sweep + reconcile + revision bump + cache retired + refreshed listing,UpdateIDpublished).Validation
cargo test -p vuio-core: 842 passed, 8 ignored, 0 failedcargo fmt -p vuio-core -p vuio-cli -p vuio-cast -p vuio-web --check: cleancargo clippy -p vuio-cli -p vuio-core -p vuio-cast -p vuio-web --all-targets: 0 warnings