Fix watcher rebuild storm: skip re-indexing gitignored directories

[acarl005]

Description

Fixes a bug where the repository file watcher re-indexed entire gitignored directories on every filesystem change. On Windows each re-walk is slow, blocking I/O that runs on the background-executor's worker threads, so the walks saturate the entire background thread pool and starve other background work - most visibly, new terminal sessions bootstrap far more slowly because their startup tasks queue behind the walks.

On a git repository, an incremental watcher update (compute_file_tree_mutations in crates/repo_metadata/src/local_model.rs) called Entry::build_tree_with_standing_queries for every newly-created directory, even gitignored ones. Inside the tree walk, evaluate_entry checks gitignore against the literal path only (check_ancestors = false), so a deep ignored path such as target/debug/.fingerprint was not recognized as ignored and its whole subtree was walked. During a cargo build this re-walked ~27,000 files per event (50-90s each), fanned across the background-executor pool.

The background-executor is a tokio runtime with one worker thread per logical core (num_cpus). On the repro machine (16 logical cores) the dump showed all 16 workers parked in the walk, so the entire runtime was busy and nothing else scheduled on it could run until a walk finished. (Task Manager showed only ~10-20% CPU because the walks are I/O-bound and mostly waiting on disk - that number understates the saturation.)

The fix: in compute_file_tree_mutations, when an added directory is gitignored (using the ancestor-aware path_is_ignored), insert an unloaded placeholder instead of walking it, exactly how the initial index represents ignored directories (IgnoredPathStrategy::IncludeLazy). Force-included paths (e.g. skill-provider dirs) are exempted so they still materialize.

Root cause was found by symbolicating a Windows process dump (all 16 background-executor workers were in repo_metadata::entry::Entry::build_tree_with_force_included_paths_and_ancestor doing filesystem syscalls, starving other background tasks) and confirmed with temporary watcher instrumentation that is removed in this PR.

Created with Warp Agent Mode: https://app.warp.dev/conversation/3ac9dd74-82ae-4401-aea2-ae0e8bfc8b0d

Linked Issue

Closes #13153

Testing

Ran the app locally (OSS build via cargo run) with repositories open and triggered heavy target/ churn with a full cargo build:

• Before the fix: a single change under target/debug/.fingerprint produced full-subtree rebuilds (files=27245 elapsed_ms=53021-style, 50-90s each) that occupied every worker thread; new sessions were slow to bootstrap and background updates lagged.
• After the fix: the same build churned thousands of target/ paths with zero subtree rebuilds; the pool stayed free and sessions/background work were unaffected.

Confirmed with temporary [watcher-diag] logging that was used to capture the before/after and then removed in this PR. cargo check and cargo clippy pass on repo_metadata.

No new automated tests were added; the change brings the incremental watcher path back in line with the initial index's existing ignored-directory handling. Happy to add a compute_file_tree_mutations unit test (ignored dir under a git repo gives an unloaded placeholder, not a walk) if reviewers prefer.

• I have manually tested my changes locally with ./script/run (via cargo run)

Screenshots / Videos

N/A - no UI changes.

CHANGELOG-BUG-FIX: [Windows] Fixed background work (including new terminal sessions) being starved and slow to start, plus sustained disk usage, caused by repeatedly re-indexing gitignored directories (e.g. build output like target/) when many files changed at once.

[oz-for-oss]

@acarl005

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[oz-for-oss]

Overview

This PR changes incremental repo metadata watcher updates so newly-added gitignored directories are represented as unloaded placeholders instead of recursively re-indexing their subtrees, while preserving the force-included path escape hatch.

Concerns

• The force-included exception can materialize a subtree below an ignored parent without preserving the inherited ignored state, so the incremental tree can disagree with the initial index for paths ignored only by an ancestor rule.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[oz-for-oss]

⚠️ [IMPORTANT] This exception preserves materialization for force-included paths, but build_tree_with_standing_queries starts with ancestor_is_ignored = false; when the path is ignored only because of a parent rule such as .agents/, the subtree is inserted with ignored=false, unlike the initial index. Pass the inherited ignored state into the builder and add a regression test for force-included dirs under ignored parents before bypassing the placeholder.