Vocabulary cache staging file: pid-based name collides across containers, and leaks when the rename fails

[lstein]

Summary

The vocabulary cache's atomic-write staging file is named after os.getpid(), and it is never cleaned up if the rename fails. Two consequences:

1. PIDs are not globally unique. Two processes sharing a db_dir — two containers on the same volume (each app is PID 1), or two hosts on an NFS databases directory — compute an identical staging path and can corrupt each other's output. That is the exact failure the code comment says the staging file prevents.
2. A genuine os.replace failure leaks the staging file every run, with no try/finally unlink.

Found during adversarial review of #111. Related: #119 (lock scope), #120 (stale caches never pruned).

Where

invokeai/app/services/image_index/image_index_default.py::_build_vocab_embeddings:

# Written aside and renamed: two processes sharing a db_dir can
# first-run at once, and a kill mid-write would otherwise leave
# a truncated archive that costs another full re-embed.
staging_path = cache_path.with_name(f"{cache_path.name}.{os.getpid()}.tmp.npz")
np.savez(staging_path, embeddings=embeddings, fingerprint=np.str_(fingerprint))
os.replace(staging_path, cache_path)
except Exception:
    ...warning...

_vocab_lock gives no cross-process protection, so the comment's multi-process claim rests entirely on the pid being distinct.

1. Colliding PIDs publish a corrupt cache

Reproduced at the fd level: process A opens the staging path and is mid-write; process B (same pid in its own namespace) np.savez to the same path, truncating that inode, then os.replace publishes it as cache_path. A's remaining writes land inside the already-published file; A's own os.replace then fails on the staging path that no longer exists.

B published: 5204752 bytes
A's replace -> FileNotFoundError   (swallowed by the handler)
*** PUBLISHED CACHE IS UNREADABLE -> BadZipFile: Bad CRC-32 for file 'embeddings.npy'

Severity is capped because it self-heals: the next run's np.load raises, hits "Discarding unreadable cluster vocabulary cache", and rebuilds. But it costs a full re-embed and the stated contract does not hold.

2. Staging file leaks when the rename fails

With os.replace raising PermissionError, the directory is left holding only cluster_vocab_….<pid>.tmp.npz and no cache file — so the run both leaks ~5 MB and persists nothing, repeating every restart.

A concrete Windows trigger: instance A is inside with np.load(cache_path) (the NpzFile holds the handle open) while instance B calls os.replace onto it. Python opens without FILE_SHARE_DELETE, so Windows raises PermissionError. That reproduces #111's headline symptom — cache never persists, one orphan per run — on Windows multi-instance setups.

EXDEV is not reachable, since staging and target share a directory by construction.

Also worth noting

• Orphans from before fix(image-index): persist the cluster vocabulary cache #111 are not reclaimed. A leftover <cache>.<pid>.tmp.npz survives a successful build. It is inert (nothing globs that directory) but costs 5.1 MB per run of the buggy build. A one-off sweep would clear them.
• No fsync. SIGKILL is survived as the comment claims (page cache), but power loss on ext4 data=writeback/XFS/btrfs can make the rename durable before the data, publishing a zero-length or partial archive. Recovery is the same discard-and-rebuild path, so this is low severity — but the comment's durability framing overreaches.

Suggested direction

Use tempfile.mkstemp(dir=cache_path.parent, suffix=".npz") or append uuid4().hex instead of the pid, and unlink the staging file in a finally. The dir= argument matters: staging in the system temp dir would put it on a different mount from db_path.parent in many deployments, where os.replace raises EXDEV every run and silently reproduces #111.

If the multi-process case is judged out of scope, the alternative is to weaken the comment so it does not claim a guarantee the code does not provide.