Slow nix store garbage collection

[chrissound]

Is there some way to make garbage collection faster for the guest vm?

My understanding is it's going through my entire /nix/store from the host, or likely doing a ton of IO via the 9p mount which is very slow.

I've tried various scripts that look at /nix/.rw-store/ but not having much luck.

[chrissound]

I've used this script to attempt to hack around it... I think I've used this approach before and ended up with a corrupt nix store though.

#!/usr/bin/env bash
set -euo pipefail

# Nuke dead upper-only nix store directories directly from the overlay.
# Bypasses nix-store --delete (which is glacially slow over 9p).
#
# Safe because:
# - Only removes dirs that exist ONLY in upper (not in lower/host store)
# - Only removes dirs with NO GC roots
# - Files are skipped (only ~4MB total, not worth the query time)

UPPER=/nix/.rw-store/upper
LOWER=/nix/.ro-store
MODE="${1:-}" # --dry-run or empty for real run

echo "=== BEFORE ==="
du -sh "$UPPER"
df -h /
echo

# Collect dead upper-only directories
echo "Finding dead upper-only dirs (this takes ~2 min for root queries)..."
dead=()
skip_lower=0
skip_live=0

for d in $(find "$UPPER" -mindepth 1 -maxdepth 1 -type d -printf '%f\n'); do
  if [ -e "$LOWER/$d" ]; then
    skip_lower=$((skip_lower + 1))
    continue
  fi
  roots=$(nix-store --query --roots "/nix/store/$d" 2>/dev/null || true)
  if [ -n "$roots" ]; then
    skip_live=$((skip_live + 1))
    continue
  fi
  dead+=("$d")
done

echo "  In lower (skip): $skip_lower"
echo "  Live roots (skip): $skip_live"
echo "  Dead (will nuke): ${#dead[@]}"
echo

if [ "${#dead[@]}" -eq 0 ]; then
  echo "Nothing to clean!"
  exit 0
fi

if [ "$MODE" = "--dry-run" ]; then
  echo "DRY RUN — would delete ${#dead[@]} dirs. Run without --dry-run to execute."
  exit 0
fi

# Nuke 'em
echo "Removing ${#dead[@]} dead dirs from upper..."
removed=0
for d in "${dead[@]}"; do
  rm -rf "$UPPER/$d"
  removed=$((removed + 1))
  [ $((removed % 50)) -eq 0 ] && echo "  ... $removed / ${#dead[@]}"
done
echo "  Removed $removed dirs."
echo

# Verify/repair nix DB (removes stale entries for now-missing paths)
echo "Repairing nix DB..."
nix-store --verify --repair 2>&1 | tail -3
echo

echo "=== AFTER ==="
du -sh "$UPPER"
df -h /
echo "Done!"

[Mic92]

harmonia has a faster gc. Not sure how much it helps. We also switched to virtiofs.