Scan tree, snapshots and what grew since last time

.github/workflows/ci.yml

@@ -65,6 +65,10 @@ jobs:
         run: |
           cargo clippy --all-targets -- -D warnings
           cargo build --release
+      # The UI's pure helpers (formatting, breadcrumbs, treemap layout, diff
+      # indexing) are plain JS with Node unit tests; no npm install needed.
+      - name: UI unit tests
+        run: node --test gui/ui/lib.test.js
 
   deny:
     name: cargo-deny (supply chain)

README.md

@@ -21,7 +21,15 @@ Find where your space went, hunt down byte-identical duplicates, and reclaim the
   target), refuses a drive root, dry-run by default, reports how long it took.
 - **Scan options** — exclude globs (`node_modules`, `*.tmp`), max depth,
   follow-symlinks.
-- **Live progress + cancel** — long scans report files/bytes and can be stopped.
+- **Live progress + cancel** — long scans report files/bytes and can be stopped;
+  a cancelled scan is clearly labelled as a partial result.
+- **One walk, whole tree** — the scan builds an in-memory tree, so the GUI
+  drills into any sub-folder and back up instantly, without touching the disk
+  again. Deleting from the GUI updates the tree in place.
+- **What grew since last time** — save a scan as a snapshot and compare a
+  later one against it: which folders grew or shrank, which are new, which are
+  gone. Every directory is recorded (not just the biggest), so the comparison
+  works at any depth. The GUI keeps a history per scanned folder automatically.
 - **Three faces, one Rust core**:
   - a **CLI** with `--json` output for scripts and agents (headless);
   - a **modern desktop GUI** (Tauri) with a treemap, folder picker, drag & drop
@@ -62,25 +70,42 @@ diskghost rm "D:\old_project\target"            # shows how many files/dirs/byte
 diskghost rm "D:\old_project\target" --apply    # permanently delete (files unlinked in parallel)
 diskghost rm "D:\old_project\target" --trash --apply   # send to the OS recycle bin instead
 #   never follows symlinks (can't escape the target); refuses a drive root
+
+# What grew since last time: save a snapshot, compare against it later
+diskghost scan "C:\Users\me" --save before.json
+diskghost scan "C:\Users\me" --since before.json          # scan + what grew/shrank/appeared/vanished
+diskghost scan "C:\Users\me" --since before.json --save after.json
+diskghost diff before.json after.json                      # compare two saved snapshots
+diskghost diff before.json after.json --at "C:\Users\me\Downloads"   # ...at a sub-folder
 ```
 
+Snapshots are plain JSON, so a scheduled task (cron, Task Scheduler) that runs
+`diskghost scan <dir> --since last.json --save last.json --json` gives you a
+daily "what grew" report with nothing else to set up.
+
 ### Headless / agent mode
 
 Add `--json` to any command for machine-readable output:
 
 ```bash
-diskghost scan . --json          # total_size, children, top_files, root_files, skipped …
+diskghost scan . --json          # total_size, children, top_files, root_files, skipped, cancelled …
 diskghost dupes . --min-mb 10 --json   # duplicate groups with reclaimable bytes
+diskghost scan . --since old.json --json   # the report plus a "diff" object (grown/shrunk/added/removed)
+diskghost diff old.json new.json --json    # just the diff
 ```
 
 Perfect for wiring into an automation/agent that decides what to clean.
 
 ## Desktop app (GUI)
 
 A modern Tauri UI lives in `gui/` — **Browse** or drag a folder onto the window,
-hit **Scan size** for a treemap + biggest folders/files (click a folder to drill
-in), or **Find duplicates** to sort/filter groups and reclaim their space with one
-click. Scans show live progress and can be cancelled. From `gui/src-tauri`:
+hit **Scan size** for a treemap + biggest folders/files (click a tile or a
+breadcrumb to drill in and back up — served from the in-memory tree, so it is
+instant; **Rescan** walks the disk again), or **Find duplicates** to sort/filter
+groups and reclaim their space with one click. Scans show live progress and can
+be cancelled. Every complete scan is kept as a snapshot; pick an earlier one
+under **Since** and **Show changes** colours the treemap and folder bars by what
+grew or shrank and lists new and removed folders. From `gui/src-tauri`:
 
 ```bash
 cargo tauri dev      # run the app
@@ -90,7 +115,8 @@ cargo tauri build    # bundle an installer
 ## Why it's fast — and correct
 
 - Parallel directory walk across all cores; a **single pass** counts files *and*
-  directories.
+  directories, and builds the whole tree in memory so navigating it costs no
+  further I/O.
 - Duplicate detection never hashes more than it must: size buckets first, hard
   links collapsed, a cheap first-block pre-hash, then BLAKE3 — streamed, so large
   files never load into RAM.
@@ -102,9 +128,9 @@ cargo tauri build    # bundle an installer
 ```
 Diskghost/
 ├── crates/
-│   ├── diskghost-core/   the engine: scan + duplicate detection + reclaim (a library)
+│   ├── diskghost-core/   the engine: scan tree, duplicates, reclaim, snapshots + diff (a library)
 │   └── diskghost-cli/    the `diskghost` command
-└── gui/                  Tauri desktop app (modern dark UI)
+└── gui/                  Tauri desktop app (modern dark UI; ui/lib.js has Node unit tests)
 ```
 
 ## Roadmap
@@ -115,16 +141,18 @@ Diskghost/
 - [x] Modern GUI: treemap, folder picker, drag &amp; drop, one-click reclaim
 - [x] Delete files/folders (CLI `rm` + GUI) — permanent or to the OS trash, parallel unlink
 - [x] Drive free/total space + scan &amp; delete timings (CLI + GUI)
-- [x] CI (fmt/clippy/test/bench on 3 OS) + signed release binaries + GUI installers
-- [ ] Interactive treemap drill-up / breadcrumbs
-- [ ] Scheduled scans &amp; "what grew since last time"
+- [x] CI (fmt/clippy/test/bench on 3 OS) + checksummed, provenance-attested release binaries + GUI installers
+- [x] Interactive treemap drill-up / breadcrumbs, served from an in-memory tree
+- [x] Snapshots &amp; "what grew since last time" (CLI `--save`/`--since`/`diff`, GUI history)
+- [ ] Code-signed GUI installers
 
 ## Development
 
 ```bash
 cargo test --workspace                                   # unit + integration tests
 cargo clippy --workspace --all-targets -- -D warnings    # lints (CI gate)
 cargo bench -p diskghost-core                             # criterion benchmarks
+node --test gui/ui/lib.test.js                            # GUI helper unit tests
 ```
 
 ## License

crates/diskghost-cli/Cargo.toml

@@ -15,4 +15,5 @@ path = "src/main.rs"
 [dependencies]
 diskghost-core = { path = "../diskghost-core" }
 clap = { version = "4", features = ["derive"] }
+serde = { version = "1", features = ["derive"] }
 serde_json = "1"

crates/diskghost-cli/src/main.rs

@@ -11,10 +11,11 @@ use std::sync::Arc;
 
 use clap::{Args, Parser, Subcommand, ValueEnum};
 use diskghost_core::{
-    find_duplicates_with_progress, human_size, reclaim, remove_path, scan_with_progress,
-    validate_globs, DupGroup, Options, Progress, ReclaimAction, RemoveMode, RemoveReport,
-    ScanReport,
+    diff, find_duplicates_with_progress, format_timestamp, human_delta, human_size, reclaim,
+    remove_path, validate_globs, DiffReport, DupGroup, Options, Progress, ReclaimAction,
+    RemoveMode, RemoveReport, ScanReport, ScanTree, Snapshot,
 };
+use serde::Serialize;
 
 /// Run `job` with a `Progress`, printing a live line to stderr while it works
 /// (only when stderr is a terminal, so JSON/pipe output stays clean).
@@ -80,6 +81,26 @@ enum Command {
         top: usize,
         #[command(flatten)]
         walk: WalkArgs,
+        /// Save the scan as a snapshot (JSON) for a later `--since` or `diff`.
+        #[arg(long, value_name = "FILE")]
+        save: Option<PathBuf>,
+        /// Compare against a snapshot saved earlier: what grew, shrank,
+        /// appeared or vanished since then.
+        #[arg(long, value_name = "FILE")]
+        since: Option<PathBuf>,
+    },
+    /// Compare two snapshots saved with `scan --save`.
+    Diff {
+        /// The older snapshot.
+        old: PathBuf,
+        /// The newer snapshot.
+        new: PathBuf,
+        /// Compare at this sub-folder instead of the scanned root.
+        #[arg(long, value_name = "DIR")]
+        at: Option<PathBuf>,
+        /// How many entries to show per list.
+        #[arg(long, default_value_t = 20)]
+        top: usize,
     },
     /// Find duplicate (byte-identical) files; optionally reclaim their space.
     Dupes {
@@ -160,22 +181,91 @@ fn main() -> ExitCode {
     let cli = Cli::parse();
 
     match cli.command {
-        Command::Scan { path, top, walk } => {
+        Command::Scan {
+            path,
+            top,
+            walk,
+            save,
+            since,
+        } => {
             if !path.is_dir() {
                 eprintln!("error: not a directory: {}", path.display());
                 return ExitCode::FAILURE;
             }
             warn_invalid_globs(&walk.exclude);
             let opts = walk.to_options();
+            // Load the old snapshot before scanning, so a bad file fails fast.
+            let old = match since.as_deref().map(Snapshot::load) {
+                Some(Ok(s)) => Some(s),
+                Some(Err(e)) => {
+                    eprintln!("error: cannot read snapshot: {e}");
+                    return ExitCode::FAILURE;
+                }
+                None => None,
+            };
             let started = std::time::Instant::now();
-            let report = with_progress(|p| scan_with_progress(&path, top, &opts, p));
+            let tree = with_progress(|p| ScanTree::build(&path, &opts, p));
             let elapsed = started.elapsed();
+            let report = tree.root_report(top);
+            let snapshot = (old.is_some() || save.is_some()).then(|| Snapshot::capture(&tree, top));
+            let changes = match (&old, &snapshot) {
+                (Some(old), Some(new)) => match diff(old, new, &report.root) {
+                    Ok(mut d) => {
+                        d.truncate(top);
+                        Some(d)
+                    }
+                    Err(e) => {
+                        eprintln!("error: {e}");
+                        return ExitCode::FAILURE;
+                    }
+                },
+                _ => None,
+            };
             if cli.json {
-                println!("{}", serde_json::to_string_pretty(&report).unwrap());
+                let out = ScanOutput {
+                    report: &report,
+                    diff: changes.as_ref(),
+                };
+                println!("{}", serde_json::to_string_pretty(&out).unwrap());
             } else {
                 print_scan(&report);
+                if let Some(d) = &changes {
+                    println!();
+                    print_diff(d);
+                }
                 println!("\nScanned in {}", fmt_duration(elapsed));
             }
+            if let (Some(file), Some(snap)) = (&save, &snapshot) {
+                if let Err(e) = snap.save(file) {
+                    eprintln!("error: cannot save snapshot: {e}");
+                    return ExitCode::FAILURE;
+                }
+                // Keep stdout clean for --json consumers.
+                eprintln!("snapshot saved to {}", file.display());
+            }
+        }
+        Command::Diff { old, new, at, top } => {
+            let (old, new) = match (Snapshot::load(&old), Snapshot::load(&new)) {
+                (Ok(o), Ok(n)) => (o, n),
+                (Err(e), _) | (_, Err(e)) => {
+                    eprintln!("error: cannot read snapshot: {e}");
+                    return ExitCode::FAILURE;
+                }
+            };
+            let at = at.unwrap_or_else(|| new.root.clone());
+            let mut report = match diff(&old, &new, &at) {
+                Ok(r) => r,
+                Err(e) => {
+                    eprintln!("error: {e}");
+                    return ExitCode::FAILURE;
+                }
+            };
+            report.truncate(top);
+            if cli.json {
+                println!("{}", serde_json::to_string_pretty(&report).unwrap());
+            } else {
+                print_diff(&report);
+            }
         }
         Command::Dupes {
             path,
@@ -241,6 +331,72 @@ fn main() -> ExitCode {
     ExitCode::SUCCESS
 }
 
+/// `scan --json` output: the report's fields at the top level, plus `diff`
+/// when `--since` was given.
+#[derive(Serialize)]
+struct ScanOutput<'a> {
+    #[serde(flatten)]
+    report: &'a ScanReport,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    diff: Option<&'a DiffReport>,
+}
+
+fn print_diff(d: &DiffReport) {
+    println!(
+        "Changes in {} since {} (now {})",
+        d.root.display(),
+        format_timestamp(d.old_taken_at),
+        format_timestamp(d.new_taken_at)
+    );
+    println!(
+        "  total: {} ({} -> {}), {:+} file(s)",
+        human_delta(d.delta),
+        human_size(d.old_size),
+        human_size(d.new_size),
+        d.files_delta
+    );
+    if d.is_unchanged() {
+        println!("  nothing changed");
+        return;
+    }
+    if !d.grown.is_empty() {
+        println!("\nGrew:");
+        for g in &d.grown {
+            println!(
+                "  {:>10}  {}  ({} -> {})",
+                human_delta(g.delta),
+                g.path.display(),
+                human_size(g.old_size),
+                human_size(g.new_size)
+            );
+        }
+    }
+    if !d.shrunk.is_empty() {
+        println!("\nShrank:");
+        for s in &d.shrunk {
+            println!(
+                "  {:>10}  {}  ({} -> {})",
+                human_delta(s.delta),
+                s.path.display(),
+                human_size(s.old_size),
+                human_size(s.new_size)
+            );
+        }
+    }
+    if !d.added.is_empty() {
+        println!("\nNew folders:");
+        for a in &d.added {
+            println!("  {:>10}  {}", human_size(a.size), a.path.display());
+        }
+    }
+    if !d.removed.is_empty() {
+        println!("\nRemoved folders:");
+        for r in &d.removed {
+            println!("  {:>10}  {}", human_size(r.size), r.path.display());
+        }
+    }
+}
+
 fn print_remove(r: &RemoveReport, trash: bool) {
     let dest = if trash { "trash" } else { "delete" };
     let what = format!(
@@ -299,6 +455,9 @@ fn print_scan(r: &ScanReport) {
     if r.skipped > 0 {
         println!("  skipped: {} unreadable entries (permissions?)", r.skipped);
     }
+    if r.cancelled {
+        println!("  CANCELLED: partial result, the numbers cover only what was scanned");
+    }
     println!();
 
     println!("Biggest sub-folders:");