Skip to content

fix: cascade folder renames to the subtree and prune ghost folders - #303

Open
Monkey7539 wants to merge 1 commit into
maathimself:mainfrom
Monkey7539:fix/folder-rename-subtree
Open

fix: cascade folder renames to the subtree and prune ghost folders#303
Monkey7539 wants to merge 1 commit into
maathimself:mainfrom
Monkey7539:fix/folder-rename-subtree

Conversation

@Monkey7539

Copy link
Copy Markdown
Contributor

Problem

Renaming a folder that has subfolders (tested via the sidebar rename, POST /folders/rename):

  1. The IMAP RENAME succeeds and moves the whole subtree server-side (per RFC 3501 §6.3.5).
  2. The endpoint only updates the DB rows for the exact old path — every child folder row (and its messages) keeps the old path.
  3. The next folder-structure sync upserts the renamed tree from LIST → the sidebar shows a duplicate tree (real renamed folders + stale ghosts with all the same subfolders).
  4. The per-folder message sync keeps iterating the stale child paths and fails to open them on every tick, foreversyncFolders never removes rows for folders that no longer exist on the server, so the ghosts are permanent.

The missing prune also means folders renamed/deleted by other IMAP clients leave the same permanent ghosts (I have one of those in my instance too — a folder that errors Reconcile: could not open … on every cycle).

Fix

/folders/rename cascades to the subtree. The path prefix is rewritten for the folder and all descendants in both folders and messages (substr-prefix matching on oldPath + delimiter, so no LIKE-escaping traps with %/_ in folder names). If a sync raced the rename and already inserted rows at the new paths, the stale old rows are deleted first instead of colliding with the unique (account_id, path) / (account_id, uid, folder) constraints.

syncFolders prunes ghosts. After upserting the LIST results, DB folder rows absent from the response are deleted, so ghosts (from external renames/deletes, or from pre-fix renames) heal on the next folder sync instead of erroring every tick. Safeguards: skipped entirely on an empty LIST (a pathological response must not wipe the account's tree), and INBOX is exempt (many servers omit it from LIST; the implicit-INBOX insert above depends on that). Message rows are deliberately left alone — in-app folder deletion already removes them explicitly, and orphans stop syncing once their folder row is gone.

Testing

  • New syncFolders pruning tests: ghosts deleted with the exact live-path list, empty LIST never prunes, upserts still precede the prune.
  • Full backend suite: 693 pass; eslint clean.
  • Reproduced the original bug against a live instance (rename a parent with children → duplicate tree + endless could not open sync errors) and verified the failure mode disappears with the cascade.

Written with AI assistance (Claude); I have reviewed and tested the changes and am authorized to submit them under the project's CLA.

🤖 Generated with Claude Code

Renaming a folder with subfolders updated only the exact-path DB rows, while
IMAP RENAME moves the whole subtree server-side. Every child folder (and its
messages) stayed under the old path: the next folder sync upserted the
renamed tree from LIST — a visible duplicate — while the per-folder message
sync kept trying, and failing, to open the stale old child paths forever.

Two-part fix:

- /folders/rename rewrites the path prefix for the folder AND all
  descendants, in both folders and messages (substr-prefix matching, no LIKE
  escaping traps). If a sync raced the rename and already inserted rows at
  the new paths, the stale old rows are dropped first instead of colliding
  with the unique constraints.

- syncFolders prunes DB folder rows absent from LIST, so ghosts from
  renames/deletes made by other clients (or pre-fix renames) heal on the
  next folder sync instead of erroring every tick. Guarded on a non-empty
  LIST; INBOX is exempt (many servers omit it from LIST); message rows are
  left to the existing deletion paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@maathimself

Copy link
Copy Markdown
Owner

Thanks, this is a real pair of bugs: the rename genuinely was not cascading to the subtree, and ghost folders did accumulate with no prune. The prune half is nicely done and well-tested.

One framing note up front: since IMAP is the source of truth and folders / messages are a re-syncable cache, none of the below loses mail on the server. The risk is wrong local state that self-heals on the next sync. So this is "needs changes," not a hard blocker, but a few are worth fixing before it goes in.

High: the collision-DELETE can delete the real source rows. The DELETE FROM folders ... WHERE (path = oldPath OR under childPrefix) AND EXISTS (a row already at the new path) assumes any row already at the target is a correct copy inserted by a racing sync. But a pre-existing ghost at the target path, exactly what this PR exists to clean up, also satisfies the EXISTS. Concrete case: the DB still has a ghost Archive row (server folder deleted externally, not yet pruned); the user renames Work to Archive; the IMAP RENAME succeeds because the server has no Archive; the DELETE then removes the real Work rows (the ghost Archive exists), and the following UPDATE finds no Work row left to rewrite. The message DELETE does the same for any Work message whose uid also exists under the ghost. Net: the real folder's rows are dropped and the ghost stays until the next sync heals it. Consider pruning ghosts first, scoping the delete to rows that are genuine duplicates, or handling the target-exists case explicitly.

Medium: no guard against a same-name (no-op) rename. newName is not checked against the current leaf, so newPath can equal oldPath. If the server accepts a same-name RENAME as a no-op rather than erroring, the collision-DELETE self-matches (EXISTS finds the row itself) and deletes the folder row, and the message DELETE (n.folder = old.folder AND n.uid = old.uid) deletes every message in the folder and subtree. Most servers reject RENAME to an existing name, so this only bites on lenient ones, but a if (newPath === oldPath) return res.json({ ok: true, newPath }) early-out removes the hazard cheaply.

Medium: the cascade is not transactional. It is now five sequential statements (two DELETEs, three UPDATEs) with no BEGIN/COMMIT. A crash or dropped connection mid-sequence leaves the subtree half-rewritten. Sync eventually reconciles, but wrapping the five in a transaction makes the rename atomic and is worth it given the added deletes.

Medium: the prune trusts any non-empty LIST. The only guard is mailboxes.length > 0, so a partial-but-non-empty LIST (a transient server hiccup) prunes the real folders missing from that response. Message rows are left alone so nothing is lost, but the affected folders vanish from the sidebar until the next full sync. An updated_at age check before pruning a row would be safer than deleting any folder absent from a single LIST.

Tests. The prune has good coverage; the rename cascade, the half with all the destructive SQL, has none. Please add rename tests: a nested subtree rewrite, a child with its own children, the ghost-at-target case, and the same-name case.

Happy to re-review once the collision-DELETE and the same-name guard are handled. Those two are the ones I would want fixed for sure; the transaction, the prune hardening, and the tests are strongly preferred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants