feat(projects): preserve managed volumes on project rename

[NeurekaSoftware]

Checklist

• This PR is not opened from my fork's main branch

What This PR Implements

When a stopped Arcane project is renamed, Arcane now keeps eligible Docker Compose project-scoped volumes aligned with the new project name.

For example, renaming a project from nginx to web can copy data from nginx_data into web_data, so the renamed project keeps using the same application data instead of starting with a new empty volume.

Fixed external volumes and volumes with a fixed Compose name: are left alone.

Fixes: #2898

Changes Made

• Added migration for Compose-managed volumes whose names are generated from the project name.
• Kept external volumes and fixed explicit volume names unchanged.
• Blocked unsafe renames unless the project is verified stopped.
• Added safety checks for existing target volumes, source volumes still attached to containers, and insufficient disk space.
• Added rollback handling for failed renames before the project record is committed.
• Added rename journals and startup recovery for interrupted renames or cleanup retries.
• Updated the project editor so a Compose top-level name: is treated as the effective project name when it is usable and not interpolated.
• Made volume migration planning use the pending saved Compose/project-file state, not only the old files already on disk.

Safety And Recovery

If anything fails before the project update is committed, Arcane rolls back the project files, directory, database state, and copied target volumes when it is safe to do so.

If the original source volume is missing during rollback, Arcane preserves the copied target volume to avoid deleting the only remaining data copy.

If the project update commits but old source-volume cleanup fails, the project stays renamed and Arcane keeps a recovery journal so cleanup can be retried later.

On startup, Arcane reads rename journals and either finishes committed rename cleanup or rolls back incomplete renames based on the current project state.

Testing Done

• cd backend && go test ./...
• pnpm -C frontend check

AI Tool Used (if applicable)

Codex assisted with implementation, test design, and PR-description cleanup.

Additional Context

Without this change, Docker Compose can create new empty project-scoped volumes after a rename. The old data may still exist in the old volume, but the renamed project no longer points at it, which can look like data loss.

Simplified Rename Flow

flowchart TD
  A["Rename requested"] --> B{"Project stopped?"}
  B -- "No" --> C["Block rename"]
  B -- "Yes" --> D["Find eligible managed volumes"]
  D --> E{"Safe to migrate?"}
  E -- "No" --> F["Stop before changing project state"]
  E -- "Yes" --> G["Copy old volume data to new volumes"]
  G --> H["Save project rename"]
  H --> I{"Save committed?"}
  I -- "No" --> J["Rollback files, directory, database, and safe volume changes"]
  I -- "Yes" --> K["Remove old source volumes"]
  K --> L{"Cleanup complete?"}
  L -- "Yes" --> M["Clear rename journal"]
  L -- "No" --> N["Keep journal for startup recovery"]

Loading

Disclaimer Greptiles Reviews use AI, make sure to check over its work.

To better help train Greptile on our codebase, if the comment is useful and valid Like the comment, if its not helpful or invalid Dislike

To have Greptile Re-Review the changes, mention greptileai.

Greptile Summary

This PR implements volume preservation during Docker Compose project renames by introducing a write-ahead journal pattern that tracks each rename phase (started → targets_copied → old_volumes_removed → project_state_committed) and supports full rollback/recovery on restart. The changes touch the project service, a new KV-backed journal, a new volume-copy pipeline using helper containers, a CLI internal-volume-helper subcommand, and compose-preview logic for pre-flight volume planning.

• Journal + recovery: project_rename_journal.go (729 lines) defines the rename phases and RecoverProjectRenameJournals runs on startup and before each UpdateProject, resuming or rolling back any interrupted rename. The rollback path correctly handles the case where both old and new directories already exist by moving the new path to a hidden conflict directory.
• Volume migration: project_volume_rename.go (777 lines) spawns helper containers to copy volume data with capacity pre-check, using stdcopy.StdCopy for demuxed stdout/stderr; the compose name: field rename path is handled via a preview directory so the new volume names are computed before any file changes are committed.
• KV service hardening: ListByPrefix now escapes \, %, and _ before appending the SQL LIKE wildcard, preventing journal-key collisions for project IDs containing those characters.

Confidence Score: 4/5

This PR is safe to merge; the journal/recovery machinery is well-tested and all previously-raised concerns appear addressed.

The implementation is thorough, with 1196-line recovery tests covering all journal phases. Every specific concern from the previous review threads (compose-name rename path, SQL wildcard escaping, walk abort, stdout/stderr mixing, recovery gating non-rename updates, directory conflict on rollback) has been resolved. The one remaining edge — a Commit() failure after project_state_committed that isn't a projectRenameSourceCleanupInternalError leaves the journal stuck until the next UpdateProject or restart — is intentional by design and recovery will resume it. No new blocking issues found.

No files require special attention; project_rename_journal.go and project_volume_rename.go are the highest-risk new files but both have strong test coverage.

Comments Outside Diff (1)

1. backend/internal/services/project_volume_rename.go, line 2001-2013 (link)

   Orphaned target volumes when old-volume restore fails during rollback

   When removedOld is non-empty (i.e., Commit() partially succeeded by deleting some old volumes) and the restore loop accumulates a restoreErr, the function returns early before calling rollbackCreatedTargets. This leaves all the new target volumes (e.g., web_data) alive in Docker while the database transaction has already been rolled back, so the project still refers to the old name — the new volumes become permanently orphaned and require manual cleanup.

   The condition is reached when Commit() removes at least one old volume then fails on a later one, the caller's DB transaction rolls back, Rollback() is triggered via the deferred handler, and the recreation/copy of any removed volume then fails. rollbackCreatedTargets should be called as a best-effort cleanup even when restoreErr != nil.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: backend/internal/services/project_volume_rename.go
   Line: 2001-2013
   
   Comment:
   **Orphaned target volumes when old-volume restore fails during rollback**
   
   When `removedOld` is non-empty (i.e., `Commit()` partially succeeded by deleting some old volumes) and the restore loop accumulates a `restoreErr`, the function returns early before calling `rollbackCreatedTargets`. This leaves all the new target volumes (e.g., `web_data`) alive in Docker while the database transaction has already been rolled back, so the project still refers to the old name — the new volumes become permanently orphaned and require manual cleanup.
   
   The condition is reached when `Commit()` removes at least one old volume then fails on a later one, the caller's DB transaction rolls back, `Rollback()` is triggered via the deferred handler, and the recreation/copy of any removed volume then fails. `rollbackCreatedTargets` should be called as a best-effort cleanup even when `restoreErr != nil`.
   
   How can I resolve this? If you propose a fix, please make it concise.

   

2. backend/internal/services/project_rename_journal.go, line 513-515 (link)

   Permanently blocks UpdateProject when both directory paths are externally deleted

   When neither oldPath nor newPath exists, the function returns a hard error. This error propagates up through rollbackProjectRenameJournalInternal → recoverProjectRenameJournalInternal → recoverProjectRenameJournalForProjectInternal, and since the journal is never cleared on this path, every subsequent UpdateProject call for that project will hit the same error indefinitely.

   This condition is reachable without any operator error: if Arcane restarts while a rename journal is in the started phase (crash before any directory rename occurred) and the project directory was deleted externally between the crash and restart, both paths are absent and the project becomes permanently unmodifiable via the API.

   Consider treating "both missing" as a warning-and-continue (the directory state is already unrecoverable regardless), rather than a hard error, so volume and database state recovery can still proceed and the journal can be cleared.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: backend/internal/services/project_rename_journal.go
   Line: 513-515
   
   Comment:
   **Permanently blocks `UpdateProject` when both directory paths are externally deleted**
   
   When neither `oldPath` nor `newPath` exists, the function returns a hard error. This error propagates up through `rollbackProjectRenameJournalInternal` → `recoverProjectRenameJournalInternal` → `recoverProjectRenameJournalForProjectInternal`, and since the journal is never cleared on this path, every subsequent `UpdateProject` call for that project will hit the same error indefinitely.
   
   This condition is reachable without any operator error: if Arcane restarts while a rename journal is in the `started` phase (crash before any directory rename occurred) and the project directory was deleted externally between the crash and restart, both paths are absent and the project becomes permanently unmodifiable via the API.
   
   Consider treating "both missing" as a warning-and-continue (the directory state is already unrecoverable regardless), rather than a hard error, so volume and database state recovery can still proceed and the journal can be cleared.
   
   How can I resolve this? If you propose a fix, please make it concise.

   

_{Reviews (36): Last reviewed commit: "fix(projects): keep rename journal on di..." | Re-trigger Greptile}

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[kmendell]

Ill have to look into this more, but doesnt just recreating the project do this?

Alos note: the name field in arcane ui is pretty much just the name for the folder and for internal refernce of it it doesnt get passed to docker (i dont think :O) anymore or at least it shouldnt

[NeurekaSoftware]

Ill have to look into this more, but doesnt just recreating the project do this?

Just an example, my "media-server" stack has 13 volumes.

Renaming the project / renaming the folder is the same as changing the stack name.

Managed volumes are named {stack}_{volume}

So renaming my "media-server" stack to "media-services" will result in brand new volumes being created when I bring it up again, unless I manually rename all previous volumes.

The exception to the stack name changing when the folder name changes is when you have a name: mystack at the top of your compose file- but if I remember correctly Arcane wasn't actually respecting that value for some reason? I would have to double check.

[NeurekaSoftware]

I just confirmed, having the stack name in the compose file should make the volumes stable but arcane doesn't respect that value for some reason.

When using docker via the CLI, you can have a name: stackname value in the compose file and you can rename the parent directory and volume names are stable. But if the name isn't in the compose file the stack name will change if the parent folder name changes.

This is why this change is needed. It adds complexity to renaming projects but it ensures nothing ever breaks when managing projects via arcane.

[NeurekaSoftware]

Reproduction Steps

Scenario 1: No name: Defined (Current Problem)

Create a Compose project:

services:

  nginx:

    image: nginx:alpine

    volumes:

      - data:/data

volumes:

  data:

Place it in a folder named:

media-server/

Bring the stack up:

docker compose up -d

Docker creates:

media-server_data

Rename the folder (or rename the stack in Arcane):

media-server/

↓

media-services/

Bring the stack up again:

docker compose up -d

Docker now creates:

media-services_data

Result:

• A brand new volume is created.

• Existing data remains in media-server_data.

• Containers start with an empty volume.

• User must manually rename or migrate volumes.

---

Scenario 2: name: Defined (Docker CLI Behavior)

Create a Compose project:

name: media-server

services:

  nginx:

    image: nginx:alpine

    volumes:

      - data:/data

volumes:

  data:

Place it in a folder named:

media-server/

Bring the stack up:

docker compose up -d

Docker creates:

media-server_data

Rename the folder:

media-server/

↓

media-services/

Bring the stack up again:

docker compose up -d

Docker still uses:

media-server_data

Result:

• No new volumes are created.

• Existing data is reused.

• Folder renames do not affect volume names.

• Stack resources remain stable.

[NeurekaSoftware]

Arcane doesn't seem to respect scenario 2 either, which probably needs its own separate bug report.

Once the bug with arcane not respecting scenario 2 is fixed, this PR should be modified to exclude migrating volumes when a project is renamed if name: mystack is both present in the compose file AND hasn't changed value.

[github-actions]

This pull request has merge conflicts. Please resolve the conflicts so the PR can stay up-to-date and reviewed.

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

Hey @kmendell, last ping from me, I swear: this is ready for review now.

This PR has changed significantly since it was first opened, so I updated the PR body to accurately reflect the current state of the branch.

Greptile is passing now, and all tests pass as well.

I also updated the Mermaid chart I posted earlier because rollback and recovery changed quite a bit. The updated chart is now in the PR body so everything is in one place.

My work here is done unless you need anything else. Just let me know.

Here is a video of everything working, including volume migration through both project renames and a Compose top-level name: rename.

Screencast_20260613_131934.webm

[kmendell]

ill try to look later today

[kmendell]

The first thing ill note juts by glancing, all of the project_volume files under services/ move those to pkg/projects as the only thing i wnat under services is the actual service files . Style nit for sure, but iot just makes things cleaner

[NeurekaSoftware]

The first thing ill note juts by glancing, all of the project_volume files under services/ move those to pkg/projects as the only thing i wnat under services is the actual service files .

All done. Let me know if you need any other changes. :)