Skip to content

refactor!: rebuild the "refresh" system into expanded "sync" system - #1501

Open
CyanVoxel wants to merge 3 commits into
devfrom
refresh-rework
Open

refactor!: rebuild the "refresh" system into expanded "sync" system#1501
CyanVoxel wants to merge 3 commits into
devfrom
refresh-rework

Conversation

@CyanVoxel

@CyanVoxel CyanVoxel commented Sep 10, 2026

Copy link
Copy Markdown
Member

Important

Blocked by #1499 in order to finalize migration and resolve failing migration tests.

Note

This PR is targeting the dev branch for release with Alpha v9.7.

Summary

This is a complete rebuild of the "refresh" system into an expanded "sync" system that removes longstanding design constraints like modal progress bars that block the app, addresses important edge cases such as Unicode normalization and filesystem case-insensitivity, and finally adds some basic amount of automatic relinking with the help of cached file stats.

Much of the performance and code consolidation was inspired by #1242 along with the NFD research stemming directly from that PR, with additional notes taken from #1495. Completes this system's task for #1460.

Half of this PR centers around the new sync system and how it stores up-to-date file metadata to aid in searches and the relinking process, and the other half is a whoooole lot of UI code for a new notification and banner that displays progress of sync operations without blocking the app. I poured a lot of polish into since I'm aiming for a v10 level of completion with this system, and two years of temporary progress bar modals was just too much.

Directory scanning, file metadata updates, and automatic relinking operations all now run in the background, letting you use TagStudio while these operations complete. Because syncing no longer has the blocking impact it did previously, I've also added a dismissable banner message that displays on app startup encouraging users to turn on the "sync library on startup" setting. Since it's still valid to not want your filesystem hit or library modified when opening a library, I've kept the setting around.

What I haven't done in this PR is rework the old "Fix Unlined Entries" modal/system, only integrated it slightly with the new sync system. That whole thing needs its own rework that's out of the scope of this PR (yes, this PR actually has a scope...)


Full Breakdown

Unicode Normalization Form D (NFD) + Windows Case-Insensitivity Enforcement

  • NFD is now enforced on the library side and for all filepath comparisons (mainly from the library cache) to fix current issues and prevent future issues with comparing non-normalized paths.
  • Filesystem case-insensitivity is now also lightly respected, with Windows users expecting to be on a case-insensitive system, and therefore filepath comparisons to filepaths with different cases will be allowed on Windows. This is a barebones implementation, but is a start to this level of filesystem awareness.
  • Because Unicode normalization was not previously enforced, there may be libraries with duplicate entries for the same path on disk. These are cases are handled or raised by the sync step, where it may be required to manually search and relink. Entries affected by this will be merged together, just as normal duplicate entries are.

Non-Blocking UI Banner

  • Syncing now runs as a background task with progress shown in a new animated banner widget at the top of the entry grid, replacing the old blocking modal progress dialogs.
  • Progress-bound operations can now be cancelled, and automatically cancel when switching or closing a library.

Reworked Refresh (Now Called Sync) System

  • Replaced the old RefreshTracker and UnlinkedRegistry with a single LibrarySyncEngine.
    • One pass now handles new file discovery, unlinked detection, automatic relinking, and stat refresh together instead of separate disconnected steps.
    • A single cache of filepaths mapped to entry IDs is now stored in the library for other systems to use (again)
    • Menu action renamed from "Refresh Directories" to "Sync Library" and still keeps the Ctrl+R shortcut (Ctrl+S to sync doesn't feel right, and Ctrl+R still feels appropreiate).
    • Renamed "Automatically Load New Files" setting to "Sync Library on Open".
    • Added a dismissable banner message that displays on app startup encouraging users to turn on the setting.
  • Entries now store date created, date modified, and file size in the database, and existing entry paths are normalized to Unicode Normalization Form D (NFD). These values are kept up to date automatically during the sync process.
Automatic Relinking
  • Added an automatic relinking phase that matches unlinked entries to new files using filename plus date modified and file size, falling back to date modified and file size alone for renamed files.
  • Only acts on matches with a single unambiguous candidate, anything ambiguous is left for the user to resolve manually through the "Fix Unlinked Entries" modal.
  • TODO: Leverage the cached file stats to narrow down ambiguous name cases where possible.

Sort by File Size, Date Created, Date Modified

  • Search results can now be sorted by date created, date modified, and file size.

Supersedes #1242 and #1495
Fixes #1252
Fixes #1279
Fixes #1450
Closes #610
Closes #1212
Closes #1218

Tasks Completed

  • Platforms Tested:
    • Windows x86
    • Windows ARM
    • macOS x86
    • macOS ARM
    • Linux x86
    • Linux ARM
  • Tested For:
    • Basic functionality
    • PyInstaller executable

@CyanVoxel CyanVoxel added this to the Alpha v9.7.x milestone Sep 10, 2026
@CyanVoxel CyanVoxel added Type: Feature New feature or request Type: Refactor Code that needs to be restructured or cleaned up labels Sep 10, 2026
@CyanVoxel CyanVoxel added Type: QoL A quality of life (QoL) enhancement or suggestion Type: UI/UX User interface and/or user experience TagStudio: Library Relating to the TagStudio library system Type: File System File system interactions Status: Blocked This issue or pull request is awaiting the outcome of another issue or pull request Type: Performance An issue or change related to performance Note: Sync Weblate Before Pulling Indicates that the Weblate translations repository needs to be up to date before pulling this code. Type: Fix A fix for a bug, typo, or other issue labels Sep 10, 2026
@CyanVoxel CyanVoxel moved this to 🏓 Ready for Review in TagStudio Development Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Note: Sync Weblate Before Pulling Indicates that the Weblate translations repository needs to be up to date before pulling this code. Status: Blocked This issue or pull request is awaiting the outcome of another issue or pull request TagStudio: Library Relating to the TagStudio library system Type: Feature New feature or request Type: File System File system interactions Type: Fix A fix for a bug, typo, or other issue Type: Performance An issue or change related to performance Type: QoL A quality of life (QoL) enhancement or suggestion Type: Refactor Code that needs to be restructured or cleaned up Type: UI/UX User interface and/or user experience

Projects

Status: 🏓 Ready for Review

Development

Successfully merging this pull request may close these issues.

1 participant