Skip to content

Prune persistentIds older than 24h so config.json stops growing - #9

Merged
cassiostp merged 1 commit into
masterfrom
cs/bounded-persistent-ids
Sep 16, 2026
Merged

cassiostp merged 1 commit into
masterfrom
cs/bounded-persistent-ids

Conversation

@cassiostp

@cassiostp cassiostp commented Aug 13, 2026

Copy link
Copy Markdown

Every delivered push appends a persistentId to config.json and the list is never trimmed, so the file grows without bound and electron-config rewrites all of it synchronously on the main process for each push. Across 46.6k native devices (log.push_receiver_config_stats, Sep 8–15) the darwin p50 is 53.7k ids / 2.2 MB and the p99 is 981k ids / 40 MB. At the median ~360 pushes per active day, a median device serialises ~0.8 GB/day of JSON on the UI thread.

Fix

Each id is 0:{microsecondTimestamp}%{hash}, so its age is known. pruneStale drops ids older than 24 h from the list seeded into listen() at start and from every append.

24 h is safe because the list only prevents the server from redelivering pushes on the next login, and the backend sends desktop pushes with TTL: 5 minutes (new-email, calendar-event) or TTL: 0 (history, userdata-history) (mail-backend/lib/notifications/notify.go). Nothing older can be replayed; the margin covers clock skew. Ids that do not parse are dropped so a format change cannot reintroduce unbounded growth.

webContents.send calls are guarded with isDestroyed().

Rollout

Publish as 2.1.10 and bump native/package.json in desktop. The first login after upgrade sends the pruned list. Rollback is a version revert; the old code reads the same array. Expected signal in log.push_receiver_config_stats: persistent_ids_count p50 from ~50k to a few hundred, config_bytes p50 from 2.2 MB to under 50 KB.

CLI-2311

@cassiostp
cassiostp force-pushed the cs/bounded-persistent-ids branch 2 times, most recently from e3dbea4 to 3c1ccfe Compare August 17, 2026 18:28
@cassiostp
cassiostp force-pushed the cs/bounded-persistent-ids branch from 3c1ccfe to 151af30 Compare September 15, 2026 20:29
@cassiostp cassiostp changed the title Bound persistentIds in their own store and mirror the client's clears Prune persistentIds older than 24h so config.json stops growing Sep 15, 2026
@linear-code

linear-code Bot commented Sep 15, 2026

Copy link
Copy Markdown

CLI-2311

@cassiostp
cassiostp force-pushed the cs/bounded-persistent-ids branch 2 times, most recently from 4cf7fbe to 2ce3473 Compare September 16, 2026 03:42
@cassiostp
cassiostp marked this pull request as ready for review September 16, 2026 03:48
Comment thread src/index.js Outdated
const savedApiKey = config.get('fcmApiKey');
if (started) {
webContents.send(NOTIFICATION_SERVICE_STARTED, (credentials.fcm || {}).token);
send(NOTIFICATION_SERVICE_STARTED, ((credentials || {}).fcm || {}).token);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep existing behaviour here where if we don't pass in credential we raise an error.

Every received push appends one id to config.json and the list is never
trimmed, so the file grows without bound - fleet median 54k ids / 2.2 MB,
p99 1M ids / 40 MB - and each push rewrites the whole file synchronously
on the main process.

Each id embeds a microsecond timestamp, so age is the natural bound: ids
older than 24 hours are pruned on start and on every write. The list
exists only to stop the server redelivering pushes on the next login, and
desktop pushes carry a 5-minute (or 0) TTL, so anything older can never be
replayed - no count cap or acked-ids signal is needed.

Unparsable ids are dropped deliberately: keeping them would be the only
path back to unbounded growth if the id format ever changes, while
dropping one costs at most a single redelivery.

CLI-2311
@cassiostp
cassiostp force-pushed the cs/bounded-persistent-ids branch from 2ce3473 to 01bcd69 Compare September 16, 2026 14:25
@cassiostp
cassiostp merged commit 993b0f8 into master Sep 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants