From 8b3a3596d41c68ae2ef7f33c5086d0c42a9f9f3a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 13:58:10 +0000 Subject: [PATCH] feat(therapy): drop card bytes as the commit parses them An import was uploaded to be read once, but pap_file kept every byte of the card after the commit had written the parsed night, and nothing has read a byte back since: the panel, the export, the share and the admin read model all go through pap_day, pap_event and pap_channel. The opening slice now deletes every file left without a day, which covers the card level files it just parsed into the import, the ones whose night was replaced and the ones no date could be read from, and fillDay deletes a night's files in the transaction that stores the night. Transactional so a slice that rolls back keeps them and the client's retry still finds a night it can parse. A night that would not parse takes its files with it either way, through the dropped row's cascade or explicitly when the card's own summary is kept. The table stays: it is where a chunked upload lands and where the commit reads from. What it no longer is, is an archive, so a parser fix can only be replayed over a card uploaded again. Co-Authored-By: Claude --- docs/therapy-storage.md | 19 +++++++++++++------ src/lib/therapy/commit.ts | 21 ++++++++++++++++----- 2 files changed, 29 insertions(+), 11 deletions(-) diff --git a/docs/therapy-storage.md b/docs/therapy-storage.md index 6e76cf7..a1f5b3f 100644 --- a/docs/therapy-storage.md +++ b/docs/therapy-storage.md @@ -28,10 +28,10 @@ GET /share/[token] redeem a link: a route handler under [locale], ## The storage model -- **Nothing reads a night by parsing bytes.** The import files stay in `pap_file` as the record of what the device - wrote, but every screen reads the parsed form: `pap_day` for the summary, `pap_event` for the scored events and - `pap_channel` for the waveforms. Re-parsing EDF on every day open is what made the panel slow, and it also meant - opening one night re-derived the whole card's `STR.edf`. +- **Nothing reads a night by parsing bytes.** `pap_file` is where an upload lands and the only thing that reads it + is the commit; every screen reads the parsed form instead: `pap_day` for the summary, `pap_event` for the scored + events and `pap_channel` for the waveforms. Re-parsing EDF on every day open is what made the panel slow, and it + also meant opening one night re-derived the whole card's `STR.edf`. - **A channel is stored the way the device wrote it.** `pap_channel.samples` holds little endian Int16 samples with the one linear mapping (`scale`, `offset`) that turns them into physical units, so what is written and what is read back are identical by construction rather than by two formulas agreeing. `scale` and `offset` are @@ -54,8 +54,15 @@ GET /share/[token] redeem a link: a route handler under [locale], A year of nights therefore never needs a single request large enough to parse all of it, and the client's own parse is never what gets stored, or a crafted request could write any number it liked into a reader's history. Every slice is idempotent, so the client retries rather than throwing away a long upload. -- **A day is replayed from the import that owns it.** `pap_file.dayId` is set at commit from the filename - timestamp, because `DATALOG/20260809/` holds the sessions of the night that _started_ on the 8th. +- **A night's files are found through the day they were written for.** `pap_file.dayId` is set in the opening + slice from the filename timestamp, because `DATALOG/20260809/` holds the sessions of the night that _started_ on + the 8th. +- **The commit consumes the bytes it parses.** A card is uploaded to be read once, so nothing keeps it afterwards: + the opening slice deletes every file that got no day, which is the card level ones it just parsed into the + import along with the undated and the replaced, and `fillDay` deletes a night's files in the same transaction + that writes the night. Transactional because a slice that rolls back has to keep them, or the retry the client + makes would find a night it can no longer parse. A committed import therefore holds no bytes at all, and the + cost of that is real: a parser fix cannot be replayed over a night already stored, only over one uploaded again. - **A night that would not parse is never written as an empty one.** `buildDigitalDay` answers a throwing loader with no sessions, and storing that would report a night somebody slept through as no usage and no AHI. So `fillDay` keeps whatever the card itself already said about that night and drops the row when the card said diff --git a/src/lib/therapy/commit.ts b/src/lib/therapy/commit.ts index d02d75a..77da3a2 100644 --- a/src/lib/therapy/commit.ts +++ b/src/lib/therapy/commit.ts @@ -177,9 +177,11 @@ async function beginCommit(userId: string, importId: string): Promise date !== null && !dates.includes(date)) .map(([path]) => path) - if (orphaned.length > 0) { - await tx.delete(papFile).where(and(eq(papFile.importId, importId), inArray(papFile.path, orphaned))) - } + + // A file that did not get a day belongs to no night this card kept: the card level ones, parsed + // into the import above and never read again, the ones whose night was replaced, and the ones no + // date could be read from. `fileCount` already records what the card held, so the bytes can go. + await tx.delete(papFile).where(and(eq(papFile.importId, importId), isNull(papFile.dayId))) await tx .update(papImport) @@ -253,8 +255,12 @@ async function fillDay( const dropped = summary ? 0 : stored.length await db.transaction(async (tx) => { - if (summary) await tx.update(papDay).set({ filledAt: new Date() }).where(eq(papDay.id, day.id)) - else await tx.delete(papDay).where(eq(papDay.id, day.id)) + if (summary) { + await tx.update(papDay).set({ filledAt: new Date() }).where(eq(papDay.id, day.id)) + await tx.delete(papFile).where(eq(papFile.dayId, day.id)) + } else { + await tx.delete(papDay).where(eq(papDay.id, day.id)) + } await tx .update(papImport) @@ -298,6 +304,11 @@ async function fillDay( await tx.delete(papEvent).where(eq(papEvent.dayId, day.id)) await tx.delete(papChannel).where(eq(papChannel.dayId, day.id)) + // The night is stored in the three tables every screen reads, so the files it was parsed from are + // dropped in the same transaction: a slice that rolls back keeps them, and a night is never left + // filled with the bytes behind it already gone. + await tx.delete(papFile).where(eq(papFile.dayId, day.id)) + const events = built.day.sessions.flatMap((session, sessionIndex) => session.events.map((event) => ({ userId,