Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/architecture/18-file-handling.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 18: File handling

**Status**: normative for the decisions it records. R64 the file core, R65 the file server, R66 the connetto seam, R67 the native client and R68 the browser client are built. R69 the demos is designed (2026-09-12) and not built. R79 the peer link and R87 the quotas are not built. Every statement carries **Decided (RN)** or an **Amended (RN)** beside it, where `RN` is the phase in `plans/master-implementation-plan.md` that owns it, and that phase's section records each decision with its rejected alternatives. Chapter 07 is the historical record of the thinking that preceded these decisions and defers to this chapter wherever the two disagree.
**Status**: normative for the decisions it records. R64 the file core, R65 the file server, R66 the connetto seam, R67 the native client and R68 the browser client are built. R69 the demos is in progress, its executable half, demo schemas, tab and worker content protocol and browser-stack wiring built 2026-09-17 as pull requests #28 to #31, with the demo surfaces and the offline and two-viewer proofs open. R79 the peer link and R87 the quotas are not built. Every statement carries **Decided (RN)** or an **Amended (RN)** beside it, where `RN` is the phase in `plans/master-implementation-plan.md` that owns it, and that phase's section records each decision with its rejected alternatives. Chapter 07 is the historical record of the thinking that preceded these decisions and defers to this chapter wherever the two disagree.

---

Expand All @@ -12,9 +12,9 @@ Applications handle files: a photo attached to an entry, a dataset a scientist c

**Decided (R24, concluded 2026-08-21).** connetto-core does not build file handling. The boundary is the crate, not the repository: the file crates live in this repository beside the demos, depend on connetto and never the reverse, and the demos carry the feature end to end per R54's rule. `connetto-core` deletes the old unimplemented `FileStore` trait and gains exactly one seam, the signer trait described under Tickets below (**Decided (R66)**).

**Amended (R69, 2026-09-12): "never the reverse" binds the libraries, and the shipped executable serves files.** `bin/connetto-server.rs` takes `connetto-file-server` behind a default-on `content` feature, mounts the four file routes beside its websocket and auth listeners, builds the ticket signer and verifier from one keypair in `main` so no key crosses a process, gives the file router its own admin and reader pools so uploads never compete with the change stream for the owner pool, and runs the sweep on a cadence. `CONNETTO_CONTENT_URL` unset keeps today's `NoSigner` and no routes. No connetto library depends on a file crate. Rejected: a leaf crate holding the executable to keep the sentence as written, and a second process now, which every demo and CI stack would start for a scale nobody has. A file-only executable over the same `Config` is additive the day a deployment scales files out.
**Amended (R69, 2026-09-12), built 2026-09-17 in #28 and #31: "never the reverse" binds the libraries, and the shipped executable serves files.** `bin/connetto-server.rs` takes `connetto-file-server` behind a default-on `content` feature, mounts the four file routes beside its websocket and auth listeners, builds the ticket signer and verifier from one keypair in `main` so no key crosses a process, gives the file router its own admin and reader pools so uploads never compete with the change stream for the owner pool, and runs the sweep on a cadence. `CONNETTO_CONTENT_URL` unset keeps today's `NoSigner` and no routes. No connetto library depends on a file crate. Rejected: a leaf crate holding the executable to keep the sentence as written, and a second process now, which every demo and CI stack would start for a scale nobody has. A file-only executable over the same `Config` is additive the day a deployment scales files out.

**Decided (R69, 2026-09-12): a browser tab stages by attaching the bytes to its own mutation.** The tab computes the file identity itself, writes the metadata row on its mirror as an ordinary captured mutation, and the bytes ride the tab's internal wire as an attachment, a `Blob` that crosses to the worker without a copy. The worker runs `stage` with the tab's changeset as the row closure, so the manifest, the outbox entry and the row commit in one transaction exactly as natively, and a file identity that disagrees between tab and worker rejects the whole mutation through the existing rollback path. Resolving is a correlated request on the same wire answered with a signed URL, a `Blob` the tab turns into an object URL it owns, or unavailable. The server protocol is untouched. Rejected: staging in the worker first and writing the row after, two transactions where a crash between them strands an outbox entry whose ticket is refused forever, and sending SQL beside the bytes, a second write vocabulary.
**Decided (R69, 2026-09-12), built 2026-09-17 in #30 and #31: a browser tab stages by attaching the bytes to its own mutation.** The tab computes the file identity itself, writes the metadata row on its mirror as an ordinary captured mutation, and the bytes ride the tab's internal wire as an attachment, a `Blob` that crosses to the worker without a copy. The worker runs `stage` with the tab's changeset as the row closure, so the manifest, the outbox entry and the row commit in one transaction exactly as natively, and a file identity that disagrees between tab and worker rejects the whole mutation through the existing rollback path. Resolving is a correlated request on the same wire answered with a signed URL, a `Blob` the tab turns into an object URL it owns, or unavailable. The server protocol is untouched. Rejected: staging in the worker first and writing the row after, two transactions where a crash between them strands an outbox entry whose ticket is refused forever, and sending SQL beside the bytes, a second write vocabulary.

The file crates own the traits with more than one implementation: `ChunkStore` (`write_chunk`, `read_chunk`, `has_chunk` and `delete_chunk` by content hash, with OPFS, `std::fs`, in-memory and object-store implementations), the application-facing content resolver, and the pin policy (**Decided (R64, R67)**).

Expand Down
2 changes: 1 addition & 1 deletion plans/master-implementation-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -4733,7 +4733,7 @@ The offline photo case runs in headless Chrome end to end, and an export taken o

## R69: files in every demo

**Status.** NOT STARTED, **designed 2026-09-12** with the maintainer. The phase as first written assumed three things that did not exist, found by reading every module of the file stack against the two steps: no `main` ran the file server's router (only `crates/connetto-file-client/tests/it/offline_photo.rs` binds one), the shipped `connetto-server` executable passed `NoSigner` (`bin/connetto-server.rs`) so no deployment could mint a ticket, and no demo schema declared the metadata table or the two SQL functions the file server's preflight requires. Six decisions below close them, each with what was rejected, and the steps are seven pull requests with their dependencies stated so three of them run at once.
**Status.** IN PROGRESS (2026-09-17), **designed 2026-09-12** with the maintainer. A is #28, B is #29 and C is #30, and #31 landed the browser-stack half of A with `examples/wasm-smoke/tests/photo_flow.rs` driving a photo through stage, commit, resolve and fetch in headless Chrome, so the executable's file half now boots for real and the online round trip is proven in CI. D, E and F remain, and F is next, the offline stage that uploads on reconnect and the second viewer refused without the grant. The phase as first written assumed three things that did not exist, found by reading every module of the file stack against the two steps: no `main` ran the file server's router (only `crates/connetto-file-client/tests/it/offline_photo.rs` binds one), the shipped `connetto-server` executable passed `NoSigner` (`bin/connetto-server.rs`) so no deployment could mint a ticket, and no demo schema declared the metadata table or the two SQL functions the file server's preflight requires. Six decisions below close them, each with what was rejected, and the steps are seven pull requests with their dependencies stated so three of them run at once.

**Blocked on nothing.** R64 to R68 are done, and the twelve questions raised before this phase (`plans/open-questions-before-r69.md`) were settled, and the nine needing code were built 2026-09-14 to 2026-09-16 as pull requests #18 to #27: the teardown list, the anonymous boot, the two permanent upload outcomes, the silence-bounded transfer, the streaming archive, the memory fallback, the keyring index, the span-chained log capture and the cursor-bounded silence assertion, each recorded in chapters 13, 14, 18 or `open-questions.md`.

Expand Down
Loading