diff --git a/docs/implementation-checklist.md b/docs/implementation-checklist.md index 8fad0af..f4316c6 100644 --- a/docs/implementation-checklist.md +++ b/docs/implementation-checklist.md @@ -1,8 +1,8 @@ # Implementation checklist Status: **active** -Current work: **[network topology lab](plans/network-lab-plan-2026-08-31.md)** — item 5, giving the peer-to-peer plan's unreachable validation gates a network to run on. Peer-to-peer transport is reachable and proved over a real network; its Phase 5 (documentation) still remains -Last updated: **2026-08-31** +Current work: **[browser client removal](plans/browser-client-removal-plan-2026-09-11.md) phase 0**, then tests on Windows, macOS and Linux (item 10, phase 0). Re-prioritised 2026-09-14 around receiver consent and status (item 11), cancel, NAT proof and cross-platform support; the order is in [`plans/README.md`](plans/README.md#suggested-order-dependencies-not-law) +Last updated: **2026-09-14** The tactical view of what is being built and what state it is in. The detailed reasoning, risks, and validation for each item live in its plan under @@ -29,6 +29,13 @@ confirmation, and transport was added. The reasoning and the cost of the swap are in [`plans/README.md`](plans/README.md#suggested-order-dependencies-not-law). +**Revised 2026-09-14** at the user's direction: receiver consent, cancel and +live status (item 11), proof of NAT traversal (items 5 and 6), and Windows, +macOS and Linux with transfers between them (item 10). 0.4.0 bundles the browser +removal (item 8), `meta_ok` key confirmation (item 3) and item 11's protocol +change, so users take one wire break, not three. The full dependency order is +in the plans index. + The network lab (item 5) was added 2026-08-31 and runs alongside rather than in that sequence. It builds nothing the other items depend on; it gives item 3's unchecked validation gates somewhere to run, so it follows transport and does @@ -155,6 +162,10 @@ cannot work. Recorded in [`decisions.md`](decisions.md) entry 10. *noticing* that entry 13 chose the prompt for does not. Fix is a fourth HKDF output under `drop/v1/confirm`, compared in constant time. Land it before the direct path ships — after that it is a wire break. + **Corrected 2026-09-14: that deadline passed.** The direct path, on by + default, shipped in v0.2.0 (`cli/src/direct.rs` is in that tag), so this + is now a wire break. It ships in 0.4.0 under the same version bump as + item 11, so users take one break, not two. - [x] Phase 4 — selection, automatic fallback, and reporting the path taken. Done 2026-08-29. `--transport p2p|relay|auto` (and `DROP_TRANSPORT`), defaulting to `auto`; a locally drawn nameplate, since a serverless send @@ -179,25 +190,10 @@ so could not express the bug. Now pinned by ## 4. Receiver preview and confirmation Plan: [`receiver-confirmation-plan-2026-08-19.md`](plans/receiver-confirmation-plan-2026-08-19.md) -Status: **proposed — needs revision** - -Moved behind encryption on 2026-08-20. Its plan is written against a cleartext -`meta` that encryption seals, so it must be revised before it is implemented, -not followed as written. - -The receiver sees name, size, type, and destination, and answers y/n before any -bytes move. - -- [ ] Phase 1 — protocol: accept/decline, `receiver_accepted`, decline as a - normal outcome, accept deadline -- [ ] Phase 2 — safe rendering of peer-supplied names -- [ ] Phase 3 — CLI prompt, ahead of destination creation, plus `--yes` -- [ ] Phase 4 — web confirmation step -- [ ] Phase 5 — decide whether `Meta` gains a file count +Status: **abandoned — superseded by item 11** on 2026-09-14 -Gate: declining leaves nothing on disk and is not counted as a failure; a -filename carrying escape sequences renders inert; a non-TTY without `--yes` -fails clearly. +Written against cleartext metadata and a browser client. Its findings carry +forward into item 11; its protocol design does not. ## 5. Network topology lab @@ -284,6 +280,19 @@ binaries and inspects what comes out. - [ ] Phase 5 — dated report under [`validation/`](validation/) and a separate CI workflow, nightly and label-triggered, never blocking pull requests +- [ ] **Found 2026-09-14: hole punching has never been exercised here.** The + peers' QUIC address discovery fails TLS against the lab helper's + self-signed certificate (`invalid peer certificate: UnknownIssuer`), so + neither learns its public address (`global_v4: None`) and no punch is + ever attempted. A throttled 48 MiB run showed the payload on the + rendezvous link for all 27 seconds, and conntrack on both NATs showed + peers dialling each other's *private* addresses only. The full-cone + failure is this, not iroh failing to punch. The helper's comment and the + rendezvous plan's risk entry both claimed iroh skips that verification, + and neither is true for 1.0.3. The fix needs a decision: item 6 phase 3. + Separately, `authentication failed` still appears intermittently (2 of 4 + runs today) and is unexplained. + Gate: every topology fails when its defining condition is removed, demonstrated once per topology and recorded. A lab that passes either way is measuring nothing, which is the failure the peer-to-peer plan's loopback tests already @@ -295,7 +304,7 @@ absent, and now it can. ## 6. Self-hosted rendezvous Plan: [`self-hosted-rendezvous-plan-2026-09-10.md`](plans/self-hosted-rendezvous-plan-2026-09-10.md) -Status: **phase 1 done** +Status: **phases 1 and 2 done; phase 3 awaiting a decision** `DROP_RENDEZVOUS_RELAY` and `DROP_RENDEZVOUS_BOOTSTRAP` point the direct path at an iroh relay and DHT nodes a deployment runs itself, instead of n0's relays and @@ -320,6 +329,15 @@ the public mainline routers compiled in. Unset, nothing changes. matters — this machine can reach the real DHT, so a passing transfer alone would not show which one carried the rendezvous. +- [ ] Phase 3 — proposed 2026-09-14, **needs the user's decision** because it + changes what the CLI trusts. A relay with a certificate from a private CA + silently gets no address discovery, so no hole punching: transfers still + complete over the relay and `--status` still says `path=p2p`. Either + `DROP_RENDEZVOUS_CA` adds an operator's CA for the rendezvous relay only, + or the docs say traversal needs a publicly trusted certificate. Both come + with a warning when a custom relay yields no discovered address. The lab + needs the former to prove traversal at all. + Why this is a feature and not a knob added for a test is argued in the plan and in entry 15: a self-hoster can already run their own relay, but rendezvous was compiled in, so the direct path could not work at all inside an egress-filtered @@ -376,6 +394,145 @@ justifies every entry it has. Both are pure Rust, which is the standing bar for the four prebuilt targets. Release binary went 26,988,848 → 27,482,144 bytes, **+493 KB (+1.8%)**, so the crossterm-only fallback is not needed. +## 8. Browser client removal + +Plan: [`browser-client-removal-plan-2026-09-11.md`](plans/browser-client-removal-plan-2026-09-11.md) +Status: **active** — open questions answered 2026-09-14 + +Delete `web/` and `crypto-wasm/` and every dependent — the server routes that +serve the client, the CI job that builds it, the Docker stage that bundles it, +and the documentation that describes it. **The relay stays**: entry 16 removed +the browser's reason for it, not the UDP-blocked network's, and `netlab` covers +that one. + +- [ ] Phase 0 — move `install.sh` out of `web/public/`. **Release-critical and + lands alone.** `release.yml` sparse-checks it out at line 130 and + publishes it at 153 under `fail_on_unmatched_files: true`, so deleting + `web/` first fails the next tag in `publish`, after the whole build matrix + has already succeeded. +- [ ] Phase 1 — the server stops serving a browser: the `/` and `/assets` + routes, and the `index_serves_the_drop_entrypoint` test that asserts the + Svelte entrypoint. +- [ ] Phase 2 — delete `web/` and `crypto-wasm/`, the workspace member, and + `.cargo/config.toml`'s wasm32 rustflag. +- [ ] Phase 3 — the `web` CI job, `Dockerfile.fullstack`, and the ignore-file + entries. +- [ ] Phase 4 — documentation across nine files, `decisions.md` entry 17, and + entry 11 marked superseded rather than deleted. + +Gate: the workspace is green with two members rather than three, a CLI-to-CLI +transfer over `--transport relay` still completes, and a release built from the +resulting tree publishes `install.sh` from its new path. The third cannot be +checked by running tests and has to be read against the workflow before tagging. + +Open questions answered 2026-09-14, all as the plan leaned: remove +`DROP_ALLOWED_ORIGINS`; `GET /` answers a plain-text 404 naming the project; +the browser claim rules in `AGENTS.md` stay, reworded as dormant; ships in 0.4.0 +with `meta_ok` confirmation and item 11. The old `chore/remove-web` branch +deletes `install.sh` first, which is the exact thing phase 0 exists to prevent, +so it is not the base for this work. + +## 9. Browser client on iroh + +Plan: [`browser-on-iroh-plan-2026-09-11.md`](plans/browser-on-iroh-plan-2026-09-11.md) +Status: **proposed — not scheduled** + +Rebuild the browser client as an iroh node compiled to WebAssembly, speaking the +same conversation the CLI speaks, so the relay stops translating between two +dialects — it renames and invents control frames today, which is item 3 phase +1's finding and the last consumer of that is the browser. + +Recorded so it is not rediscovered as new. **Nothing here is committed work**, +and open question 1 — whether a browser client that can never be direct is worth +its maintenance — should be answered before any of it is started. + +- [ ] Blocker 1 — `Transport`'s futures are all declared `Send` and wasm futures + are not. Wants n0-future's conditional-`Send` approach, and it is a + refactor of shipped tested code for the benefit of code that does not + exist yet. +- [ ] Blocker 2 — `discovery-pkarr-dht` cannot run in a browser. Rendezvous + needs pkarr over HTTP relay, which `DROP_RENDEZVOUS_BOOTSTRAP`'s + `host:port` shape does not describe, and which has to inherit entry 15's + no-silent-fallback rule somewhere a user cannot read an error. +- [ ] Blocker 3 — `peers_enforce_one_guess` needs a third answer, since a + browser has neither a Drop relay refusing a second claim nor a terminal to + ask. A decisions entry, not a plumbing choice. +- [ ] Blocker 4 — item 3's `meta_ok` confirmation lands first, or the wire + breaks twice. +- [ ] Blocker 5 — `send::run` and `recv::run` spool and write to a filesystem a + browser does not have. The middle of the stack is shared; both ends are + per-platform. + +A browser peer is **permanently relayed** — iroh cannot hole-punch from a +sandbox, and WebTransport and WebRTC are both unimplemented there — so a browser +transfer is never the direct path and must never be described as one. + +## 10. Windows, macOS and Linux + +Plan: [`cross-platform-plan-2026-09-14.md`](plans/cross-platform-plan-2026-09-14.md) +Status: **proposed** + +Install and run on all three, and a file or folder sent between any two of them +arrives intact or says exactly what could not be reproduced. **Today: Linux +works, macOS is built but has never had a test run on it, Windows has no +build.** CI runs on Ubuntu only, so no `cfg(not(unix))` branch has ever +compiled. + +- [ ] Phase 0 — the Rust job on `ubuntu-24.04`, `macos-14` and `windows-2025`, + and record what fails before fixing any of it. +- [ ] Phase 1 — a Windows receiver: a symlink it cannot create is a warning, + not an abort (today it aborts every Linux-to-Windows folder with a + symlink in it); names Windows reads differently (`a:b` is an NTFS stream, + `CON` a device, `report.` loses its dot) are rewritten with a warning; + an uncreatable entry on any platform warns and continues. +- [ ] Phase 2 — a Windows sender: portable symlink targets, spool cleanup when + the console is closed, VT processing for the progress line. +- [ ] Phase 3 — `x86_64`/`aarch64-pc-windows-msvc` release zips with a static + CRT, `install.ps1`, README install lines. +- [ ] Phase 4 — archives produced on each OS and extracted on the others in CI, + nine pairings. +- [ ] Phase 5 — manual Windows ↔ Linux checklist on the user's machine, both + carriers, the interface, Ctrl-C, closing the window, the firewall dialog. +- [ ] Phase 6 — documentation and a decision entry for the Windows name policy. + +Gate: CI green on three OSes; hostile Windows names contained and honest ones +rewritten with warnings; Windows assets install; nine archive pairings green; the +manual checklist recorded. + +## 11. Receiver consent, cancel, and live status + +Plan: [`receiver-consent-and-status-plan-2026-09-14.md`](plans/receiver-consent-and-status-plan-2026-09-14.md) +Status: **proposed** + +The receiver sees name, type, size and where it will be saved, and accepts +before a byte is written. Either side can cancel and the other is told in words. +The sender sees the receiver connect, pass the code, review, accept or decline, +receive, finish. + +- [ ] Phase 0 — decisions entry; consent before bytes, `--yes` required without + a terminal (user decision 2026-09-14), reasons as enumerations, version 2, + exit codes. +- [ ] Phase 1 — display sanitisation, alone. **Live bug**: a received filename + reaches `eprintln!` unfiltered today, so an escape sequence in it can + redraw the terminal. +- [ ] Phase 2 — protocol and relay: `meta_ok` on both carriers, `accept`, + `decline`, receiver `cancel`, `finishing`; the relay refuses chunks + before `accept` and stops counting declines and cancels as failures; + `ENVELOPE_VERSION` and `DROP_ALPN` to 2. +- [ ] Phase 3 — receiver consent in the CLI: destination planned but not + created, preview with a receiver-derived type and a program warning, + 120 s deadline, refusal without a terminal before connecting. +- [ ] Phase 4 — cancel through both transfer paths, two-stage Ctrl-C, sender + state lines and `drop-status: state=`, exit codes 3 and 4. +- [ ] Phase 5 — review and transfer screens with Accept/Decline and Cancel, + with item 7 phase 3. +- [ ] Phase 6 — documentation; release notes lead with `--yes`. + +Gate: declining leaves the destination unchanged and the sender exits 3; a +hostile filename renders inert everywhere; either side's cancel reaches the other +in words on both carriers; no terminal without `--yes` refuses before contacting +anything; 0.3.0 against 0.4.0 fails with a sentence, not a hang. + ## Not scheduled Recorded so they are not rediscovered as new ideas. None are committed work. diff --git a/docs/plans/README.md b/docs/plans/README.md index 2fa318b..78ad170 100644 --- a/docs/plans/README.md +++ b/docs/plans/README.md @@ -40,9 +40,72 @@ the plan contract unmissable and indexes what is here. Phase 2 is the QUIC transport, and starts with a decision the refactor surfaced — the relay renames and invents control frames, and a direct connection has nobody to do that. +- [`browser-client-removal-plan-2026-09-11.md`](browser-client-removal-plan-2026-09-11.md) + — **active 2026-09-14**, open questions answered; phase 0 is next and lands + alone. Delete `web/` and `crypto-wasm/` and every dependent, keeping the relay. + [`../decisions.md`](../decisions.md) entry 16 removed the browser client's + audience: with no hosted relay and no compiled-in default, only someone + self-hosting the fullstack image can reach it. Phase 0 stands alone and is + release-critical — `release.yml` publishes `web/public/install.sh`, so the + script moves to `scripts/` before anything is deleted or the next tag fails + in `publish` after a full build matrix. The relay is explicitly not in scope: + its browser justification is gone, its UDP-blocked one is real and `netlab` + covers it. +- [`network-lab-plan-2026-08-31.md`](network-lab-plan-2026-08-31.md) + — **2026-09-14 finding:** hole punching has never been exercised in the lab. + QUIC address discovery fails TLS against the lab's self-signed helper + (`UnknownIssuer`), so no peer learns its public address and no punch is + attempted. The full-cone row fails for that reason, not because iroh cannot + punch. The same gap hits a self-hosted relay with a private-CA certificate; + see the self-hosted rendezvous plan's phase 3. Background: + a `netlab/` directory that runs the real binaries inside Linux network + namespaces against constructed topologies, so the peer-to-peer plan's + validation gates stop being unreachable by hand. Two findings shape it: the + lab needs no root, because an unprivileged user namespace grants + `CAP_NET_ADMIN` inside itself; and the direct path cannot run hermetically as + the code stands, because rendezvous needs the public DHT, n0's relays, and an + address that [`../decisions.md`](../decisions.md) entry 14 deliberately + refuses to publish. The relay-path topologies are unblocked and come first. +- [`self-hosted-rendezvous-plan-2026-09-10.md`](self-hosted-rendezvous-plan-2026-09-10.md) + — phases 1 and 2 shipped in 0.3.0. Phase 3, proposed 2026-09-14 and awaiting + the user's decision: trust an operator's CA for the rendezvous relay, or + document that traversal needs a publicly trusted certificate. ## Proposed +- [`receiver-consent-and-status-plan-2026-09-14.md`](receiver-consent-and-status-plan-2026-09-14.md) + — the receiver sees name, type, size and destination and accepts before a + byte is written; either side can cancel and the other is told in words; the + sender sees the receiver's state throughout. Supersedes the 2026-08-19 + confirmation plan. Three findings: a received filename can put escape + sequences on the terminal **today**, so display sanitisation lands first and + alone; Ctrl-C tells the peer nothing and the relay counts a cancel as a + failure; and the sender's progress is already acknowledgement-driven, so + "how much the receiver has" needs no new frame. A wire change, bumping + `ENVELOPE_VERSION` and `DROP_ALPN` to 2 together with `meta_ok` confirmation, + so 0.3.0 against 0.4.0 fails with a sentence instead of a hang. +- [`cross-platform-plan-2026-09-14.md`](cross-platform-plan-2026-09-14.md) + — Windows, macOS and Linux, and transfers between them. Today Linux is the + only platform tests have ever run on, macOS is built but untested, and + Windows has no build. Phase 0 puts the test suite on all three runners first, + because every `cfg(not(unix))` branch has never compiled. Findings from + reading: a symlink in a folder aborts every Linux-to-Windows folder transfer; + names like `a:b`, `CON` and `report.` mean something else to Windows (an NTFS + stream, a device, a stripped dot); closing the Windows console leaves spool + files in `%TEMP%`. Cross-OS confidence comes from archives produced on each OS + and extracted on the others in CI, plus a manual Windows ↔ Linux checklist on + the user's machine. The wire itself is platform-neutral. + +- [`browser-on-iroh-plan-2026-09-11.md`](browser-on-iroh-plan-2026-09-11.md) + — **not scheduled.** Rebuild the browser client as an iroh node compiled to + wasm, so a browser speaks the same conversation as the CLI and the relay + stops translating between two dialects. Recorded so it is not rediscovered as + new. Blocked on three things: the transport trait declares every future + `Send` and wasm futures are not, `discovery-pkarr-dht` cannot run in a + browser so rendezvous needs an HTTP relay the entry 15 variables do not + describe, and `meta_ok` must land first or the wire breaks twice. A browser + peer is permanently relayed — iroh cannot hole-punch from a sandbox — so it + is never the direct path. - [`interactive-terminal-ui-plan-2026-09-10.md`](interactive-terminal-ui-plan-2026-09-10.md) — make `drop send` and `drop recv` the only two things a person needs to know: typed bare on a terminal each opens a small full-screen interface, and @@ -52,25 +115,18 @@ the plan contract unmissable and indexes what is here. unwinding, so a raw terminal is never restored on Ctrl-C. Terminal lifecycle therefore lands before a single screen is drawn. Bare `drop` opens a chooser on a terminal, and still prints usage and exits 1 anywhere else. - - -- [`network-lab-plan-2026-08-31.md`](network-lab-plan-2026-08-31.md) - — a `netlab/` directory that runs the real binaries inside Linux network - namespaces against constructed topologies, so the peer-to-peer plan's - validation gates stop being unreachable by hand. Two findings shape it: the - lab needs no root, because an unprivileged user namespace grants - `CAP_NET_ADMIN` inside itself; and the direct path cannot run hermetically as - the code stands, because rendezvous needs the public DHT, n0's relays, and an - address that [`../decisions.md`](../decisions.md) entry 14 deliberately - refuses to publish. The relay-path topologies are unblocked and come first. - [`meta-ok-key-confirmation-plan-2026-08-31.md`](meta-ok-key-confirmation-plan-2026-08-31.md) — make the receiver prove it opened the sealed metadata instead of saying so. `meta_ok` carries a key-confirmation value derived from the agreed secret and compared in constant time, closing the case where an attacker who guessed wrong claims success and so suppresses the prompt entry 13 exists to raise. + +## Abandoned + - [`receiver-confirmation-plan-2026-08-19.md`](receiver-confirmation-plan-2026-08-19.md) - — show the receiver what it is about to accept and require a y/n before any - bytes move. Adds a protocol handshake and a new terminal outcome. + — superseded 2026-09-14 by the receiver consent and status plan. Written + against cleartext metadata and a browser client; its three findings carried + forward, its protocol design did not. ## Done @@ -120,3 +176,25 @@ story for the slow one. Confirmation last is otherwise unchanged, and no longer carries the caveat that it ships while the relay can forge the filename it displays. + +**Revised 2026-09-14**, at the user's direction. The priorities are now receiver +consent, cancel on both sides, and the sender seeing the receiver's state; proof +of NAT traversal; and Windows, macOS and Linux with transfers between them. The +order that serves them, by dependency: + +1. **Browser removal phase 0**, alone. Release-critical, and every later plan + edits `release.yml` or `ci.yml` after it. +2. **Cross-platform phase 0** (tests on three OSes). Cheap, and every later + change is then checked on Windows as it lands instead of all at once at + the end. +3. **Browser removal phases 1–4.** Deletes the `meta_ok` plan's browser phase and + the second protocol implementation that would otherwise need the new frames. +4. **Consent plan phase 1** (display sanitisation). A live bug and no wire + change. +5. **`meta_ok` key confirmation, then consent phases 2–4.** One wire bump, to + version 2. +6. **Cross-platform phases 1–3** (Windows receiver, sender, build and install), + then **consent phase 5** with the interface's transfer screen. +7. **Tag 0.4.0.** +8. Then the NAT proof, once self-hosted rendezvous phase 3 is decided, and + cross-platform phases 4–5. diff --git a/docs/plans/browser-client-removal-plan-2026-09-11.md b/docs/plans/browser-client-removal-plan-2026-09-11.md new file mode 100644 index 0000000..ccfdb79 --- /dev/null +++ b/docs/plans/browser-client-removal-plan-2026-09-11.md @@ -0,0 +1,379 @@ +# Browser client removal plan + +Status: **active** — open questions answered 2026-09-14, phase 0 next +Created: **2026-09-11** +Last updated: **2026-09-14** + +## Goal + +Delete `web/` and `crypto-wasm/`, and every dependent that assumes a browser +client exists — the server routes that serve it, the CI job that builds it, the +Docker stage that bundles it, and the documentation that describes it. + +**The relay stays.** This plan removes the browser client, not `api`. The two +are easy to confuse because [`../decisions.md`](../decisions.md) entry 16 left +the relay with only one stated reason to exist, and that reason was the browser. +It has another one, it is tested, and [What does not change](#what-does-not-change) +argues it. + +## Why now + +Four reasons, in descending order of how much they would survive an argument. + +**1. Entry 16 already removed the browser client's audience.** With +`DEFAULT_SERVER` deleted and no hosted relay, a browser user has nothing to +connect to out of the box. The only person who can use `web/` today is someone +who self-hosts `Dockerfile.fullstack` — that is, someone who has already built a +deployment. Everyone else reaches a client that cannot reach a relay. + +**2. It has never been exercised by a browser.** This is the repository's own +admission, not a new finding — +[`../implementation-checklist.md`](../implementation-checklist.md) item 2 +records it: *"`App.svelte` itself. `tsc` does not check `.svelte` files and the +interop tests drive the envelope and the protocol from Node, not the UI. The +browser flows have been exercised by the build and by their shared envelope, not +by a browser."* An untested second implementation of the protocol is a liability +whether or not anyone uses it. + +**3. It costs every week.** The `web` CI job installs Node, a pinned +`wasm-pack`, and the `wasm32-unknown-unknown` target, then builds the workspace +binaries again for the interop test — the single most expensive job in +[`../../.github/workflows/ci.yml`](../../.github/workflows/ci.yml) at a +25-minute timeout against 15 for Rust. It generates dependabot traffic that is +pure noise once the client is gone (PR #61 is open at the time of writing). It +holds four items in [`release-checklist.md`](../release-checklist.md) that +nobody can honestly tick. + +**4. It costs almost nothing against the future.** The obvious counter-argument +is that a browser client is worth having and deleting it forfeits the work. It +mostly does not: see +[`browser-on-iroh-plan-2026-09-11.md`](browser-on-iroh-plan-2026-09-11.md), +which would replace `web/src/api.ts` (a WebSocket client to a relay this plan +keeps but that plan stops using) and `web/src/envelope.ts` (a wrapper around +`crypto-wasm`) in their entirety. What survives a rebuild is UI judgement, and +that is recoverable from git history at any time. + +## The branch that already exists + +`chore/remove-web` at `f30bcfb`, one commit, 22 files, 3,819 deletions, **28 +commits behind `main`**. No pull request is open for it. + +Its commit message is accurate about its own scope: *"Delete the browser client, +and nothing that depended on it yet."* Every dependent below is still present on +that branch, and **merging it as it stands breaks the next release** — see +phase 0. + +Either rebase it onto `main` and build the phases on top, or restart from `main` +and let it go. Rebasing keeps the deletion reviewable as one commit, which is +worth something; the branch is old enough that either is defensible. + +## What is being removed + +| Path | What it is | Action | +| --- | --- | --- | +| `web/` | Svelte and TypeScript browser client | delete | +| [`../../crypto-wasm/`](../../crypto-wasm/) | `drop-crypto` compiled to WebAssembly, bindings only | delete | +| [`../../Dockerfile.fullstack`](../../Dockerfile.fullstack) | builds the frontend and the server into one image | delete | +| `web/public/install.sh` | the CLI installer | **move**, see phase 0 | + +## What does not change + +- **The relay.** `api`, `src/ws/`, the session store, the rate limiter, the + budget accounting: all stay. Its browser justification is gone; its other one + is real and tested. `--transport relay` is what works inside a network where + UDP never gets out, `netlab`'s `udp_blocked` topology covers exactly that, and + entry 16 explicitly kept `DROP_SERVER` as the way a deployment runs its own. + Deleting the browser must not be allowed to drift into deleting the relay — + that is a separate decision with a separate argument, and this plan does not + make it. +- **`drop-crypto`.** `api` depends on it for version and limit constants, and + the CLI depends on it for the envelope. Only the wasm *bindings* crate goes. +- **The wire.** No protocol change, no framing change, no envelope change. A + `drop` binary built before this lands interoperates with one built after it. + This is the one reason the removal can ship independently of everything else + on the checklist. +- **The four release targets** and the install path. Phase 0 moves the script; + it does not change what it does or where a user gets it. +- **`DROP_RENDEZVOUS_*`, `--transport`, `--status`.** Untouched. + +## Phase 0 — move `install.sh` before anything is deleted + +**This phase is release-critical and lands on its own, ahead of the rest.** + +[`release.yml`](../../.github/workflows/release.yml) sparse-checks-out +`web/public/install.sh` at line 130 and publishes it at line 153 under +`fail_on_unmatched_files: true` at line 154. Deleting `web/` without moving the +script first means the next `v*` tag fails in the `publish` job — after the +build matrix has succeeded, so the failure arrives late and looks unrelated to +the commit that caused it. + +The script has nothing to do with the browser client. It downloads binaries from +GitHub releases and verifies them against `checksums.txt`; it lived under +`web/public/` only because entry 8's split deployment served it as a static file +from the frontend host. Entry 16 ended that arrangement. + +- [ ] Move `web/public/install.sh` to `scripts/install.sh`. `web/dist/install.sh` + is the committed build output of the same file and goes with `web/` in + phase 2. +- [ ] Update [`release.yml`](../../.github/workflows/release.yml) lines 123, 130 + and 153 to the new path. +- [ ] Remove the `/install.sh` route at + [`src/lib.rs:47-49`](../../src/lib.rs#L47-L49). The relay served it so + `curl https://drop.lifbom.com/install.sh` worked; that host is gone, and + [`../../README.md`](../../README.md) already points at + `github.com/op-q/drop/releases/latest/download/install.sh`. +- [ ] Check the installer still resolves: `DROP_VERSION=v0.3.0 sh scripts/install.sh` + into a scratch `DROP_INSTALL_DIR`. + +Landing this alone, before any deletion, means the release path is never broken +even for one commit — and if the rest of the plan stalls, nothing is left +half-done. + +## Phase 1 — the server stops serving a browser + +- [ ] Delete the routes at [`src/lib.rs:43`](../../src/lib.rs#L43) (`/` → + `web/dist/index.html`) and [`src/lib.rs:57`](../../src/lib.rs#L57) + (`/assets` → `web/dist/assets`). The `/install.sh` route went in phase 0. +- [ ] Drop the now-unused `ServeDir` / `ServeFile` imports at + [`src/lib.rs:28`](../../src/lib.rs#L28), and `get_service` from the + `axum::routing` import at [`src/lib.rs:19`](../../src/lib.rs#L19) if + nothing else uses it. +- [ ] Delete `index_serves_the_drop_entrypoint` at + [`tests/health_check.rs:59-76`](../../tests/health_check.rs#L59-L76). It + asserts the response body contains `
` and `./assets/`, + both of which are the Svelte entrypoint. +- [ ] Decide what `GET /` returns now. It must not 404 silently into a + monitoring gap — `/health` and `/ready` exist for probes, so the honest + options are a 404 with a one-line body naming the project, or a redirect + to the repository. See [open question 2](#open-question-2--what-does-get--return). + +## Phase 2 — delete the client and its wasm bindings + +- [ ] Delete `web/`. +- [ ] Delete `crypto-wasm/`. +- [ ] Remove `crypto-wasm` from `members` at + [`Cargo.toml:2`](../../Cargo.toml#L2). +- [ ] Delete [`.cargo/config.toml`](../../.cargo/config.toml). Its only content + is the `wasm32-unknown-unknown` `getrandom_backend` rustflag, and its own + comment says it is scoped to that target on purpose. With no wasm target + in the workspace the file has no remaining job. **Check first** that + nothing else was added to it since this plan was written. +- [ ] `cargo update --workspace` or equivalent so `Cargo.lock` drops + `drop-crypto-wasm` and the wasm-only dependency tree under it. + +## Phase 3 — build, CI, and container surface + +- [ ] Delete the `web` job, [`ci.yml:51-102`](../../.github/workflows/ci.yml#L51-L102). + That removes the Node setup, the pinned `wasm-pack` download, the wasm32 + target, `npm audit`, and the extra `cargo build --workspace --bins` that + existed to stop the interop test skipping silently. +- [ ] Delete [`Dockerfile.fullstack`](../../Dockerfile.fullstack) and point + [`docker-compose.yml:5`](../../docker-compose.yml#L5) at + [`Dockerfile`](../../Dockerfile), which already builds `api` alone and + needs no change. +- [ ] Remove `web/node_modules` and `web/dist` from + [`.dockerignore:6-7`](../../.dockerignore#L6-L7). +- [ ] Remove the web entries from [`.gitignore:9-12`](../../.gitignore#L9-L12) + and fix the section comment at line 1 (`# Rust and web build state`). +- [ ] Remove `*.svelte text` at + [`.gitattributes:4`](../../.gitattributes#L4) and the Vite whitespace + exemption at lines 8-10. Keep `*.ts text` only if any TypeScript remains; + after this plan, none does. + +## Phase 4 — documentation + +The largest phase by file count and the easiest to leave half-done. Every item +here is a claim that becomes false the moment phase 2 lands, and +[AGENTS.md](../../AGENTS.md) requires README claims to track tested behavior. + +- [ ] [`AGENTS.md`](../../AGENTS.md): delete the web build rule and the + `tsc`/`.svelte` rule at lines 71-74, and the five `npm --prefix web` + commands at lines 86-90. **The two claim invariants at lines 33-40 are a + separate question** — see [open question 3](#open-question-3--the-claim-invariants). +- [ ] [`architecture.md`](../architecture.md): drop the `crypto-wasm/` and + `web/` rows from the crate table (lines 47-48), redraw the dependency + diagram at lines 55-56, drop the Browser row from the transfer-work table + at line 87, and reword line 28 — *"It stays as the fallback for browsers, + for UDP-blocked networks, and for the NAT cases hole-punching cannot + solve"* — to the two reasons that survive. Line 37's "four members plus a + web client" becomes three members. +- [ ] [`commands.md`](../commands.md): delete the npm block at lines 14-18, the + wasm toolchain setup at lines 27-38, the interop note at lines 40-41, the + second npm block at 65-66, and the Vite dev-server section at 78-81. +- [ ] [`deployment.md`](../deployment.md): drop Node.js and npm from + Requirements (lines 10-11), fix "Run it from source" (line 17) to + `cargo run` alone, remove `VITE_BACKEND_ORIGIN` from the configuration + table (line 38), rewrite the Docker section (lines 81-83), and delete the + "Split deployment" section (lines 89-100) — a frontend/backend split with + no frontend is not a shape anyone can deploy. +- [ ] [`release-checklist.md`](../release-checklist.md): delete the three npm + commands (lines 33-35) and the `npm audit` item (42), and the browser + smoke tests at lines 54, 58 and 62. Line 62's direct-to-disk item refers + to the File System Access API and goes with them. +- [ ] [`security.md`](../security.md): lines 24-47 are the CLI-versus-browser + section and lines 223-229 the known-weaknesses entries that depend on it. + Subject to [open question 3](#open-question-3--the-claim-invariants). +- [ ] [`docs/README.md`](../README.md): line 106's browser caveat. +- [ ] [`k8s/README.md`](../../k8s/README.md): lines 78-83 justify + `WS_MAX_MESSAGE_BYTES` by *"The browser client sends 64 KiB chunks, so the + cap leaves four times the headroom it needs."* **That paragraph is already + wrong and this is a good moment to fix it**: the constant is + `RECOMMENDED_CHUNK_BYTES + 64 * 1024` at + [`src/config.rs:47`](../../src/config.rs#L47) — 1 MiB + 64 KiB, not the + 256 KiB the text claims. Re-justify it against the CLI's 1 MiB chunk. No + code change; the cap is already keyed to the shared constant rather than + to the browser. +- [ ] [`README.md`](../../README.md): no change needed. Its only match on + "browser" is *"a file browser for `send`"*, which is the terminal UI. + Confirm rather than assume. +- [ ] [`decisions.md`](../decisions.md): add **entry 17**, recording the removal + and its reasoning. Mark **entry 11** (the browser runs the envelope as + WebAssembly) superseded, the way entry 8 was marked by 16, rather than + deleting it. Entry 11 is the record of why `crypto/` is a separate crate, + and `crypto/Cargo.toml:15`, `cli/src/lib.rs:9` and `cli/tests/protocol.rs:57` + all cite that reason in comments — they need rewording to say the split is + kept for the envelope's own sake and for whatever compiles it next. + +## Risks + +**The release breaks if phase 0 is skipped or reordered.** The whole reason +phase 0 exists. `fail_on_unmatched_files: true` means the failure is loud, but +it lands in the `publish` job after a full build matrix has run. + +**Self-hosters of `Dockerfile.fullstack` lose their browser UI.** This is the +one genuinely breaking change for a real user, and it deserves release notes +saying so plainly rather than a line in a changelog. It is a minor-version +change: no wire change, no CLI change, a removed deployment shape. + +**UI work is discarded.** Mitigated but worth stating: `App.svelte` is 1,072 +lines and recoverable from `main` at `a1e84d6` or from `f30bcfb^`. Name that +revision in entry 17 so a future reader does not have to bisect for it. + +**Scope drift into deleting the relay.** Called out in +[What does not change](#what-does-not-change) because it is the plausible +mistake here, not a theoretical one — entry 16 genuinely did remove the relay's +*stated* reason to exist, and a reader who finds that entry first will conclude +the relay is next. + +## Validation + +Run the full set from [`commands.md`](../commands.md), less the npm half that +this plan deletes: + +```bash +scripts/check-secrets.sh +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings +cargo test --workspace --all-targets +``` + +- [ ] The workspace builds and tests green with two members, not three. Record + the test count; it should drop by exactly one (`index_serves_the_drop_entrypoint`). +- [ ] `git grep -n 'web/'` returns nothing outside `docs/plans/` and the + superseded `decisions.md` entries. Historical records keep their + references; live documentation does not. +- [ ] `git grep -ni 'svelte\|wasm-pack\|vite\|npm '` returns nothing outside + those same two places. +- [ ] `docker compose up --build` starts and `/health` answers. +- [ ] `kubectl kustomize k8s/overlays/local` and `.../gke` still render — the + `kubernetes` CI job covers this, but the manifests reference the image + built by the compose file. +- [ ] `netlab` still passes. It drives the real binaries and the real relay and + never touched the browser, so this should be a no-op — which is the point + of checking. +- [ ] A CLI-to-CLI transfer over `--transport relay` completes. The relay is the + thing most at risk of being damaged by a plan about removing something + else. + +**Gate:** the workspace is green with `web/` and `crypto-wasm/` gone, a relayed +CLI-to-CLI transfer still completes, and a release built from the resulting tree +publishes `install.sh` from its new path. The third is the one that cannot be +checked by running tests, so check it against the workflow file by hand before +tagging. + +## Open questions + +### Open question 1 — does `DROP_ALLOWED_ORIGINS` survive? + +CORS has exactly one consumer today and it is the browser client. +[`src/config.rs:123-137`](../../src/config.rs#L123-L137) builds the layer, +[`src/lib.rs:65`](../../src/lib.rs#L65) applies it, and nothing else sends an +`Origin` header — the CLI does not. After this plan it is configuration for +nobody, documented in a table that +[`release-checklist.md`](../release-checklist.md) requires to list every +variable the code reads. + +Note that it stays dead under +[`browser-on-iroh-plan-2026-09-11.md`](browser-on-iroh-plan-2026-09-11.md) too: +a browser iroh node talks to an *iroh* relay, not to `api`, so it would not make +a cross-origin request to this server either. + +Leaning: remove the layer and the variable, and say so in entry 17. Against: +someone may be serving their own frontend against a self-hosted relay today, and +this silently breaks them. Cheap compromise: keep it, and let the table say it +has no in-tree consumer. + +**Answered 2026-09-14, at the user's direction: remove it.** The layer and the +variable both go, and entry 17 and the release notes say so, so a self-hoster +serving their own frontend finds out from the notes rather than from a browser +console. + +### Open question 2 — what does `GET /` return? + +Options: 404 with a one-line body naming the project and linking the repository; +a 308 to the repository; or leaving the route absent so axum's default 404 +answers. A bare unstyled 404 at the root of a self-hosted relay reads as a +broken deployment, which matters because the operator is the only person who +will ever see it. + +Leaning: a 404 with a short plain-text body. It is honest, it needs no new +dependency, and it cannot be mistaken for a redirect loop. + +**Answered 2026-09-14: the leaning.** `GET /` answers 404 with one plain-text +line naming the project and the repository. + +### Open question 3 — the claim invariants + +[AGENTS.md](../../AGENTS.md) lines 33-40 and +[`security.md`](../security.md) lines 24-47 carry two rules that exist because a +browser client exists: *"Browser transfers never qualify"* for the +peer-to-peer claim, and *"Browser transfers are encrypted in the browser but are +only as strong as the code the site delivered."* + +With no browser client, both are rules about nothing. Deleting them is tidy and +is also how the claim gets made wrong the next time somebody ships a browser +client — and +[`browser-on-iroh-plan-2026-09-11.md`](browser-on-iroh-plan-2026-09-11.md) is +that next time, where **both rules still hold unchanged**, because a browser +still runs code the site delivered no matter which transport carries the bytes. + +Leaning: keep both, reworded to name their dormancy — one sentence saying no +browser client currently ships and that these bind any future one. A rule that +survives the thing it described is cheaper than rediscovering it. + +**Answered 2026-09-14: the leaning.** Both rules stay, reworded as dormant. + +### Open question 4 — which release + +The removal is not a wire change, so it can ship alone as `0.4.0`. `meta_ok` +key confirmation *is* a wire change and is the other candidate for that version. +Shipping them together gives users one disruptive upgrade instead of two; +shipping the removal alone gets a smaller change in front of people sooner. + +Not this plan's decision, but it should be made before either is tagged. + +**Answered 2026-09-14: together, as `0.4.0`.** The removal ships with `meta_ok` +key confirmation and with the receiver consent and status work in +[`receiver-consent-and-status-plan-2026-09-14.md`](receiver-consent-and-status-plan-2026-09-14.md), +which is a wire change too. One disruptive upgrade instead of three. Doing the +removal first also deletes phase 3 of the `meta_ok` plan, which is browser work. + +## Kickoff prompt + +> Read `docs/plans/browser-client-removal-plan-2026-09-11.md` and +> `AGENTS.md`. Verify its file and line references against the current source +> before relying on them — the plan records intent at writing time. Start with +> phase 0 only, on a topic branch off `main`, and stop for review before phase 1: +> phase 0 is release-critical and is meant to be reviewable and mergeable alone. +> Answer open question 1 and 2 before phase 1, and open question 3 before +> phase 4. diff --git a/docs/plans/browser-on-iroh-plan-2026-09-11.md b/docs/plans/browser-on-iroh-plan-2026-09-11.md new file mode 100644 index 0000000..f6ffc7f --- /dev/null +++ b/docs/plans/browser-on-iroh-plan-2026-09-11.md @@ -0,0 +1,255 @@ +# Browser client on iroh plan + +Status: **proposed — not scheduled** +Created: **2026-09-11** +Last updated: **2026-09-11** + +## Goal + +Rebuild the browser client as an **iroh node compiled to WebAssembly**, speaking +the same Drop conversation the CLI speaks, so that a browser transfer and a CLI +transfer differ in their carrier and in nothing else. + +This is recorded now so the idea is not rediscovered as new. It is **not +scheduled**, it is blocked on work that has not happened, and +[Blockers](#blockers-in-the-order-they-have-to-be-solved) is the honest list of +why. + +## Why this is worth doing: it removes a translator + +The argument is not "browsers are nice to have." It is that the current design +has two dialects and a middlebox reconciling them, and this collapses that to +one conversation. + +The relay does not merely forward. It **renames and invents control frames** — +recorded as a finding of item 3 phase 1 in +[`../implementation-checklist.md`](../implementation-checklist.md), and stated +in the transport trait's own documentation at +[`transport/mod.rs:100-106`](../../cli/src/transport/mod.rs#L100-L106): + +> `receiver_connected` is a sentence the relay invents; no peer ever sends it. +> A path that blocked on it would be a path that only works over a relay, which +> is exactly what this phase is undoing. + +[`../decisions.md`](../decisions.md) entry 12 settled that the control +vocabulary is the peer's and a relay only embellishes it. The browser client is +the last consumer of the embellished version. Removing it removes the reason the +embellishment has to keep working. + +Under this plan: + +```text +Browser (iroh compiled to wasm) + Drop control frames + sealed chunks ← identical to the CLI + iroh QUIC connection + WebSocket ─────────────► iroh relay ──UDP──► CLI peer + knows nothing about Drop +``` + +Against today: + +```text +Browser + Drop control frames + sealed chunks, relay dialect + WebSocket ─────────────► Drop relay (api) ─────► CLI peer + knows codes and sessions, + invents control frames +``` + +The WebSocket does not disappear — it moves **beneath** the Drop protocol +instead of being it. It becomes the browser's substitute for the UDP socket the +sandbox will not give it, which is what iroh already uses it for. + +## What a browser can and cannot do + +Verified against iroh's documentation on 2026-09-11. This section is the reason +the plan is shaped the way it is, so it is worth re-checking before building: +iroh's browser support is moving. + +**Can:** compile to `wasm32-unknown-unknown` via `wasm-bindgen` and join the +iroh network as an ordinary endpoint, with connections end-to-end encrypted such +that the relay cannot read them. + +**Cannot, and this is not a temporary gap:** open a UDP socket, and therefore +hole-punch. iroh's own wording is *"we can't port our hole-punching logic in +iroh to browsers"* and *"all connections from browsers to somewhere else need to +flow via a relay server."* WebTransport with `serverCertificateHashes` and +WebRTC are both named as possible future routes to a direct browser connection +and neither is implemented. + +**So a browser peer is permanently relayed.** Drop's headline claim is unharmed +— "no Drop server" stays precisely true, because an iroh relay is not a Drop +server — but a browser transfer can never be the direct path, and no amount of +later work in this repository changes that. It must never be described as +peer-to-peer, which is what [AGENTS.md](../../AGENTS.md) already requires. + +## Blockers, in the order they have to be solved + +### 1. The transport trait's futures are `Send` + +The largest structural blocker, and it sits inside the abstraction that was +built to make this kind of thing possible. + +Every method on `Transport` declares `+ Send` on its returned future — +[`transport/mod.rs:108`](../../cli/src/transport/mod.rs#L108), +[`116`](../../cli/src/transport/mod.rs#L116), +[`122`](../../cli/src/transport/mod.rs#L122), +[`131`](../../cli/src/transport/mod.rs#L131), +[`136`](../../cli/src/transport/mod.rs#L136) — and the trait's doc comment at +[line 73](../../cli/src/transport/mod.rs#L73) explains that it is deliberate: +the transfer paths are spawned onto a multi-threaded runtime by the CLI's own +tests, and leaving `Send` to inference would produce errors at the call site +rather than here. + +On `wasm32-unknown-unknown` there are no threads and JS-backed futures are +`!Send`. A wasm transport cannot satisfy the bound. + +This is a known shape rather than a novel problem — n0 hit it in iroh itself and +solved it with `n0-future`'s conditional `Send` aliases, which resolve to `Send` +on native targets and to nothing on wasm. Adopting the same approach is the +likely fix, and it touches every implementation of the trait plus the two +transfer paths. **It is a refactor of shipped, tested code for the benefit of +code that does not exist yet**, which is the main reason this plan is not +scheduled. + +### 2. The browser cannot reach the DHT + +Rendezvous today is `pkarr` with `features = ["dht"]`, talking to mainline +directly. iroh documents `discovery-pkarr-dht` as **incompatible with wasm**, +and the underlying reason is the same one as blocker 1's: no UDP. + +A browser needs pkarr over HTTP relay instead. That is a second piece of +infrastructure, and it does not fit the existing configuration surface: +`DROP_RENDEZVOUS_BOOTSTRAP` is documented as comma-separated `host:port` DHT +nodes, which is not what a pkarr HTTP relay is. + +Worse, it has to inherit entry 15's load-bearing rule — **a malformed value is +an error rather than a silent return to the public default** — in a context +where the failure is even quieter, because a browser user cannot read a terminal +error. An operator who meant to keep rendezvous inside their network and +silently got the public relay has lost exactly what they configured. + +### 3. `peers_enforce_one_guess` needs a third answer + +[`transport/mod.rs:94`](../../cli/src/transport/mod.rs#L94) has no default *on +purpose*: `false` on a direct connection is an unlimited guessing oracle, and +`true` over the relay fails every transfer. A new carrier must answer or fail to +compile. + +A browser-over-iroh transport is a genuinely third case. There is no Drop relay +refusing a second claim — an iroh relay knows nothing about Drop sessions — so +the relay's `false` is wrong. But `true` means the browser runs entry 13's +one-guess checkpoint, including `AskTheTerminal`, which has no terminal to ask. + +**This is a security decision requiring a decisions.md entry, not a plumbing +choice**, and it is why blocker 4 is ordered where it is. + +### 4. `meta_ok` must land first + +The checkpoint the answer to blocker 3 depends on is the one +[`meta-ok-key-confirmation-plan-2026-08-31.md`](meta-ok-key-confirmation-plan-2026-08-31.md) +exists to fix. That plan's own note — *"Land it before the direct path ships — +after that it is a wire break"* — is already overdue; the direct path shipped in +`v0.2.0`. + +Building a browser transport against the unfixed frame means taking the same +wire break twice, once for the CLI and once for the browser. Doing `meta_ok` +first costs nothing here and saves a second breaking release. + +### 5. The transfer paths touch a filesystem + +`send::run` and `recv::run` are not reusable as they stand: + +- [`payload.rs:104`](../../cli/src/payload.rs#L104) spools a compressed payload + to `std::env::temp_dir()`, with the whole `SpoolFile` lifecycle and its + signal-handler deletion path built around a real file. +- [`recv.rs:516`](../../cli/src/recv.rs#L516) creates the destination with + `fs::File::create`, and `untar.rs` writes a tree. + +A browser has neither. It has the File System Access API where available, and +memory where not — a distinction the current client already contends with, per +the direct-to-disk item in [`release-checklist.md`](../release-checklist.md). + +So the reuse is real but partial: the **middle** of the stack — framing, the +envelope, the control conversation, the transport trait — is shared, and **both +ends** are per-platform. That is a defensible architecture, but it should be +understood as one before starting, not discovered in phase 3. + +### 6. Toolchain and manifest + +`iroh` requires `default-features = false` on wasm, which drops metrics. It also +sets `rust-version = "1.91"`, whereas +[`crypto/Cargo.toml:5`](../../crypto/Cargo.toml#L5) deliberately stays at +`1.85` *because* it compiles to wasm — a property that a new wasm crate pulling +in iroh does not inherit. + +## Dependencies + +This plan is blocked on, in order: + +1. [`browser-client-removal-plan-2026-09-11.md`](browser-client-removal-plan-2026-09-11.md) + — start from a clean slate rather than converting `App.svelte` in place. + Almost nothing in the current client survives contact with this design, and + keeping it alive during the rebuild means maintaining two browser clients. +2. [`meta-ok-key-confirmation-plan-2026-08-31.md`](meta-ok-key-confirmation-plan-2026-08-31.md) + — blocker 4. +3. The conditional-`Send` refactor of the transport trait — blocker 1. Worth + scoping as its own change against the existing CLI, where it is reviewable + with the current tests, rather than as phase 1 of a wasm project. + +## What does not change + +- **The claim invariants.** A browser still runs code the site delivered, so + [AGENTS.md](../../AGENTS.md)'s rule that browser transfers are only as strong + as the delivered code holds exactly as written, under any transport. This plan + is a reason to **keep** those rules through the removal, not to retire them — + see that plan's open question 3. +- **"No Drop server."** An iroh relay is not a Drop server, so the claim stays + precise. But a browser needs a site to be served from and a relay it can + reach, so the browser path always involves somebody's infrastructure and must + say so. +- **The envelope.** `drop-crypto` compiled to wasm is how the browser gets the + envelope today and would still be how it gets it. Entry 11's reasoning — one + implementation, not two — is the part of the current design that is right and + survives. +- **The CLI.** Nothing here changes `drop send` or `drop recv`. To a CLI peer, a + browser is a peer that never punches through, which iroh already handles. + +## Open questions + +### Open question 1 — is a browser client wanted at all? + +The honest prior question. A browser client that can never be direct, needs two +pieces of relay infrastructure, and duplicates a CLI that installs in one +command may not earn its maintenance. The case for it is reach: no install, and +a receiver who has never heard of Drop. + +This should be answered before any of the blockers are worked, because blocker 1 +is a refactor of shipped code and is only worth doing if the answer is yes. + +### Open question 2 — who runs the relays? + +A browser needs an iroh relay and a pkarr HTTP relay. n0 runs public ones; a +deployment that cares runs its own via the entry 15 variables, once blocker 2 +extends them. Neither is a Drop server, but "no install, no configuration" +quietly means "n0's infrastructure" for the default browser user, and that +belongs in the README rather than in a plan. + +### Open question 3 — does `api` survive this? + +If the browser stops using the Drop relay, `api`'s remaining job is +`--transport relay` for UDP-blocked networks. That is real and `netlab` covers +it. But it is worth asking whether an iroh relay — which already carries +connections that cannot hole-punch, and which entry 15 already lets a deployment +self-host — subsumes it. If it does, Drop ends with one relay concept instead of +two, which is the same simplification this plan makes for the browser. + +Explicitly **not** proposed here. Noted because it is the natural next question +and should be asked deliberately rather than drifted into. + +## Sources + +- [Iroh — WebAssembly and Browsers](https://docs.iroh.computer/deployment/wasm-browser-support) +- [Iroh & the Web](https://www.iroh.computer/blog/iroh-and-the-web) +- [Tracking: WebAssembly support for iroh, n0-computer/iroh#2799](https://github.com/n0-computer/iroh/issues/2799) +- [Implementing a WebRTC Transport, n0-computer/iroh#4024](https://github.com/n0-computer/iroh/discussions/4024) diff --git a/docs/plans/cross-platform-plan-2026-09-14.md b/docs/plans/cross-platform-plan-2026-09-14.md new file mode 100644 index 0000000..cb318f7 --- /dev/null +++ b/docs/plans/cross-platform-plan-2026-09-14.md @@ -0,0 +1,475 @@ +# Cross-platform plan: Windows, macOS and Linux, and transfers between them + +Status: **proposed** +Created: **2026-09-14** +Last updated: **2026-09-14** + +## Goal + +`drop` installs and runs on Windows, macOS and Linux, and a transfer between any +two of them — a file or a folder, over either path — arrives intact, or tells +the receiver exactly what it could not reproduce and why. + +The user set this as very important on 2026-09-14. At the time of writing +the honest description is: **Linux works, macOS builds and has never had a test +run on it, and Windows has no build at all.** + +## Where things stand, verified 2026-09-14 + +| Platform | Built at release | Tests run in CI | Installer | Notes | +| --- | --- | --- | --- | --- | +| Linux x86_64 / aarch64 (musl) | yes | yes, Ubuntu only | `install.sh` | the only platform anything has been proven on | +| macOS x86_64 / aarch64 | yes | **no** | `install.sh` | release only runs `drop --version` on it | +| Windows | **no** | **no** | **none** | `install.sh` stops with "unsupported operating system" | + +- [`release.yml`](../../.github/workflows/release.yml) builds four targets, all + Linux or macOS. Its "Verify the binary runs" step is the only thing that has + ever executed a macOS binary, and all it runs is `--version`. +- [`ci.yml`](../../.github/workflows/ci.yml)'s `rust` job runs on + `ubuntu-24.04` alone. Every `#[cfg(not(unix))]` branch in the CLI has + therefore **never been compiled**, let alone run or linted. +- `web/public/install.sh` detects the platform with `uname -s` and supports + `Linux` and `Darwin` only. Phase 0 of the browser removal plan moves it to + `scripts/install.sh`. + +The code is not Unix-only by accident. Someone thought about Windows: there are +`cfg(not(unix))` fallbacks for file modes, spool file permissions, signals and +symlinks, and the interface already drops key-release events because "Windows +reports both edges of a key" ([`ui/app.rs:146-158`](../../cli/src/ui/app.rs#L146-L158)). +What is missing is anything that proves those branches work, plus a handful of +places where Windows behaves differently in a way nobody has accounted for. + +## Findings + +Each one is from reading the source, not from running on Windows, which nobody +can do yet. Phase 0 exists to turn them into observed behaviour. + +### Finding 1 — a symlink in a folder breaks every Linux-to-Windows folder transfer + +[`untar.rs:417-423`](../../cli/src/untar.rs#L417-L423) makes `create_symlink` +return `Unsupported` on anything that is not Unix, and +[`untar.rs:273`](../../cli/src/untar.rs#L273) propagates that error with `?`. +The whole extraction stops at the first symlink, mid-transfer, and leaves a +partial tree behind. Symlinks are common in real folders: `node_modules/.bin`, +Python virtual environments, most build output. So in practice **most folders +sent from Linux or macOS to Windows fail**. + +The fix is small: a symlink the receiver cannot create becomes a warning, the +same way an existing file already does at +[`untar.rs:266-269`](../../cli/src/untar.rs#L266-L269). Creating links on Windows +needs Developer Mode or an elevated process, so trying and falling back is worse +than not trying. + +### Finding 2 — names a peer chooses mean different things to Windows + +The extractor judges a path safe with `Path::components` +([`tar.rs:468-497`](../../cli/src/tar.rs#L468-L497)) and then checks it against +the filesystem. Both checks are sound on Unix. On Windows a *normal* path +component can still be interpreted by the filesystem API: + +| Name in the archive | What Windows does with it | Consequence | +| --- | --- | --- | +| `notes.txt:hidden` | NTFS **alternate data stream** `hidden` on `notes.txt` | Data attached invisibly to a file that may already exist and belong to the receiver | +| `notes.txt::$DATA` | the *main* stream of `notes.txt` | Can reach an existing file under a name the exists-check at [`untar.rs:279`](../../cli/src/untar.rs#L279) does not match on text, so the no-replace rule's protection is unproven | +| `CON`, `NUL`, `AUX`, `PRN`, `COM1`–`COM9`, `LPT1`–`LPT9`, including `nul.txt` and any case | a **device**, not a file | Content written to a device, or a creation error that aborts the extraction | +| `report.` or `report ` (trailing dot or space) | silently stripped, so it becomes `report` | The name validated is not the name written | +| `<>"|?*` or a control character | an invalid name | Creating it fails, and `?` aborts the extraction | + +The single-file path has the same exposure one level up: +[`recv.rs:503-507`](../../cli/src/recv.rs#L503-L507) keeps the final component of +a name the sender chose. `report:v2.pdf` from a Linux sender creates a stream, +not a file. + +Not all of this is hostile. Colons are ordinary on Linux and macOS, and a folder +full of `10:30 standup.md` is exactly what an honest user sends. So the rule +cannot be "refuse". See [open question 1](#open-question-1--rewrite-or-refuse-a-name-windows-cannot-store). + +The same failure also happens on Linux and macOS when a receiver extracts onto +an exFAT or NTFS USB drive. That case needs no rewriting, only for an +uncreatable entry to become a warning rather than an abort. + +### Finding 3 — a Windows sender puts Windows syntax in symlink targets + +[`tar.rs:146`](../../cli/src/tar.rs#L146) records `fs::read_link` output +verbatim. On Windows that is `..\shared\config` or `C:\Users\...`. A Linux +receiver sees one normal component containing backslashes, finds it inside the +destination, and creates a link that never resolves. It is harmless, since the +target stays inside the destination by construction, but it is wrong. Relative +targets should be written with `/`. An absolute one should be skipped with a +warning, as it would dangle on any other machine. + +### Finding 4 — the progress line assumes a VT terminal + +[`progress.rs:88`](../../cli/src/progress.rs#L88) writes `\r\x1b[2K`. Windows +Terminal handles that. The older console that `cmd.exe` and PowerShell 5 still +open on many machines handles it only after a program turns on VT processing, +and nothing does. Without it the screen shows `←[2K` before every update. +crossterm already carries the Windows code to turn VT processing on, and the +interface depends on crossterm. + +### Finding 5 — closing the window leaves the user's bytes in `%TEMP%` + +[`payload.rs:85-88`](../../cli/src/payload.rs#L85-L88) waits only for Ctrl-C on +non-Unix. Closing the console window, logging off and shutting down send +`CTRL_CLOSE_EVENT`, `CTRL_LOGOFF_EVENT` and `CTRL_SHUTDOWN_EVENT`, which nobody +handles. The process dies and a compressed send's spool file survives. That file +is a copy of the user's data in a temporary directory, which is exactly what the +spool cleanup exists to prevent. `tokio::signal::windows` exposes all three. +Windows allows a few seconds after a close event, which is enough to delete one +file. + +### Finding 6 — executable bits do not survive a Windows sender + +[`tar.rs:207-214`](../../cli/src/tar.rs#L207-L214) records `0o644`/`0o755` from +the read-only flag, because Windows has no executable bit. A script sent from +Windows to Linux arrives non-executable. Nothing can recover a bit that was +never recorded, so this is a documented limitation, not a bug. + +### Finding 7 — tests that can only fail on the platform they were written on + +Seven tests in [`cli/tests/archive.rs`](../../cli/tests/archive.rs) and one in +[`cli/tests/transfer.rs:366`](../../cli/tests/transfer.rs#L366) are +`#[cfg(unix)]`. That is correct, since they plant symlinks. But Windows has no +equivalent tests for the hostile names in finding 2. Those threats exist only +there, so today they are covered nowhere. + +### Finding 8 — Windows-only behaviour outside the code + +- **Firewall.** The first time `drop.exe` binds a UDP socket, Windows Defender + Firewall asks whether to allow it on private and public networks. Answering + Cancel blocks unsolicited inbound traffic. Outbound still works, so a + direct transfer usually still succeeds, sometimes only through iroh's relay. + The dialog appears in the middle of a transfer and has to be documented or it + reads as a bug. +- **SmartScreen.** An unsigned `.exe` downloaded through a browser gets "Windows + protected your PC". A binary fetched with PowerShell's `Invoke-WebRequest` + carries no Mark of the Web and does not. So the installer route avoids it and + a manual download does not. Signing is out of scope, see + [open question 3](#open-question-3--code-signing). +- **C runtime.** A default `*-pc-windows-msvc` build links the Visual C++ + runtime dynamically, and a clean machine without the redistributable refuses + to start it. `-C target-feature=+crt-static` removes the dependency, the same + reason the Linux targets are musl. + +### What does *not* differ, and why no cross-OS protocol work is needed + +The wire is platform-neutral by construction. Control frames are JSON, the +envelope's integers are big-endian ([`protocol.md`](../protocol.md), *Sealing*), +chunks are opaque bytes, and nothing on the wire is a path. A Windows peer and a +Linux peer run the same pure-Rust envelope. **Everything platform-specific is at +the two ends: how a sender reads a tree, and how a receiver writes one.** That is +where this plan spends its effort. + +## Constraints + +- The archive invariants in [AGENTS.md](../../AGENTS.md) do not relax on any + platform. Absolute paths, `..`, drive and UNC prefixes and anything that + traverses a link are still refused. Any rewriting in phase 1 happens to one + normal component and must never introduce a separator. +- Nothing already on the receiver's disk is replaced unless `--force` was + given, on every platform. +- No C toolchain dependencies. The existing bar in + [`cli/Cargo.toml`](../../cli/Cargo.toml) stays: pure Rust, so every target + builds on its own native runner. +- The wire does not change. A Windows build interoperates with a Linux build + of the same version and nothing more is promised. + +## Phases + +### Phase 0 — tests on all three operating systems + +Nothing below can be verified until this exists, so it comes first and its +failures are the input to the rest. + +- [ ] `ci.yml`: turn the `rust` job into a matrix over `ubuntu-24.04`, + `macos-14` (arm64) and `windows-2025`. Formatting stays on Linux only, + because it cannot vary by platform. Clippy runs on all three, since + `cfg(windows)` code is otherwise never linted. +- [ ] Record what fails on the first run in this plan, dated, before fixing + any of it. +- [ ] Make the suite pass on all three without skipping anything that is not + genuinely inapplicable. Every new `cfg` on a test gets a comment saying + why the test cannot run there. +- [ ] Mark `.github/workflows/ci.yml` checkouts `autocrlf`-safe: + `.gitattributes` already sets `eol=lf`, so fixture bytes should match + across platforms. Confirm on the Windows runner rather than assume. + +**Gate:** `cargo test --workspace --all-targets` green on all three runners. + +### Phase 1 — a Windows receiver writes what it can and reports the rest + +- [ ] Finding 1: a symlink the platform cannot create becomes a warning, + `skipped symlink {name}: this system cannot create symbolic links`, and + extraction continues. +- [ ] Finding 2, archives: a pure function that takes one normal component and + returns what Windows can store, compiled and unit-tested on **every** + platform and applied only when the receiver runs on Windows. Rewrite + `: < > " | ? *` and control characters to `_`, a trailing dot or space + to `_`, and a reserved device name to the same name with `_` after its + stem (`CON` → `CON_`, `nul.txt` → `nul_.txt`). A rewritten name produces a + warning naming both forms. Existence and link checks run on the + **rewritten** path, so the no-replace rule and the link checks judge the + path that is actually written. +- [ ] Finding 2, single files: the same function applied to the name in + [`recv.rs:503-507`](../../cli/src/recv.rs#L503-L507), before collision + numbering, so `report:v2.pdf` saves as `report_v2.pdf` and the preview in + the consent plan shows that name. +- [ ] Every platform: an entry whose creation fails with an invalid-name or + permission error becomes a warning, not an abort, so a Linux receiver + writing to an exFAT drive gets every file it can store. Disk-full and other + I/O errors still abort, because continuing would only fail again. +- [ ] Case-insensitive collisions (`Makefile` and `makefile` on Windows or + macOS): no code change is expected, since the second is "already exists" + and is skipped. Pin it with a test that runs on the two platforms where it + applies. +- [ ] Windows-only extraction tests, the counterpart of the Unix symlink tests, + covering `a:b`, `a::$DATA` next to an existing `a`, `CON`, `nul.txt`, + `report.`, `report `, `a\..\b`, `C:x`, `\\?\C:\x` and `\\server\share\x`. Each + asserts both what was written and that nothing outside the destination or + in an existing file changed. + +**Gate:** a hostile archive built from every name above extracts on a Windows +runner without writing outside the destination or into an existing file, and +an honest archive from a Linux tree with symlinks and colons extracts with +warnings and every representable file intact. + +### Phase 2 — a Windows sender produces a portable archive, and cleans up + +- [ ] Finding 3: on Windows, write relative link targets with `/`, and skip + absolute targets with a warning. Tested by a pure function on every + platform. +- [ ] Finding 5: `wait_for_termination` also resolves on `ctrl_close`, + `ctrl_logoff` and `ctrl_shutdown` on Windows. Test: the existing spool + cleanup test already runs on every platform after phase 0. Add a Windows + test that the spool file can be deleted while the payload still holds it + open. `std` opens files with `FILE_SHARE_DELETE`, but that is exactly the + kind of thing to pin rather than trust. +- [ ] Finding 4: turn VT processing on once at startup on Windows. If the + console refuses, fall back to `\r` and padding without escapes. + +### Phase 3 — build, package and install for Windows + +- [ ] `release.yml`: add `x86_64-pc-windows-msvc` on `windows-2025`, and + `aarch64-pc-windows-msvc` on `windows-11-arm` (see risks). Package as + `drop-.zip` holding `drop.exe`, `LICENSE` and `README.md`, since + `Expand-Archive` is built in and `tar.gz` is not a Windows habit. The + checksum step covers `drop-*.zip` as well as `drop-*.tar.gz`. +- [ ] Static CRT for both Windows targets through + `[target.'cfg(all(windows, target_env = "msvc"))'] rustflags`. **Note for + the browser removal plan's phase 2:** that phase deletes + `.cargo/config.toml`, and must keep this section if phase 3 has landed. +- [ ] The "Verify the binary runs" step works unchanged on a Windows runner + with `shell: bash`. Confirm it, and confirm the binary does not depend on + `VCRUNTIME140.dll` (`dumpbin /dependents`). +- [ ] `scripts/install.ps1`: detect the architecture, download the zip and + `checksums.txt`, check with `Get-FileHash -Algorithm SHA256`, install to + `%LOCALAPPDATA%\Programs\drop\drop.exe`, and add that directory to the + **user** `PATH` only if it is absent. Honour `DROP_VERSION`, + `DROP_INSTALL_DIR` and `DROP_RELEASE_BASE` to match `install.sh`. Publish + it as a release asset beside `install.sh`. +- [ ] `install.sh` on `MINGW*`/`MSYS*`/`CYGWIN*`: point at the PowerShell + installer by name rather than printing "unsupported operating system". +- [ ] `README.md`: a Windows install line, + `irm https://github.com/op-q/drop/releases/latest/download/install.ps1 | iex`, + and a platform table that matches phase 0's CI rather than intent. + +**Gate:** a release built from a `workflow_dispatch` on an existing tag's +successor publishes both zips and `install.ps1`, and `install.ps1` installs a +binary that runs `drop --version` on the Windows runner. + +### Phase 4 — archives produced on one OS, extracted on the others, in CI + +The wire is platform-neutral (see above), so an archive written on one OS and +extracted on another is the cross-OS risk that remains. It can be tested +without networking two runners together. + +- [ ] A separate workflow, `cross-os.yml`, triggered on pull requests that touch + `cli/src/tar.rs`, `cli/src/untar.rs`, `cli/src/payload.rs` or + `cli/src/recv.rs`, and nightly. Job `produce`, a matrix over three OSes, + builds a fixed tree using every name that platform can hold: nested + folders, an empty folder, a large file, a zero-byte file, Unicode names, + a name longer than 100 bytes, a relative symlink where the platform + allows one, an executable script, and colons where the platform allows + them. It writes the archive bytes `TarPlan` produces, uncompressed and + gzipped, and uploads them with a manifest of expected contents. +- [ ] Job `consume`, needing `produce`, is also a matrix over three OSes. It + downloads all three archives and extracts each through the real receive + path, not only through `TarExtractor`, so the gzip and naming code runs + too. It then checks against the manifest: identical bytes for every + representable file, and the expected warning for every file that is not. +- [ ] Driven by `#[ignore]`d tests reading `DROP_FIXTURE_OUT` / + `DROP_FIXTURE_IN`, so the logic lives in Rust with the rest of the tests + and the workflow is only plumbing. + +Nine pairings, three of which are same-OS controls. + +Considered and **rejected**: a live transfer between two CI runners. The runners +have no channel to swap a code mid-run, and making one would mean a test-only +way to fix a transfer code in advance. That is a knob that weakens the one +secret the protocol has, and it would exist in shipped binaries. + +### Phase 5 — a real Windows machine against a real Linux one + +CI runners are clean virtual machines without the firewall prompt, antivirus +scanning or a legacy console. The user has a Windows machine and this Linux one. + +- [ ] Run the checklist below. Record results in + `docs/validation/cross-os-windows-linux-.md`, with the Windows + version, the terminal used and anything that surprised. + +```text +Build drop.exe from the phase 3 release (or a CI artifact), drop on Linux from the same commit. + +Direct path (no DROP_SERVER set) +[ ] Linux -> Windows: one file, 1 GiB+ saved intact (compare SHA-256) +[ ] Windows -> Linux: one file, 1 GiB+ saved intact +[ ] Linux -> Windows: folder with symlinks, colons, CON.txt, Unicode, an empty dir + warnings name each rewrite/skip; the rest intact +[ ] Windows -> Linux: folder with nested dirs, Unicode, a read-only file + intact; permissions as documented +[ ] First run on Windows: note the firewall dialog, and what happens on Allow vs Cancel + +Relay path (api running on the Linux box, DROP_SERVER=http://: on both) +[ ] Windows -> Linux and Linux -> Windows, one file each, --transport relay + +Interface and lifecycle on Windows +[ ] `drop send` and `drop recv` bare in Windows Terminal: every screen, arrow keys move one row +[ ] the same in cmd.exe / conhost: no raw escape codes on screen +[ ] Ctrl-C during a send and during a receive: terminal restored, peer told, no partial file +[ ] Close the console window mid-send of a compressed folder: no drop-* spool file left in %TEMP% +[ ] install.ps1 on a machine without drop: installs, PATH updated, new shell finds `drop` +``` + +**Gate:** every box ticked, or its failure recorded with an item created for it. + +### Phase 6 — documentation and the decision + +- [ ] `README.md`: platforms, install per platform, and known limitations: + executable bits from Windows (finding 6), symlinks on Windows (finding 1), + rewritten names (finding 2), the firewall dialog and SmartScreen + (finding 8). +- [ ] [`security.md`](../security.md): Windows name handling as part of the + hostile-archive section, including why streams and device names are + rewritten and not merely refused. +- [ ] [`decisions.md`](../decisions.md): an entry for the Windows name policy, + once open question 1 is settled. +- [ ] [`release-checklist.md`](../release-checklist.md): the Windows assets and + `install.ps1` in the publish check. + +## Files + +| File | Change | +| --- | --- | +| `.github/workflows/ci.yml` | OS matrix for the Rust job | +| `.github/workflows/cross-os.yml` | new: produce/consume archive matrix | +| `.github/workflows/release.yml` | Windows targets, zip packaging, checksums, `install.ps1` | +| `.cargo/config.toml` | static CRT for msvc targets | +| `cli/src/untar.rs` | symlink and invalid-name failures become warnings; Windows names | +| `cli/src/tar.rs` | Windows component rewrite function; portable link targets | +| `cli/src/recv.rs` | single-file names through the same function | +| `cli/src/payload.rs` | Windows console close, logoff, shutdown | +| `cli/src/progress.rs` or `cli/src/main.rs` | VT processing on Windows | +| `cli/tests/archive.rs`, `cli/tests/transfer.rs` | Windows hostile-name tests; fixture producer and consumer | +| `scripts/install.ps1`, `scripts/install.sh` | new installer; pointer from MSYS | +| `README.md`, `docs/security.md`, `docs/decisions.md`, `docs/release-checklist.md` | as phase 6 | + +## Risks + +- **Phase 0 finds more than the findings list.** Likely, not a failure of the + plan. The findings come from reading; the runner is the first thing that has + ever executed this code on Windows. Budget for it and record what it finds. +- **`aarch64-pc-windows-msvc` may not build.** iroh's TLS stack uses `ring`, + and `ring` on Arm Windows has at times needed `clang` on the runner. If it + fails, ship x86_64 only and say so. Windows 11 on Arm runs x86_64 binaries + under emulation, and a missing native build is a performance note, not a + missing platform. +- **Rewriting names is a policy and can be wrong for someone.** A receiver who + expected `a:b` gets `a_b`. The warning is what keeps this honest, and it has + to be in front of the user, not buried. The interface's transfer screen + should list it. +- **Rewriting creates collisions.** `a:b` and `a_b` in the same folder map to + one name. The no-replace rule makes the second a skip with a warning, which is + safe. A later version could number the collision instead. Pinned by a test + so it is a known behaviour, not an accident. +- **Windows path length.** Paths over 260 characters fail unless long paths + are enabled on the machine. A deep folder from Linux can exceed it. Rust's + `std` adds the `\\?\` prefix for absolute paths and avoids the limit in most + cases. Include a deep path in the phase 4 fixture tree so this is observed + rather than assumed. +- **Antivirus.** Real-time scanning on a real Windows machine can hold a + just-written file open, which turns `remove_file` in the extractor's + replace-safely step ([`untar.rs:294`](../../cli/src/untar.rs#L294)) into a + sharing violation. It only shows on a real machine, which is one reason + phase 5 exists. +- **CI cost.** Windows and macOS runners are slower, and on private repositories + they bill at a multiple. The repository is public, so the minutes are free, + but a Windows `cargo test` from cold is realistically 15-25 minutes. Use a + dependency cache and set `timeout-minutes` from what the first run takes, not + from the Linux job's 15. + +## Validation + +```bash +scripts/check-secrets.sh +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings # on all three runners +cargo test --workspace --all-targets # on all three runners +``` + +- [ ] CI green on Linux, macOS and Windows (phase 0 gate) +- [ ] Hostile Windows names contained, honest ones rewritten with warnings (phase 1 gate) +- [ ] Windows zips, checksums and `install.ps1` published and installing (phase 3 gate) +- [ ] Nine producer/consumer pairings green (phase 4) +- [ ] Manual Windows ↔ Linux checklist recorded (phase 5 gate) + +## Open questions + +### Open question 1 — rewrite or refuse a name Windows cannot store + +[`tar.rs:463-467`](../../cli/src/tar.rs#L463-L467) sets the precedent: a +dangerous path is *refused, not normalized*, "because a rewritten path silently +changes where a hostile archive lands". That argument is about paths that would +escape, where normalizing `..` away changes the directory an entry lands in. A +character rewritten inside one component changes the leaf name, never the +directory. Nothing is introduced that could be a separator, and the filesystem +checks run afterwards on the result. + +Refusing would drop every file with a colon from an honest Linux folder, which +is data loss in the common case to prevent a threat the rewrite also prevents. + +**Leaning: rewrite, with a warning per entry, and record it in `decisions.md` +as a deliberate, narrow exception to the refuse-don't-normalize rule.** 7-Zip +makes the same choice on Windows. + +### Open question 2 — `aarch64-pc-windows-msvc` in the first Windows release, or later + +Leaning: attempt it in phase 3 and drop it without ceremony if `ring` needs a +toolchain the runner lacks. See risks. + +### Open question 3 — code signing + +An Authenticode certificate costs money yearly and needs a secret in CI. The +installer path already avoids SmartScreen. **Leaning: not now.** Document the +SmartScreen dialog for manual downloads and revisit if people report it. + +## Dependencies on other plans + +- **Browser client removal, phase 0** moves `install.sh` to `scripts/`. Phase 3 + here adds `install.ps1` beside it and edits the same `release.yml` publish + step, so it lands after phase 0. +- **Browser client removal, phase 2** deletes `.cargo/config.toml`. If phase 3 + here has already landed, keep the Windows section. +- **Receiver consent and status** shows the name a file will be saved under, + and after phase 1 that is the rewritten name on Windows. Whichever lands + second wires them together. + +## Kickoff prompt + +```text +Read docs/plans/cross-platform-plan-2026-09-14.md and AGENTS.md. Verify the file +and line references against the current source first. Start with phase 0 only: +add the OS matrix to the Rust CI job on a topic branch, push it, and record +what fails on macOS and Windows in the plan, dated, before fixing anything. +The archive invariants in AGENTS.md apply on every platform; any Windows name +rewriting touches one normal component and never introduces a separator. +``` diff --git a/docs/plans/interactive-terminal-ui-plan-2026-09-10.md b/docs/plans/interactive-terminal-ui-plan-2026-09-10.md index ffa3213..c90b6af 100644 --- a/docs/plans/interactive-terminal-ui-plan-2026-09-10.md +++ b/docs/plans/interactive-terminal-ui-plan-2026-09-10.md @@ -263,6 +263,11 @@ clears the most recent one. Settings already toggled survive it. - [ ] Receiver-side cancel (Finding 4): send `cancel`, and teach the sender to end cleanly on it. - [ ] Esc during a transfer cancels and closes both sides. +- Note 2026-09-14: the cancel protocol, the receiver's review screen and the + sender's state list are designed in + [`receiver-consent-and-status-plan-2026-09-14.md`](receiver-consent-and-status-plan-2026-09-14.md) + (its phases 4 and 5). This phase builds the transfer screen and the progress + sink those phases fill in; do not design a second cancel here. ### Phase 4 — The guess prompt as a screen diff --git a/docs/plans/meta-ok-key-confirmation-plan-2026-08-31.md b/docs/plans/meta-ok-key-confirmation-plan-2026-08-31.md index d6b32b3..09c2159 100644 --- a/docs/plans/meta-ok-key-confirmation-plan-2026-08-31.md +++ b/docs/plans/meta-ok-key-confirmation-plan-2026-08-31.md @@ -2,7 +2,7 @@ Status: **proposed** Created: **2026-08-31** -Last updated: **2026-08-31** +Last updated: **2026-09-14** ## Goal @@ -179,7 +179,14 @@ impl SessionKeys { - **Wire compatibility.** A new-sender/old-receiver pair on the direct path fails: the receiver sends a bare `meta_ok` and the sender charges an attempt. The direct path is unreleased, so this is acceptable now and will - not be later. Land it before the QUIC path ships. + not be later. Land it before the QUIC path ships. **Corrected 2026-09-14: the QUIC path shipped in v0.2.0**, on by default, so + this is already a wire break. It now lands under the version bump in + [`receiver-consent-and-status-plan-2026-09-14.md`](receiver-consent-and-status-plan-2026-09-14.md) + (`ENVELOPE_VERSION` and `DROP_ALPN` to 2), so an old peer is refused with a + sentence rather than charged a failed attempt. Phase 3 (the browser) + disappears once the browser client removal lands first, which it is ordered + to do. That plan also makes `meta_ok` travel over the relay, so the "relay + path must not change" risk below is replaced by the relay forwarding it. - **The relay path must not change.** The relay parses receiver frames into a closed set and treats an unknown one as fatal. The confirmation must ride only where `peers_enforce_one_guess()` is true. Decision 13's commit records diff --git a/docs/plans/network-lab-plan-2026-08-31.md b/docs/plans/network-lab-plan-2026-08-31.md index 436fde7..a79d3fe 100644 --- a/docs/plans/network-lab-plan-2026-08-31.md +++ b/docs/plans/network-lab-plan-2026-08-31.md @@ -2,7 +2,7 @@ Status: **in progress** — phases 0 to 4 done, 5 outstanding Created: **2026-08-31** -Last updated: **2026-09-10** +Last updated: **2026-09-14** ## Goal @@ -642,6 +642,76 @@ and something about dialling an endpoint whose record was published from behind this NAT. The next step is packet capture on both sides of the NAT namespace, not another assertion. +#### Second look, 2026-09-14 — **the lab cannot punch, because address discovery never works in it** + +Re-run on `main` at `a1e84d6`: plain LAN and symmetric NAT pass, and full cone +fails. This time it failed by staying relayed, not with `authentication failed`: +the rendezvous link carried 33.36 MiB, 2.09 times the 16 MiB payload. Then three +experiments, run from a scratch test that was not committed: + +1. **Is the punch just late?** Throttle the core router's egress to the + rendezvous host to 16 Mbit, send 48 MiB, and sample the link counter every + half second. It climbed at a steady ~1.85 MiB per half second for the whole + 27-second transfer and never flattened. **No punch at any point**, so it is + not a matter of the lab timing out too early. +2. **Is it a conntrack clash on a NAT with no firewall?** A dump of + `/proc/net/nf_conntrack` on both NATs mid-transfer shows each peer sending + to the other's **private** address, such as `10.10.0.2 → 10.20.0.2:42277`. + Those packets die at the core, which has no route to either inner subnet. + There is **no entry at all toward the other NAT's public address** + (`10.60.0.2`), so nobody ever tried the mapped address. A stateful WAN + input firewall on both NATs, the next hypothesis, changed nothing useful. + That run failed with `authentication failed` instead. See below. +3. **Did the peers learn their public address?** A temporary tracing + subscriber in the CLI (`iroh=debug`) answers it directly. On both peers: + + ```text + iroh::_events::direct_addrs: addrs={DirectAddr { addr: 10.10.0.2:60090, typ: Local }} + iroh::net_report: QADv4: probe failed: QUIC connection failed: the cryptographic + handshake failed: error 48: invalid peer certificate: UnknownIssuer + iroh::net_report: net_report generated report=Report { udp_v4: false, ..., + global_v4: None, global_v6: None, ... } + ``` + +**Root cause: QUIC address discovery (QAD) against the lab's rendezvous helper +fails TLS verification.** With no `global_v4`, a peer's only candidates are its +private interface addresses, so hole punching has nothing to work with. Every +direct-path topology here has been relay-or-LAN since phase 4 was written. The +full-cone row was never testing iroh's traversal. It was testing an endpoint +that could not discover itself. + +**The helper's own premise is wrong.** `netlab/rendezvous/src/main.rs:44-48` +and `:77` say the self-signed certificate works "because iroh's QUIC client +does not verify relays against a public root". iroh 1.0.3 does verify QAD +against its CA roots, and the log above is the rejection. The self-hosted +rendezvous plan's risk list repeats the claim and is wrong in the same way. + +**This is also a product gap, not only a lab one.** `DROP_RENDEZVOUS_RELAY` +pointed at an operator's relay with a certificate from a private CA gets the +same outcome: transfers still complete over the relay, `--status` still says +`path=p2p`, and **no hole is ever punched**. Nothing anywhere says so. Only +operators whose relay has a publicly trusted certificate get traversal. See +[`self-hosted-rendezvous-plan-2026-09-10.md`](self-hosted-rendezvous-plan-2026-09-10.md) +phase 3. + +**The fix is not in this plan.** It changes which certificates the CLI trusts, +and that needs the user's decision first. The candidate: a +`DROP_RENDEZVOUS_CA` variable naming a PEM file of extra roots for the +rendezvous relay only, and a helper that issues a CA and signs its leaf with it, +since a self-signed leaf used as its own trust anchor is rejected as +`UnknownIssuer` too. **Rejected: skipping verification in the lab**, even as an +experiment. It would make the lab exercise a code path that production never +runs. + +**`authentication failed` is still unexplained, and is separate.** It now +appears intermittently rather than every time: two of four runs today, one of +them with no firewall. It is not the missing QAD, which is missing in the +passing runs too. It needs its own traced run, with the full sender and +receiver logs kept, once QAD works and the traversal path is actually being +exercised. Separately, the receiver logs +`Endpoint dropped without calling Endpoint::close` on its dial-timeout path, a +small cleanup bug in `direct::dial_sender`. + Topology 2's checkbox stays `[x]` because the topology and its measurements are written and running. What is unproven is the claim it was built to test. diff --git a/docs/plans/receiver-confirmation-plan-2026-08-19.md b/docs/plans/receiver-confirmation-plan-2026-08-19.md index 3c8c6b0..7636c3a 100644 --- a/docs/plans/receiver-confirmation-plan-2026-08-19.md +++ b/docs/plans/receiver-confirmation-plan-2026-08-19.md @@ -1,6 +1,6 @@ # Receiver confirmation plan -Status: **proposed** +Status: **abandoned** — superseded 2026-09-14 by [`receiver-consent-and-status-plan-2026-09-14.md`](receiver-consent-and-status-plan-2026-09-14.md), which redesigns it for sealed metadata, both carriers and no browser Created: **2026-08-19** Last updated: **2026-08-19** diff --git a/docs/plans/receiver-consent-and-status-plan-2026-09-14.md b/docs/plans/receiver-consent-and-status-plan-2026-09-14.md new file mode 100644 index 0000000..d312ffe --- /dev/null +++ b/docs/plans/receiver-consent-and-status-plan-2026-09-14.md @@ -0,0 +1,517 @@ +# Receiver consent, cancel, and live status plan + +Status: **proposed** +Created: **2026-09-14** +Last updated: **2026-09-14** + +Supersedes [`receiver-confirmation-plan-2026-08-19.md`](receiver-confirmation-plan-2026-08-19.md). +That plan was written against cleartext metadata and a browser client, and the +checklist has marked it "needs revision" since encryption landed. Its three +findings (hostile display names, the file created before any prompt, and the +three clocks) all still hold and are carried forward here. Its protocol design +does not. + +## Goal + +Three things the user asked for on 2026-09-14, designed together because they +are one conversation: + +1. **The receiver sees what is coming before anything is written**: the name it + will be saved under, its type, its size, and for a folder how many files. + They then accept or decline. +2. **Either side can cancel at any moment**, from a key in the interface or + Ctrl-C at a command, and the other side is told plainly rather than finding + out from a dropped connection. +3. **The sender sees the receiver's state throughout**: connected, entered the + code correctly, reviewing, accepted or declined, how much the receiver has, + finishing, done, or cancelled. + +## What exists, verified against `main` at `a1e84d6` + +- The receiver opens the sealed metadata before anything touches disk, + [`recv.rs:247-265`](../../cli/src/recv.rs#L247-L265), and only then calls + `open_target` at [`recv.rs:295`](../../cli/src/recv.rs#L295). **The consent + point already has a natural home between those two lines.** +- `Metadata` is `{filename, mime_type, plaintext_size}`, + [`crypto/src/envelope.rs:39-47`](../../crypto/src/envelope.rs#L39-L47), and it + is sealed. The old plan's worry that a file count "widens the cleartext + metadata surface" is gone: anything added here is encrypted. +- The sender's progress is **already the receiver's progress**. It advances on + acknowledgements, not on bytes written to the socket, + [`send.rs:634-646`](../../cli/src/send.rs#L634-L646). Status item 3's + "how much the receiver has" needs no new frame. +- On the direct path the receiver sends `meta_ok` after opening the metadata, + [`recv.rs:272-274`](../../cli/src/recv.rs#L272-L274). Over the relay it + cannot, because the relay parses receiver frames into a closed set, + [`src/domain/messages.rs:26-40`](../../src/domain/messages.rs#L26-L40), and + fails the session on anything else, + [`download_ws.rs:322-338`](../../src/routes/download_ws.rs#L322-L338). +- **Cancel exists in one direction only.** The sender sends `cancel` on a stream + failure, [`send.rs:370-376`](../../cli/src/send.rs#L370-L376). The relay + answers by reporting `cancelled` to the *sender* and an `error` to the + receiver, and **counts it as a failed transfer**, + [`upload_ws.rs:549-565`](../../src/routes/upload_ws.rs#L549-L565) and + [`transfer_service.rs:56-59`](../../src/services/transfer_service.rs#L56-L59). + The receiver has no cancel frame at all. Its only way out is `error`, which + the relay reports as "receiver could not save the file". +- **Ctrl-C tells nobody.** Both `run`s install a handler that restores the + terminal, deletes spool files and calls `process::exit(130)`, + [`send.rs:117-128`](../../cli/src/send.rs#L117-L128) and + [`recv.rs:133-137`](../../cli/src/recv.rs#L133-L137). The peer learns from a + closed socket, and over the relay that is reported as a failure. +- The receiver's name handling: + [`recv.rs:503-535`](../../cli/src/recv.rs#L503-L535) keeps the final component + and numbers collisions with `create_new`. Printing it goes straight to + `eprintln!`, [`recv.rs:282-286`](../../cli/src/recv.rs#L282-L286). **A + filename with an escape sequence can redraw the terminal today, before any + consent prompt exists.** +- The interface (item 7) has a chooser, file browser, options and code entry. + Its transfer screen is phase 3 of + [`interactive-terminal-ui-plan-2026-09-10.md`](interactive-terminal-ui-plan-2026-09-10.md), + and it has no cancel path yet (its finding 4). + +## Design + +### The conversation after this plan + +```text +sender receiver + │ key_exchange ─────────────────────────────────────▶ │ + │ ◀───────────────────────────────────── key_exchange │ + │ meta (sealed) ────────────────────────────────────▶ │ opens metadata + │ ◀──────────────────────── meta_ok {confirmation} │ proves the code, both carriers + │ sender: "The receiver entered the code. They are reviewing the transfer." + │ │ shows preview, asks + │ ◀──────────────────────────── accept │ or: decline {reason} + │ chunk, chunk, … ──────────────────────────────────▶ │ + │ ◀─────────────────────────── chunk_ack {bytes} │ sender progress = receiver's + │ complete ─────────────────────────────────────────▶ │ + │ ◀──────────────────── finishing │ large archives: extraction tail + │ ◀──────────────────── complete {bytes_received} │ + │ + │ at any point after connecting, either side: + │ ◀─────────────────────────────▶ cancel {reason} +``` + +### New and changed peer frames + +| Frame | From | Fields | Meaning | +| --- | --- | --- | --- | +| `meta_ok` | receiver | `confirmation` (hex) | **Changed**: both carriers now, with the key confirmation from [`meta-ok-key-confirmation-plan-2026-08-31.md`](meta-ok-key-confirmation-plan-2026-08-31.md) | +| `accept` | receiver | — | **New**. The sender may stream | +| `decline` | receiver | `reason`: `"declined"` or `"timed_out"` | **New**. Terminal, and not a failure | +| `cancel` | either | `reason`: `"user"`, `"write_failed"`, `"integrity"` or `"too_large"` | **Changed**: receivers may send it, and it carries a reason | +| `finishing` | receiver | — | **New**, optional. All bytes are in, and the receiver is closing the file or finishing extraction | + +**Reasons are enumerations, not text.** A peer's message would land on the other +terminal verbatim, which is the escape-sequence problem again in the other +direction. Each side maps a reason to its own sentence, and an unknown reason +reads as `"user"`. + +`meta_ok` on both carriers is what lets the sender say "the receiver entered the +code correctly" over the relay too, and it is the frame the key confirmation +already needs. `peers_enforce_one_guess()` keeps its job, **which carrier's +failed checkpoint consumes an attempt and may be retried**. It no longer +decides whether `meta_ok` is sent. Its doc comment, and the sender test +`the_relay_path_is_not_asked_to_pass_a_checkpoint`, change with it. Over the +relay a failed confirmation ends the transfer: the relay has already burned the +session, so there is nothing to retry. + +### The relay + +- `ReceiverMessage` gains `MetaOk { confirmation }`, `Accept`, + `Decline { reason }`, `Cancel { reason }` and `Finishing`. It forwards each to + the sender **verbatim**, by [`decisions.md`](../decisions.md) entry 12's rule + that the peer's words are canonical. It cannot verify `confirmation` and does + not try. +- `SenderMessage::Cancel` gains `reason` and is forwarded to the receiver as + `cancel`, replacing today's `error: sender cancelled`. +- **Ordering the relay enforces**, which is cheap and bounds what a modified + peer can make it do: `accept`/`decline` only after `meta` was forwarded, and + **binary chunks refused until `accept` has been forwarded**. A sender + that streams early is failed, and the relay budget is never spent on bytes + nobody agreed to. The receiver checks the same thing itself (below), so this + is defence in depth, not the defence. +- `decline` and `cancel` end the session **without** `record_transfer_failed`. + `/metrics` gains `total_transfers_declined` and `total_transfers_cancelled`, + which is additive to the JSON snapshot. +- Clocks. The accept deadline is 120 s, well under `SESSION_TTL_SECS` = 300 s, + and the relay refreshes `last_activity` on every forwarded frame, so an open + prompt never outlives the session. + +### Versioning: this is a wire break, and it should fail loudly + +A 0.3.0 peer against a 0.4.0 peer would otherwise hang. The new sender waits for +`accept` from an old receiver that is waiting for chunks, and both sit there +until a timeout that says nothing useful. So: + +- `ENVELOPE_VERSION` goes from 1 to 2. It is already checked at both ends and at + the relay, and a mismatch is already deliberately fatal. The receiver + already fails with `UnsupportedVersion { found }`. Its message becomes "the + sender is running a different version of drop (protocol 1, this is 2)". +- `DROP_ALPN` goes from `drop/transfer/1` to `drop/transfer/2`. The direct path + then refuses at the QUIC handshake. Map the ALPN mismatch error to the same + sentence rather than surfacing a TLS error. +- The version field now versions the conversation, not only the sealing. + **Record that in `decisions.md`**, because entry 12 and the envelope section + of `protocol.md` both describe it as the envelope's version. +- `meta_ok` key confirmation lands under the same bump. Both are unreleased until + 0.4.0, so one bump covers both, whichever lands first makes it. + +### The receiver + +**Order of operations**, replacing [`recv.rs:231-295`](../../cli/src/recv.rs#L231-L295): + +1. Exchange keys, wait for `meta`, check the version, open the metadata. +2. Send `meta_ok { confirmation }`. +3. **Resolve the destination without creating it**: the sanitized name, the + collision-numbered name it would get, and whether it is a file or an archive + extraction. This splits `open_target` into `plan_target` (pure, reads the + directory) and `create_target` (writes). The name shown is then the name + used. +4. Ask. Accept sends `accept`. Decline or the deadline sends `decline` and + returns normally, **with nothing created**. +5. `create_target`. If the name was taken between asking and creating, it is + numbered again and the new name is printed. `create_new` makes that race + safe, and it is rare enough not to re-ask. +6. Receive. A chunk before step 4 finished is a protocol violation: send + `cancel { reason: "integrity" }` and stop. +7. After the last chunk, send `finishing` before `finish()`, then `complete`. + +**The preview**, command surface: + +```text +Incoming transfer + Name quarterly-report.pdf + Type PDF document (.pdf) + Size 2.4 MiB + Save as ./quarterly-report-1.pdf (quarterly-report.pdf already exists) +Accept? [y/N] +``` + +```text +Incoming transfer + Folder holiday-photos/ 128 files, 1.2 GiB unpacked + Size 1.1 GiB to download + Save to ./holiday-photos/ +Accept? [y/N] +``` + +- **Name**: through `sanitize_for_display`, which strips C0/C1 control + characters, DEL and bidirectional controls (U+200E, U+200F, U+061C, + U+202A–202E, U+2066–2069) and collapses whitespace runs to one space. It caps + at 80 columns with an ellipsis **in the middle**, so the extension stays + visible. Every other peer-chosen name reaching a terminal goes through it too: + the `Receiving` line, extractor warnings, and the "already exists" note. +- **Type** comes from the receiver's own reading of the final extension, not + from the sender's `mime_type`. The sender chooses the MIME type, so it is a + claim, while the extension of the name that will be written is a fact. + `report.pdf` padded to `report.pdf .exe` shows `Type Program (.exe)`. +- **A warning line for programs and scripts**: `.exe .msi .bat .cmd .com .scr + .ps1 .vbs .js .jar .app .dmg .pkg .sh .command .desktop .lnk`, and on + Windows any extension in `%PATHEXT%`. It reads: "This is a program. Only open + it if you trust the sender." +- **Folders**: `Metadata` gains optional `entry_count` and `unpacked_size`. + Sealed, so no metadata leak. They are the sender's claims, and are shown as + that: the expansion guard at + [`recv.rs:55-88`](../../cli/src/recv.rs#L55-L88) still bounds what is + actually written. After the transfer, report the **actual** file count + alongside. `serde(default)` on both fields is not needed for compatibility, + since the version bump already refuses old peers, but keeps the JSON tolerant. +- The existing `-f/--force` changes the Save-as line to "replacing". + +**When nobody is there**: `-y`/`--yes` accepts without asking. **With no +terminal on stdin and no `--yes`, `drop recv` refuses before it connects**, +with "no terminal to ask for consent; pass --yes to accept whatever the sender +sends". Refusing before connecting matters: a refusal after key exchange would +burn a code the sender cannot reuse. (Decided by the user on 2026-09-14: require +`--yes`, no silent auto-accept.) `netlab/runner.py` passes `--yes`. So do the +Rust integration tests that drive the binary. + +**Deadline**: 120 s from showing the prompt. On expiry, send `decline { reason: +"timed_out" }`, print "No answer in 2 minutes; declined.", and exit 0. The +prompt reads stdin on the blocking pool the way `AskTheTerminal` already does, +[`send.rs:455-461`](../../cli/src/send.rs#L455-L461). + +### The sender + +What it prints, command surface, one line per state change on stderr: + +```text +Waiting for the receiver to connect... +Receiver connected. +The receiver entered the code. Waiting for them to accept... +Accepted. Sending... +Sending 45.0% 1.1 MiB / 2.4 MiB ... ← already acknowledgement-driven +The receiver has everything and is finishing up... +Done. The receiver confirmed all 2.4 MiB. +``` + +or `The receiver declined.` (exit 3), `The receiver didn't answer in time.` (exit +3), `The receiver cancelled.` (exit 4), `The receiver couldn't write the file, so +the transfer stopped.` (exit 4). + +- **Waiting for `accept`**: up to 150 s, which is the receiver's 120 s plus + margin for a slow link. Every wait loop in `send.rs` — + `await_meta_checkpoint`, the new `await_consent`, `next_acknowledgement` and + `await_completion` — handles `cancel` and `decline` rather than skipping + unknown frames. Today they skip them, which is how a cancel would currently + be ignored for the rest of a transfer. +- `--status`/`DROP_STATUS` gains a second kind of line, + `drop-status: state=`, + so netlab and scripts can assert the conversation, not only the path. Existing + `drop-status: path=` lines are unchanged. +- **Exit codes** become part of the program-facing surface: 0 done, 1 error, + 3 declined or timed out, 4 cancelled by the peer, 130 cancelled here. Only the + sender distinguishes 3 and 4. A receiver that declines exits 0, because it + did what it was asked. See open question 2. + +### Cancel + +One mechanism for both sides and both surfaces: a `Cancel` token (a +`tokio::sync::watch` or `CancellationToken`) passed into `send_transfer` and +`receive_transfer`. Each `transport.receive()` wait becomes a `select!` between +the frame and the token. + +- **On cancel**: send `cancel { reason: "user" }` best-effort, bounded at 1 s, + then `close()`. The receiver discards a partial single file exactly as it does + on an integrity failure, + [`recv.rs:442-447`](../../cli/src/recv.rs#L442-L447). A partial extraction is + left in place and reported: "Cancelled. 37 of 128 files were already + extracted into ./holiday-photos/." That keeps the existing reasoning about not + deleting a tree the receiver may already have had files in. +- **Ctrl-C at a command**: the first press fires the token, and the handler + stops calling `process::exit` directly. A **second** Ctrl-C, or the token not + resolving within 2 s, exits immediately as today, with terminal restore and + spool cleanup first. That keeps the "a signal must never leave the terminal + raw" guarantee from item 7 phase 1. +- **The interface**: the review screen has `Accept` / `Decline` buttons (←/→, + Enter; `y`/`n` shortcuts). The transfer screen, from item 7 phase 3, has a + `Cancel` button, with Esc and `c`. Cancelling during a transfer asks "Cancel + the transfer? The receiver will be told." (Enter confirms, Esc returns), since + one stray Esc should not end a 4 GiB transfer at 99%. +- **Cancel-safety of reads**: `iroh` reads are not cancel-safe mid-frame. That + is harmless here for the reason `await_meta_checkpoint` already states, + [`send.rs:540-549`](../../cli/src/send.rs#L540-L549): after cancelling, nothing + reads that stream again. Writing the `cancel` frame uses the send half, which + the abandoned read does not touch. + +### The sender's interface screen + +Item 7 phase 3 builds the transfer screen. This plan supplies what it shows: + +```text +┌ Sending quarterly-report.pdf ─────────────────────────────────┐ +│ Code 7F2A91-crossover-clockwork-ridge │ +│ Path peer-to-peer (no Drop server) │ +│ │ +│ ✓ Receiver connected │ +│ ✓ Code verified │ +│ ✓ Accepted │ +│ ● Sending ██████████░░░░░░░░░░ 45% 1.1 / 2.4 MiB 8 MiB/s│ +│ ○ Receiver finishing │ +│ │ +│ [ Cancel ] │ +└───────────────────────────────────────────────────────────────┘ +``` + +The state list is fed by a `SenderEvent` channel that `send_transfer` emits +into. The command surface prints the same events as lines, so the two surfaces +cannot drift. This is the "sink" item 7 phase 3 already calls for, given a +concrete type. + +## Phases + +### Phase 0 — decisions + +- [ ] `decisions.md` entry: consent before bytes, the receiver-owned type + label, `--yes` required without a terminal, reasons as enumerations, + version 2 versions the conversation, and exit codes 3 and 4. +- [ ] Settle open questions 1–3 below. + +### Phase 1 — display safety, on its own + +It fixes a live bug independent of consent: an escape sequence in a received +filename reaches the terminal today. + +- [ ] `sanitize_for_display` in a new `cli/src/display.rs`, with unit tests: + CSI sequence, OSC 8 hyperlink, C1 control, each bidirectional control, + collapsed whitespace padding, middle ellipsis keeping the extension, and a + legitimate non-ASCII name (`Łódź 東京 🎉.txt`) passing through unchanged. +- [ ] Applied to every peer-chosen string that reaches a terminal: + `recv.rs:282-286`, `recv.rs:526-532`, extractor warnings printed at + `recv.rs:662-664`, and the sender's printing of paths it scanned (its own + files, but a hostile filename can already exist on a shared disk). +- [ ] `type_label(name)` and `is_program(name)`, tested against the padding + trick above. + +### Phase 2 — protocol and relay + +Bumps the version; lands with or after `meta_ok` key confirmation. + +- [ ] `crypto`: `ENVELOPE_VERSION = 2`; `Metadata` gains `entry_count` and + `unpacked_size`, both `Option`. +- [ ] `quic.rs`: `DROP_ALPN = b"drop/transfer/2"`, and the ALPN-mismatch error + mapped to the version sentence. +- [ ] Relay: the new `ReceiverMessage` variants, forwarding, ordering + enforcement (no chunk before `accept`), `decline`/`cancel` counted apart + from failures, `last_activity` refreshed on forwarded frames. +- [ ] Relay tests in `tests/websocket_transfer.rs`: accept then transfer; + decline ends the session with no failure metric; a chunk before accept + fails the session; receiver cancel mid-stream reaches the sender as + `cancel`; sender cancel reaches the receiver as `cancel`, not `error`. +- [ ] `protocol.md`: the conversation diagram, both frame tables, the relay's + new ordering rules, and the version section. + +### Phase 3 — receiver consent in the CLI + +- [ ] Split `open_target` into `plan_target` / `create_target`. +- [ ] `ConsentPrompt` trait, the counterpart of `AnotherAttempt`, with + `AskTheTerminal`-style and scripted implementations, so the policy is + tested without a terminal. +- [ ] Preview rendering for file and folder; `-y/--yes`; no-terminal refusal + **before connecting**; the 120 s deadline. +- [ ] Sender: `payload.rs` fills `entry_count` and `unpacked_size` from + `TarPlan`; `await_consent`. +- [ ] Tests over `ScriptedTransport` and in `cli/tests/transfer.rs` over a real + relay and over the direct pair: + - decline leaves the destination directory **byte-for-byte unchanged** + (listing and mtimes) + - timeout declines + - accept completes + - a chunk before accept is refused + - `--yes` never prompts + - no terminal without `--yes` exits non-zero **without contacting the relay** +- [ ] `netlab/runner.py` passes `--yes`. + +### Phase 4 — cancel and status + +- [ ] The `Cancel` token through both transfer paths; every wait loop handles + `cancel`/`decline`. +- [ ] Ctrl-C: first press cancels politely, second press or 2 s exits hard. +- [ ] `SenderEvent` channel, and the command-surface lines and + `drop-status: state=` lines from it. +- [ ] `finishing` from the receiver. +- [ ] Exit codes 3 and 4. +- [ ] Tests: + - receiver cancel mid-stream ends the sender with exit 4 over both carriers + - sender cancel mid-stream leaves no partial single file on the receiver + - a partial extraction is reported with its count + - Ctrl-C test on Unix: send SIGINT to a child `drop send` mid-transfer and + assert the receiver prints "The sender cancelled." +- [ ] netlab: assert the `state=` sequence in the relayed and plain-LAN + topologies, so the lab checks the conversation as well as the carrier. + +### Phase 5 — the interface + +Coordinated with item 7 phase 3, which builds the transfer screen this plan +fills in. + +- [ ] Receiver review screen with Accept/Decline, the same fields as the + command preview, and the program warning. +- [ ] Sender transfer screen with the state list above and Cancel. +- [ ] Receiver transfer screen with progress and Cancel. +- [ ] Cancel confirmation during a transfer. +- [ ] Both screens show the peer's cancel or decline in words and close on a + key, not instantly, so the person sees what happened. + +### Phase 6 — documentation + +- [ ] `README.md`: consent, `--yes`, cancelling, exit codes; release notes say + scripts piping `drop recv` must add `--yes`. +- [ ] `docs/commands.md`: the preview and the status lines. +- [ ] `security.md`: display sanitisation, the receiver-owned type label, and + the relay's new ordering enforcement. +- [ ] Mark [`receiver-confirmation-plan-2026-08-19.md`](receiver-confirmation-plan-2026-08-19.md) + superseded, pointing here. + +## Files + +| File | Change | +| --- | --- | +| `crypto/src/envelope.rs` | version 2, `entry_count`, `unpacked_size` | +| `src/domain/messages.rs`, `src/routes/upload_ws.rs`, `src/routes/download_ws.rs`, `src/services/transfer_service.rs`, `src/telemetry/metrics.rs` | new receiver frames, forwarding, ordering, metrics | +| `cli/src/display.rs` | new: sanitisation, type label, program detection | +| `cli/src/recv.rs` | plan/create split, consent, cancel, `finishing` | +| `cli/src/send.rs` | `await_consent`, cancel handling in every wait, `SenderEvent`s, exit codes | +| `cli/src/payload.rs` | `entry_count`, `unpacked_size` | +| `cli/src/transport/quic.rs` | ALPN version | +| `cli/src/main.rs` | `--yes`, exit codes, Ctrl-C two-stage | +| `cli/src/ui/app.rs` | review and transfer screens | +| `cli/tests/transfer.rs`, `tests/websocket_transfer.rs`, `netlab/runner.py`, `netlab/test_transfer.py` | as phases 2–4 | +| `docs/protocol.md`, `docs/security.md`, `docs/decisions.md`, `docs/commands.md`, `README.md` | as phase 6 | + +## Risks + +- **Breaking scripted receivers.** Deliberate, and chosen by the user. The + error has to name `--yes`, and the release notes have to say it first. +- **A preview that lies by omission.** The sender controls `entry_count` and + `unpacked_size`. Labelling them as the sender's claims and reporting the + actual counts afterwards keeps the preview honest. The expansion guard keeps + it safe. +- **Name race between preview and creation.** Handled by renumbering, and by + printing the name actually used. A test pins it. +- **Three clocks.** Accept deadline 120 s, sender consent wait 150 s, relay + session TTL 300 s. In that order and pinned by a test that reads the + constants, so a later edit that inverts them fails in CI instead of in front + of a user. +- **Cancel racing completion.** A cancel that crosses a `complete` in flight + must not turn a finished transfer into a reported failure. Rule: the receiver + counts a transfer done once it has sent `complete`, and ignores a later + `cancel`. The sender counts it done once it has received `complete`. Tested + with a scripted transport that delivers both. +- **Two version bumps colliding.** `meta_ok` confirmation and this plan both + change the wire. Whichever lands first bumps to 2, the other rides on it, and + nothing is released in between. + +## Validation + +```bash +scripts/check-secrets.sh +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings +cargo test --workspace --all-targets +netlab/.venv/bin/python -m pytest netlab/ +``` + +- [ ] Declining leaves the destination unchanged, and the sender exits 3 +- [ ] A hostile filename renders inert in the preview and in every other line +- [ ] Either side's cancel reaches the other in words, over both carriers +- [ ] No terminal without `--yes` refuses before the relay is contacted +- [ ] The sender's state lines appear in order in netlab's relayed and LAN runs +- [ ] 0.3.0 against 0.4.0 fails with the version sentence in both directions, on + both carriers + +## Open questions + +### Open question 1 — does the sender learn the receiver's saved name? + +It would be reassuring ("saved as report-1.pdf"). It also tells the sender +something about the receiver's disk, since a numbered name means the file +already existed. **Leaning: no.** The sender learns accepted, declined and done, +and nothing about the receiver's filesystem. + +### Open question 2 — exit codes + +Proposed: 3 declined or timed out, 4 cancelled by the peer, 130 cancelled +locally. Is 3 for "didn't answer" right, or should a timeout be 4? **Leaning: +3.** From the sender's side both mean the receiver did not take it, and a +script retrying on 3 handles both correctly. + +### Open question 3 — should the receiver be able to see the sender's name for itself? + +No identity exists in the protocol. The code is the only credential. Out of +scope, and worth writing down, because "who is this from?" is the obvious next +request after a preview exists. The honest answer is the channel the code was +shared over. + +## Kickoff prompt + +```text +Read docs/plans/receiver-consent-and-status-plan-2026-09-14.md, docs/protocol.md, +docs/decisions.md entries 12 and 13, and AGENTS.md. Verify line references +first. Start with phase 1 (display safety) on its own branch — it fixes a live +bug and has no protocol change. Phase 2 bumps ENVELOPE_VERSION and DROP_ALPN; +coordinate with the meta_ok key confirmation plan so the wire changes once. +Nothing may be created in the receiver's destination before it accepts. +``` diff --git a/docs/plans/self-hosted-rendezvous-plan-2026-09-10.md b/docs/plans/self-hosted-rendezvous-plan-2026-09-10.md index b6460cb..d9aa61d 100644 --- a/docs/plans/self-hosted-rendezvous-plan-2026-09-10.md +++ b/docs/plans/self-hosted-rendezvous-plan-2026-09-10.md @@ -1,8 +1,8 @@ # Self-hosted rendezvous plan -Status: **in progress** — phase 1 outstanding +Status: **active** — phases 1 and 2 landed 2026-09-10 and shipped in 0.3.0; phase 3 proposed 2026-09-14 and awaiting a decision Created: **2026-09-10** -Last updated: **2026-09-10** +Last updated: **2026-09-14** ## Goal @@ -125,22 +125,22 @@ interface for it. ### Phase 1 — The two knobs -- [ ] `Rendezvous` in `direct.rs`: parsed from `DROP_RENDEZVOUS_RELAY` and +- [x] `Rendezvous` in `direct.rs`: parsed from `DROP_RENDEZVOUS_RELAY` and `DROP_RENDEZVOUS_BOOTSTRAP`, with a malformed value **failing loudly** rather than falling back to the default. Silently ignoring a typo in a relay URL would send an operator's traffic to n0 while they believed it was staying inside their network, which is the one outcome worse than an error. -- [ ] `QuicEndpoint::bind` takes a `RelayMode`; `bind_without_relays` stays as +- [x] `QuicEndpoint::bind` takes a `RelayMode`; `bind_without_relays` stays as the name for the LAN-only case so existing tests keep reading correctly. -- [ ] `MainlineDirectory::new` takes bootstrap nodes. -- [ ] `publish_sender` and `dial_sender` take the `Rendezvous` they bind with. -- [ ] Unit tests for parsing: unset, set, a malformed URL, a malformed +- [x] `MainlineDirectory::new` takes bootstrap nodes. +- [x] `publish_sender` and `dial_sender` take the `Rendezvous` they bind with. +- [x] Unit tests for parsing: unset, set, a malformed URL, a malformed bootstrap entry, and that an empty variable is treated as unset rather than as "no bootstrap nodes at all" — the second would silently disable the DHT. -- [ ] `ENVIRONMENT` block in `USAGE`, naming both and saying what unset means. -- [ ] `security.md`: what an operator takes on, from the section above. +- [x] `ENVIRONMENT` block in `USAGE`, naming both and saying what unset means. +- [x] `security.md`: what an operator takes on, from the section above. Deliberately **not** in phase 1: a pkarr relay. pkarr's HTTP relay client is excluded today by `default-features = false`, which `cli/Cargo.toml` says is @@ -155,6 +155,44 @@ Not work in this plan, and listed so the dependency is visible: Phase 4 of relay and a mainline testnet in a namespace and points both variables at them. That is where this gets exercised end to end. +### Phase 3 — a relay whose certificate a private CA issued + +Proposed 2026-09-14, from the netlab finding. **Not started. It needs the +user's decision**, because it changes what the CLI trusts. + +**The gap.** QUIC address discovery against the rendezvous relay is verified +against iroh's CA roots. A deployment whose relay uses a certificate from its +own CA fails that check silently, and that is the ordinary case inside an +egress-filtered network, the exact audience phase 1 was built for. Transfers +still complete, over the relay. `--status` still says `path=p2p`. Hole punching +is never attempted, and nothing tells the operator. + +- [ ] Decide whether this is wanted at all. The alternative is documenting that + the relay needs a publicly trusted certificate for traversal to work, and + stopping there. +- [ ] If wanted: `DROP_RENDEZVOUS_CA`, a path to a PEM file whose certificates + are **added to** the default roots, never replacing them, and only for the + endpoint's relay and discovery connections. A malformed, unreadable or + empty file is an error, by this plan's own no-silent-fallback rule. + Setting it without `DROP_RENDEZVOUS_RELAY` is also an error, so an + operator cannot believe they configured something they did not. +- [ ] What it does **not** change: peer authentication. Peers still + authenticate each other by endpoint id and by the transfer code. A hostile + CA in this file can impersonate the *relay*, which already sees connection + metadata and nothing more (see Risks). `security.md` says so. +- [ ] Detect the silent failure whether or not the variable is set. When a + custom relay is configured and the endpoint comes online with no + discovered public address, print one warning naming the likely cause. +- [ ] Lab: the helper issues a CA and signs its leaf with it, writes the CA to + a file, and the runner passes `DROP_RENDEZVOUS_CA`. A self-signed leaf + used as its own trust anchor is rejected too, so the helper needs a real + CA. Then the full-cone row finally tests traversal, and the symmetric + row's negative control becomes meaningful instead of trivially true. + +**Rejected:** skipping certificate verification in the lab, even as an +experiment. It would make the lab exercise a trust path production never runs, +which is the lab's defining failure mode. + ## Risks - **An operator points this at something hostile.** Addressed above: the @@ -171,7 +209,10 @@ That is where this gets exercised end to end. is ever learned and no punch is attempted. The lab's helper therefore serves QUIC address discovery on a self-signed certificate, which iroh's client accepts because it installs a custom verifier — it authenticates by endpoint - id, not by certificate chain. The port is not configurable: `RelayMode::custom` + id, not by certificate chain. **Corrected 2026-09-14: it does not.** iroh + 1.0.3 rejects that certificate with `UnknownIssuer`, so address discovery has + never worked in the lab and no punch has ever been attempted there. See + phase 3, and the netlab plan's "Second look, 2026-09-14". The port is not configurable: `RelayMode::custom` gives every entry `RelayQuicConfig::default()`, so the client probes 7842 and nowhere else. Worth knowing if a second relay URL is ever added. - **Scope creep into a discovery framework.** Two values, read once, with no