From 95074aaef216af45bb1a4d83206fd5f85f4a0105 Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Sat, 5 Sep 2026 21:59:56 +0000 Subject: [PATCH] Add the LogicSRC OpenSwarm specification family BitTorrent distributes bytes better than any CDN and has never been a product: nobody is paid to seed, and nothing in it is private. Every Profullstack media property answers that with a central HTTP proxy and a pass system bolted on, and pays for every byte it serves. OpenSwarm is an add-on to BitTorrent, carried as BEP 10 extension messages, that fixes both. The swarm carries AES-256-CTR ciphertext whose integrity is pinned by two SHA-256 merkle roots in a signed manifest, so a tracker or DHT node learns an infohash and a size. A leecher buys a pass over x402 in USDC (the same exchange x402-gateway runs for crawl passes, settled by CoinPay), a seeder serves inside a bounded credit window, and the leecher signs a cumulative voucher for every verified batch. Whoever seeds gets paid. Vanilla clients remain valid members and browsers remain first-class peers. Adding a file mints a key pair for it, derived from one publisher seed by default so there is one thing to back up: the public half is the file's identity and its BEP 46 key, the private half signs the manifest and authorises grants and payout changes, and a separate content key encrypts the bytes and is sealed to paying peers. The family: a core (records, keys, hashing, transports, discovery, events), ipfile, ippay, ipdb (a signed hash-chained catalogue with heads on the DHT), ipaudio, ipvideo, iplive (paid relays with backpressure) and ipname (Moshpit pins and DNS TXT). Plus the c0mpute.com integration with seven workload types and thirteen use cases, a proposed ip CLI, conformance profiles, a security model and an FAQ. Registered on the site the way OpenCreds is: nav entry, docs registry, sitemap, a /openswarm landing page, PRD 0005, and the MCP prd_next_id expectation moved to 0006. Specs only. No code, no schemas, no reference implementation. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2 --- apps/logicsrc-web/src/app/openswarm/page.tsx | 237 ++++++++ apps/logicsrc-web/src/app/sitemap.ts | 1 + .../src/components/site-shell.tsx | 1 + apps/logicsrc-web/src/lib/docs.ts | 1 + docs/openswarm.md | 190 ++++++ docs/openswarm/c0mpute.md | 390 +++++++++++++ docs/openswarm/cli.md | 170 ++++++ docs/openswarm/conformance.md | 157 +++++ docs/openswarm/faq.md | 89 +++ docs/openswarm/ipaudio.md | 313 ++++++++++ docs/openswarm/ipdb.md | 386 +++++++++++++ docs/openswarm/ipfile.md | 545 ++++++++++++++++++ docs/openswarm/iplive.md | 378 ++++++++++++ docs/openswarm/ipname.md | 187 ++++++ docs/openswarm/ippay.md | 464 +++++++++++++++ docs/openswarm/ipvideo.md | 271 +++++++++ docs/openswarm/security.md | 133 +++++ docs/openswarm/spec.md | 510 ++++++++++++++++ packages/logicsrc-mcp/src/standards.test.ts | 2 +- prd/0005-add-logicsrc-openswarm-spec.md | 184 ++++++ prd/README.md | 1 + 21 files changed, 4609 insertions(+), 1 deletion(-) create mode 100644 apps/logicsrc-web/src/app/openswarm/page.tsx create mode 100644 docs/openswarm.md create mode 100644 docs/openswarm/c0mpute.md create mode 100644 docs/openswarm/cli.md create mode 100644 docs/openswarm/conformance.md create mode 100644 docs/openswarm/faq.md create mode 100644 docs/openswarm/ipaudio.md create mode 100644 docs/openswarm/ipdb.md create mode 100644 docs/openswarm/ipfile.md create mode 100644 docs/openswarm/iplive.md create mode 100644 docs/openswarm/ipname.md create mode 100644 docs/openswarm/ippay.md create mode 100644 docs/openswarm/ipvideo.md create mode 100644 docs/openswarm/security.md create mode 100644 docs/openswarm/spec.md create mode 100644 prd/0005-add-logicsrc-openswarm-spec.md diff --git a/apps/logicsrc-web/src/app/openswarm/page.tsx b/apps/logicsrc-web/src/app/openswarm/page.tsx new file mode 100644 index 0000000..b9b83be --- /dev/null +++ b/apps/logicsrc-web/src/app/openswarm/page.tsx @@ -0,0 +1,237 @@ +import type { ReactNode } from "react"; +import type { Metadata } from "next"; +import { SiteShell } from "@/components/site-shell"; +import { card, mono, pre, table, td, th } from "../openontology/ui"; + +export const metadata: Metadata = { + title: "OpenSwarm · LogicSRC", + description: + "OpenSwarm is an open specification family for paid, encrypted, peer-to-peer distribution of files and media: BitTorrent plus payment plus encryption as extension messages, with audio, video, live streams and a replicated catalogue on top, and c0mpute.com nodes that seed, relay and index for money.", + alternates: { canonical: "/openswarm" } +}; + +const PROTOCOLS: Array<[name: string, line: string]> = [ + ["ipfile", "Paid, encrypted file swarms as BitTorrent extension messages"], + ["ippay", "Passes and vouchers over x402 and CoinPay: how a leecher pays and a seeder is paid"], + ["ipdb", "A replicated, append-only catalogue of manifests and metadata, discoverable over the DHT"], + ["ipaudio", "Audio releases and tracks on ipfile swarms: renditions, seek tables, gapless, royalties"], + ["ipvideo", "Video on demand: CMAF renditions, segment index, subtitles, thumbnails, an HLS bridge"], + ["iplive", "Live streams: segment fan-out over paid relays with backpressure"], + ["ipname", "How a Moshpit name or a domain resolves to a publisher key and a catalogue"] +]; + +const ROLES: Array<[role: string, work: string, paid: string]> = [ + ["Storage", "Pin an ipfile swarm for a period and seed it", "Pin job plus vouchers per byte served"], + ["Relay", "Fan out iplive segments to viewers", "Vouchers per byte served"], + ["Transcode", "Produce ipaudio and ipvideo renditions as ipfile swarms", "Job price"], + ["Index", "Replicate ipdb feeds, answer queries, serve a gateway", "Job price plus x402 per query"], + ["Keeper", "Hold a content key and grant it to paying peers", "Share of each key grant"] +]; + +const DOCS: Array<[slug: string, title: string, blurb: string]> = [ + ["openswarm", "Overview", "What the family defines, how the pieces stack, and what already exists."], + ["opencreds", "OpenCreds", "Where the publisher seed should live."] +]; + +export default function OpenSwarmPage(): ReactNode { + return ( + +
+
+

LogicSRC standards surface

+

OpenSwarm

+

+ An open specification family for paid, encrypted, peer-to-peer distribution{" "} + of files and media. It is an add-on to BitTorrent, not a new transport. A swarm carries + ciphertext, the tracker and the DHT learn nothing about the content, every peer that + serves a verified piece gets paid for it, and the same primitives carry files, audio, + video, live streams and the catalogue that describes them. +

+
+

+ It exists because BitTorrent solved distribution and never solved the two things that + keep it from being a product: nobody is paid to seed, and nothing in it is private. Every + Profullstack media property has rebuilt the same paid-access layer on top of a central + HTTP proxy because the swarm could not carry the payment. +

+

+ Status: 0.1 draft. No reference implementation yet. The full documents + live in the repo under docs/openswarm/. +

+
+ +
+
+

The family

+

Seven protocols over one core. Reuse is the default posture: a primitive is specified once.

+
+
+ + + + + + + + + {PROTOCOLS.map(([name, line]) => ( + + + + + ))} + +
ProtocolOne line
{name}{line}
+
+ 
{`  +---------------------------------------------------------------+
+  |  ipaudio      ipvideo      iplive         ipdb (catalogue)    |
+  +---------------------------------------------------------------+
+  |  ipfile: manifest, per-file key pair, encrypted pieces,       |
+  |          key grants, credit window, vouchers per served piece |
+  +-------------------------------+-------------------------------+
+  |  ippay: passes and vouchers   |  ipname: Moshpit name -> key  |
+  +-------------------------------+-------------------------------+
+  |  BitTorrent wire (BEP 3) + extension protocol (BEP 10)        |
+  |  DHT (BEP 5, 44, 46), trackers, WebRTC peers, webseeds,       |
+  |  optional Moshpit MTP/1 tunnel                                |
+  +---------------------------------------------------------------+`}
+
+ +
+
+

One swarm shape, one key model, one payment loop

+
+
+
+ The swarm carries ciphertext. +

+ A hybrid BEP 52 torrent whose single file is AES-256-CTR ciphertext. The v1 infohash + makes it reachable from WebTorrent in a browser, the v2 merkle tree makes every 16 KiB + block verifiable, and the plaintext root in the signed manifest makes the decrypted + result verifiable. A vanilla client can join and, if the publisher allows, download + bytes it cannot read. +

+
+
+ Adding a file mints a key pair for it. +

+ Derived from one publisher seed by default, so there is one thing to back up. The + public half is the file’s identity and the BEP 46 key under which the latest + manifest is published; the private half signs the manifest and authorises key grants + and payout changes. A separate random content key encrypts the bytes and is sealed to + paying peers. +

+
+
+ Seeders are paid per verified piece. +

+ A leecher buys a pass over x402 in USDC. A seeder serves inside a bounded credit + window. After verifying each batch the leecher signs a cumulative voucher. The seeder + redeems the latest voucher at a hub, which splits it between publisher, seeder and + itself. Nobody pays for bytes they did not verify, and nobody serves more than the + window unpaid. +

+
+
+ 
{`leecher                                   seeder
+  |  hello                                  |
+  |---------------------------------------->|
+  |<------------------------------- hello   |
+  |  pass                                   |
+  |---------------------------------------->|   verifies hub signature, scope, cap
+  |<----------------------------- unchoke   |
+  |  request / piece ... (vanilla)          |
+  |<=======================================>|   up to the credit window unpaid
+  |<------------------------------ credit   |
+  |  voucher (cumulative, signed)           |
+  |---------------------------------------->|   unpaid resets
+  |  key_req                                |
+  |---------------------------------------->|
+  |<------------------------------- grant   |   content key sealed to the leecher`}
+
+ +
+
+

How c0mpute.com nodes take part

+

+ A node already advertises roles and takes work through a gossipsub auction. OpenSwarm + adds workload types to bid on and a per-byte income that needs no auction at all. +

+
+
+ + + + + + + + + + {ROLES.map(([role, work, paid]) => ( + + + + + + ))} + +
RoleWorkPaid by
{role}{work}{paid}
+
+
+ +
+
+

Using it (proposed CLI)

+
+ 
{`# One publisher identity, reused for every file
+ip init
+
+# Add a file: derives the file key pair, encrypts, builds the hybrid torrent,
+# signs the manifest, starts seeding
+ip file add ./interview.flac --per-gib 0.01 --key-price 0.50
+
+# Publish an audio release that references it
+ip audio publish ./release.json
+
+# Fetch as a paying peer (buys a pass, gets the key, verifies, decrypts)
+ip file get ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d --out ./interview.flac
+
+# Ask the catalogue
+ip db query --type ipaudio.track --where 'record.artist == "Ada"' --limit 20`}
+
+ +
+
+

What already exists

+
+
+
+ bittorrented.com runs the browser WebTorrent player, a hybrid seeder on + wss trackers, and a DHT crawl. It speaks the vanilla wire this family extends. +
+
+ x402-gateway and CoinPay sell passes with the x402 v2 offer and verify + and settle the proof. ippay reuses that exchange unchanged. +
+
+ c0mpute has the blake3 chunk store, erasure coding, hardware transcode, + a gateway, and a live-stream design that already swarms segments. +
+
+ moshpit-transport and the Moshpit registry provide the post-quantum + tunnel and the name pins. +
+
+
+ {DOCS.map(([slug, title, blurb]) => ( + + {title} +

{blurb}

+ + ))} +
+
+ + ); +} diff --git a/apps/logicsrc-web/src/app/sitemap.ts b/apps/logicsrc-web/src/app/sitemap.ts index 46b9682..77dc0e5 100644 --- a/apps/logicsrc-web/src/app/sitemap.ts +++ b/apps/logicsrc-web/src/app/sitemap.ts @@ -18,6 +18,7 @@ const STATIC_ROUTES: Array<{ { path: "/docs", changeFrequency: "weekly", priority: 0.9 }, { path: "/openontology", changeFrequency: "weekly", priority: 0.9 }, { path: "/opencreds", changeFrequency: "weekly", priority: 0.9 }, + { path: "/openswarm", changeFrequency: "weekly", priority: 0.9 }, { path: "/openprd", changeFrequency: "weekly", priority: 0.9 }, { path: "/openontology/explore", changeFrequency: "daily", priority: 0.7 }, { path: "/openspec", changeFrequency: "weekly", priority: 0.8 }, diff --git a/apps/logicsrc-web/src/components/site-shell.tsx b/apps/logicsrc-web/src/components/site-shell.tsx index 9375896..582355d 100644 --- a/apps/logicsrc-web/src/components/site-shell.tsx +++ b/apps/logicsrc-web/src/components/site-shell.tsx @@ -11,6 +11,7 @@ const NAV: Array<{ href: string; label: string; external?: boolean }> = [ { href: "/credential-sharing", label: "Credentials" }, { href: "/openontology", label: "OpenOntology" }, { href: "/opencreds", label: "OpenCreds" }, + { href: "/openswarm", label: "OpenSwarm" }, { href: "/openprd", label: "OpenPRD" }, { href: "/#cli", label: "CLI" }, { href: "/docs", label: "Docs" }, diff --git a/apps/logicsrc-web/src/lib/docs.ts b/apps/logicsrc-web/src/lib/docs.ts index 1c02bb7..6db3b35 100644 --- a/apps/logicsrc-web/src/lib/docs.ts +++ b/apps/logicsrc-web/src/lib/docs.ts @@ -8,6 +8,7 @@ const DOCS_DIR = resolve(process.cwd(), "../../docs"); // Curated, public-facing reference docs. Internal notes (roadmap, positioning, // arcade) are intentionally excluded. export const DOC_SLUGS = [ + "openswarm", "opencreds", "openprd", "openontology", diff --git a/docs/openswarm.md b/docs/openswarm.md new file mode 100644 index 0000000..61393a4 --- /dev/null +++ b/docs/openswarm.md @@ -0,0 +1,190 @@ +# OpenSwarm + +Status: 0.1 draft. No reference implementation yet; see "What already exists" below. + +Slug: `openswarm` + +OpenSwarm is a LogicSRC OpenSpec family for **paid, encrypted, peer-to-peer +distribution of files and media**. It is an add-on to BitTorrent, not a new +transport. A swarm carries ciphertext, the tracker and the DHT learn nothing +about the content, every peer that serves a verified piece gets paid for it, +and the same swarm primitives carry files, audio, video, live streams and the +catalogue that describes them. + +It exists because BitTorrent solved distribution and never solved the two +things that keep it from being a product: nobody is paid to seed, and nothing +in it is private. Every Profullstack media property (bittorrented.com, +tipoffwatch, genrewatch, p0dcasters) has rebuilt the same paid-access layer on +top of central HTTP proxies because the swarm could not carry the payment. + +Working name during design was "IPMedia". The family is called OpenSwarm +because the swarm is the one object every member shares, because "media" +undersells `ipfile` and `ipdb`, and because "IP media" reads as IPTV, which is +a different product. The member protocols keep their `ip` names. + +## The family + +| Protocol | One line | Document | +| --- | --- | --- | +| Core | Records, identifiers, keys, signatures, transports, discovery, events | [`spec.md`](./openswarm/spec.md) | +| `ipfile` | Paid, encrypted file swarms as BitTorrent extension messages | [`ipfile.md`](./openswarm/ipfile.md) | +| `ippay` | Passes and vouchers: how a leecher pays and a seeder is paid, over x402 and CoinPay | [`ippay.md`](./openswarm/ippay.md) | +| `ipdb` | A replicated, append-only catalogue of manifests and metadata, discoverable over the DHT | [`ipdb.md`](./openswarm/ipdb.md) | +| `ipaudio` | Audio releases and tracks on `ipfile` swarms: renditions, seek tables, gapless, royalties | [`ipaudio.md`](./openswarm/ipaudio.md) | +| `ipvideo` | Video on demand on `ipfile` swarms: CMAF renditions, segment index, subtitles, thumbnails | [`ipvideo.md`](./openswarm/ipvideo.md) | +| `iplive` | Live streams: segment fan-out over peers, paid relays, backpressure | [`iplive.md`](./openswarm/iplive.md) | +| `ipname` | How a Moshpit name resolves to a publisher key and a catalogue | [`ipname.md`](./openswarm/ipname.md) | + +Supporting documents: + +- c0mpute.com integration and use cases: [`c0mpute.md`](./openswarm/c0mpute.md) +- Proposed `ip` CLI: [`cli.md`](./openswarm/cli.md) +- Conformance: [`conformance.md`](./openswarm/conformance.md) +- Security model: [`security.md`](./openswarm/security.md) +- FAQ: [`faq.md`](./openswarm/faq.md) + +## How the pieces stack + +``` + +---------------------------------------------------------------+ + | ipaudio ipvideo iplive ipdb (catalogue) | + | releases titles channels feeds, entries | + +---------------------------------------------------------------+ + | ipfile: manifest, per-file key pair, encrypted pieces, | + | key grants, credit window, vouchers per served piece | + +-------------------------------+-------------------------------+ + | ippay: passes and vouchers | ipname: Moshpit name -> key | + | x402 purchase, CoinPay hub | and catalogue | + +-------------------------------+-------------------------------+ + | BitTorrent wire (BEP 3) + extension protocol (BEP 10) | + | DHT (BEP 5, 44, 46), trackers, WebRTC peers (WebTorrent), | + | webseeds (BEP 19), optional Moshpit MTP/1 tunnel | + +---------------------------------------------------------------+ +``` + +Reading it upward: a video title is a set of `ipfile` swarms plus an index. +An `ipfile` swarm is an ordinary hybrid v1/v2 torrent whose payload is +ciphertext. Payment and key delivery ride on BEP 10 extension messages that a +vanilla client never sees. The catalogue that says what a swarm contains is +itself replicated as `ipfile` swarms and pointed at from the DHT. + +## Reuse is the default posture + +Every building block is specified once, with a stable interface, and reused: + +- **Chunking and hashing** are BitTorrent v2 (BEP 52): 16 KiB blocks, SHA-256 + merkle trees, power-of-two piece lengths. `ipaudio`, `ipvideo` and `iplive` + do not define their own. +- **Manifests** are one record shape (core §3) with a `type`. A track, a + title and a channel are records that reference `ipfile` manifests. +- **Keys** are one identity model (core §4). A publisher key is reused for + every file that publisher adds; per-file keys are derived from it and can be + recovered from it. A fresh unrelated pair is minted only on request. +- **Swarm encryption** is `ipfile`'s AES-256-CTR over pieces, keyed per file. + Audio and video renditions are encrypted the same way, by being files. +- **Payment** is `ippay`'s pass and voucher pair. A relay serving live + segments is paid with the same voucher a seeder gets for a piece. +- **Relay** is one role. A c0mpute node that relays `iplive` segments uses + the same credit window and the same voucher as a seeder. + +A member protocol that needs something the core lacks adds a record type or +an extension message. It does not redefine a primitive. + +## How c0mpute.com nodes participate + +A c0mpute node already advertises roles (`storage`, `transcode`, `gateway`, +`verifier`) and takes work through a gossipsub auction. OpenSwarm adds +workload types the node can bid on and a second income path that needs no +auction at all: + +| Role | Work | Paid by | +| --- | --- | --- | +| Storage | Pin an `ipfile` swarm for a period, seed it | Job price (pinning) plus vouchers per byte served | +| Relay | Fan out `iplive` segments to viewers | Vouchers per byte served | +| Transcode | Produce `ipaudio` / `ipvideo` renditions, publish them as `ipfile` swarms | Job price | +| Index | Replicate `ipdb` feeds, answer queries, serve a gateway | Job price plus x402 per query | +| Keeper | Hold a file's content key and grant it to paying peers | Share of each key grant | + +Details, thirteen worked use cases and the proposed CLI invocations are in +[`c0mpute.md`](./openswarm/c0mpute.md). + +## What it defines + +**One swarm shape.** A hybrid BEP 52 torrent whose single payload file is +AES-256-CTR ciphertext. The v1 infohash makes it reachable from WebTorrent in a +browser, the v2 merkle tree makes every 16 KiB block verifiable, and the +plaintext root in the signed manifest makes the decrypted result verifiable. + +**One key model.** A publisher key pair (Ed25519). Adding a file mints a +per-file pair derived from it: the public half is the file's identity and the +BEP 46 key under which the latest manifest is published; the private half +signs the manifest and authorises key grants and payout changes. A separate +random content key encrypts the bytes and is wrapped to paying peers. + +**One payment loop.** A leecher buys a pass (x402, USDC, CoinPay as the +reference hub). A seeder serves pieces inside a bounded credit window. After +verifying each piece the leecher signs a cumulative voucher. The seeder +redeems the latest voucher at the hub, which splits it between seeder, +publisher and hub. Nobody pays for bytes they did not verify, and nobody +serves more than the window unpaid. + +**One catalogue.** An `ipdb` feed is a signed, hash-linked log of records. +Its head is a BEP 44 mutable item under the feed key, so any DHT node can find +the latest catalogue of any publisher with one `get`. + +## What it does not define + +A media player. Transcoding settings beyond what a manifest must declare. A +token. A blockchain. Content moderation. A search ranking. DRM in the sense +of controlling a device after the key is delivered; a paying peer holds the +key and the spec says so. + +## What already exists + +| Piece | Where | Relationship | +| --- | --- | --- | +| Browser WebTorrent player, hybrid Node seeder, wss trackers | `profullstack/media-streamer` (bittorrented.com) | Speaks the vanilla wire this family extends; needs the `ipfile` extension to pay | +| DHT crawl (bitmagnet) and `/dht` browse | `profullstack/media-streamer`, `dht-infohash-crawler` | Observes `ipfile` swarms as opaque infohashes; `ipdb` is how it would learn what they are | +| Pay-per-pass grants, HLS manifest sealing | `media-streamer` IPTV and seedbox rails | The central-proxy version of what `ippay` moves into the swarm | +| x402 v2 offer, `X-PAYMENT` proof, verify and settle | `profullstack/x402-gateway`, CoinPay | `ippay` pass purchase reuses it unchanged | +| CloudEvents 1.0 + Standard Webhooks signing | `profullstack/autoblog` | Every OpenSwarm event uses the same envelope and headers | +| MTP/1 post-quantum transport, name pins | `profullstack/moshpit-transport`, `moshpit-proxy` | Optional tunnel for native peer links; `ipname` pin kind | +| blake3 chunk store, RS 10/14 erasure, ffmpeg transcode, live-stream DIP 0019 | `profullstack/c0mpute` | The node that seeds, relays, transcodes and indexes | + +New in this family: the extension messages, the manifest and key model, the +pass and voucher protocol, the catalogue log, and the media record types. + +## Quick start (proposed CLI) + +```bash +# One publisher identity, reused for every file +ip init + +# Add a file: derives the file key pair, encrypts, builds the hybrid torrent, +# signs the manifest, starts seeding +ip file add ./interview.flac --per-gib 0.01 --key-price 0.50 + +# Publish an audio release that references it +ip audio publish ./release.json + +# Fetch as a paying peer (buys a pass, gets the key, verifies, decrypts) +ip file get ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d --out ./interview.flac + +# Ask the catalogue +ip db query --type ipaudio.track --where 'record.artist == "Ada"' --limit 20 +``` + +## Relationship to the other LogicSRC specs + +- **OpenCreds** ([opencreds.md](./opencreds.md)) stores the publisher seed as + a `key` item. Losing that seed is the one unrecoverable event in this family, + and OpenCreds is where it should live. +- **OpenContext** ([opencontext.md](./opencontext.md)) governs what an agent + may read. An agent holding a pass for one swarm is entitled to that swarm's + key and nothing else; the pass scope is the OpenContext decision. +- **OpenOntology** ([openontology.md](./openontology.md)) names the entities + an `ipdb` entry refers to. An `ipaudio.track`'s `artist` is an ontology + entity where one is in use. +- **Credential Sharing** ([credential-sharing.md](./credential-sharing.md)) + moves a hub API key or a payout address between providers; OpenSwarm never + stores either. diff --git a/docs/openswarm/c0mpute.md b/docs/openswarm/c0mpute.md new file mode 100644 index 0000000..ac99e55 --- /dev/null +++ b/docs/openswarm/c0mpute.md @@ -0,0 +1,390 @@ +# OpenSwarm on c0mpute.com + +Version: **0.1** (draft) +Status: draft. Describes how a c0mpute node takes part in OpenSwarm swarms +and how it is paid. The node facts below are from the `profullstack/c0mpute` +repo as of 2026-09-05; the workload types and flows are proposed. + +## 1. What c0mpute is, in the terms this document needs + +A c0mpute node is a machine running the `c0mpute` Rust binary as a worker. +Its identity is a libp2p Ed25519 keypair at `~/.config/c0mpute/identity.key`. +Nodes join a Kademlia DHT under `/c0mpute/kad/1.0.0` from a bootstrap list +at `https://c0mpute.com/bootstrap.json`, and advertise capability tags +(`c0mpute:role:storage`, `c0mpute:role:transcode`, `c0mpute:role:gateway`, +`c0mpute:role:verifier`, one of `c0mpute:gpu:nvidia|amd|apple` or +`c0mpute:cpu`) on gossipsub every 60 seconds. + +Work is a gossipsub auction. A buyer publishes a `JobOffer` on +`c0mpute/jobs/` with `required_capabilities`, +`max_price_usd` and a `deadline_unix_ms`; workers publish `JobBid`s; after +an 8 second window the buyer publishes `JobAccept`; the winner runs it and +publishes a `JobReceipt` with `output_hash` and `status`. Payment is +CoinPay escrow in USD-equivalent (USDC among others); workers keep the full +agreed price. Storage today is a blake3-addressed chunk store with +Reed-Solomon 10 data + 4 parity shards, priced at $0.008 per GB-month and +$0.005 per GB of internet egress. Transcode is in-process ffmpeg with +hardware encoders. A gateway serves `/chunks/` on port 7777. The +live-stream DIP 0019 already describes RTMP/SRT ingest to 4 second segments +swarmed BitTorrent-style. + +OpenSwarm adds nothing to how a node joins or bids. It adds workload types +the node can bid on, a second identity the node registers as a payee, and +per-byte income that arrives without any auction. + +## 2. Node identity and payout + +A node derives an OpenSwarm identity from its libp2p key: + +``` +seed = HKDF-SHA256(ikm = libp2p Ed25519 private key bytes, salt = "", info = "openswarm:node:v1") +``` + +From that seed the node has a publisher identity (core §4.1): a signing key +for `hello` messages, vouchers it signs as a payer when it pulls from other +peers, and a box key for grants. At first start with OpenSwarm enabled it +registers as a payee (`ippay` §6.1) at every hub in its configuration, +with the same payout address `c0mpute worker register` already holds for +CoinPay. `c0mpute worker start --openswarm` is the proposed flag; the node +then adds `c0mpute:openswarm:seed`, `c0mpute:openswarm:relay`, +`c0mpute:openswarm:keeper`, `c0mpute:openswarm:index` and +`c0mpute:openswarm:gateway` to its capability tags according to which +roles it runs. + +## 3. Workload types + +Each is a `module.toml` `[module.workloads]` entry in a proposed `openswarm` +plugin. `spec_inline` shapes are given as the buyer sends them in +`JobOffer`; every `spec_inline` carries the records the worker needs, so a +worker never has to resolve anything before bidding. + +| Workload type | Capability | What the worker does | Priced by | +| --- | --- | --- | --- | +| `ipfile.pin` | `c0mpute:role:storage`, `c0mpute:openswarm:seed` | Fetch a swarm, hold it for a period, seed it, answer storage challenges | GB-month, plus vouchers while serving | +| `ipfile.keep` | `c0mpute:openswarm:keeper` | Hold a content key under a delegating grant, answer `key_req` and the key URL | Flat per period, plus `keeperBps` per grant | +| `ipfile.gateway` | `c0mpute:role:gateway`, `c0mpute:openswarm:gateway` | Serve a swarm as a BEP 19 webseed and the manifest and layer routes | Flat per period, plus vouchers | +| `ipvideo.transcode` | `c0mpute:role:transcode` | Produce a CMAF ladder from a source `ipfile`, publish each rendition as an `ipfile`, return the segment indexes | Per job, as `ffmpeg.transcode` | +| `ipaudio.transcode` | `c0mpute:role:transcode` | Produce audio renditions with gapless values and seek indexes | Per job | +| `iplive.relay` | `c0mpute:openswarm:relay` | Join a channel as a relay for a period with a guaranteed `maxDownstream` | Per hour floor, plus vouchers | +| `ipdb.index` | `c0mpute:openswarm:index` | Follow named feeds, keep a replica current, answer `POST /db/query` | Per feed-month, plus x402 per query if the buyer sets a price | + +### 3.1 `ipfile.pin` + +```json +{ + "job_id": "5b8c1f3e-8a7d-4e2b-9c11-0f2a6d3e4b5c", + "workload_type": "ipfile.pin", + "buyer_peer_id": "12D3KooWQYhTNQdmr3ArTeUHRYzFg94BKuRMVuA2v6LzCDXkQmEA", + "buyer_did": "did:coinpay:buyer:5d292428", + "spec_hash": "blake3:9f4c2a1b7e6d5c4b3a2f1e0d9c8b7a6f5e4d3c2b1a0f9e8d7c6b5a4f3e2d1c0b", + "spec_inline": { + "manifest": { "openswarm": "0.1", "type": "ipfile.manifest", "sigs": [] }, + "days": 30, + "minPeersSeen": 1, + "erasure": { "k": 10, "parity": 4 }, + "challenge": { "everyHours": 6 } + }, + "required_capabilities": ["c0mpute:role:storage", "c0mpute:openswarm:seed"], + "max_price_usd": 0.18, + "deadline_unix_ms": 1757104800000, + "published_at_ms": 1757101200000 +} +``` + +(`manifest` is the full record in a real offer.) The price is for holding +734 MB for 30 days at the storage rate plus a margin. The worker fetches +the swarm as a paying peer if it has no free source (it needs a pass; the +buyer includes one scoped to the file in `spec_inline.pass` when +`pricing.perGib` is non-zero), stores the ciphertext however it likes, and +seeds. A `c0mpute:role:verifier` node runs the existing storage challenge +against random pieces; a failed challenge is a `JobReceipt` with `status: +"failed"` and a reputation slash, as today. + +The receipt's `output_hash` is the swarm's `infohashV2`. Vouchers the node +earns while seeding are separate from the job and go through the hub. + +### 3.2 `iplive.relay` + +```json +{ + "workload_type": "iplive.relay", + "spec_inline": { + "channel": { "openswarm": "0.1", "type": "iplive.channel", "sigs": [] }, + "hours": 3, + "maxDownstream": 16, + "regions": ["us-west"] + }, + "required_capabilities": ["c0mpute:openswarm:relay"], + "max_price_usd": 0.60 +} +``` + +(Other `JobOffer` fields as in §3.1.) The floor pays the relay to be there +for the whole event even if nobody watches; vouchers pay per byte when they +do. The origin adds accepted relays to `relays.allow` and to the head's +`relays` list. + +### 3.3 `ipdb.index` + +```json +{ + "workload_type": "ipdb.index", + "spec_inline": { + "feeds": ["ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default"], + "days": 30, + "query": { "public": true, "priceUsd": "0.000000" }, + "gateway": true + }, + "required_capabilities": ["c0mpute:openswarm:index", "c0mpute:role:gateway"], + "max_price_usd": 0.50 +} +``` + +The node follows the feeds, serves `POST /db/query` and the head route, and +publishes its gateway URL in its `hello` and capability ad so clients and +`ipname` records can point at it. + +## 4. How a node gets paid + +Two paths, both landing in the node's CoinPay payout: + +1. **Jobs.** As today: `JobAccept` names the price, CoinPay escrow releases + on a `completed` receipt. Pins, keeps, gateways, transcodes, relays and + index jobs all pay this way for the commitment. +2. **Vouchers.** For every byte the node serves as a seeder, relay, keeper + gateway or query gateway, the paying peer signs a voucher to the node's + OpenSwarm identity. The node redeems the latest voucher per (pass, + payee, swarm) at the hub when a session ends or every 15 minutes, + whichever is sooner, and the hub pays out per its schedule. + +A node that runs no jobs at all, only seeding swarms it chose to hold, is +still paid by vouchers. That is the "plug in and earn" path and needs no +buyer. + +## 5. Use cases + +Each use case is a flow plus the CLI invocations, with the `ip` CLI as +proposed in [cli.md](./cli.md) and the existing `c0mpute` CLI as it is. + +### 5.1 A podcast episode, seeded by the network + +Publisher: p0dcasters.com on behalf of a show. Goal: listeners pay per +episode, seeders are paid, the show gets 70 percent. + +```bash +ip audio publish ./ep42.json --renditions opus-64,opus-96 --per-gib 0.005 --key-price 0.25 +ip file pin ed25519: --days 90 --min-peers 3 --max-price 0.40 +``` + +`ip audio publish` transcodes locally (or with `--on c0mpute`), adds two +`ipfile`s, writes the track record, appends all three to the feed. `ip +file pin` publishes an `ipfile.pin` offer; three storage nodes win and +seed. A listener's app buys a pass scoped to the publisher for 2 USD once, +then streams every episode against it. p0dcasters renders the RSS feed from +the release record through a keeper gateway for apps that only speak HTTP. + +### 5.2 A film, pay-per-view, no server in the path + +Publisher: an independent distributor on bittorrented.com. Goal: viewers +watch in the browser, nobody proxies the bytes. + +```bash +ip video publish ./film.mov --ladder default --on c0mpute --max-price 6.00 +ip file pin --title sha256: --days 180 --min-peers 5 +``` + +Four `ipvideo.transcode` jobs run on GPU nodes; each rendition is added as +an `ipfile` from the worker and seeded there. The bittorrented.com player +(WebTorrent plus the `ipfile` extension plus MSE) buys a pass in the page +with CoinPay Wallet, fetches pieces from WebRTC-capable seeders, decrypts, +plays. The distributor's share arrives daily. + +### 5.3 A live game with paid relays + +Publisher: tipoffwatch.com carrying a stream it has the rights to relay. + +```bash +ip live create tipoff-court1 --latency low --per-gib 0.02 --key-price 1.00 +ip live relay-hire ed25519:<channel key> --hours 3 --relays 6 --max-downstream 16 --max-price 4.00 +ip live start tipoff-court1 --input srt://:9000 --record +``` + +Six `iplive.relay` jobs win. Viewers buy a 1 USD key and stream; each +viewer pulls from two relays; relays earn 35 percent of vouchers. When +the game ends the recording is published as an `ipvideo.title` +automatically and the same pass, scoped to the publisher, plays it. + +### 5.4 Software release distribution, free to download + +Publisher: a vendor shipping a 4 GB installer. Goal: downloads free, +seeding paid by the vendor, nothing on a CDN. + +```bash +ip file add ./installer-4.2.0.dmg --public --per-gib 0 --key-price 0 +ip file pin ed25519:<file key> --days 365 --min-peers 10 --erasure 10,4 +``` + +`--public` sets `keys.public` so any peer decrypts. The vendor pays ten +pins at the storage rate; downloaders pay nothing; vanilla BitTorrent +clients with the manifest's key can also decrypt with a small tool. The +BEP 46 pointer under the file key is a permanent "latest version" link: +`ip file get ed25519:<file key>` always fetches the current release. + +### 5.5 Encrypted off-site backup + +Publisher: a small company. Goal: 200 GB held on three independent nodes, +readable by nobody else. + +```bash +ip file add ./backup-2026-09.tar --standalone-key --private --per-gib 0.01 --key-price 0 +ip file pin ed25519:<file key> --days 30 --min-peers 3 --erasure 10,4 +``` + +`--private` sets `swarm.private` (no DHT, no PEX) and `visibility: pass`. +No grant is ever issued to anyone but the owner; `keys.owner` in the +manifest plus the standalone seed in the company's OpenCreds vault is the +recovery path. Pins are verified by storage challenges. Restore is `ip +file get` with the seed loaded. + +### 5.6 A dataset sold to agents by the gigabyte + +Publisher: a data vendor. Buyer: a crawler or an AI agent with an x402 +wallet. Goal: sell to bots, the same motivation as the crawl paywall. + +```bash +ip file add ./corpus-2026q3.parquet --per-gib 0.50 --key-price 25.00 +``` + +```bash +# the buyer, unattended +x402 pay "https://coinpayportal.com/api/openswarm/passes?payer=<key>&file=<file key>&cap=60&grant=1" +ip file get ed25519:<file key> --pass ./pass.json --out ./corpus.parquet +``` + +`@profullstack/x402-client` buys the pass exactly as it buys a crawl pass. +The agent then fetches 100 GB from whichever nodes pinned it, paying 50 USD +in vouchers as it goes. Seeders that pinned the dataset earn 25 percent of +that; the vendor earns 70. + +### 5.7 An album with a royalty split + +Publisher: a label. Goal: three contributors paid automatically. + +```bash +ip audio publish ./album/*.flac --release ./album.json --renditions flac,opus-160 \ + --royalty 0xCC3b...41E5:6000 --royalty 0x7E5F...5Bdf:3000 --royalty 0x1234...abcd:1000 +``` + +Every voucher's publisher share is split 60/30/10 by the hub after it looks +up the track record in `ipdb`. No play counts, no statements: a payout per +day per address. + +### 5.8 Running an index for a label's catalogue + +Operator: a c0mpute node with a public IP. Buyer: the label. + +```bash +c0mpute worker start --roles gateway --openswarm +ip db index-hire ed25519:<label key>/default ed25519:<label key>/releases --days 30 --max-price 1.00 +``` + +The node follows both feeds and answers queries at +`https://<node>/db/query`. The label's `ipname` record names the node as +its gateway. bittorrented.com's `/dht` page can query it to show what an +`ipfile` infohash it crawled actually is. + +### 5.9 Browser-only listening on bittorrented.com + +No node, no install. The site loads WebTorrent, `wire.use(ipfile)`, a +WebCrypto decryptor and an `<audio>` element fed by MSE. The listener buys +a pass with CoinPay Wallet through the x402 browser flow. Everything else +is peers. The hybrid Node seeder media-streamer already runs is the one +peer that is always there. + +### 5.10 A creator seeding their own work from a laptop + +```bash +ip init +ip file add ./set.mp3 --per-gib 0.01 --key-price 0.50 +ip file seed --all +``` + +No c0mpute at all. The laptop seeds, gets paid by vouchers, and goes to +sleep at night. Listeners who fetched it keep seeding if they leave the +app open, and they are paid too. Pins on c0mpute are what the creator buys +when the laptop is not enough. + +### 5.11 Two organisations exchanging files over a post-quantum link + +Both run native nodes with Moshpit identities and names. + +```bash +ip name set chovy.hacker --gateway https://gw.example --mtp-pin <pin> +ip file add ./contract-bundle.zip --private --per-gib 0 +ip file share ed25519:<file key> --to ip://preshy.nigeria --grant +ip file get ip://chovy.hacker/default/file:ed25519:<file key> --tunnel mtp +``` + +`ip file share` seals a grant to the counterparty's box key from its +`ipname` record and sends it out of band. `--tunnel mtp` runs the +BitTorrent connection through an MTP/1 session pinned to the counterparty's +`mtp` pin. Nothing in the transfer is decryptable by a future quantum +adversary who recorded the link. + +### 5.12 Enriching the DHT crawl + +bitmagnet behind bittorrented.com/dht records every infohash it sees. An +`ipfile` swarm appears as a one-file torrent named by a hex string. With an +`ipdb` replica, `/dht/<infohash>` can look up `swarm.infohashV1` across +followed feeds and render the manifest's `name`, the track or title record, +the price and a play button, instead of "unknown, 734 MB". A feed the crawl +has never seen stays opaque, which is the privacy property working as +designed. + +### 5.13 A keeper for a publisher who is not always online + +```bash +ip file keep-hire ed25519:<file key> --days 90 --max-price 0.30 +``` + +The winning node receives a delegating grant sealed to its box key, +answers `key_req` in the swarm and serves the key URL. It earns +`keeperBps` of every `keyUsd`. The publisher's laptop can be off; buyers +still get keys. + +## 6. What changes in c0mpute + +Proposed, not built: + +1. An `openswarm` plugin (`plugins/openswarm/module.toml`) declaring the + seven workload types in §3 with `mode = "in-process"` for pin, keep, + gateway, relay and index, and `mode = "subprocess"` wrapping the `ip` + CLI for the two transcodes until they are ported. +2. Capability tags `c0mpute:openswarm:*` in `capabilities.rs`, gated by + `--openswarm` flags on `worker start`. +3. A BitTorrent peer in the node: a Rust client that speaks BEP 3, 10, 52, + BEP 44 and the `ipfile`, `ipdb`, `iplive` extensions, with WebRTC via + the same `node-datachannel` approach media-streamer uses in Node, so + browsers can reach it. +4. The chunk store gains a piece index: `(infohashV2, pieceIndex) -> + blake3 chunk`, so a swarm piece is served from the existing store and + erasure shards are reconstructed on demand. +5. `c0mpute-gateway` gains `/swarm/<infohashV2>/{manifest,layer,data}`, + `/db/query`, `/audio/...` and `/video/...` bridge routes with the + `ippay` §7 headers, next to the existing `/chunks/<hash>`. +6. The verifier's storage challenge accepts an `ipfile` manifest and + challenges random pieces by SHA-256 piece root rather than blake3. +7. The status contract (`/api/status`) gains `openswarm: { swarms_seeded, + bytes_served_24h, vouchers_redeemed_24h }`. + +None of this changes the auction, escrow, DID or reputation formula. + +## 7. Open questions + +- Whether `ipfile.pin` should be priced by the existing $0.008 per GB-month + rate automatically, with the buyer setting only `days`. +- Whether the verifier should also spot-check that a pinned node actually + serves (a paid fetch by the verifier, refunded by the hub) rather than + only that it holds. +- Whether c0mpute wants to be a hub itself, so a node's job income and + voucher income arrive on one statement. diff --git a/docs/openswarm/cli.md b/docs/openswarm/cli.md new file mode 100644 index 0000000..b145352 --- /dev/null +++ b/docs/openswarm/cli.md @@ -0,0 +1,170 @@ +# The `ip` CLI (proposed) + +Version: **0.1** (draft) +Status: proposal. No binary exists. Names and flags here are the contract +an implementation would meet; they are not a promise that this is the +final surface. + +## 1. Shape + +One binary, `ip`, with a noun-verb tree. Every command accepts `--json` +for machine output and returns exit code 0 on success, 2 on a usage error, +3 on a verification failure (signature, hash, chain), 4 on a payment +failure (no pass, cap reached, hub refused), 5 on a network failure. + +Configuration lives at `~/.config/ip/config.json`; the keystore at +`~/.config/ip/keys/` with mode 0600, or in an OpenCreds vault when +`ip init --vault` is used. The local `ipdb` replica is +`~/.local/share/ip/db.sqlite`. + +## 2. Commands + +### 2.1 Identity + +``` +ip init [--vault] [--name <label>] +ip key show [--box] +ip key export --out <file> # the seed, encrypted with a passphrase +ip key import <file> +ip key rotate # mints a new seed, revises every file (core §4.5) +ip key bind-pq <mldsa65 key file> # publishes an openswarm.binding +``` + +### 2.2 Files + +``` +ip file add <path> [--per-gib <usd>] [--key-price <usd>] [--public] [--private] + [--standalone-key] [--piece-length <bytes>] [--tracker <url>]... [--webseed <url>]... + [--hub <url>]... [--pay-to <network>:<address>] [--split <pub>,<seed>,<hub>] + [--keeper <ed25519:...>]... [--meta.<k> <v>]... [--feed <name>] +ip file get <file key | infohash | ip:// url | magnet> --out <path> [--pass <file>] [--tunnel mtp] [--stream] +ip file revise <file key> [--per-gib ...] [--tracker ...] [--pay-to ...] # metadata-only revision +ip file reencrypt <file key> # new content key, new swarm +ip file rotate <file key> # successor file key +ip file seed [<file key>... | --all] [--vanilla ciphertext|deny] +ip file share <file key> --to <ip:// name | x25519 key> [--grant] [--delegate --expires <days>] +ip file pin <file key> --days <n> [--min-peers <n>] [--erasure <k>,<p>] [--max-price <usd>] +ip file keep-hire <file key> --days <n> [--max-price <usd>] +ip file status <file key> # manifest, revs, peers seen, bytes served, earnings +``` + +`ip file add` prints the file key, both infohashes, the manifest id and +the `magnet:` and `ip://` forms. With `--json` it prints the manifest. + +### 2.3 Payment + +``` +ip pass buy --hub <url> (--file <key> | --publisher <key>) --cap <usd> [--grant] [--days <n>] +ip pass list +ip pass show <pass id> +ip payee register --hub <url> --pay-to <network>:<address> +ip payee balance [--hub <url>] +ip voucher redeem [--all | --swarm <infohash>] # normally automatic; manual for audit +ip hub info <url> +``` + +`ip pass buy` performs the x402 exchange with the key from +`X402_PRIVATE_KEY` or `--key-file`, the same as `x402 pay`. + +### 2.4 Audio + +``` +ip audio publish <track.json | source files...> [--release <release.json>] [--renditions <ids>] + [--on c0mpute --max-price <usd>] [--royalty <address>:<bps>]... [file add flags] +ip audio play <track id | ip:// url> [--rendition <id>] +ip audio rss <release id> --gateway <url> --out <feed.xml> +``` + +### 2.5 Video + +``` +ip video publish <source> [--ladder default | <ids>] [--subtitle <lang>:<file.vtt>]... + [--thumbnails] [--chapters <file>] [--on c0mpute --max-price <usd>] [file add flags] +ip video play <title id | ip:// url> +ip video hls <title id> --gateway <url> # prints the master playlist URL +``` + +### 2.6 Live + +``` +ip live create <name> [--latency normal|low] [--segment-ms <n>] [--per-gib <usd>] [--key-price <usd>] + [--rendition <id>:<codecs>:<w>x<h>:<kbps>]... [--relays any | <keys>] +ip live start <name> --input <rtmp:// | srt:// | file> [--record] [--max-downstream <n>] +ip live stop <name> +ip live watch <channel key | ip:// url> [--rendition <id>] [--out <file>] +ip live relay <channel key> [--max-downstream <n>] +ip live relay-hire <channel key> --hours <n> --relays <n> [--max-downstream <n>] [--max-price <usd>] +``` + +### 2.7 Catalogue + +``` +ip db put <key> <record.json> [--feed <name>] +ip db del <key> [--feed <name>] +ip db get <ip:// url | feed/key> +ip db query [--type <t>] [--feed <ref>]... [--where '<path> <op> <value>']... [--order <path>:asc|desc] [--limit <n>] +ip db follow <feed ref> +ip db unfollow <feed ref> +ip db head [<feed ref>] +ip db segment [--feed <name>] # seal a segment now +ip db index-hire <feed ref>... --days <n> [--max-price <usd>] +``` + +`--where` takes one clause per flag; the value is JSON (`'"Ada"'`, `120000`). + +### 2.8 Names + +``` +ip name set <name> [--gateway <url>] [--hub <url>]... [--mtp-pin <pin>] [--feed <ref>]... +ip name resolve <name> +ip name pin <name> # prints the registry pin value and the TXT record +``` + +### 2.9 Node + +``` +ip node status +ip node hello <peer address> # diagnostic: handshake and print the peer's hello +``` + +A c0mpute worker embeds the same library; `c0mpute worker start --openswarm` +is the daemon form and `ip` is the operator's tool. + +## 3. Output examples + +``` +$ ip file add ./interview.flac --per-gib 0.01 --key-price 0.50 +file ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d +manifest sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb +infohash v1 a3ce2180413415d7cf4268fb892b8ffd539e8459 + v2 4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342 +size 734003200 (700 pieces of 1048576) +magnet magnet:?xs=urn:btpk:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d&s=ipfile +url ip://ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d?file +seeding yes (2 trackers, dht) +``` + +``` +$ ip file get ed25519:0d87e0...295d --out ./interview.flac +pass reused sha256:39655d...02fd (cap 2.000000, spent 0.000000) +peers 3 (2 paid, 1 gateway) +progress 734003200 / 734003200 verified plainRoot ok +vouchers 3 signed, 0.006836 USD +key granted by ed25519:d2d05f...6c94 (keeper) +``` + +## 4. Environment + +| Variable | Meaning | +| --- | --- | +| `IP_HOME` | Overrides the config directory. | +| `IP_HUB` | Default hub URL. | +| `X402_PRIVATE_KEY` | EVM key used to buy passes, shared with `x402-client`. | +| `IP_TRACKERS` | Comma list of default trackers for `file add`. | +| `MOSHPIT_RESOLVE_MODE` | Passed through to name resolution (`ipname` §4.2). | + +## 5. Conformance + +A CLI claiming this contract implements every command in §2 with these +exit codes, prints the fields in §3 for `add` and `get`, and never writes a +seed or a content key to stdout unless asked with an explicit `--reveal`. diff --git a/docs/openswarm/conformance.md b/docs/openswarm/conformance.md new file mode 100644 index 0000000..089d2cc --- /dev/null +++ b/docs/openswarm/conformance.md @@ -0,0 +1,157 @@ +# OpenSwarm Conformance + +Version: **0.1** (draft) + +Conformance is claimed per **profile**. An implementation names the +profiles it meets; it does not have to meet all of them. Each profile is +a checklist over the member documents plus the fixture suite that will +ship with the reference implementation. + +## 1. Profiles + +| Profile | Who | Documents | +| --- | --- | --- | +| `core` | Anything that reads or writes records | spec.md | +| `publisher` | Creates manifests, tracks, titles, channels, feeds | spec, ipfile §3 and §8, ipdb §4, plus the media documents it emits | +| `seeder` | Serves `ipfile` pieces for payment | spec, ipfile §5 and §6, ippay §4.4 | +| `leecher` | Fetches and pays | spec, ipfile §5 and §10, ippay §3 and §4 | +| `keeper` | Grants keys | ipfile §5.5, ippay §5.4 | +| `gateway` | HTTP webseed, bridges, queries | ipfile §6.4, ippay §7, ipaudio §7, ipvideo §5, ipdb §6.3 | +| `hub` | Sells passes, redeems vouchers, pays out | ippay §3, §5, §6 | +| `origin` | Runs a live channel | iplive §3, §4, §6 | +| `relay` | Relays live segments | iplive §4, §5 | +| `viewer` | Watches live | iplive §7 | +| `replica` | Holds and queries feeds | ipdb §3, §5, §6 | +| `resolver` | Resolves names | ipname §4 | +| `cli` | The `ip` command | cli.md | + +## 2. Checklists + +### 2.1 `core` + +- [ ] C1. Canonicalises with RFC 8785 and computes record ids per spec §3.3. +- [ ] C2. Verifies `ed25519` signatures with the domain prefix of spec §3.4. +- [ ] C3. Ignores `mldsa65` entries it cannot verify without failing the record. +- [ ] C4. Preserves unknown fields on round trip. +- [ ] C5. Derives file, feed, channel and box keys per spec §4 so that the + fixture seed produces the fixture public keys. +- [ ] C6. Computes BEP 52 pieces roots and hybrid infohashes per spec §5 + matching the fixture torrent. +- [ ] C7. Reads and writes the BEP 44 pointer of spec §7.2. + +### 2.2 `publisher` + +- [ ] P1. Emits `ipfile.manifest` with both signatures, correct roots and a + `keys.owner` the fixture seed can open. +- [ ] P2. Revisions increase `rev`, keep `fileId`, and metadata-only + revisions keep `cipherRoot` and `keyId`. +- [ ] P3. Appends an `ipdb.entry` for every record it creates, with no + sequence gap. +- [ ] P4. Media records reference only files it also published, with + matching `size`. +- [ ] P5. Audio and video renditions are aligned (ipaudio §4.3, ipvideo §3.1). + +### 2.3 `seeder` + +- [ ] S1. Sends `hello` before any other `ipfile` message and refuses + messages from a peer that has not. +- [ ] S2. Keeps a peer choked until a valid pass arrives, or treats it as + vanilla per `policy.vanilla`. +- [ ] S3. Never exceeds the credit window unpaid; sends `credit` when it + chokes for that reason. +- [ ] S4. Validates vouchers per ippay §4.4 including recomputing `usd`. +- [ ] S5. Rejects a voucher not signed to its own identity. +- [ ] S6. Serves the plaintext piece layer on `layer_req`. +- [ ] S7. Redeems the latest voucher per triple at the hub. + +### 2.4 `leecher` + +- [ ] L1. Verifies the manifest against the file key it started from. +- [ ] L2. Buys or reuses a pass in scope before sending `pass`. +- [ ] L3. Signs vouchers only for verified bytes, on the interval, with + cumulative `bytes` and correct `usd`. +- [ ] L4. Verifies the piece layer against `plainRoot` before trusting an + entry; verifies every decrypted piece. +- [ ] L5. Checks `sha256(key) == keyId` on every grant. +- [ ] L6. Sends `bye done`. + +### 2.5 `keeper` + +- [ ] K1. Grants only against a valid pass with the file in `grants`, or a + delegating grant. +- [ ] K2. Seals to the requester's `box` from its `hello`. +- [ ] K3. Reports every grant to the hub. + +### 2.6 `gateway` + +- [ ] G1. Serves `/swarm/<infohashV2>/{manifest,layer,data}` with Range. +- [ ] G2. Applies the pass and voucher headers of ippay §7 and returns + `X-OpenSwarm-Credit` on every 206. +- [ ] G3. If a keeper: audio and video bridges decrypt only for a valid pass. +- [ ] G4. If a replica: answers `POST /db/query` with `asOf`. + +### 2.7 `hub` + +- [ ] H1. Publishes a signed hub record at the well-known path. +- [ ] H2. Sells passes over x402 with the CoinPay v2 offer, binds the payer + by `X-OpenSwarm-Payer`, and records nonce to resource. +- [ ] H3. Redeems vouchers with the arithmetic of ippay §4.2 and §5.3 + exactly; the fixture receipts match byte for byte. +- [ ] H4. Enforces the cap per pass across payees and returns `partial` and + `cap-exceeded` correctly. +- [ ] H5. Records grants once per pass and file. +- [ ] H6. Pays out on schedule and emits the events of ippay §8. + +### 2.8 `origin`, `relay`, `viewer` + +- [ ] O1. Signs a head per announced unit, chained by `prev`. +- [ ] O2. Rotates the content key per epoch and announces one segment early. +- [ ] O3. Writes a final head with `live: false`. +- [ ] R1. Verifies a segment root against a signed head before forwarding. +- [ ] R2. Honours `window` and never serves below `live - buffer`. +- [ ] R3. Answers `deny rate` with the current head when full. +- [ ] V1. Drops to the live edge rather than stalling. +- [ ] V2. Vouchers every peer that served it. +- [ ] V3. Requests a new grant on `keyId` change. + +### 2.9 `replica` + +- [ ] D1. Verifies chains and embedded signatures; refuses a `subject` + mismatch. +- [ ] D2. Detects forks and freezes state until resolved. +- [ ] D3. Replicates by segments and gossip. +- [ ] D4. Implements every query operator of ipdb §6.1. + +### 2.10 `resolver` + +- [ ] N1. Applies clearnet-first before any registry call. +- [ ] N2. Verifies the pin, the signature and the `name` match. +- [ ] N3. Reports the anchor kind. + +## 3. Fixture suite + +The reference implementation will publish, under `packages/openswarm/ +fixtures/`, a deterministic set built from one seed: + +| Fixture | Contents | +| --- | --- | +| `seed.json` | A 32-byte seed and every public key derived from it. | +| `file-small/` | A 3 MiB plaintext, its content key, ciphertext, piece layers, hybrid `.torrent`, and manifest. | +| `records/` | One valid record of every registered type, plus one invalid variant each (bad signature, bad chain, bad `usd`). | +| `session.jsonl` | A scripted `ipfile` session: handshake dicts, `hello`s, pass, three vouchers, grant, in order. | +| `receipts.json` | The receipts a hub must return for `session.jsonl`. | +| `queries.json` | Queries over `records/` and their exact results. | +| `names.json` | Registry responses and the expected resolutions, including a pin mismatch. | + +A profile is conformant when every checklist item is met and the relevant +fixtures pass. Until the reference implementation exists, the checklists +are the whole of conformance, and a claim is a claim. + +## 4. Interoperability + +As with OpenCreds, a round trip inside one implementation proves only that +the code agrees with itself. The interop test that matters is: a swarm +seeded by implementation A, paid for by a leecher from implementation B, +with vouchers redeemed at hub C, decrypts to `plainRoot`. The first three +implementations to exist should be tested that way before any of them +claims 1.0. diff --git a/docs/openswarm/faq.md b/docs/openswarm/faq.md new file mode 100644 index 0000000..ee23321 --- /dev/null +++ b/docs/openswarm/faq.md @@ -0,0 +1,89 @@ +# OpenSwarm FAQ + +**Is this IPFS?** +No. IPFS is content addressing plus a DHT plus its own wire protocol, with +no payment and no encryption. OpenSwarm is BitTorrent plus payment plus +encryption, carried as BEP 10 extension messages, so every existing +torrent client, tracker and crawler still understands the swarm. A file's +identifier is a BEP 52 pieces root rather than a CID because that is the +tree a BitTorrent v2 client already verifies. + +**Why not a blockchain, a token or a payment channel?** +Because a 16 KiB block at the reference price is worth 0.00000015 USD and +no chain settles that. Passes are bought on chain once (x402, USDC); +vouchers are off-chain IOUs redeemed at a hub; payouts are batched. The +hub is trusted like a payment processor is trusted, and the spec says so +rather than pretending otherwise. + +**Who runs a hub?** +Anyone. CoinPay is the proposed reference because it already does the x402 +verify and settle half. A manifest lists the hubs its publisher accepts, +and a seeder refuses passes from any other. Two hubs on a manifest is the +recommended posture. + +**Can a seeder get paid for bytes it did not send?** +No. A voucher is signed by the payer for bytes the payer already verified. + +**Can a leecher get bytes without paying?** +One credit window's worth, per identity, per session. Then it is choked. +That is the design: bounded loss instead of a trustless protocol nobody +would implement. + +**What happens to a vanilla client that finds the swarm?** +It sees a one-file torrent named by a hex string. If the publisher set +`policy.vanilla` to `ciphertext`, it can download the ciphertext for free +and cannot read it. If `deny`, it is choked. It never receives an +extension message. + +**Does this work in a browser?** +Yes. WebTorrent supports BEP 10 extensions, WebCrypto has Ed25519, X25519 +and AES-CTR, and MSE plays CMAF. bittorrented.com's player plus three +additions is the reference browser peer. A browser can only reach WebRTC +peers, so a swarm needs one hybrid seeder that announces to a wss tracker. + +**Why derive file keys from one publisher key?** +So there is one thing to back up. A publisher seed plus any manifest +recovers that file's private key and its content key. A standalone key is +available for a file that will change hands. + +**What if I lose the seed?** +Nothing becomes unreadable and nothing can be changed: prices, payout +address and trackers are frozen at the last signed manifest. Store the +seed in an OpenCreds vault. + +**Can a key be revoked after delivery?** +No. A delivered key is held by the peer. Re-encryption under a new key +protects future readers only. Live streams rotate keys every epoch for +exactly this reason. + +**Why AES-CTR and not an AEAD?** +Length-preserving and seekable: a block decrypts without the rest of the +piece. Integrity comes from two signed merkle trees, which is stronger than +a per-piece tag because it is bound to the publisher's signature. + +**How does a Moshpit name fit?** +A name's owner registers a pin of kind `openswarm` over its publisher key +and serves a signed record at `/.well-known/openswarm.json`. A resolver +checks the pin, then the signature. Ordinary domains use a TXT record. +Clearnet wins by default, as everywhere in Moshpit. + +**How does c0mpute fit?** +A node bids on pin, keep, gateway, transcode, relay and index jobs, and is +paid by vouchers for every byte it serves regardless of jobs. See +[c0mpute.md](./c0mpute.md). + +**Is the catalogue private?** +A feed is public by construction. A private catalogue is a feed segment +published as a paid `ipfile`, readable only by pass holders. + +**Why is the family called OpenSwarm and the protocols `ip*`?** +The family name follows the LogicSRC convention and names the shared +object. The protocol names are the ones the tools will be called by. The +working name was IPMedia; see the overview for why it changed. + +**What exists today?** +The x402 sale and CoinPay settlement, the browser player and hybrid seeder, +the DHT crawl, CloudEvents and Standard Webhooks, the MTP/1 transport, the +Moshpit registry, and c0mpute's store, transcode, gateway and live-stream +design. Everything that mentions a pass, a voucher, a grant, a manifest, a +feed or a head is new. diff --git a/docs/openswarm/ipaudio.md b/docs/openswarm/ipaudio.md new file mode 100644 index 0000000..aaefc24 --- /dev/null +++ b/docs/openswarm/ipaudio.md @@ -0,0 +1,313 @@ +# ipaudio: Audio on ipfile Swarms + +Version: **0.1** (draft) +Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)) and rides +on [ipfile](./ipfile.md), [ippay](./ippay.md) and [ipdb](./ipdb.md). +Nothing below is implemented. + +## 1. Scope + +`ipaudio` defines the records that make an `ipfile` playable as music or +speech: + +1. A **track**: renditions (each one an `ipfile`), a seek index, gapless + trim values, contributors and royalty split. +2. A **release**: an ordered set of tracks with artwork and release metadata. +3. **Streaming rules**: how a player maps time to pieces, buffers, switches + bitrate and plays two tracks without a gap. +4. A **gateway bridge** for players that only speak HTTP. + +Everything about bytes, keys, payment and discovery is `ipfile` and `ippay`. +This document adds no wire messages. + +### Non-goals + +A player. Loudness normalisation. A recommendation system. DRM beyond +`ipfile` key delivery. Radio (a continuous stream is `iplive`). + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **rendition** | One encoding of a track, stored as one `ipfile`. | +| **anchor** | A `[timeMs, byteOffset]` pair at which decoding can start. | +| **seek index** | The list of anchors for a rendition. | +| **gapless trim** | Encoder delay and end padding in samples, so tracks join without silence. | + +## 3. Data model + +### 3.1 Track + +```json +{ + "openswarm": "0.1", + "type": "ipaudio.track", + "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "title": "Interview with Ada", + "artist": "Ada", + "artists": [{ "name": "Ada", "role": "primary" }], + "durationMs": 2941200, + "isrc": null, + "language": "en", + "artwork": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "renditions": [ + { + "id": "opus-96", + "codec": "opus", + "container": "ogg", + "bitrateKbps": 96, + "sampleRate": 48000, + "channels": 2, + "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "size": 35294400, + "gapless": { "encoderDelay": 312, "endPadding": 1216 }, + "index": [[0, 0], [2000, 24012], [4000, 48160], [6000, 72301]] + }, + { + "id": "flac", + "codec": "flac", + "container": "flac", + "bitrateKbps": 0, + "sampleRate": 48000, + "channels": 2, + "file": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be", + "size": 734003200, + "gapless": { "encoderDelay": 0, "endPadding": 0 }, + "index": [[0, 8192], [2000, 507904], [4000, 1007616], [6000, 1503232]] + } + ], + "royalties": [ + { "label": "Ada", "network": "eip155:8453", "address": "0xCC3b072391AE7A8d10cF00DdC5F61DB2cA5541E5", "bps": 8000 }, + { "label": "Producer", "network": "eip155:8453", "address": "0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf", "bps": 2000 } + ], + "createdAt": "2026-09-05T18:05:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +(`index` is truncated to four anchors here. A real one has one anchor per +two seconds of audio.) + +| Field | Rule | +| --- | --- | +| `publisher` | Signs the record. MUST equal the `publisher` of every referenced `ipfile` manifest. | +| `durationMs` | Of the plaintext audio. | +| `artwork` | File key of an `ipfile` holding a JPEG or PNG. MAY be null. | +| `renditions[].codec` | One of `opus`, `aac`, `flac`, `mp3`. | +| `renditions[].container` | `ogg` for Opus, `mp4` for AAC (fragmented, CMAF), `flac`, `mp3`. | +| `renditions[].bitrateKbps` | Target bitrate; `0` for lossless. | +| `renditions[].file` | The `ipfile` manifest's file key. | +| `renditions[].size` | MUST equal that manifest's `size`. | +| `renditions[].gapless` | Samples to trim at the start and end, as the encoder reported. `0` for FLAC. | +| `renditions[].index` | Anchors, ascending by time, first at `[0, o]` where `o` is the byte offset of the first decodable unit after headers. | +| `royalties` | How the publisher's share of every voucher is split (§5). | + +A track MUST have at least one rendition. Every rendition of a track MUST +have anchors at the same set of `timeMs` values (§4.3). + +### 3.2 Release + +```json +{ + "openswarm": "0.1", + "type": "ipaudio.release", + "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "title": "Conversations, Volume 1", + "artist": "Ada", + "kind": "podcast", + "releasedAt": "2026-09-05", + "upc": null, + "artwork": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "tracks": [ + { + "position": 1, + "subject": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", + "feed": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default" + } + ], + "description": "Long-form interviews.", + "createdAt": "2026-09-05T18:06:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +`kind` is one of `album`, `ep`, `single`, `podcast`, `audiobook`, +`compilation`. `tracks[].subject` is the record id of the track's first +revision, which is also its `ipdb` key (`track:<id>`), so a track can be +revised without breaking the release. + +### 3.3 Where records live + +Both records are `ipdb` entries in the publisher's feed under +`track:<id>` and `release:<id>`. The renditions are `ipfile` manifests under +`file:<file key>` in the same feed. A client that finds a release has +everything it needs in one feed. + +## 4. Streaming + +### 4.1 Time to pieces + +To play from time `t` on rendition `r`: + +``` +anchor = the last index entry with timeMs <= t +offset = anchor.byteOffset +piece = floor(offset / manifest.pieceLength) +``` + +Request pieces from `piece` onward in order. Decrypt each as it completes +(`ipfile` §4.1), verify against the plaintext piece layer, and feed the +decoder from `offset`. The first decodable unit is guaranteed at the anchor +because the index was built from container boundaries (Ogg page, MP4 +fragment, MP3 frame, FLAC frame). + +### 4.2 Buffering + +A player SHOULD hold the pieces covering the next 30 seconds and request +the next piece when the buffer drops below 15 seconds. It SHOULD send +vouchers on the `ipfile` schedule regardless of playback state; a paused +player that has received bytes still owes for them. + +With 256 KiB pieces and a 96 kbps rendition a piece is about 21 seconds of +audio. That is why `ipaudio` recommends 256 KiB: one piece is one buffer +step. + +### 4.3 Adaptive bitrate + +Because every rendition shares the same anchor times, a switch is: finish +the current anchor interval on rendition A, start the next interval on +rendition B from its anchor at the same `timeMs`. The publisher guarantees +alignment by encoding every rendition with the same segment duration +(2000 ms recommended). A player picks the highest `bitrateKbps` whose piece +download rate over the last 10 seconds exceeds 1.5 times the bitrate. + +Switching renditions means joining a second swarm. The pass is per file, so +a pass scoped by `publishers` rather than `files` is what a player wants +(`ippay` §3.1). + +### 4.4 Gapless + +When track A ends and track B begins, the player trims `endPadding` samples +from A's decoded tail and `encoderDelay` samples from B's decoded head, and +concatenates the sample streams. It MUST have B's first piece decrypted +before A's last anchor interval starts, which means it holds a grant for B +already. A release player therefore requests the grant for track `n + 1` +when track `n` starts. + +## 5. Royalties + +An `ipfile` manifest names one `pricing.payTo` for the publisher's share. +`ipaudio` splits that share further with `royalties`: a list of recipients +and basis points summing to 10000. A hub applies the split when it can see +the track record for a file; the `ipfile` manifest's `meta.track` field +carries the track's record id so the hub can look it up in `ipdb`. Where +the hub cannot find the track, the whole publisher share goes to `payTo` +and a `com.logicsrc.openswarm.ippay.split.unresolved.v1` event is emitted. + +A contributor is paid per byte served of any rendition of the track. That +is the whole royalty model: no plays counted, no per-stream rate, no +minimum. It is a consequence of `ippay`, not a separate system. + +## 6. Publishing procedure + +`ip audio publish <track.json>` where the input names source files: + +1. For each rendition requested (`--renditions opus-96,flac`), transcode + with ffmpeg or submit an `ipaudio.transcode` job to c0mpute + (`c0mpute.md` §3). Record `encoderDelay` and `endPadding` from the + encoder. +2. Build the seek index by walking container boundaries at 2000 ms steps. +3. `ip file add` each rendition with `--piece-length 262144`, + `--meta.track <placeholder>`; collect file keys. +4. Write the track record, sign, `ip db put track:<id>`. Revise each + rendition manifest's `meta.track` to the track id (metadata-only revision). +5. Optionally write or update a release. + +## 7. Gateway bridge + +A gateway that is also a keeper (holds the content key under a delegating +grant) MAY serve decrypted audio to pass holders: + +``` +GET /audio/<track record id>/<rendition id> +Authorization: Bearer <base64url pass> +Range: bytes=0- +``` + +It returns `206` with the plaintext bytes, `Content-Type` from the +container, `Accept-Ranges: bytes`, and `X-OpenSwarm-Credit` per `ippay` §7. +This is what lets a podcast app, a car, or bittorrented.com's existing +`<AudioPlayer>` (which plays a plain URL) consume an `ipaudio` track. The +gateway is the payee for those bytes and is paid by voucher like any +seeder; the client is the pass holder's HTTP library, which sends vouchers +in `X-OpenSwarm-Voucher`. + +A gateway MAY also render an RSS 2.0 feed for a `podcast` release with +`<enclosure>` URLs of the form above, so a release is subscribable in any +podcast client that can send a bearer header, and previewable in any +client at all if the publisher sets `perGib` and `keyUsd` to zero. + +## 8. Recommended encodings + +| Rendition id | Codec | Container | Settings | +| --- | --- | --- | --- | +| `opus-64` | Opus | Ogg | 64 kbps VBR, 48 kHz, speech | +| `opus-96` | Opus | Ogg | 96 kbps VBR, 48 kHz | +| `opus-160` | Opus | Ogg | 160 kbps VBR, 48 kHz, music | +| `aac-128` | AAC-LC | fragmented MP4 | 128 kbps, for Safari and CarPlay | +| `flac` | FLAC | FLAC | Lossless, source sample rate | + +Ogg Opus and FLAC carry their own seek structure; the index is still +REQUIRED so a player never has to read the container to find a piece. + +## 9. Security and privacy + +Nothing beyond `ipfile`. Two notes: a rendition list reveals bitrates and +sizes to anyone who has the track record, and a seek index reveals the +container's page structure; neither reveals audio. A publisher who does +not want the catalogue public uses a paid `ipdb` segment (`ipdb` §9). + +## 10. Events + +| Type | Emitted by | +| --- | --- | +| `com.logicsrc.openswarm.ipaudio.track.published.v1` | publisher | +| `com.logicsrc.openswarm.ipaudio.release.published.v1` | publisher | + +Play counts are not events in this family. A client MAY report them to a +publisher's webhook as `com.logicsrc.openswarm.ipaudio.play.v1` with +`{ "track", "rendition", "ms" }`; nothing in the payment loop depends on it. + +## 11. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| Plain-URL audio player, range streaming, HLS fallback | Yes | `media-streamer` `<AudioPlayer>`, `/api/stream`, `/api/stream/hls` | The gateway bridge is what it would point at | +| Podcast catalogue and RSS | Yes | `p0dcasters.com`, `media-streamer` podcasts lib | Rendering RSS from a release record | +| Per-rendition transcode on the network | Yes (video) | `c0mpute` `ffmpeg.transcode` | An audio preset with gapless values in the result | +| Track and release records, seek index, gapless join, royalty split | No | | All of it | + +## 12. Conformance + +An `ipaudio` player maps time to pieces per §4.1, switches only at shared +anchors per §4.3, trims per §4.4, and holds a valid pass for every rendition +it fetches. A publisher emits records per §3 with aligned anchors. A gateway +implementing §7 applies `ippay` §7 headers. + +## 13. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/ipdb.md b/docs/openswarm/ipdb.md new file mode 100644 index 0000000..75292c7 --- /dev/null +++ b/docs/openswarm/ipdb.md @@ -0,0 +1,386 @@ +# ipdb: The Replicated Catalogue + +Version: **0.1** (draft) +Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)). Nothing +below is implemented. + +## 1. Scope + +`ipdb` is the catalogue: the place a client asks "what is this swarm", "what +has this publisher released", "what is in this playlist". It defines: + +1. A **feed**: a signed, hash-linked, append-only log owned by one key. +2. An **entry**: one log item that puts or deletes a record under a key. +3. **Heads**: how the latest state of a feed is announced on the DHT. +4. **Replication**: how feeds move between peers, as `ipfile` swarms and as + gossip on swarm connections. +5. **Queries**: a JSON query shape a local replica and a gateway both answer. + +It is a database of records, not of bytes. A manifest is in `ipdb`; the +file it describes is in an `ipfile` swarm. + +### Non-goals + +Multi-writer documents. A feed has one writer. Two people editing one +playlist is two feeds and a view over both, or a future CRDT record type. +Global search ranking. Consistency across feeds; each feed is consistent +with itself and that is all that is promised. + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **feed** | A log identified by `(feed key, name)`. | +| **feed key** | An Ed25519 key. A publisher's default feed uses the publisher key itself. | +| **entry** | A record of type `ipdb.entry`, sequenced and chained. | +| **key** | A string unique within a feed that later entries overwrite. | +| **head** | The latest entry of a feed. | +| **segment** | A batch of entries stored as one file for replication. | +| **replica** | A peer holding some feeds, verified, queryable. | +| **follow** | A replica's decision to keep a feed current. | + +## 3. Data model + +### 3.1 Feed keys + +The default feed is the publisher key with name `default`. Named feeds +derive from the publisher seed the same way files do: + +``` +feedSeed = HKDF-SHA256(ikm = publisher seed, salt = utf8(name), info = "openswarm:feed:v1") +feed key = Ed25519 keypair from feedSeed +``` + +A feed reference is written `ed25519:<hex>/<name>`; the default feed is +`ed25519:<hex>/default`. Names match `^[a-z0-9][a-z0-9-]{0,63}$`. + +### 3.2 Entry + +```json +{ + "openswarm": "0.1", + "type": "ipdb.entry", + "feed": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "name": "default", + "seq": 12, + "prev": "sha256:ba8cd1a97b865a05cdc5840c63b086dd463624a831874c1ba0d7c4ce6f806f7a", + "op": "put", + "key": "file:ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "kind": "ipfile.manifest", + "subject": "sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb", + "record": { + "openswarm": "0.1", + "type": "ipfile.manifest", + "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "rev": 1, + "createdAt": "2026-09-05T18:00:00.000Z", + "sigs": [] + }, + "createdAt": "2026-09-05T18:00:05.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +(The embedded manifest is abbreviated here; in a real entry it is the +complete record with its signatures.) + +| Field | Rule | +| --- | --- | +| `feed`, `name` | The feed. The entry MUST be signed by `feed`. | +| `seq` | 1 for the first entry, then `prev.seq + 1`. | +| `prev` | Record id of entry `seq - 1`, or `null` for `seq` 1. | +| `op` | `put` or `del`. | +| `key` | The logical key. Convention: `<noun>:<identifier>`, e.g. `file:ed25519:...`, `track:sha256:...`, `playlist:my-mix`. | +| `kind` | The embedded record's `type`. Absent for `del`. | +| `subject` | The embedded record's id. Absent for `del`. A replica MUST recompute it and refuse a mismatch. | +| `record` | The complete embedded record. Absent for `del`. Its own signatures MUST verify. | + +An entry is at most 256 KiB. A record larger than that is published as an +`ipfile` and the entry embeds an `ipfile.manifest` pointing at it, with +`kind` naming the payload type in `record.meta.kind`. + +### 3.3 State + +The state of a feed is the map `key -> latest put record` after applying +entries in `seq` order, with `del` removing the key. A replica exposes both +the log (for audit and replication) and the state (for queries). A record +that was overwritten stays in the log; that is what makes a manifest +revision history free. + +Across feeds there is no merge. A query over several feeds returns matches +from each; the `feed` of every result is returned with it. + +### 3.4 Head + +```json +{ + "openswarm": "0.1", + "type": "ipdb.head", + "feed": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "name": "default", + "seq": 12, + "head": "sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb", + "segments": [ + { "from": 1, "to": 10, "file": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94" } + ], + "createdAt": "2026-09-05T18:00:05.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +`segments` lists the `ipfile` swarms that hold sealed ranges of the log +(§5.1). Entries after the last segment are fetched by gossip (§5.2). + +## 4. Publishing + +`ip db put <key> <record.json>` appends an entry. `ip file add` appends one +automatically under `file:<file key>` in the default feed. `ip audio +publish`, `ip video publish` and `ip live create` do the same for their +records. A publisher never edits a feed by hand; every tool that creates a +record appends it. + +Rules for the writer: + +- `seq` MUST be strictly sequential. A gap is a fork. +- Two entries with the same `seq` from the same feed are a **fork**. A + replica that observes a fork MUST keep both, mark the feed forked from that + `seq`, and stop applying either branch to state until the feed publishes + an `ipdb.entry` with `op: "put"`, `key: "ipdb:resolve"`, and a record + naming the surviving branch head. Forks happen when a seed is restored on + two machines; they are detected, not prevented. +- A `del` of a key that was never put is valid and does nothing. + +## 5. Replication + +### 5.1 Segments as ipfile swarms + +Every 1000 entries, or daily, the writer seals entries `from..to` as one +file: newline-delimited canonical JSON, one entry per line. It publishes the +file as an `ipfile` with `pricing.perGib` and `keyUsd` at `0.000000` and +`keys.public` set to the content key, so the swarm is readable by anyone +while staying an ordinary `ipfile` for seeders, gateways and c0mpute pin +jobs. The head lists the segment. Readers verify each line's chain as they +apply it, so a segment served by a stranger is safe. + +### 5.2 Gossip on swarm connections + +Peers that advertise `ipdb` in the BEP 10 handshake exchange heads on any +swarm connection: + +| `t` | Payload | Meaning | +| --- | --- | --- | +| `heads` | `{ "f": [ { "k": <32 bytes>, "n": string, "s": int } ] }` | Feeds I follow and the seq I have. At most 64 per message. | +| `entries_req` | `{ "k": <32 bytes>, "n": string, "from": int, "to": int }` | Send me entries in the range. At most 100. | +| `entries` | `{ "r": [bytes] }` | Entries as JSON, in seq order. | +| `head` | `{ "r": bytes }` | An `ipdb.head` record, when the sender has a newer one. | + +A peer sends `heads` once after `hello` and again whenever a followed feed +advances. A peer that receives a `heads` with a higher `s` for a feed it +follows sends `entries_req`. Gossip fills the tail after the last segment +and keeps replicas close to real time without any peer being special. + +### 5.3 DHT announcement + +The head is a BEP 44 mutable item (core §7.3): + +``` +k = feed key +salt = "ipdb:" + name +seq = head seq +v = bencoded { "h": <head record id, 32 bytes>, "s": <seq> } +``` + +A client with only a feed reference does `get` on the DHT, obtains `h`, +then asks any peer or gateway for the head record by id, then segments and +tail. A gateway also serves `GET /db/feeds/<key>/<name>/head`. + +### 5.4 Following + +`ip db follow ed25519:<hex>/default` marks a feed followed: the replica +keeps its head current by DHT polling (every 10 minutes, or on `head` +gossip), fetches segments, and applies entries. A c0mpute node with role +`index` follows every feed it is paid to follow (`c0mpute.md` §3). + +## 6. Queries + +### 6.1 Shape + +```json +{ + "type": "ipaudio.track", + "feeds": ["ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default"], + "where": [ + ["record.artist", "==", "Ada"], + ["record.durationMs", ">=", 120000] + ], + "orderBy": [["record.createdAt", "desc"]], + "limit": 20, + "cursor": null +} +``` + +| Field | Rule | +| --- | --- | +| `type` | The embedded record type to match. REQUIRED. | +| `feeds` | Feed references to search. `null` means every followed feed. | +| `where` | Conjunction of `[path, op, value]`. Paths dot into the entry; `record.` reaches the embedded record. | +| `orderBy` | List of `[path, "asc"|"desc"]`. | +| `limit` | 1 to 200. | +| `cursor` | Opaque, from a previous result. | + +Operators: `==`, `!=`, `<`, `<=`, `>`, `>=`, `in`, `contains` (array +membership or substring), `prefix`. No joins, no aggregation in 0.1; a +rating average is computed by the caller from the entries it gets back. + +### 6.2 Result + +```json +{ + "results": [ + { + "feed": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default", + "key": "track:sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", + "seq": 14, + "subject": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", + "record": { "openswarm": "0.1", "type": "ipaudio.track", "sigs": [] } + } + ], + "cursor": "eyJzZXEiOjE0fQ", + "asOf": { "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default": 27 } +} +``` + +`asOf` reports the head seq of each feed the answer was computed from, so a +caller can tell a stale replica from an empty one. + +### 6.3 Over HTTP + +`POST /db/query` on a gateway with the query as the body. A gateway MAY +require a pass (`Authorization: Bearer`) and MAY sell one over x402 for +queries the way a hub sells swarm passes; the reference price is +`0.000100` USD per query. A public gateway with no charge is the common case +and is what bittorrented.com's `/dht` browse would become for `ipfile` +swarms it can resolve. + +## 7. Record kinds carried + +`ipdb` carries any record type. These are the ones the family defines and +the keys they use: + +| Kind | Key convention | Defined in | +| --- | --- | --- | +| `ipfile.manifest` | `file:<file key>` | ipfile | +| `ipaudio.release` | `release:<record id of first revision>` | ipaudio | +| `ipaudio.track` | `track:<record id of first revision>` | ipaudio | +| `ipvideo.title` | `title:<record id of first revision>` | ipvideo | +| `iplive.channel` | `channel:<channel key>` | iplive | +| `ipname.pin` | `name:<moshpit name>` | ipname | +| `ipdb.playlist` | `playlist:<slug>` | this document §7.1 | +| `ipdb.rating` | `rating:<subject record id>` | this document §7.2 | + +### 7.1 Playlist + +```json +{ + "openswarm": "0.1", + "type": "ipdb.playlist", + "slug": "late-night", + "title": "Late night", + "items": [ + { "kind": "ipaudio.track", "subject": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", "feed": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default" } + ], + "createdAt": "2026-09-05T19:00:00.000Z", + "sigs": [] +} +``` + +A playlist item references a record by id and the feed it was found in, so +a reader can fetch it even if the playlist author's replica is gone. The +media-streamer `collections` table maps onto this one to one. + +### 7.2 Rating + +```json +{ + "openswarm": "0.1", + "type": "ipdb.rating", + "subject": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", + "value": 4, + "createdAt": "2026-09-05T19:05:00.000Z", + "sigs": [] +} +``` + +`value` is 1 to 5. One rating per subject per feed, by the key convention. +An aggregate is `query type ipdb.rating where record.subject == ...` over +the feeds a caller trusts; a gateway MAY cache the count and mean. + +## 8. Local storage + +A replica stores entries in a local database. The reference layout is +SQLite with tables `feeds(key, name, seq, head, forked_at)`, +`entries(feed, name, seq, id, prev, op, key, kind, subject, json)`, +`state(feed, name, key, seq, kind, subject)`, and one FTS table over +`record.title`, `record.artist`, `record.name` for the `contains` +operator. This is the layout the `ip` CLI would use; a gateway on Postgres +does the same with jsonb. Nothing in the wire depends on it. + +## 9. Security notes + +- **Authenticity.** Every entry is signed by the feed key and chained. A + replica cannot insert, reorder or drop an entry without breaking the chain + to the announced head. +- **Freshness.** The DHT head carries `seq`; BEP 44 refuses a lower `seq`. + A replica that shows an old head is stale, not lying, and `asOf` says so. +- **Embedded records** verify on their own signatures. A feed cannot put + someone else's manifest under its own key and have it trusted as theirs; + the manifest's `publisher` signature is the authority, the feed entry is + the announcement. +- **Spam.** A gateway that follows any feed anyone asks for will be filled. + Following is a decision; `c0mpute` index jobs are paid per followed feed + for that reason. +- **Privacy.** A feed is public by construction. A private catalogue is a + segment published as a paid `ipfile` (no `keys.public`), and the entries + are then readable only by pass holders. The head still reveals the seq. + +## 10. Events + +| Type | Emitted by | +| --- | --- | +| `com.logicsrc.openswarm.ipdb.entry.appended.v1` | writer | +| `com.logicsrc.openswarm.ipdb.head.published.v1` | writer | +| `com.logicsrc.openswarm.ipdb.fork.detected.v1` | replica | + +## 11. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| Infohash catalogue with browse and search | Yes (central) | `media-streamer` `dht_torrents`, `/api/dht/browse`, `services/dht-search-api` | Learning what an infohash is from a signed feed instead of BEP 9 names | +| Collections, playlists, favourites | Yes (central) | `media-streamer` `collections`, `collection_items` | The `ipdb.playlist` record as the portable form | +| Signed append-only log | Partly | Moshpit `moshpit_tld_log` (server side, `seq` cursor) | The feed as a client-verifiable chain with DHT heads | +| Entry, head, segments, gossip, query shape | No | | All of it | + +## 12. Conformance + +A replica verifies chains and embedded signatures per §3, detects forks +per §4, replicates by segments and gossip per §5, and answers every +operator in §6.1. A writer never emits a gap. A gateway serves `POST +/db/query` and the head route in §5.3. + +## 13. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/ipfile.md b/docs/openswarm/ipfile.md new file mode 100644 index 0000000..f4c3ac8 --- /dev/null +++ b/docs/openswarm/ipfile.md @@ -0,0 +1,545 @@ +# ipfile: Paid, Encrypted File Swarms + +Version: **0.1** (draft) +Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)). Nothing +below is implemented. + +## 1. Scope + +`ipfile` is BitTorrent with three additions carried as extension messages: + +1. **Encryption.** The swarm carries ciphertext. The content key is sold + separately and delivered only to a peer that has paid. +2. **Payment.** A seeder serves pieces inside a bounded credit window and a + leecher signs a cumulative voucher for every verified piece batch. Whoever + seeds gets paid. +3. **Ownership.** Every file has a key pair. The manifest that names the + swarm, the price and the payout address is signed by it. + +It is not a new transport. The wire is BEP 3, the extension is BEP 10, the +hashes are BEP 52, discovery is BEP 5 and BEP 44, browsers are WebTorrent. A +client that knows none of this can still join the swarm and, if the manifest +allows, download ciphertext it cannot read. + +### Non-goals + +Plaintext swarms (use a torrent). Hiding the size of a file. Hiding that a +swarm exists (use `swarm.private` to keep it off the DHT, but a tracker still +knows). Revoking a delivered key without re-encrypting. Streaming media +semantics (those are `ipaudio`, `ipvideo`, `iplive`). + +## 2. Terminology + +Core terms apply. In addition: + +| Term | Meaning | +| --- | --- | +| **manifest** | The `ipfile.manifest` record: identity, roots, swarm, price, key delivery, policy. | +| **revision** | A manifest with a higher `rev` under the same file key. | +| **keeper** | A peer holding the content key with a `delegate: true` grant, allowed to grant it onward. | +| **credit window** | Bytes a seeder will serve beyond the last voucher before choking. | +| **voucher interval** | Bytes after which a leecher owes a new voucher. | +| **vanilla peer** | A peer that did not advertise `ipfile` in its extension handshake. | + +## 3. Data model + +### 3.1 The manifest + +```json +{ + "openswarm": "0.1", + "type": "ipfile.manifest", + "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "box": "x25519:4c4055604fe6bd8781cab6835093ad896f68f676b472d5bd6811e52f28cb18c3", + "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "fileId": "165aa20f2088f71a1880fe42c48d211b", + "keyDerivation": "derived", + "rev": 1, + "name": "interview-2026-09-05.flac", + "size": 734003200, + "contentType": "audio/flac", + "pieceLength": 1048576, + "plainRoot": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", + "cipherRoot": "sha256:ba8cd1a97b865a05cdc5840c63b086dd463624a831874c1ba0d7c4ce6f806f7a", + "cipher": { + "alg": "aes-256-ctr", + "keyId": "sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb" + }, + "swarm": { + "infohashV1": "sha1:a3ce2180413415d7cf4268fb892b8ffd539e8459", + "infohashV2": "sha256:4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342", + "private": false, + "trackers": [ + "wss://tracker.openwebtorrent.com", + "wss://tracker.webtorrent.dev", + "udp://tracker.opentrackr.org:1337/announce" + ], + "webseeds": [ + "https://gw.c0mpute.com/swarm/4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342/data" + ] + }, + "visibility": "public", + "policy": { + "vanilla": "ciphertext", + "creditBytes": 4194304, + "voucherBytes": 1048576 + }, + "pricing": { + "currency": "USD", + "perGib": "0.010000", + "keyUsd": "0.500000", + "split": { "publisherBps": 7000, "seederBps": 2500, "hubBps": 500 }, + "keeperBps": 1000, + "hubs": ["https://coinpayportal.com/api/openswarm"], + "payTo": { "network": "eip155:8453", "address": "0xCC3b072391AE7A8d10cF00DdC5F61DB2cA5541E5" } + }, + "keys": { + "modes": ["peer", "url"], + "url": "https://keys.example.com/openswarm", + "keepers": ["ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94"], + "owner": "HjYr1barG9BmJK6DEBSm9Rl8iGeYTZFGwfyG8AJy5g3Wk8uP9nyG1IeDaHuOT/VgYoNMmGvlCnaWw4olY6xM/0GKdzr05DqWt49benOyvAs=" + }, + "replication": { "min": 3, "erasure": { "k": 10, "parity": 4 } }, + "meta": { "duration": 2941.2, "language": "en" }, + "createdAt": "2026-09-05T18:00:00.000Z", + "updatedAt": "2026-09-05T18:00:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + }, + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `file` | The file public key. REQUIRED. The record's first signature MUST be by this key. | +| `box` | The file box public key (core §4.2). REQUIRED. | +| `publisher` | The publisher identity. REQUIRED. A second signature by this key is REQUIRED. | +| `fileId` | 16 bytes hex. REQUIRED. Stable across revisions. | +| `keyDerivation` | `derived` (from the publisher seed, core §4.2) or `standalone`. | +| `rev` | Integer, starts at 1, strictly increasing per file key. | +| `predecessor`, `successor` | File keys, for rotation (core §4.5). OPTIONAL. | +| `name` | Display name. Visible to anyone who obtains the manifest. MAY be empty. | +| `size` | Plaintext byte length. Ciphertext has the same length. | +| `contentType` | Media type of the plaintext. | +| `pieceLength` | Power of two, at least 16384. | +| `plainRoot`, `cipherRoot` | BEP 52 pieces roots (core §5.1). | +| `cipher.alg` | `aes-256-ctr` in this version. | +| `cipher.keyId` | `sha256:` of the content key. Identifies which key a grant carries without revealing it. | +| `swarm.infohashV1`, `swarm.infohashV2` | Of the hybrid torrent whose one file is the ciphertext (core §5.3). | +| `swarm.private` | BEP 27 flag. | +| `swarm.trackers`, `swarm.webseeds` | Announce URLs and BEP 19 URLs. | +| `visibility` | `public`: any peer gets the manifest on request. `pass`: only a peer presenting a valid pass in scope. | +| `policy.vanilla` | `ciphertext`: serve pieces to vanilla peers free. `deny`: choke them. | +| `policy.creditBytes` | Credit window. Default 4194304. | +| `policy.voucherBytes` | Voucher interval. Default 1048576. MUST be at most `creditBytes / 2`. | +| `pricing` | See §7. `keeperBps` is taken from `keyUsd`, the `split` is applied to per-byte vouchers. | +| `keys.modes` | Any of `peer` (a key holder in the swarm grants on request), `url` (an HTTPS key service). | +| `keys.keepers` | Identities delegated to grant. Each MUST hold an `ipfile.grant` with `delegate: true` signed by the file key. | +| `keys.owner` | The content key sealed to `box`. Lets the owner recover the key from seed plus manifest. | +| `keys.public` | OPTIONAL. The content key itself, hex. Set only when the publisher wants anyone to decrypt (a free release, an `ipdb` segment). With it present, `keyUsd` MUST be `0.000000` and no grant is ever needed. | +| `replication` | Publisher's request: at least `min` independent seeders. `erasure` is advice to storage nodes, never a wire format (§9). | +| `meta` | Free-form. Member protocols define what they put here: `ipaudio` sets `meta.track`, `ipvideo` sets `meta.title`, `ipdb` sets `meta.kind` on segment files. | + +`bps` values are basis points; the three in `split` MUST sum to 10000. + +### 3.2 The grant + +A grant carries the content key to one recipient: + +```json +{ + "openswarm": "0.1", + "type": "ipfile.grant", + "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "keyId": "sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb", + "grantor": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "grantee": "x25519:de3c5a1791131231f797726a59b46b5b877628f0d10ae97fb5f119faa1ad1dc5", + "pass": "sha256:39655de63a29b02c90b236fa870fe6f8dfad1f52a8119fe4df85e88cd6ab02fd", + "delegate": false, + "expiresAt": null, + "sealed": "KrGWWxD4blHwoh7UmJedgv/CdxWz2J9cbRZpMY+09mm6FT7exyU7KjEAXmWRTLh2Xr8INCGKhv6dpjScKFGCkCrPZDyfAHE2s4E5/VXc2zQ=", + "createdAt": "2026-09-05T18:01:30.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `grantor` | The file key, or a keeper listed in the manifest. A grant by anyone else is invalid. | +| `grantee` | The recipient's box public key. | +| `pass` | Record id of the pass that paid `keyUsd`. `null` only when `delegate` is true. | +| `delegate` | True makes the grantee a keeper. Only the file key MAY sign a delegating grant. | +| `expiresAt` | Keeper delegation SHOULD expire. A paid grant is perpetual: the peer holds the key. | +| `sealed` | `crypto_box_seal(contentKey, grantee)`, base64. 80 bytes for a 32-byte key. | + +A grantee decrypts `sealed` with its box private key and checks +`sha256(key) == keyId` before using it. + +## 4. Encryption + +### 4.1 Piece encryption + +The plaintext is split into pieces of `pieceLength`. Piece `i` is encrypted +independently under AES-256-CTR: + +``` +iv(i) = HKDF-SHA256(ikm = contentKey, salt = "", info = "openswarm:ipfile:iv:v1:" + decimal(i), L = 12) || 0x00000000 +cipher(i) = AES-256-CTR(contentKey, iv(i), plain(i)) +``` + +The 32-bit counter is big-endian and starts at zero, which is the WebCrypto +`AES-CTR` shape with `length: 32`. A piece is at most 2^32 blocks of 16 +bytes, so the counter never wraps below 64 GiB per piece. + +CTR is chosen over an AEAD because it is length-preserving and seekable: a +16 KiB block at offset `o` in piece `i` decrypts from counter `o / 16` +without touching the rest of the piece. Integrity does not come from the +cipher. It comes from two merkle trees: + +- The swarm verifies every block of ciphertext against `cipherRoot` through + the torrent's own v2 piece layers. A tampered block is rejected by any + BitTorrent v2 client before this specification is involved. +- The decrypting client verifies every plaintext piece against the plaintext + piece layer (§5.6), whose root is `plainRoot` in the signed manifest. + +A malleable cipher whose output is pinned by a signed hash tree is not +malleable in practice: any bit flip fails one of the two checks. + +### 4.2 The content key + +32 random bytes per file version. A new `cipherRoot` MUST use a new content +key; reusing a key across two plaintexts under the same IV derivation would +leak their XOR. The key never leaves a peer except inside a grant's sealed +box, over a key service response, or in `keys.owner`. + +## 5. Wire protocol + +### 5.1 Extension handshake and `hello` + +After the BitTorrent handshake, both peers send the BEP 10 extension +handshake. A peer that supports this specification includes `ipfile` in `m` +and `"openswarm": "0.1"`. Immediately after, it sends `hello`: + +``` +{ "t": "hello", + "v": 1, + "id": <32 bytes, Ed25519 public key>, + "box": <32 bytes, X25519 public key>, + "nonce": <16 bytes>, + "roles": ["seed"], + "hubs": ["https://coinpayportal.com/api/openswarm"], + "credit": 4194304, + "interval": 1048576, + "mtp": <optional 32 bytes, SHA-256 of the peer's MTP/1 SPKI>, + "sig": <64 bytes> } +``` + +`sig` is Ed25519 over: + +``` +"openswarm:hello:v1" || infohashV2 (32 bytes) || own peer_id (20 bytes) || remote peer_id (20 bytes) || nonce +``` + +Binding both `peer_id`s and the infohash ties the identity to this +connection in this swarm; a captured `hello` replays nowhere. `roles` is any +of `seed`, `leech`, `keeper`, `gateway`. `credit` and `interval` are the +window and the voucher interval this seeder will apply, which MUST be at +least the manifest's `policy` values. + +A peer MUST NOT send any other `ipfile` message before it has sent `hello` +and received a valid one. + +### 5.2 Messages + +All payloads are bencoded dictionaries. `r` carries a record as UTF-8 JSON. + +| `t` | Direction | Payload | Meaning | +| --- | --- | --- | --- | +| `hello` | both | §5.1 | Identity and terms. | +| `manifest_req` | any | `{ "rev": int or absent }` | Ask for the manifest, optionally a specific revision. | +| `manifest` | any | `{ "r": bytes }` | The manifest record. Sent unsolicited by a seeder after `hello` when `visibility` is `public`. | +| `layer_req` | leecher | `{}` | Ask for the plaintext piece layer. | +| `layer` | seeder | `{ "d": bytes }` | 32 bytes per piece, concatenated (§5.6). | +| `pass` | leecher | `{ "r": bytes }` | Present an `ippay.pass`. | +| `credit` | seeder | `{ "unpaid": int, "limit": int, "owed": string }` | Bytes served since the last accepted voucher, the window, and the amount owed. | +| `voucher` | leecher | `{ "r": bytes }` | An `ippay.voucher`. | +| `key_req` | leecher | `{ "keyId": bytes }` | Ask a key holder for a grant. | +| `grant` | key holder | `{ "r": bytes }` | An `ipfile.grant` sealed to the requester's `box`. | +| `deny` | any | `{ "what": string, "reason": string }` | Refuse `pass`, `key_req` or `manifest_req` with a registered reason. | +| `bye` | any | `{ "reason": string }` | About to disconnect. | + +Registered `deny` and `bye` reasons: `no-pass`, `pass-invalid`, +`pass-expired`, `pass-scope`, `pass-cap`, `unpaid`, `voucher-invalid`, +`voucher-stale`, `not-keeper`, `no-key`, `visibility`, `rate`, `done`. + +### 5.3 The paid session + +``` +leecher seeder + | BT handshake, BEP 10 handshake | + |---------------------------------------->| + |<----------------------------------------| + | hello | + |---------------------------------------->| + |<------------------------------- hello | + |<---------------------------- manifest | (public visibility) + | pass | + |---------------------------------------->| seeder verifies hub signature, + |<----------------------------- unchoke | scope, expiry, cap + | request / piece ... (vanilla) | + |<=======================================>| up to `credit` bytes unpaid + |<------------------------------ credit | unpaid >= interval: seeder asks + | voucher (cumulative) | + |---------------------------------------->| seeder verifies, unpaid resets + | request / piece ... | + |<=======================================>| + | key_req | + |---------------------------------------->| if seeder is a key holder + |<------------------------------- grant | + | bye done | + |---------------------------------------->| +``` + +Rules: + +1. A seeder MUST keep a leecher choked until a valid `pass` arrives, unless + `policy.vanilla` is `ciphertext`, in which case an unpaid peer is treated + as vanilla (§6) and gets no priority. +2. A seeder MUST NOT serve more than `credit` bytes beyond the bytes covered + by the last accepted voucher. When the window is full it chokes and sends + `credit`. It MAY disconnect with `bye unpaid` after 60 seconds choked. +3. A leecher MUST send a voucher when unpaid bytes reach `interval`, and + only for bytes whose blocks it has verified against the v2 piece layer. A + voucher is cumulative for this (pass, payee, swarm) triple (`ippay` §4). +4. A seeder MUST verify the voucher's signature, its monotonic `seq`, that + `bytes` is at least the previous voucher's `bytes`, and that the increase + does not exceed what it actually served. A voucher that claims more than + served is accepted for what was served and flagged; one that claims less + is `voucher-stale` and rejected. +5. A leecher MUST NOT sign a voucher for bytes it has not verified. It MAY + sign for fewer than it has received; the seeder then chokes sooner. That + is the only lever a leecher has against a slow seeder, and it costs + nothing but throughput. +6. Vouchers cover ciphertext bytes. Payment does not depend on the leecher + ever obtaining the key. + +### 5.4 Who may pay and who is paid + +The voucher's `payer` is the pass's `payer`. The voucher's `payee` is the +seeder's `hello.id`. A seeder MUST reject a voucher whose `payee` is not its +own identity. The hub pays whoever redeems a voucher signed to them, so an +identity is a payout account. A seeder that wants to be paid registers its +identity and payout address at the hub once (`ippay` §6). + +### 5.5 Key delivery + +**Peer mode.** Any peer whose `roles` include `keeper`, or that is the file +key itself, answers `key_req`. It MUST verify that the requester's pass is +valid, in scope for this file, and has `grant: true` (meaning `keyUsd` was +paid). It then signs and sends a grant sealed to the requester's `box`. A +keeper MUST report the grant to the hub (`ippay` §5.4) to earn `keeperBps` +and to let the hub mark the pass's grant as consumed; a pass buys one grant +per file. + +**URL mode.** `GET <keys.url>/grant?file=<file key hex>&box=<x25519 hex>` +with `Authorization: Bearer <pass token>` returns the grant record as JSON. +Without a pass it returns 402 with an x402 offer for `keyUsd`, so a client +that has no hub relationship can still buy a key directly (`ippay` §3.4). + +**Owner recovery.** The owner decrypts `keys.owner` with the file box private +key derived from the seed. + +### 5.6 The plaintext piece layer + +The plaintext v2 piece layer is the sequence of per-piece SHA-256 merkle +roots (32 bytes each) whose root is `plainRoot`. A seeder holds it because it +had the plaintext or received it with the swarm. A leecher fetches it with +`layer_req` (or from a gateway at `/swarm/<infohashV2>/layer`) and MUST +verify that it hashes to `plainRoot` before trusting a single entry. After +decrypting a piece the leecher checks it against the corresponding entry. +For a 1 GiB file with 1 MiB pieces the layer is 32 KiB. + +## 6. Coexistence with vanilla BitTorrent + +A vanilla client sees a hybrid torrent with one file named by a hex string. +It can download it if seeders allow. Specifically: + +- The extension handshake without `ipfile` marks the peer as vanilla. +- With `policy.vanilla: "ciphertext"`, a seeder treats it as any BitTorrent + peer: normal choking algorithm, no credit accounting, no payment. It gets + bytes it cannot decrypt. This is a free choice a publisher makes to widen + the swarm; vanilla peers that then seed are unpaid redundancy. +- With `policy.vanilla: "deny"`, a seeder keeps it choked and MAY disconnect. + It still answers `ut_metadata` so the peer can learn it has nothing to gain. +- A vanilla peer never receives an `ipfile` message, because BEP 10 forbids + sending an extension a peer did not advertise. +- The DHT, trackers and PEX carry infohashes only. A public tracker learns a + swarm exists and how many peers it has, exactly as today. +- WebTorrent peers are vanilla until the page loads the `ipfile` extension. + bittorrented.com's player would add it with `wire.use()`. + +Encryption of the wire itself (the RC4 "message stream encryption" some +clients offer) is neither required nor relied on. Confidentiality of content +comes from piece encryption; confidentiality of the link, where wanted, comes +from the MTP/1 tunnel (core §6.5). + +## 7. Pricing and settlement hook + +The manifest's `pricing` is the whole price list: + +| Field | Meaning | +| --- | --- | +| `perGib` | USD per GiB of ciphertext served. Vouchers are computed from it (`ippay` §4.2). | +| `keyUsd` | USD for one grant. `0.000000` makes the key free to any pass holder. | +| `split` | Where each voucher's value goes: publisher, the seeder who served, the hub. | +| `keeperBps` | Share of `keyUsd` paid to a keeper that grants. The rest of `keyUsd` goes to the publisher. | +| `hubs` | Hubs the publisher accepts passes from. A seeder MUST reject a pass from a hub not listed. | +| `payTo` | Where the publisher's share is paid. CAIP-2 network and an address. | +| `royalties` | OPTIONAL. A list of `{ "label", "network", "address", "bps" }` summing to 10000 that splits the publisher's share further. `ipaudio` and `ipvideo` records carry the same list; when both exist the manifest's wins, because the manifest is what the file key signed. | + +Passes are bought over x402 exactly as `@profullstack/x402-gateway` sells +crawl passes today: a 402 whose body is CoinPay's x402 v2 offer, a proof in +`X-PAYMENT`, verify and settle through CoinPay. `ippay` §3 has the exchange. +A publisher who wants no payment at all sets `perGib` and `keyUsd` to +`0.000000`, and seeders serve for free with vouchers that carry zero; the +accounting still runs so the publisher sees who served what. + +## 8. Publishing procedure + +`ip file add <path>` does, in order: + +1. Load the publisher seed. Generate `fileId`. Derive the file key and file + box (core §4.2), or generate a standalone seed with `--standalone-key`. +2. Generate the content key. Compute `plainRoot` and the plaintext piece + layer. Encrypt piece by piece; compute `cipherRoot`. +3. Build the hybrid torrent: one file named by `cipherRoot` hex, `piece + length` as chosen, `private` per flag, `announce-list` from + configuration, `url-list` for any gateway. Compute both infohashes. +4. Seal the content key to the file box (`keys.owner`). +5. Write the manifest with `rev: 1`. Sign with the file key, then the + publisher key. +6. Publish the BEP 44 pointer under the file key (core §7.2). +7. Append an `ipdb.entry` for the manifest to the publisher's default feed + (`ipdb` §4). +8. Start seeding. Announce to trackers. If any keeper is configured, send it + a delegating grant. + +A revision repeats steps 2 to 7 with `rev + 1` and the same `fileId`. A +metadata-only revision (price, trackers, keepers) skips step 2 and keeps +`cipherRoot`, `keyId` and the swarm. + +## 9. Storage and serving on a c0mpute node + +A c0mpute storage node stores whatever it likes internally: today that is +blake3-addressed chunks with Reed-Solomon 10 data + 4 parity shards spread +over hosts. On the wire it MUST present whole ciphertext pieces verified +against `cipherRoot`. Erasure coding is a storage policy. `replication. +erasure` in the manifest is the publisher saying what durability it is +paying for in a pin job; it never changes what a peer sends. + +A node that holds only shards reconstructs the piece before serving it. A +node advertises `c0mpute:role:storage` and bids on `ipfile.pin` jobs +(`c0mpute.md` §3). Pin income and voucher income are separate: the pin job +pays for holding the bytes for a period, vouchers pay for serving them. + +## 10. Retrieval procedure + +`ip file get <file key | infohashV2 | ip:// url>`: + +1. Resolve to a manifest: BEP 44 `get` under the file key, or `manifest_req` + from a peer, or a gateway. +2. Verify both signatures. Verify `sha256` of the canonical bytes matches + any record id the caller supplied. +3. Choose a hub from `pricing.hubs` the client has an account with. Buy or + reuse a pass in scope (`ippay` §3). Include `grant: true` if the key is + needed. +4. Join the swarm. For each peer: exchange `hello`, send `pass`, download + with sequential priority for streaming or rarest-first for bulk, sign + vouchers on schedule. +5. Fetch and verify the plaintext piece layer. Obtain a grant from a key + holder or the key URL. Decrypt pieces as they complete; verify each + against the layer. +6. On completion, verify the whole against `plainRoot`. Send `bye done`. +7. Keep seeding if configured. The client is now a seeder and earns. + +## 11. Security notes specific to ipfile + +- **Freeloading leecher.** Bounded by `creditBytes`. A seeder loses at most + one window per leecher identity per session and MAY refuse an identity it + has seen freeload (`ippay` §5.5 reputation). +- **Freeloading seeder.** Cannot happen: a voucher is only for bytes already + received and verified. A seeder that stops after being paid was paid for + what it sent. +- **Voucher inflation.** A seeder cannot forge a voucher; it lacks the + payer's key. A payer that over-states `bytes` pays more than it must; that + is its own loss. +- **Double spend across seeders.** A pass has a `capUsd`. A payer could sign + vouchers to many seeders totalling more than the cap. The hub honours + redemptions in order until the cap is reached and reports the rest as + `cap-exceeded`; each seeder's exposure is still one window. A payer that + does this is reported and its passes are refused (`ippay` §5.5). +- **Content key exposure.** A paying peer holds the key. Any peer can leak + it. This family prices access, it does not prevent copying. Rotation is the + remedy for a leaked key going forward, and only forward. +- **Metadata.** Size, piece length and both infohashes are public. `name` + and `meta` are visible to anyone who obtains the manifest; `visibility: + "pass"` restricts that to pass holders. +- **Manifest substitution.** The BEP 44 pointer is signed by the file key + and carries `seq`; a stale pointer is refused by the DHT itself. A + manifest from a peer is verified against the file key the client started + from. + +## 12. Events + +| Type | Emitted by | `data` | +| --- | --- | --- | +| `com.logicsrc.openswarm.ipfile.manifest.published.v1` | publisher | `record` | +| `com.logicsrc.openswarm.ipfile.manifest.revised.v1` | publisher | `record` | +| `com.logicsrc.openswarm.ipfile.grant.issued.v1` | key holder | `record` (with `sealed` removed) | +| `com.logicsrc.openswarm.ipfile.session.closed.v1` | seeder | `{ "swarm", "peer", "bytes", "vouchers", "reason" }` | + +## 13. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| Hybrid seeder announcing to wss trackers, browser WebTorrent player, range streaming | Yes | `media-streamer` (`src/lib/streaming/streaming.ts`, `src/hooks/use-webtorrent.ts`) | The `ipfile` extension in `wire.use()`, piece decryption in the player | +| DHT observation of infohashes | Yes | `media-streamer/services/dht-search-api` (bitmagnet), `dht-infohash-crawler` | Nothing; an `ipfile` swarm is an opaque infohash to a crawler | +| x402 pass sale and verification | Yes | `x402-gateway`, CoinPay `/api/x402/verify` and `/settle` | Pass scope and vouchers (`ippay`) | +| Content-addressed chunk store, erasure coding, gateway by hash | Yes | `c0mpute` (`c0mpute-store`, `c0mpute-gateway`) | Serving pieces by index verified by SHA-256, the pin workload | +| Sealed box key wrapping | Yes | `logicsrc credentials` team vaults, OpenCreds `team` profile | The grant record | +| Manifest, file key derivation, piece encryption, `hello`, credit window, voucher exchange | No | | All of it | + +## 14. Conformance + +An `ipfile` implementation: + +1. Produces manifests per §3.1 with both signatures and a correct BEP 44 + pointer. +2. Encrypts and decrypts per §4 and verifies against both roots. +3. Speaks every message in §5.2, refuses out-of-order messages per §5.1, and + enforces §5.3 rules 2 to 5. +4. Treats vanilla peers per §6 according to `policy.vanilla`. +5. Issues grants only against a valid pass or a delegating grant per §5.5. +6. Passes the published fixture swarm end to end: manifest, pass, three + vouchers, grant, decrypted output matching `plainRoot`. + +## 15. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/iplive.md b/docs/openswarm/iplive.md new file mode 100644 index 0000000..f3db2a3 --- /dev/null +++ b/docs/openswarm/iplive.md @@ -0,0 +1,378 @@ +# iplive: Live Streams over Paid Peers + +Version: **0.1** (draft) +Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)). Reuses +`ipfile`'s encryption, grants and credit window and `ippay`'s passes and +vouchers. Adds one BEP 10 extension, `iplive`. Nothing below is +implemented. + +## 1. Scope + +`iplive` moves a live stream from one origin to many viewers through peers +that are paid to relay it: + +1. A **channel** record: identity, renditions, latency mode, price, relay + policy. +2. A **head** record: signed every segment, naming the segments that exist + and the key epoch in force. +3. The **`iplive` extension**: announcing, requesting and fanning out + encrypted segments with backpressure. +4. **Roles**: origin, relay, viewer, and how a relay is paid. +5. **Recording**: how a finished stream becomes an `ipvideo` title. + +### Non-goals + +Ingest protocols (RTMP and SRT are the encoder's business; c0mpute's DIP +0019 covers them). Sub-second latency; the floor is one part duration. +Chat. Ads. + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **channel** | A named live source with a key pair derived from the publisher seed. | +| **channel id** | 20 bytes: SHA-256 of `"openswarm:iplive:v1" || channel public key`, truncated. Used where the DHT and trackers want an infohash. | +| **origin** | The peer that encodes, encrypts, signs heads and holds the content key. | +| **relay** | A peer that forwards segments it has verified and is paid per byte. | +| **viewer** | A peer that plays. It MAY also relay. | +| **segment** | One CMAF fragment, 2000 ms by default, encrypted as one unit. | +| **part** | In low-latency mode, a sub-segment chunk (500 ms default) announced before the segment completes. | +| **key epoch** | The span of segments encrypted under one content key. | +| **live edge** | The newest segment the origin has announced. | + +## 3. Data model + +### 3.1 Channel + +The channel key derives like a feed: + +``` +channelSeed = HKDF-SHA256(ikm = publisher seed, salt = utf8("live:" + name), info = "openswarm:live:v1") +``` + +```json +{ + "openswarm": "0.1", + "type": "iplive.channel", + "channel": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "box": "x25519:4c4055604fe6bd8781cab6835093ad896f68f676b472d5bd6811e52f28cb18c3", + "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "name": "courtside", + "title": "Courtside, live", + "latency": "low", + "segmentMs": 2000, + "partMs": 500, + "epochSegments": 900, + "dvrSegments": 1800, + "renditions": [ + { "id": "720p", "codecs": "avc1.64001f,mp4a.40.2", "width": 1280, "height": 720, "frameRate": 30, "bitrateKbps": 2800 }, + { "id": "360p", "codecs": "avc1.64001e,mp4a.40.2", "width": 640, "height": 360, "frameRate": 30, "bitrateKbps": 800 } + ], + "swarm": { + "trackers": ["wss://tracker.openwebtorrent.com", "udp://tracker.opentrackr.org:1337/announce"], + "private": false + }, + "relays": { "allow": "any", "maxDownstream": 8, "minRelays": 2 }, + "policy": { "vanilla": "deny", "creditBytes": 8388608, "voucherBytes": 2097152 }, + "pricing": { + "currency": "USD", + "perGib": "0.020000", + "keyUsd": "1.000000", + "split": { "publisherBps": 6000, "seederBps": 3500, "hubBps": 500 }, + "keeperBps": 0, + "hubs": ["https://coinpayportal.com/api/openswarm"], + "payTo": { "network": "eip155:8453", "address": "0xCC3b072391AE7A8d10cF00DdC5F61DB2cA5541E5" } + }, + "keys": { "modes": ["peer"], "keepers": [] }, + "createdAt": "2026-09-05T20:00:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + }, + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `channel`, `box` | Channel signing and box keys. Signed by both channel and publisher, as an `ipfile` manifest is. | +| `latency` | `normal`: segments announced when complete. `low`: parts announced as they are encoded. | +| `segmentMs`, `partMs` | 2000 and 500 by default. 4000 is allowed and matches c0mpute DIP 0019. | +| `epochSegments` | Segments per content key. 900 at 2 s is 30 minutes. | +| `dvrSegments` | Segments the origin and relays keep for late joiners and scrubbing. | +| `relays.allow` | `any`, or a list of identities allowed to pull from the origin. | +| `relays.maxDownstream` | Peers one relay will feed at once. | +| `relays.minRelays` | Relays the origin wants before it stops feeding viewers directly. | +| `policy`, `pricing`, `keys` | As `ipfile` §3.1 and §7, with the same fields and meanings. `seederBps` pays relays. | + +The record lives in `ipdb` under `channel:<channel key>`. + +### 3.2 Head + +Signed by the channel key every time a segment (or, in low-latency mode, +a part) is announced: + +```json +{ + "openswarm": "0.1", + "type": "iplive.head", + "channel": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "seq": 4187, + "prev": "sha256:ba8cd1a97b865a05cdc5840c63b086dd463624a831874c1ba0d7c4ce6f806f7a", + "epoch": 4, + "keyId": "sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb", + "live": true, + "segments": [ + { + "seq": 4187, + "t": 8374000, + "d": 2000, + "renditions": [ + { "id": "720p", "root": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", "size": 702113, "parts": [175528, 175530, 175527, 175528] }, + { "id": "360p", "root": "sha256:4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342", "size": 200418, "parts": [50104, 50105, 50104, 50105] } + ] + } + ], + "init": [ + { "id": "720p", "root": "sha256:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b", "size": 1524 }, + { "id": "360p", "root": "sha256:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be", "size": 1498 } + ], + "relays": ["ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94"], + "createdAt": "2026-09-05T22:19:34.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `seq` | Segment number of the live edge. Starts at 1 when the stream starts. | +| `prev` | Id of the previous head, so a viewer can detect a skipped announcement. | +| `epoch`, `keyId` | Which content key encrypts segments from this seq on. A key change is announced one full segment before it applies. | +| `segments` | The newest segment, or the newest few. A head is small: relays fill history from `have`. | +| `segments[].renditions[].root` | BEP 52 pieces root of the segment ciphertext, with the segment as one piece. That is what a relay verifies before forwarding. | +| `segments[].renditions[].parts` | Byte lengths of each part, in order, for low-latency requests. | +| `init` | Roots and sizes of each rendition's init segment, which is not encrypted. | +| `relays` | Identities the origin currently feeds; a viewer prefers these. | +| `live` | `false` in the final head, which marks the end of the stream. | + +Heads are not stored in `ipdb`. The final head is, under +`channel:<key>:end:<seq>`, alongside the recording (§8). + +### 3.3 Segment encryption + +As `ipfile` §4.1, with the segment as a single piece and the IV derived with +`info = "openswarm:iplive:iv:v1:" + decimal(seq)`. The content key changes +each epoch; a grant carries `keyId` and a viewer requests a new grant when +the head announces a new `keyId`. A pass expiring mid-stream stops at the +next epoch, which is why epochs are short. + +## 4. Wire protocol + +### 4.1 Finding peers + +Peers announce the channel id to the DHT (`announce_peer`) and to the +channel's trackers exactly as they would an infohash. Vanilla clients that +see it find a swarm whose `ut_metadata` returns nothing; they leave. +Browsers find WebRTC peers through wss trackers as with any WebTorrent +swarm. + +### 4.2 Handshake + +The BitTorrent handshake uses the channel id as the infohash. The BEP 10 +handshake advertises `iplive`. Both sides send `hello` with the `ipfile` +§5.1 fields (identity, box, nonce, roles, hubs, credit, interval, sig) +where `roles` is any of `origin`, `relay`, `viewer`, and the signature +covers the channel id in place of `infohashV2`. + +### 4.3 Messages + +| `t` | Payload | Meaning | +| --- | --- | --- | +| `hello` | as `ipfile` | Identity and terms. | +| `channel` | `{ "r": bytes }` | The channel record. Sent by origin and relays after `hello`. | +| `head` | `{ "r": bytes }` | A head record. Forwarded to every downstream peer within 100 ms of receipt. | +| `have` | `{ "from": int, "to": int, "r": [string] }` | Segment range and rendition ids the sender holds. | +| `window` | `{ "buffer": int, "live": int, "down": int, "max": int }` | Segments buffered, live edge, downstream count, `maxDownstream`. Sent after `hello` and whenever it changes. | +| `want` | `{ "seq": int, "r": string, "part": int or absent }` | Request one segment, or one part. | +| `seg` | `{ "seq": int, "r": string, "part": int or absent, "o": int, "d": bytes }` | A 16 KiB chunk of the requested unit at offset `o`. | +| `init` | `{ "r": string, "d": bytes }` | An init segment. | +| `pass`, `credit`, `voucher`, `key_req`, `grant`, `deny`, `bye` | as `ipfile` | Payment and keys, unchanged. | + +`seg` chunks are 16 KiB so that a WebRTC data channel and an MTP/1 record +both carry one without fragmentation. + +### 4.4 Session + +``` +viewer relay + | hello | + |---------------------------------------->| + |<------------------------------- hello | + |<----------------------------- channel | + |<------------------------------ window | buffer=30 live=4187 down=3 max=8 + | pass | + |---------------------------------------->| + |<-------------------------------- head | seq 4187 + |<-------------------------------- have | 4158..4187 + | want 720p init | + |---------------------------------------->| + |<-------------------------------- init | + | want 4185 720p | + |---------------------------------------->| + |<--------------------------------- seg | x 43 chunks + | want 4186 720p ... | + | key_req | + |---------------------------------------->| relay is not a keeper: + |<-------------------------------- deny | not-keeper; ask origin or keys.url + | voucher | + |---------------------------------------->| +``` + +Rules: + +1. A viewer starts `segmentMs * 2` behind the live edge in `normal` mode + and one part behind in `low` mode. +2. A peer MUST NOT `want` a seq below `live - buffer` of the peer it asks. + If it has fallen further behind it jumps to `live - 1` and discards. + Stalls are resolved by dropping, never by holding a relay's buffer. +3. A relay forwards a segment only after verifying its ciphertext against + the `root` in a head signed by the channel key. A relay never needs the + content key. +4. A relay serving `down >= max` answers new `hello`s with `deny rate` and + the current `head`, whose `relays` list points elsewhere. +5. Payment is `ipfile` §5.3 rules 2 to 6, with a relay as payee. The + default window is larger (8 MiB) because a segment is bigger than a + piece and a choke at the live edge is a stall. +6. Keys come from the origin or a keeper listed in the channel, never from + a plain relay. A relay MAY be a keeper if the publisher delegates to it. + +### 4.5 Low latency + +In `low` mode the origin announces a head as soon as the first part of a +segment is encoded, with `parts` filled in as they complete (lengths for +finished parts, `0` for pending). A viewer requests parts by index and +appends them to MSE as they arrive. A part is encrypted as part of the +segment's CTR stream: part `k` begins at counter `offset(k) / 16`, so a +relay can forward parts without waiting for the segment and a viewer can +decrypt them in order. The segment `root` is only known when the segment +completes; a relay in `low` mode forwards parts on the strength of the +signed head naming the part lengths and verifies the root when the last +part lands, disconnecting any upstream whose completed segment fails. + +## 5. Roles and payment + +| Role | Holds key | Pays | Is paid | +| --- | --- | --- | --- | +| Origin | Yes | Nothing | Publisher share of every voucher, `keyUsd` per grant | +| Relay | No | Vouchers to its upstream for bytes it pulled | Vouchers from downstream for bytes it served | +| Viewer | Yes, after a grant | Vouchers to whoever served it | Vouchers if it also relays | + +A relay's margin is `seederBps` of what it serves minus what it pays +upstream. With `perGib = 0.020000`, a relay feeding eight viewers from one +upstream pull earns `8 * 0.35 * 0.02 = 0.056` and pays `0.02` per GiB +pulled, before the hub's share. That is the incentive to relay, and a +c0mpute node bids on `iplive.relay` jobs for the guaranteed floor +(`c0mpute.md` §3). + +The origin is a peer that pays nobody; `relays.allow` and `minRelays` are +how it limits its own upload. A publisher who wants more reach than its +relays provide buys pins on c0mpute. + +## 6. Publishing procedure + +`ip live create courtside --latency low` writes the channel record and +puts it in `ipdb`. `ip live start courtside --input srt://:9000` then: + +1. Derives the channel key. Generates epoch 1's content key. +2. Runs the encoder (ffmpeg) producing CMAF segments per rendition, and + for `low` mode, parts. +3. For each completed unit: encrypts, computes the root, signs a head, + announces to the DHT and trackers, sends `head` to every connected peer. +4. Answers `key_req` with grants for pass holders. +5. Rotates the content key every `epochSegments`, announcing the new + `keyId` one segment early. +6. On stop: signs a final head with `live: false`, and if `--record`, + assembles the recording (§8). + +## 7. Viewer procedure + +`ip live watch ed25519:<channel key>` or a browser page: + +1. Resolve the channel record from `ipdb` or a peer. +2. Buy or reuse a pass scoped to the channel key with `grant: true`. +3. Find peers; prefer identities in the latest head's `relays`; connect to + two. +4. Get the init segment, the head, a grant. Start at the live edge minus + the offset in §4.4 rule 1. +5. Pull each segment from the connected peer with the smaller `live - seq` + lag; on a `deny rate` or a stall over one segment duration, replace the + peer. +6. Send vouchers on schedule to each peer that served bytes. + +## 8. Recording + +With `--record`, the origin keeps every segment's plaintext, concatenates +each rendition into one CMAF file, and publishes an `ipvideo.title` whose +segment index is exactly the head history. Because segments were 2 or 4 +seconds and keyframe-aligned, the recording is a valid `ipvideo` ladder +with no re-encode. The title's `ipdb` entry references the channel in +`meta.channel` and the final head's id. + +## 9. Security and privacy + +- **Relays see ciphertext and heads.** They learn segment sizes and + timing, never content. +- **Head forgery** needs the channel key. A relay forwarding an unsigned or + wrongly signed head is dropped by every downstream peer. +- **Freeloading** is bounded per `ipfile` §11 with the larger window. +- **Denial by relay count**: an origin with `relays.allow: any` can be + surrounded by relays that pull and never serve. It pays them nothing (it + pays nobody) and they earn nothing; the cost is its upload. `maxDownstream` + on the origin is the cap. +- **Late key**: a viewer whose pass expires keeps the current epoch's key + and loses the next. Epoch length is the publisher's tolerance. + +## 10. Events + +| Type | Emitted by | +| --- | --- | +| `com.logicsrc.openswarm.iplive.channel.created.v1` | publisher | +| `com.logicsrc.openswarm.iplive.stream.started.v1` | origin | +| `com.logicsrc.openswarm.iplive.stream.ended.v1` | origin, with the final head id and recording title id | +| `com.logicsrc.openswarm.iplive.relay.joined.v1` | origin | + +## 11. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| RTMP/SRT ingest to HLS segments, segment swarming in the DHT, relay nodes rebuilding playlists, LL-HLS opt-in | Designed | `c0mpute` `dips/0019-live-stream-plugin.md` | Encrypted segments, signed heads, paid relays: the same design with `ippay` attached | +| Live TV playback in the browser (hls.js, mpegts.js) | Yes | `media-streamer` `hls-player-modal.tsx`, tipoffwatch and genrewatch players | MSE append of decrypted CMAF parts from peers instead of an HLS URL | +| Pay-per-game passes with live session caps | Yes (central) | `media-streamer` IPTV resale (`iptv_share_sessions`) | Concurrency is no longer the owner's provider line; it is the origin's `maxDownstream` | +| Channel and head records, `iplive` messages, relay economics, recording to `ipvideo` | No | | All of it | + +## 12. Conformance + +An origin signs a head per unit, rotates keys per epoch, and announces a +key change one segment early. A relay verifies roots before forwarding, +honours `window` rules, and never serves below `live - buffer`. A viewer +drops rather than stalls, vouchers every peer that served it, and requests +a grant on a `keyId` change. + +## 13. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/ipname.md b/docs/openswarm/ipname.md new file mode 100644 index 0000000..0f52d72 --- /dev/null +++ b/docs/openswarm/ipname.md @@ -0,0 +1,187 @@ +# ipname: Names for Publishers + +Version: **0.1** (draft) +Status: draft. A thin bridge between the OpenSwarm core +([spec.md](./spec.md)) and the Moshpit registry. Nothing below is +implemented. + +## 1. Scope + +`ipname` answers one question: given a name a person can type, what +publisher key and what catalogue does it mean? It defines: + +1. An `ipname.pin` record a publisher signs, served from the name's host. +2. A Moshpit registry pin kind, `openswarm`, that anchors that record. +3. A DNS TXT form for ordinary domains. +4. The `ip://` URL. + +It does not define a naming system. Moshpit names exist (`label.tld`, +registry at `pit.moshcode.sh`) and DNS exists. `ipname` maps both to keys. + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **name** | A Moshpit name (`chovy.hacker`) or a DNS host (`example.com`). | +| **pin** | In the Moshpit registry, `base64(SHA-256(SubjectPublicKeyInfo))` of a key, with a `kind`. | +| **openswarm pin** | A registry pin of kind `openswarm` over the publisher's Ed25519 SPKI (RFC 8410). | +| **name record** | The `ipname.pin` record at `/.well-known/openswarm.json`. | + +## 3. The name record + +```json +{ + "openswarm": "0.1", + "type": "ipname.pin", + "name": "chovy.hacker", + "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "feeds": [ + "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default", + "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/podcasts" + ], + "gateway": "https://gw.c0mpute.com", + "hubs": ["https://coinpayportal.com/api/openswarm"], + "mtp": "sK8m2eLwG4l3iF5oT9uYQ6c1Vb0nZxA7hE2dR8pJkLw=", + "createdAt": "2026-09-05T18:30:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `name` | The name this record is served for. A record served under a different host MUST be refused. | +| `publisher` | The key that signs the record and every feed it lists. | +| `feeds` | Feed references (`ipdb` §3.1). The first is the default. | +| `gateway` | An HTTP gateway that serves this publisher's swarms and `POST /db/query`. MAY be null. | +| `hubs` | Hubs the publisher sells passes through. | +| `mtp` | The publisher's MTP/1 pin, if it runs a Moshpit transport endpoint. MAY be null. | + +The record is also put in `ipdb` under `name:<name>` in the default feed, +so a replica that has the feed can answer the reverse question: what names +does this key claim. + +## 4. Resolution + +### 4.1 Moshpit names + +A Moshpit name has exactly two labels of letters and digits. The registry +holds a `target` host and a list of pins per name. `ipname` uses both: + +1. `GET https://pit.moshcode.sh/api/moshpit/pins?name=chovy.hacker` returns + `{ "name", "pins": [...], "target" }`. Pins are objects or strings + depending on the registry version; the resolver looks for entries of + kind `openswarm`. +2. Fetch `https://<target>/.well-known/openswarm.json` (through + `moshpit-proxy` when the target is itself a Moshpit name, plain HTTPS + when it is a clearnet host). +3. Compute `base64(SHA-256(SPKI(record.publisher)))` and require it to equal + an `openswarm` pin. Verify the record signature. Require `record.name` + to equal the queried name. + +The registry pin is the anchor. The registry is not trusted for the key +itself; it is trusted to say which hash the name's owner registered, and +the owner's key is whatever hashes to that. This is the same argument +`moshpit-proxy` makes for TLS pins and `qrypt.chat` makes for `mtp` pins, +and the pin format is byte-identical so one registry field carries all +three kinds. + +Writing the pin is one registry call by the name's owner: + +``` +POST https://app.moshcode.sh/api/moshpit/tlds/hacker/pins +Authorization: Bearer <MOSHPIT_API_KEY> +{ "label": "chovy", "pin": "base64...", "kind": "openswarm", "note": "OpenSwarm publisher key" } +``` + +`kind: "openswarm"` does not exist in the registry today; adding it is a +one-line change to the accepted kinds and is listed as an open question. + +### 4.2 House policy: clearnet first + +Moshpit's `prefer` rule applies unchanged: a name that resolves on the +public DNS root is a clearnet host by default, and the registry is +consulted only in `fallback` or `moshpit` mode. `ipname` MUST use +`@moshcoder/moshpit-resolve`'s `decideResolution` (or an equivalent) before +touching the registry, so that `example.com` is never redirected to a pit +record because someone claimed `.com` as an ending. + +### 4.3 DNS hosts + +For a clearnet host, the anchor is a TXT record: + +``` +_openswarm.example.com. IN TXT "v=openswarm1; key=ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e" +``` + +The resolver fetches `https://example.com/.well-known/openswarm.json` and +requires `record.publisher` to equal the TXT key. Without the TXT record +the resolver MAY accept the well-known record on the strength of TLS alone +and MUST report the lower assurance to the caller (`"anchor": "tls"` +rather than `"anchor": "dns"` or `"anchor": "moshpit"`). + +### 4.4 Caching + +A resolved name is cached for the shorter of 300 seconds and the registry's +own TTL, matching `moshpit-proxy`'s pin cache. A negative result is cached +for 5 seconds. + +## 5. The ip URL + +``` +ip://<name or key>[/<feed>[/<key>]] +``` + +| Example | Meaning | +| --- | --- | +| `ip://chovy.hacker` | The publisher's default feed. | +| `ip://chovy.hacker/podcasts` | A named feed. | +| `ip://chovy.hacker/default/track:sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff` | One record by `ipdb` key. | +| `ip://ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e` | The same publisher by key; no name resolution. | +| `ip://ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d?file` | A file by its file key: BEP 44 lookup, no feed. | + +A gateway maps `ip://` to HTTPS: `https://<gateway>/ip/<name or key>/...` +with the same path. A `magnet:` link with `xs=urn:btpk:<file key>&s=ipfile` +is the BitTorrent-native equivalent of the last row and opens in any BEP 46 +client, which then needs the `ipfile` extension to do anything with it. + +## 6. Security notes + +- **Registry compromise** can change a pin. A resolver that has seen a name + before SHOULD keep the last verified publisher key and warn on change, the + way `moshpit-proxy` handles a pin rotation with `MOSHPIT_PROXY_TOFU`. +- **Host compromise** can serve a different record, which fails the pin. +- **Name squatting** is the registry's problem, and only the ending owner + can mint names under it. +- **Privacy**: resolving a name tells the registry which names a client + asks about, as DNS does. A client that already holds a key never resolves. + +## 7. Events + +None. Name changes surface as `ipdb.entry.appended` for `name:<name>`. + +## 8. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| Registry with per-name pins of kind `tls` and `mtp` | Yes | `pit.moshcode.sh`, `moshpit_name_pins` | The `openswarm` kind | +| Pin lookup, cache, TOFU handling | Yes | `moshpit-proxy` `lib/pins.ts` | Reuse as the resolver's registry client | +| Clearnet-first decision | Yes | `@moshcoder/moshpit-resolve` | Called before any registry lookup | +| Name record, TXT form, `ip://` URL | No | | All of it | + +## 9. Conformance + +A resolver implements §4.1 to §4.4, refuses a record whose `name` or pin +does not match, and reports the anchor kind. A publisher serves the record +at the well-known path over HTTPS and keeps it in `ipdb`. + +## 10. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/ippay.md b/docs/openswarm/ippay.md new file mode 100644 index 0000000..721b23b --- /dev/null +++ b/docs/openswarm/ippay.md @@ -0,0 +1,464 @@ +# ippay: Passes, Vouchers and Settlement + +Version: **0.1** (draft) +Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)). The x402 +half exists (`@profullstack/x402-gateway`, CoinPay); the pass and voucher +half does not. + +## 1. Scope + +`ippay` defines how money moves in an OpenSwarm swarm: + +1. A **pass**: a hub-signed record that lets one payer spend up to a cap on + named files, bought over x402 in USDC. +2. A **voucher**: a payer-signed, cumulative IOU to one payee for verified + bytes, exchanged on the swarm wire or over HTTP. +3. A **hub**: the settlement service that sells passes, redeems vouchers, + splits the value between publisher, payee and itself, and pays out. +4. **Reputation**: the minimum a hub records so a seeder can bound its risk. + +It reuses the x402 v2 offer and proof exactly as CoinPay speaks them and as +`x402-gateway` already sells crawl passes. It does not define a token, a +chain, or an on-chain payment channel. + +### Non-goals + +Trustless settlement. A hub is trusted to pay out what it accepted, the way +a payment processor is. Anyone may run a hub; a manifest lists which hubs a +publisher trusts. Per-request on-chain payments: a 16 KiB block is worth +about 0.00000015 USD at the reference price, and no chain settles that. + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **payer** | The identity that bought a pass and signs vouchers. Usually a leecher or viewer. | +| **payee** | The identity a voucher is signed to. A seeder, relay, keeper or gateway. | +| **cap** | The most a pass can spend across all payees. | +| **standing** | The hub's record of a payer's or payee's behaviour. | + +## 3. Passes + +### 3.1 Record + +```json +{ + "openswarm": "0.1", + "type": "ippay.pass", + "hub": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b", + "payer": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be", + "scope": { + "files": ["ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d"], + "publishers": [] + }, + "capUsd": "2.000000", + "grants": ["ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d"], + "paidUsd": "2.500000", + "payment": { + "network": "eip155:8453", + "nonce": "0x2f0a9c1b8d7e6f5a4b3c2d1e0f9a8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a" + }, + "issuedAt": "2026-09-05T18:00:40.000Z", + "expiresAt": "2026-10-05T18:00:40.000Z", + "createdAt": "2026-09-05T18:00:40.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `hub` | The hub's signing key. A seeder verifies the signature against the key in the hub record (§6.1), fetched once and cached. | +| `payer` | The identity that will sign vouchers. Bound at purchase time (§3.3). | +| `scope.files` | File keys this pass may spend on. `["*"]` means any file whose manifest lists this hub. | +| `scope.publishers` | Publisher keys; any file by one of them is in scope. Either list MAY be empty; at least one MUST be non-empty. | +| `capUsd` | Spending cap across all vouchers. | +| `grants` | Files for which `keyUsd` was paid. A pass buys one grant per listed file. | +| `paidUsd` | What was paid: `capUsd` plus each granted file's `keyUsd`. | +| `payment.nonce` | The EIP-3009 nonce of the proof that bought it. Lets the pass be traced to a settlement. | +| `expiresAt` | Hubs SHOULD issue 30 day passes. Unspent cap is not refunded in 0.1. | + +A pass is self-contained. Presented over HTTP it is +`Authorization: Bearer <base64url(canonical JSON of the record including sigs)>`. +A verifier needs the hub's public key and a clock; no round trip. + +### 3.2 Purchase over x402 + +A client asks the hub for a pass. The hub answers 402 with CoinPay's x402 v2 +offer. The client pays and retries. This is the same exchange +`@profullstack/x402-gateway` runs for `/crawl`. + +``` +GET /passes?payer=00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be + &file=0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d + &cap=2.000000&grant=1 +Host: coinpayportal.com +Accept: application/json +``` + +``` +HTTP/1.1 402 Payment Required +Content-Type: application/json; charset=utf-8 +Cache-Control: no-store +Vary: Accept, X-Payment + +{ + "x402Version": 2, + "accepts": [ + { + "scheme": "exact", + "network": "eip155:8453", + "amount": "2500000", + "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913", + "payTo": "0xCC3b072391AE7A8d10cF00DdC5F61DB2cA5541E5", + "resource": "https://coinpayportal.com/api/openswarm/passes?payer=00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be&file=0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d&cap=2.000000&grant=1", + "description": "OpenSwarm pass: 2.000000 USD cap plus 1 key grant", + "mimeType": "application/json", + "maxTimeoutSeconds": 300, + "extra": { "name": "USD Coin", "version": "2" } + } + ], + "pass": { "capUsd": "2.000000", "keyUsd": "0.500000", "expiresInDays": 30 } +} +``` + +`amount` is in the asset's smallest unit: 2.5 USDC is `2500000`. The client +signs an EIP-3009 `transferWithAuthorization` for that amount to `payTo` +(the `@profullstack/x402-client` `pay()` call, or CoinPay Wallet in a +browser) and retries with the proof: + +``` +GET /passes?payer=...&file=...&cap=2.000000&grant=1 +X-PAYMENT: <base64 JSON proof> +PAYMENT-SIGNATURE: <same, v2 header name> +``` + +The hub calls CoinPay `/api/x402/verify` with `expected: { amount, resource, +payTo, asset }` from the entry it offered, then `/api/x402/settle`. On +`settled: true` it returns `200` with the pass record as the body and +`X-OpenSwarm-Pass: <base64url record>` as a header. A replayed proof returns +the pass it already bought, with `replayed: true` in a wrapper, and never a +second pass. + +The `payer` key is in the resource URL on purpose. EIP-3009 cannot bind a +resource; a proof for one resource can be presented for another from the +same merchant at the same or lower price. The hub therefore records +`nonce -> resource` at verify time and refuses a nonce for any other +resource, which closes the gap for passes. + +### 3.3 Binding the payer + +The pass names an Ed25519 `payer`. The buyer proves control of it by signing +the request: header `X-OpenSwarm-Payer: <base64 Ed25519 signature over +"openswarm:passreq:v1" || resource URL>`. Without it the hub MUST refuse to +issue; otherwise anyone who observed a proof could bind the pass to their +own key. + +### 3.4 Direct key purchase + +A key service (`ipfile` §5.5, URL mode) MAY sell a grant without a hub: its +402 offers `keyUsd` alone. That is a publisher running the x402-gateway +pattern on its own key endpoint. It gets no per-byte income that way; it is +the choice for a publisher who pays for distribution with free webseeds. + +## 4. Vouchers + +### 4.1 Record + +```json +{ + "openswarm": "0.1", + "type": "ippay.voucher", + "pass": "sha256:39655de63a29b02c90b236fa870fe6f8dfad1f52a8119fe4df85e88cd6ab02fd", + "payer": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be", + "payee": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "swarm": "sha256:4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342", + "seq": 3, + "bytes": 268435456, + "usd": "0.002500", + "createdAt": "2026-09-05T18:04:02.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Rule | +| --- | --- | +| `pass` | Record id of the pass. | +| `payer` | MUST equal the pass's `payer`. The signature MUST be by this key. | +| `payee` | The identity being paid. From the peer's `hello`. | +| `file`, `swarm` | The file key and `infohashV2` the bytes belong to. | +| `seq` | Starts at 1 per (pass, payee, swarm) and increases by one. | +| `bytes` | Cumulative verified ciphertext bytes received from this payee on this swarm under this pass. Never decreases. | +| `usd` | Cumulative value, computed per §4.2 from `bytes` and the manifest's `perGib`. | + +A voucher is cumulative, so only the latest one per (pass, payee, swarm) +matters. A payee stores the latest and discards the rest. Redemption is one +record per triple, however long the session was. + +### 4.2 Value + +``` +perGibMicro = perGib * 1_000_000 (exact: the string has 6 places) +usdMicro = floor(bytes * perGibMicro / 2^30) +usd = usdMicro formatted with 6 places +``` + +Integer arithmetic only. For `perGib = "0.010000"` and `bytes = 268435456` +(256 MiB): `268435456 * 10000 / 1073741824 = 2500`, so `"0.002500"`. A +payee MUST recompute `usd` and reject a voucher whose `usd` differs from the +computed value, so a payer cannot understate by editing the number. + +### 4.3 Exchange + +On the swarm wire a voucher travels in the `ipfile` `voucher` message +(`ipfile` §5.2) or the `iplive` `voucher` message. Over HTTP it travels as +`X-OpenSwarm-Voucher: <base64url record>` on the request that would push +unpaid bytes past the interval (§7). + +### 4.4 Validation by the payee + +1. Signature by `payer`; `payer` equals the pass's payer. +2. Pass signature by a hub in the manifest's `pricing.hubs`, pass not + expired, file in scope. +3. `payee` is this peer. `swarm` is this swarm. +4. `seq` is exactly previous `seq + 1` (or 1 for the first). +5. `bytes` is at least previous `bytes` and at most bytes actually served. +6. `usd` recomputes. + +A voucher failing 1 to 4 or 6 is rejected with `voucher-invalid`. One +failing 5 by claiming too much is accepted at the served amount and the +excess is reported to the hub as `overclaim`. One claiming less than +previous is `voucher-stale`. + +## 5. The hub + +### 5.1 Hub record + +Published at `GET <base>/.well-known/openswarm-hub.json`: + +```json +{ + "openswarm": "0.1", + "type": "ippay.hub", + "key": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b", + "name": "CoinPay", + "base": "https://coinpayportal.com/api/openswarm", + "networks": ["eip155:8453", "eip155:137", "eip155:1"], + "assets": { + "eip155:8453": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913", + "eip155:137": "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359", + "eip155:1": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48" + }, + "minHubBps": 300, + "payout": { "minUsd": "1.000000", "schedule": "daily" }, + "passDays": 30, + "createdAt": "2026-09-05T00:00:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +The record is self-signed; trust in the hub key comes from the manifest that +lists the hub's `base`, from TLS to that base, and optionally from an MTP/1 +pin. `minHubBps` is the least `split.hubBps` a manifest may set for this hub +to sell passes for it. + +### 5.2 API + +All bodies are JSON. Records are verified on receipt. + +| Method and path | Auth | Purpose | +| --- | --- | --- | +| `GET /.well-known/openswarm-hub.json` | none | Hub record. | +| `GET /passes?payer&file|publisher&cap&grant` | x402 + `X-OpenSwarm-Payer` | Buy a pass (§3.2). | +| `GET /passes/<id>` | none | `{ "pass", "spentUsd", "grantsUsed": [...], "status": "active|expired|revoked" }`. | +| `POST /vouchers` | none | Body: one voucher or an array. Returns receipts (§5.3). | +| `POST /grants` | none | Body: `{ "grant": <record without sealed>, "manifest": <record> }`. Keeper reports a grant. | +| `POST /payees` | signed | Body: `{ "payee", "payTo": { "network", "address" }, "sig" }`, `sig` by the payee key over `"openswarm:payee:v1" || address`. | +| `GET /payees/<key>` | none | `{ "payTo", "balanceUsd", "paidOutUsd", "standing" }`. | +| `GET /payers/<key>/standing` | none | `{ "passes", "overclaims", "capExceeded", "freeloads", "since" }`. | +| `POST /reports` | signed | `{ "kind": "freeload", "payer", "swarm", "unpaidBytes", "reporter", "sig" }`. | +| `POST /webhooks` | publisher signed | Register a CloudEvents endpoint for a publisher key. | + +### 5.3 Redemption and receipts + +For each voucher the hub: + +1. Validates §4.4 steps 1, 2 and 6 (it cannot check 3 to 5; those were the + payee's job, and the payee signed nothing, so an inflated voucher only + hurts the payer). +2. Loads the latest accepted voucher for the triple. Accepts only the + increase in `usd` since it, so re-submitting an old voucher pays nothing. +3. Applies the cap: `spentUsd + increase <= capUsd`. If not, accepts the + remainder up to the cap with status `partial`, or `cap-exceeded` if + nothing remains. +4. Fetches the manifest for `file` (from `ipdb`, a gateway, or the DHT) and + applies `split`. +5. Credits the payee, the publisher's `payTo`, and itself. +6. Returns a receipt: + +```json +{ + "openswarm": "0.1", + "type": "ippay.receipt", + "voucher": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", + "pass": "sha256:39655de63a29b02c90b236fa870fe6f8dfad1f52a8119fe4df85e88cd6ab02fd", + "payee": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "status": "accepted", + "accepted": { "bytes": 268435456, "usd": "0.002500" }, + "split": { "publisherUsd": "0.001750", "payeeUsd": "0.000625", "hubUsd": "0.000125" }, + "createdAt": "2026-09-05T18:10:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +Split arithmetic is in micro-USD, integer, with the remainder after +`floor` for each share going to the publisher. `2500 * 7000 / 10000 = 1750`, +`2500 * 2500 / 10000 = 625`, `2500 * 500 / 10000 = 125`. + +### 5.4 Key grants + +`POST /grants` lets a keeper collect `keeperBps` of `keyUsd` and marks the +grant on the pass as used. The hub verifies the grant signature is by the +file key or a keeper in the manifest, that the pass listed the file in +`grants`, and that no grant was recorded for that pass and file before. A +second grant for the same pass and file is refused; the peer already holds +the key. + +### 5.5 Standing + +A hub keeps, per payer: passes issued, overclaims, cap-exceeded redemptions, +freeload reports. Per payee: vouchers redeemed, overclaims accepted against +it. A seeder MAY consult `GET /payers/<key>/standing` before extending a +window larger than the manifest minimum, and MUST NOT refuse a peer for +standing alone when the manifest's `creditBytes` window is what it offers. +Standing is advisory; the credit window is the guarantee. + +### 5.6 Payout + +The hub pays each payee's balance to its registered `payTo` in USDC on the +payee's network when it reaches `payout.minUsd`, on `payout.schedule`. The +publisher's share goes to the manifest's `pricing.payTo`. The hub keeps its +share. A payout emits `com.logicsrc.openswarm.ippay.payout.sent.v1`. + +Where the hub is CoinPay: today's x402 rail pays a merchant's own wallet +directly and collects no platform fee. A hub is different: it is the +merchant for the pass sale, holds the balance, and pays out. This is a +custodial position and is flagged as an open question in the PR. + +## 6. Payees + +### 6.1 Registering + +A seeder, relay, keeper or gateway that wants to be paid registers once: + +```json +{ + "payee": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "payTo": { "network": "eip155:8453", "address": "0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf" }, + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" +} +``` + +Vouchers redeemed before registration accumulate and pay out once an +address exists. A c0mpute node registers its libp2p-adjacent OpenSwarm +identity this way at first start (`c0mpute.md` §2). + +### 6.2 Relationship to the file key + +A publisher's payout address is in its signed manifest and can only be +changed by a manifest revision, that is, by the file key. A payee's payout +address is registered at the hub by the payee key. The two never mix: a +publisher that also seeds its own file is paid twice, once as publisher and +once as payee, to two addresses it may set to the same value. + +## 7. HTTP presentation + +A gateway or key service applies the same rules over HTTP: + +| Header | Direction | Meaning | +| --- | --- | --- | +| `Authorization: Bearer <base64url pass>` | request | The pass. | +| `X-OpenSwarm-Voucher: <base64url voucher>` | request | A voucher covering bytes received so far. | +| `X-OpenSwarm-Credit: unpaid=<bytes>; limit=<bytes>; owed=<usd>` | response | Sent with every 206 so the client knows when to voucher. | +| `402` with `X-OpenSwarm-Credit` and no x402 body | response | Window exhausted; send a voucher and retry. | + +The gateway's `payee` is its identity key, published in the hub record it +registered with and returned in `X-OpenSwarm-Payee` on every response. + +## 8. Events + +| Type | Emitted by | +| --- | --- | +| `com.logicsrc.openswarm.ippay.pass.issued.v1` | hub | +| `com.logicsrc.openswarm.ippay.voucher.redeemed.v1` | hub | +| `com.logicsrc.openswarm.ippay.voucher.rejected.v1` | hub | +| `com.logicsrc.openswarm.ippay.grant.recorded.v1` | hub | +| `com.logicsrc.openswarm.ippay.payout.sent.v1` | hub | +| `com.logicsrc.openswarm.ippay.report.filed.v1` | hub | + +Publishers subscribe with `POST /webhooks`. Delivery is per core §8.2. + +## 9. Security notes + +- **Hub compromise.** A hub key signs passes. A stolen key mints free + passes until the manifest's `hubs` list is revised. Hubs SHOULD keep the + signing key in an HSM and publish a successor key in the hub record before + rotating. +- **Pass theft.** A pass is bearer for HTTP presentation but useless without + the payer's private key on the swarm, since vouchers must be signed by + `payer`. A gateway that accepts a pass without a voucher is serving one + window on trust, same as a seeder. +- **Voucher replay.** Cumulative and per-triple; a replay pays nothing. +- **Cap races.** Two seeders redeem at once past the cap: the hub serialises + per pass, so exactly one gets the remainder. Each seeder's exposure was one + window regardless. +- **Payee address change.** Signed by the payee key. A hub SHOULD hold + payouts for 24 hours after a change and notify the previous address's + webhook. + +## 10. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| x402 v2 offer, `X-PAYMENT` proof, verify and settle, HMAC passes | Yes | `x402-gateway` (`src/x402.js`, `src/pass.js`), CoinPay `/api/x402/*` | The pass record replaces the HMAC token; scope, cap, grants | +| Node payer with EIP-3009 signing | Yes | `x402-client` | Nothing; it can buy a pass unchanged | +| Browser payer | Yes | CoinPay Wallet, `@profullstack/coinpay` `x402-browser.js` | Nothing | +| Pay-per-pass grants with concurrency caps | Yes (central) | `media-streamer` IPTV and seedbox rails | The swarm version does not need the session table; the window is the cap | +| Vouchers, receipts, split, payees, standing, payout batching | No | | All of it | + +## 11. Conformance + +A payee implementation validates vouchers per §4.4 and redeems per §5.2. +A payer implementation computes `usd` per §4.2 and never signs for bytes it +has not verified. A hub implements every row of §5.2, the receipt of §5.3, +and the arithmetic of §4.2 and §5.3 exactly. + +## 12. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/ipvideo.md b/docs/openswarm/ipvideo.md new file mode 100644 index 0000000..2997f27 --- /dev/null +++ b/docs/openswarm/ipvideo.md @@ -0,0 +1,271 @@ +# ipvideo: Video on Demand on ipfile Swarms + +Version: **0.1** (draft) +Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)) and rides +on [ipfile](./ipfile.md), [ippay](./ippay.md) and [ipdb](./ipdb.md). +Nothing below is implemented. + +## 1. Scope + +`ipvideo` defines the records that make a set of `ipfile` swarms playable +as a film, an episode or a clip: + +1. A **title**: renditions (CMAF fragmented MP4, one `ipfile` each), a + segment index, subtitles, thumbnails, chapters, artwork. +2. **Playback rules**: segment-aligned adaptive bitrate over pieces, MSE in + the browser. +3. A **gateway bridge** that renders HLS for players that speak nothing + else. +4. **Transcode on c0mpute**: how the renditions get made. + +Bytes, keys, payment and discovery are `ipfile` and `ippay`. No new wire +messages. + +### Non-goals + +Live (that is `iplive`). Interactive video. Ad insertion (c0mpute's DAI +DIPs may reference a title; this document does not). DRM in the device +sense. + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **rendition** | One encoding ladder step, stored as one `ipfile` in CMAF form. | +| **segment** | A fragmented MP4 `moof + mdat` pair, starting with a keyframe. | +| **init segment** | The `ftyp + moov` bytes a decoder needs before any segment. | +| **segment index** | The list of `{ t, d, o, l }` for every segment of a rendition. | +| **ladder** | The set of renditions of a title, aligned on segment boundaries. | + +## 3. Data model + +### 3.1 Title + +```json +{ + "openswarm": "0.1", + "type": "ipvideo.title", + "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "title": "The Interview", + "kind": "movie", + "series": null, + "season": null, + "episode": null, + "year": 2026, + "durationMs": 5400000, + "language": "en", + "artwork": { + "poster": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "backdrop": null + }, + "renditions": [ + { + "id": "1080p", + "container": "cmaf", + "codecs": "avc1.640028,mp4a.40.2", + "width": 1920, + "height": 1080, + "frameRate": 24, + "bitrateKbps": 5000, + "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "size": 3375000000, + "init": { "o": 0, "l": 1524 }, + "segmentsFile": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be" + }, + { + "id": "480p", + "container": "cmaf", + "codecs": "avc1.64001e,mp4a.40.2", + "width": 854, + "height": 480, + "frameRate": 24, + "bitrateKbps": 1400, + "file": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", + "size": 945000000, + "init": { "o": 0, "l": 1498 }, + "segments": [ + { "t": 0, "d": 4000, "o": 1498, "l": 702113 }, + { "t": 4000, "d": 4000, "o": 703611, "l": 698402 } + ] + } + ], + "audio": [ + { "id": "en", "language": "en", "label": "English", "channels": 2, "default": true } + ], + "subtitles": [ + { "id": "en-cc", "language": "en", "kind": "captions", "format": "vtt", "file": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b" } + ], + "thumbnails": { + "file": "ed25519:4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342", + "vtt": "ed25519:ba8cd1a97b865a05cdc5840c63b086dd463624a831874c1ba0d7c4ce6f806f7a", + "intervalMs": 10000 + }, + "chapters": [ + { "t": 0, "title": "Opening" }, + { "t": 1830000, "title": "The question" } + ], + "royalties": [], + "createdAt": "2026-09-05T18:20:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +(The `480p` segment list is truncated to two entries; a real one has one +entry per segment.) + +| Field | Rule | +| --- | --- | +| `kind` | `movie`, `episode`, `clip`, `trailer`. `series`, `season`, `episode` are set for `episode`. | +| `renditions[].container` | `cmaf` in this version: fragmented MP4 per ISO 23000-19, one segment per `moof`. | +| `renditions[].codecs` | RFC 6381 string as MSE `isTypeSupported` wants it. | +| `renditions[].file` | The `ipfile` manifest's file key. `size` MUST match. | +| `renditions[].init` | Byte range of the init segment within the plaintext. | +| `renditions[].segments` | Inline index, or `segmentsFile`: file key of a free `ipfile` (`keys.public` set) whose plaintext is the JSON array. Inline when under 64 KiB, file otherwise. | +| `segments[].t`, `d` | Presentation start and duration in ms. `o`, `l`: byte offset and length in the plaintext. | +| `audio` | Audio tracks muxed in the renditions. Alternate-language audio is a rendition with `codecs` of audio only and `video: false`. | +| `subtitles[].format` | `vtt` in this version. Each is a free `ipfile` unless the publisher prices it. | +| `thumbnails` | A sprite sheet `ipfile` and a WebVTT file mapping times to sprite regions, as hls.js and video.js expect. | +| `royalties` | As `ipaudio` §5. Empty means the whole publisher share goes to `payTo`. | + +Every rendition MUST share the same `t` list. Every segment MUST start with +a keyframe. Segment duration SHOULD be 4000 ms, matching c0mpute's +live-stream DIP (`-hls_time 4`) so a recorded live stream and a VOD title +use one ladder. + +### 3.2 Where records live + +`ipdb` under `title:<id>` in the publisher's feed. Renditions, subtitles +and thumbnails are `ipfile` manifests in the same feed with +`meta.title = <title record id>`. + +## 4. Playback + +### 4.1 Segment to pieces + +To play segment `s` of rendition `r`: + +``` +first piece = floor(s.o / pieceLength) +last piece = floor((s.o + s.l - 1) / pieceLength) +``` + +Request those pieces; decrypt; verify against the plaintext piece layer; +slice `[s.o, s.o + s.l)`; append to the MSE `SourceBuffer` after the init +segment. With 1 MiB pieces and a 5 Mbps rendition a 4 s segment is about +2.5 MiB, so two to three pieces. + +### 4.2 Adaptive bitrate + +Switch at segment boundaries only. The next segment is fetched from the +rendition whose `bitrateKbps` is the highest below 0.8 times the measured +piece throughput over the last 10 seconds, with a two-segment hysteresis +before stepping up. Because `t` lists are shared, the MSE buffer stays +contiguous across a switch; a new init segment is appended before the +first segment of the new rendition, as in any CMAF player. + +### 4.3 Buffering and priority + +Hold 30 seconds ahead. Request the pieces for the next segment with +sequential priority (what media-streamer's `torrent.select` does for range +streaming today). Seeking to time `t`: find the segment with the largest +`t <= target`, drop outstanding requests, start there. A player MUST keep +sending vouchers for bytes received while seeking, including bytes it then +discards; discarded bytes were still served. + +### 4.4 Browser + +A browser peer is a WebTorrent client with the `ipfile` extension, a +WebCrypto AES-CTR decryptor, and MSE. This is bittorrented.com's player +with three additions and no server in the path. Where MSE is unavailable +(some iOS contexts), the gateway bridge in §5 applies. + +## 5. Gateway bridge + +A keeper gateway renders HLS from the title record: + +``` +GET /video/<title id>/master.m3u8 +GET /video/<title id>/<rendition id>/index.m3u8 +GET /video/<title id>/<rendition id>/data (plaintext, Range) +Authorization: Bearer <base64url pass> +``` + +The media playlist uses `#EXT-X-MAP` for the init range and +`#EXT-X-BYTERANGE` for every segment against the single `data` URL, so the +gateway serves plaintext by range from decrypted pieces and never has to +split files. Subtitles become `#EXT-X-MEDIA` entries pointing at +`/file/<file key>/data`. Credit and voucher headers are as `ippay` §7; a +player that cannot send vouchers (a bare `<video>` tag) is limited to the +pass's first credit window unless the gateway is configured to trust the +pass for the title's full size. + +HLS's own encryption (`EXT-X-KEY` with AES-128-CBC) is not used. The +gateway is a keeper; it decrypts. A publisher who does not want any +gateway holding a key does not delegate to one, and browser MSE is the +only path. + +## 6. Transcode on c0mpute + +`ip video publish <source> --ladder default` submits one c0mpute job per +rendition. The existing `ffmpeg.transcode` workload takes +`TranscodeSpec { codec, bitrate_bps, width, height, keyframe_interval, +hardware_pref, extra_ffmpeg_args }`; `ipvideo` adds `extra_ffmpeg_args` +that force CMAF output and a 4 s keyframe interval, and expects the result +to include the segment index. The proposed `ipvideo.transcode` workload +(`c0mpute.md` §3) is that: a wrapper whose `TranscodeResult` carries +`segments`, `init`, `codecs` and gapless-irrelevant fields, and whose +output is handed to `ip file add` on the worker so the rendition is seeded +from where it was made. + +Default ladder: + +| id | codecs | size | bitrate | +| --- | --- | --- | --- | +| `1080p` | `avc1.640028,mp4a.40.2` | 1920x1080 | 5000 kbps | +| `720p` | `avc1.64001f,mp4a.40.2` | 1280x720 | 2800 kbps | +| `480p` | `avc1.64001e,mp4a.40.2` | 854x480 | 1400 kbps | +| `360p` | `avc1.64001e,mp4a.40.2` | 640x360 | 800 kbps | +| `1080p-av1` | `av01.0.08M.08,opus` | 1920x1080 | 3000 kbps, optional | + +## 7. Security and privacy + +As `ipfile`. A segment index reveals segment sizes, which for a +constant-quality encode leak scene complexity over time; a publisher who +cares uses constant bitrate. Subtitle files are small free swarms and +reveal the dialogue to anyone; price them if that matters. + +## 8. Events + +| Type | Emitted by | +| --- | --- | +| `com.logicsrc.openswarm.ipvideo.title.published.v1` | publisher | +| `com.logicsrc.openswarm.ipvideo.transcode.completed.v1` | c0mpute worker | + +## 9. Implementations + +| Piece | Exists | Where | What is new | +| --- | --- | --- | --- | +| Browser WebTorrent player, MSE-compatible fMP4 output, HLS fallback, range seeking | Yes | `media-streamer` (`media-player-modal.tsx`, `transcoding.ts`, `/api/stream/hls`) | Piece decryption before MSE; the title record instead of `vod_titles` | +| Hardware ffmpeg transcode job with codec ladder | Yes | `c0mpute` (`c0mpute-transcode`, `TranscodeSpec`) | CMAF output flags and the segment index in the result | +| VOD catalogue with per-title price and pass window | Yes (central) | `media-streamer` `vod_titles`, `vod_providers` | The title record and `ipfile` pricing replace both tables | +| Thumbnail VTT, chapters, subtitle tracks | Partly | hls.js and video.js conventions | The record fields | +| Title record, segment index file, HLS bridge from records | No | | All of it | + +## 10. Conformance + +A player fetches by segment per §4.1, switches at boundaries per §4.2, +and vouchers for discarded bytes per §4.3. A publisher emits aligned +ladders with keyframe-aligned segments. A gateway implementing §5 renders +byte-range HLS and applies `ippay` §7. + +## 11. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. | diff --git a/docs/openswarm/security.md b/docs/openswarm/security.md new file mode 100644 index 0000000..697c8cd --- /dev/null +++ b/docs/openswarm/security.md @@ -0,0 +1,133 @@ +# OpenSwarm Security Model + +Version: **0.1** (draft) + +This document collects the security claims the member documents make and +states plainly what the family does not protect. + +## 1. What is protected, by what + +| Asset | Protection | Where | +| --- | --- | --- | +| File contents | AES-256-CTR under a random content key; integrity from two SHA-256 merkle trees pinned by a signed manifest | ipfile §4 | +| Content key in transit | X25519 sealed box to the recipient's box key | ipfile §3.2 | +| Content key at rest for the owner | Sealed to the file box key; recoverable from seed plus manifest | core §4.3 | +| Manifest authenticity | Ed25519 by the file key and the publisher key, with a domain prefix | core §3.4 | +| Latest-manifest pointer | BEP 44 signature by the file key, `seq` monotonic | core §7.2 | +| Catalogue integrity | Per-entry signature plus hash chain; head on the DHT | ipdb §3 | +| Passes | Ed25519 by the hub; payer bound by a request signature | ippay §3 | +| Vouchers | Ed25519 by the payer; cumulative; `usd` recomputed by the payee | ippay §4 | +| Live segments | Same cipher; root in a signed head per unit; relays verify before forwarding | iplive §3 | +| Name to key | Registry pin (SHA-256 of SPKI) or DNS TXT, plus a signed record | ipname §4 | +| Peer identity on a connection | `hello` signature over infohash and both peer ids | ipfile §5.1 | +| Link confidentiality (optional) | MTP/1: X25519 + ML-KEM-768, ML-DSA-65, ChaCha20-Poly1305 | core §6.5 | +| Events | Standard Webhooks HMAC-SHA256 with timestamp tolerance | core §8 | + +## 2. Key material + +| Key | Type | Lives | Backs up as | +| --- | --- | --- | --- | +| Publisher seed | 32 bytes | Keystore or OpenCreds `key` item | The one thing to back up | +| File seed (derived) | HKDF from publisher seed and `fileId` | Never stored | Recomputed | +| File seed (standalone) | 32 random bytes | Keystore | Its own OpenCreds item | +| Content key | 32 random bytes | Keystore, `keys.owner`, keepers, grantees | `keys.owner` | +| Box keys | X25519 from the seeds | Derived | Recomputed | +| Hub signing key | Ed25519 | Hub HSM | Hub's problem | +| Payer EVM key | secp256k1 | `X402_PRIVATE_KEY`, wallet | Wallet | +| Node identity | libp2p Ed25519 | `~/.config/c0mpute/identity.key` | c0mpute's problem | + +## 3. Threats and answers + +**A public tracker or DHT node wants to know what a swarm is.** It gets an +infohash, a size, a piece length, a peer count, and a file named by a hex +string. The manifest is not on the DHT; the pointer to it is signed and +opaque. With `swarm.private` it gets nothing from the DHT at all. + +**A peer wants bytes without paying.** It gets one credit window per +identity per session, then a choke. With `policy.vanilla: "ciphertext"` it +can get all the ciphertext free, by design, and cannot read it. + +**A seeder wants to be paid for bytes it did not serve.** It cannot; the +voucher is signed by the payer for bytes the payer verified. Its only lever +is to stop serving after being paid for what it already sent. + +**A payer signs vouchers past its cap to many seeders.** Each seeder was +exposed for one window. The hub pays in redemption order until the cap +and refuses the rest; the payer's standing records it and future passes +are refused. + +**A payer or seeder replays a voucher.** It is cumulative and per triple; +replay pays zero. + +**A hub key is stolen.** Free passes until manifests drop the hub. The hub +record SHOULD carry a successor key and publishers SHOULD list two hubs. + +**A hub does not pay out.** It is a trusted processor; the answer is to +list a different hub. Receipts are signed, so non-payment is provable. + +**The publisher seed is stolen.** Every derived file key is compromised: +the thief can revise manifests, change payout addresses and delegate +keepers. Detection is a manifest revision the owner did not make, visible +in the `ipdb` log. Remedy is publisher rotation (core §4.5) from a machine +the thief does not control; `successor` links require the new key too, so +the thief cannot forge the rotation. + +**The publisher seed is lost.** Core §4.7. Nothing is unreadable; nothing +can be changed. + +**A grantee leaks the content key.** Anyone with the key and the ciphertext +reads the file. The family prices access and does not prevent copying. A +leaked key is answered by re-encryption, which protects only future +readers. This is the same position every streaming service is in once the +frame is on a screen, stated rather than hidden behind a DRM promise. + +**A relay forwards a bad segment.** Every downstream peer verifies the root +against a signed head and drops the relay. + +**A Sybil of relays surrounds an origin.** They pull, earn nothing (nobody +pays them), and cost the origin upload up to `maxDownstream`. Pins and +hired relays are how an origin buys reach it controls. + +**A registry hands out a wrong pin.** The record fails the pin check; +first-seen pinning in the resolver warns on change. + +**An EIP-3009 proof is presented for a different resource.** Known +limitation of x402 v2: the struct cannot bind a resource. The hub records +nonce to resource at verify time and refuses reuse (ippay §3.2). + +**A gateway is asked to serve plaintext to a bare `<video>` tag.** It serves +one credit window against the pass and then 402s; a gateway configured to +trust the pass for the whole title is taking the whole title as its +exposure, and the configuration says so. + +## 4. What is deliberately visible + +- File size, piece length, both infohashes, tracker list, peer count. +- Manifest `name` and `meta`, to anyone who obtains the manifest (public + visibility) or any pass holder (pass visibility). +- The existence and sequence length of a feed; the content of a public + feed. +- Segment sizes and timing of a live stream, to relays. +- Who paid whom how much, to the hub. +- The publisher's payout address, in the signed manifest. + +## 5. Defaults an implementation MUST ship + +| Setting | Default | Reason | +| --- | --- | --- | +| `policy.creditBytes` | 4 MiB (`ipfile`), 8 MiB (`iplive`) | Bounded exposure, no choke at the live edge | +| `policy.voucherBytes` | 1 MiB, 2 MiB | Four vouchers per window | +| `policy.vanilla` | `ciphertext` | Wider swarm at zero confidentiality cost | +| `visibility` | `public` | A catalogue that cannot be read is not a catalogue | +| Pass lifetime | 30 days | Long enough to stream a release, short enough to bound a stolen key | +| Key epoch (`iplive`) | 900 segments | 30 minutes at 2 s | +| Resolver cache | 300 s positive, 5 s negative | Matches moshpit-proxy | +| Keystore mode | 0600 | As every other Profullstack keystore | + +## 6. Cryptographic agility + +`cipher.alg`, `sigs[].alg`, `kdf` labels with `:v1` suffixes, and +extension names are all versioned data. A new cipher is a new `alg` value +and a new IV label; a new signature is a new `alg`; a breaking wire change +is a new extension name. Nothing is edited in place, for the reason +OpenCreds gives: peers that shipped the old value are still running. diff --git a/docs/openswarm/spec.md b/docs/openswarm/spec.md new file mode 100644 index 0000000..1a89cf7 --- /dev/null +++ b/docs/openswarm/spec.md @@ -0,0 +1,510 @@ +# The OpenSwarm Core Specification + +Version: **0.1** (draft) +Status: draft. Nothing below is implemented yet. Wire formats are expected to +change before 1.0, and this document says where. + +The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT and MAY are to be +interpreted as described in RFC 2119. + +## 1. Scope + +The core defines what every member protocol shares: + +1. **Records** (§3): the JSON shape, canonicalisation, identifiers and + signatures of every object in the family. +2. **Keys** (§4): publisher identities, derived per-file keys, box keys, and + the rules for reuse, rotation and loss. +3. **Hashing and chunking** (§5): what a content identifier is and why it is + a BitTorrent v2 merkle root. +4. **Transports** (§6): the BitTorrent wire, the extension protocol, browser + peers, webseeds, and the optional Moshpit tunnel. +5. **Discovery** (§7): DHT, trackers, Moshpit names. +6. **Events** (§8): the CloudEvents envelope and webhook signing. +7. **Versioning and conformance** (§9, §10). + +Member protocols (`ipfile`, `ippay`, `ipdb`, `ipaudio`, `ipvideo`, `iplive`, +`ipname`) add record types and extension messages. They MUST NOT redefine +anything in this document. + +## 2. Terminology + +| Term | Meaning | +| --- | --- | +| **record** | A signed JSON object with an `openswarm` version and a `type`. | +| **record id** | `sha256:` plus the hex SHA-256 of the record's canonical bytes without `sigs`. | +| **publisher** | The Ed25519 identity that owns files and feeds. | +| **file key** | A per-file Ed25519 pair derived from the publisher key. Its public half is the file's identity. | +| **box key** | An X25519 pair used to receive sealed content keys. Every identity has one. | +| **content key** | 32 random bytes that encrypt one file version's pieces. | +| **swarm** | One BitTorrent swarm carrying one ciphertext payload. | +| **piece** | A BEP 52 piece: a power-of-two length, at least 16 KiB. | +| **block** | 16 KiB, the unit of a BitTorrent `request` and of a v2 merkle leaf. | +| **peer** | Any participant in a swarm. A **seeder** has all pieces, a **leecher** wants some, a **relay** forwards live segments. | +| **gateway** | An HTTP server that serves ciphertext by range (a BEP 19 webseed) and answers catalogue queries. | +| **hub** | The settlement service that sells passes and redeems vouchers. See `ippay`. | +| **pass** | A hub-signed record that lets a peer spend up to a cap on named swarms. | +| **voucher** | A payer-signed cumulative IOU for bytes received from one payee. | +| **grant** | A record carrying a content key sealed to one recipient's box key. | +| **feed** | An `ipdb` log owned by one key. | + +## 3. Records + +### 3.1 Shape + +Every object in the family is a record: + +```json +{ + "openswarm": "0.1", + "type": "ipfile.manifest", + "createdAt": "2026-09-05T18:00:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +| Field | Type | Rule | +| --- | --- | --- | +| `openswarm` | string | Family version the record was written under. `"0.1"`. | +| `type` | string | `<protocol>.<noun>`, lower case. Registered types are listed by each member document. | +| `createdAt` | string | RFC 3339, UTC, millisecond precision. | +| `sigs` | array | One or more signatures (§3.4). MAY be empty only for record types a member document marks as unsigned. | + +Every other field is defined by the member document for that `type`. An +implementation reading an unknown field MUST preserve it on round trip. + +### 3.2 Canonical form + +The canonical bytes of a record are the RFC 8785 (JSON Canonicalization +Scheme) serialisation of the record with the `sigs` member removed. JCS +sorts keys, strips whitespace and fixes number formatting. Amounts and hashes +are strings in this family precisely so that JCS never has to touch a float. + +### 3.3 Record id + +``` +id = "sha256:" + lowercase_hex(SHA-256(canonical bytes)) +``` + +The id is never stored inside the record. A reader computes it. Two records +with the same canonical bytes are the same record, which is what makes `ipdb` +replication idempotent. + +### 3.4 Signatures + +A signature entry is `{ "alg", "key", "sig" }`. The signed message is: + +``` +"openswarm:sig:v1:" + type + "\n" + canonical bytes +``` + +The prefix is domain separation. A signature over an `ipfile.manifest` can +not be replayed as a signature over an `ippay.voucher` with the same body. + +| `alg` | `key` prefix | Signature | Status | +| --- | --- | --- | --- | +| `ed25519` | `ed25519:` | 64 bytes, base64 | REQUIRED | +| `mldsa65` | `mldsa65:` | 3309 bytes, base64 | OPTIONAL, for post-quantum ownership | + +A record MUST carry at least one `ed25519` signature by the key the member +document names as its signer. It MAY carry an `mldsa65` signature by a key +the same identity has bound (§4.6). A verifier that does not implement +`mldsa65` MUST ignore those entries and MUST NOT treat their presence as an +error. + +Key strings are `<alg>:<hex>`. Ed25519 and X25519 public keys are 32 bytes, +so 64 hex characters. ML-DSA-65 public keys are 1952 bytes. + +### 3.5 Amounts and times + +Money is a decimal string with exactly six fractional digits, in USD: +`"0.002500"`. Six digits is USDC's precision and is what CoinPay's x402 +dialect already carries. Integer arithmetic is specified where rounding +matters (`ippay` §4). Times are RFC 3339 UTC. Byte counts are JSON integers +and MUST stay below 2^53. + +## 4. Keys + +### 4.1 Publisher identity + +A publisher identity is a 32-byte seed. From it: + +``` +publisher signing key = Ed25519 keypair from seed +publisher box key = X25519 keypair from HKDF-SHA256(seed, salt = "", info = "openswarm:box:v1") +``` + +The identity string is `ed25519:<hex of the signing public key>`. The box +public key is published in the records that need it (§4.4). + +`ip init` creates one seed. It is reused for every file, feed and channel the +publisher adds. The seed SHOULD be stored as an OpenCreds `key` item with +`keyType: "openswarm-seed"`; it is the one thing in this family that cannot +be regenerated. + +### 4.2 Per-file keys are derived, and reusable by default + +Adding a file mints a key pair for that file. By default the pair is derived, +not random: + +``` +fileId = 16 random bytes, recorded in the manifest as hex +fileSeed = HKDF-SHA256(ikm = publisher seed, salt = fileId, info = "openswarm:file:v1") +file key = Ed25519 keypair from fileSeed +file box = X25519 keypair from HKDF-SHA256(fileSeed, salt = "", info = "openswarm:box:v1") +``` + +Consequences, all intended: + +- Holding the publisher seed and any manifest (which carries `fileId`) is + enough to recover that file's private key. There is no key database to back + up beyond the seed. +- Two files never share a key. Compromise of one file seed does not expose + another, because HKDF output is one-way from the seed. +- The public half is stable across revisions of the file. A re-encode keeps + the same `fileId`, so it keeps the same identity and the same BEP 46 + pointer (§7.2). + +`ip file add --standalone-key` mints a random `fileSeed` instead. The manifest +records `"keyDerivation": "standalone"` so tooling knows the seed must be +backed up on its own. Use it for a file whose ownership will be transferred: +handing over a standalone seed transfers exactly one file. + +### 4.3 What each half does + +| Half | Used for | +| --- | --- | +| File public key | The file's identity in every reference (`"file": "ed25519:..."`), the BEP 44 key under which the latest manifest is announced, the key peers verify manifests against. | +| File private key | Signs every manifest revision. Signs `ipfile.grant` records that delegate the content key to a keeper. Signs payout changes: the `pricing.payTo` a hub pays a publisher's share to is whatever the latest signed manifest says. | +| File box key | Receives the owner's own sealed copy of the content key (`keys.owner` in the manifest), so the content key is recoverable from the manifest plus the seed. | +| Content key | Encrypts the pieces. Random, 32 bytes, per file version. Never signs anything. | + +The file key is not a wallet. It authorises where money goes; the money goes +to an EVM address the manifest names. A seeder's payout address is likewise +its own, registered at the hub against the seeder's identity key (`ippay` +§6). Nothing in this family derives a wallet from a signing key. + +### 4.4 Box keys and sealing + +Any content key that travels does so inside a sealed box: + +``` +sealed = X25519 anonymous sealed box (libsodium crypto_box_seal) to the recipient's box public key +``` + +This is the construction the OpenCreds `team` profile already uses to wrap a +vault key to a member, and it is chosen for the same reason: a seeder can hand +a key to a peer it has never met, given only that peer's box public key from +the `hello` message. When the link is already an MTP/1 session (§6.5) the +box is still applied; the transport protects the link, the box protects the +key at rest in a `grant` record. + +### 4.5 Rotation + +**Publisher rotation.** Mint a new seed. For each file, publish a manifest +revision signed by the old file key (derived from the old seed) that names +the new `publisher`, and that carries a new `fileId` and a new file key +derived from the new seed in `successor`. Readers that already hold the old +file key follow `successor` once and re-pin. The old seed is kept only to +derive old file keys during the transition. + +**File rotation.** Same mechanism for one file: a final revision under the +old file key with `successor` set. A reader MUST NOT follow a `successor` +chain more than 8 deep, and MUST refuse a successor whose own latest manifest +does not name the predecessor in `predecessor`. Both links are required so a +compromised old key cannot redirect a file to an attacker's swarm; the +attacker would also need to sign as the new key. + +**Content key rotation.** Re-encrypt under a fresh content key and publish a +new revision with a new `cipherRoot` and new swarm. Old grants stop working +for the new swarm. Rotation is the only way to revoke a key that has been +delivered; a delivered key is held by the peer, and the spec does not pretend +otherwise. + +### 4.6 Post-quantum binding + +An identity MAY bind an ML-DSA-65 key by publishing an `openswarm.binding` +record signed by both keys: + +```json +{ + "openswarm": "0.1", + "type": "openswarm.binding", + "subject": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "pq": "mldsa65:0f3a51c0a3e6d1e2f4c99b3d3b6a2c5a9f1e8d7c6b5a4f3e2d1c0b9a8f7e6d5c", + "createdAt": "2026-09-05T18:00:00.000Z", + "sigs": [ + { + "alg": "ed25519", + "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", + "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" + } + ] +} +``` + +(`pq` is truncated above for readability; a real value is 1952 bytes of +hex, and the record carries a second entry with `"alg": "mldsa65"` whose +3309-byte signature is omitted here for the same reason.) +This matches the Moshpit registry's `mtp` pin kind, which is an ML-DSA-65 +identity. A verifier that trusts the binding accepts an `mldsa65` signature +from `pq` as a signature by `subject`. + +### 4.7 Loss + +If the publisher seed is lost and no standalone seeds exist: + +- Every file's ciphertext stays servable. Seeders hold pieces, not keys. +- Every grant already delivered keeps working. +- No manifest can be revised: prices, payout address, trackers and + `successor` are frozen at the last signed revision. +- Payouts continue to the frozen `pricing.payTo`. +- No new keeper can be delegated. Existing keepers keep granting. + +That is the failure mode, stated so that it can be planned for. Store the +seed in a vault. + +## 5. Hashing and chunking + +### 5.1 The content identifier + +The identifier of a file's bytes is its **BEP 52 pieces root**: the SHA-256 +merkle root over 16 KiB leaves, padded per BEP 52. It is written +`sha256:<64 hex>`. + +This is chosen over a flat SHA-256 or an IPFS CID because a swarm peer can +verify a single 16 KiB block against it with a merkle proof, which is what +lets a leecher sign a voucher the moment a block arrives. A flat hash would +make every payment wait for the whole file. A CID would be a second tree that +no BitTorrent client checks. + +An `ipfile` manifest carries two roots: `plainRoot` over the plaintext and +`cipherRoot` over the ciphertext. The swarm verifies against `cipherRoot` +through the torrent's own hashes; the client verifies its decrypted output +against `plainRoot`. + +c0mpute's chunk store addresses objects by blake3 (`c0mpute://blake3:<hex>`). +That stays internal to the node. A c0mpute storage node maps piece index to +blake3 chunk however it likes; what it puts on the wire is verified by SHA-256. + +### 5.2 Piece length + +A piece length MUST be a power of two and at least 16 KiB (BEP 52). Members +recommend defaults: `ipfile` 1 MiB for files over 64 MiB and 256 KiB below; +`ipaudio` 256 KiB; `ipvideo` 1 MiB; `iplive` one segment per piece. + +### 5.3 Hybrid torrents + +A swarm's info dictionary MUST be a hybrid v1/v2 torrent per BEP 52: it +carries both `pieces` (SHA-1, v1) and `file tree` with `piece layers` +(SHA-256, v2). Two infohashes result: + +- `infohashV1`: SHA-1 of the info dictionary, 20 bytes. This is what + WebTorrent in a browser uses, and what today's trackers and the bitmagnet + crawl behind bittorrented.com/dht record. +- `infohashV2`: SHA-256 of the info dictionary, truncated to 20 bytes for + DHT and tracker use, full 32 bytes in records. + +The info dictionary MUST NOT contain plaintext file names. `name` is the +ciphertext root in hex. A crawler that fetches metadata over BEP 9 learns a +size and a hash. + +## 6. Transports + +### 6.1 The wire + +Peer connections are BitTorrent (BEP 3) over TCP, uTP, or WebRTC data +channels (the WebTorrent dialect). Every standard message keeps its meaning. +OpenSwarm never changes `choke`, `unchoke`, `have`, `request`, `piece` or the +bitfield. That is what makes a vanilla client a valid, if unpaid, participant. + +### 6.2 Extension messages + +All family messages ride on the extension protocol (BEP 10). The extension +handshake advertises: + +``` +{ "m": { "ipfile": 20, "ipdb": 21, "iplive": 22, "ut_metadata": 3, "ut_pex": 1 }, + "v": "ip/0.1", + "openswarm": "0.1" } +``` + +The numbers are local ids chosen by the sender, as BEP 10 requires. The +names are fixed: `ipfile`, `ipdb`, `iplive`. Message payloads are bencoded +dictionaries with a string key `t` naming the message. Records embedded in a +message are carried as their UTF-8 JSON bytes under key `r`, so the bencode +layer never has to understand JSON. + +A peer that did not advertise a name MUST NOT be sent that extension's +messages. A peer receiving a message for an extension it advertised but a `t` +it does not know MUST ignore it. + +### 6.3 Browser peers + +WebTorrent supports BEP 10 extensions via `wire.use()`, and the four wss +trackers bittorrented.com's player already announces to +(`wss://tracker.webtorrent.dev`, `wss://tracker.openwebtorrent.com` are the +two that handshake today) find WebRTC peers. A browser peer signs vouchers +with WebCrypto Ed25519 and seals with X25519 from the same API. Nothing in +this family requires a native binary in a browser. + +A browser can only reach WebRTC peers. A swarm that wants browser reach MUST +have at least one hybrid peer (a native seeder that also announces to a wss +tracker), which is what media-streamer's `StreamingService` already is. + +### 6.4 Webseeds and gateways + +A gateway is a BEP 19 webseed: it serves ciphertext by HTTP range under the +URL the manifest lists. A gateway MAY require a pass (`Authorization: Bearer` +with the pass's `token`) and MAY require vouchers in a request header +(`ippay` §7). A gateway that requires neither is a free webseed and is how a +publisher pays for its own distribution instead of charging for it. + +### 6.5 Moshpit tunnel + +Between two native peers that both hold Moshpit identities, the BitTorrent +connection MAY be tunnelled through an MTP/1 session +(`@profullstack/moshpit-transport`: X25519 + ML-KEM-768 key agreement, +ML-DSA-65 server identity, ChaCha20-Poly1305 records, TCP). The `hello` +message carries the peer's MTP pin (`base64(SHA-256(SPKI))`) so a peer can +reconnect over the tunnel after meeting on the plain wire. MTP/1 has no +client authentication; ownership on the tunnel is still proved by `hello` +signatures (`ipfile` §5.1), not by the transport. + +MTP/1 is TCP only and 64 KiB per record. It is not available to browsers and +it does not replace the swarm wire. It is the confidentiality option for +node-to-node links, and it is the transport the hub API SHOULD be reachable +over in addition to HTTPS. + +## 7. Discovery + +### 7.1 Peers + +A swarm is found the way any torrent is: DHT `get_peers` on `infohashV1` and +on the truncated `infohashV2` (BEP 5), tracker announces (BEP 15, HTTP, and +wss for browsers), PEX (BEP 11), LSD. A manifest lists the trackers it wants. +A manifest MAY set `swarm.private` to true (BEP 27), which disables DHT and +PEX and leaves only trackers and webseeds; that is the choice for a swarm +whose existence should not be public. + +### 7.2 Latest manifest by file key + +The latest manifest for a file is announced as a BEP 44 mutable item: + +``` +k = file public key (32 bytes) +salt = "ipfile" +seq = manifest rev +v = bencoded { "ih": <infohashV1, 20 bytes>, "ih2": <infohashV2, 32 bytes>, "m": <manifest record id, 32 bytes> } +``` + +`v` is 100 bytes, well inside the 1000 byte BEP 44 limit. The item is signed +by the file key, which is why the file key is Ed25519 and why it is the file's +identity. This is BEP 46 with an extra pointer: the magnet form is +`magnet:?xs=urn:btpk:<file key hex>&s=ipfile`. + +The manifest itself is fetched from any peer in the swarm over the `ipfile` +`manifest` message, from a gateway at `/swarm/<infohashV2>/manifest`, or from +an `ipdb` replica. + +### 7.3 Feeds and names + +An `ipdb` feed head is a BEP 44 mutable item under the feed key with salt +`ipdb:<feed name>`. A Moshpit name resolves to a publisher key and a gateway +through `ipname`. A client that starts from a name, a feed, a file key or an +infohash reaches the same manifest. + +## 8. Events + +### 8.1 Envelope + +Every event is a CloudEvents 1.0 envelope, as `@profullstack/autoblog` +already emits: + +```json +{ + "specversion": "1.0", + "id": "7f2b5a7e-2b0e-4c8f-9a4d-3d6c0b9c1e10", + "type": "com.logicsrc.openswarm.ipfile.voucher.redeemed.v1", + "source": "https://coinpayportal.com/api/openswarm", + "subject": "sha256:41d10f45e705e0526c9eeedf62b32dda3daaf552dd9b6a7b09e01776b05813bb", + "time": "2026-09-05T18:02:11.000Z", + "datacontenttype": "application/json", + "data": { + "record": { "openswarm": "0.1", "type": "ippay.voucher" } + } +} +``` + +`type` is `com.logicsrc.openswarm.<protocol>.<noun>.<verb>.v1`. `subject` is +the record id the event is about. `data.record` is the full record where the +event carries one. Each member document registers its types. + +### 8.2 Delivery + +Webhooks are Standard Webhooks as autoblog signs them: `POST` with +`content-type: application/cloudevents+json`, headers `webhook-id` (the +event id), `webhook-timestamp` (unix seconds) and `webhook-signature` +(`v1,<base64 HMAC-SHA256>` over `<id>.<timestamp>.<body>`), a five minute +tolerance window, retries at 0, 10 and 60 seconds on network error, 5xx, 408 +and 429. Receivers verify with a constant-time compare. + +## 9. Versioning + +- `openswarm` on a record is the family version. A 0.x reader MUST accept + any 0.x record and MUST preserve unknown fields. +- Each member document has its own version history. Adding a record type, + a message `t` or a field is additive and does not bump the family version. +- Wire changes that break an existing message get a new `t` name. A message + name is never redefined, for the same reason an OpenCreds label is never + edited: peers that shipped it are still running. +- Extension names in the BEP 10 handshake (`ipfile`, `ipdb`, `iplive`) are + permanent. A breaking revision would ship as `ipfile2`. + +## 10. Conformance + +A conforming implementation of the core: + +1. Reads and writes records per §3, computes ids and verifies `ed25519` + signatures with the domain prefix. +2. Derives file keys and box keys per §4 from a seed, and produces a manifest + that a second implementation given the same seed and `fileId` can sign + identically. +3. Computes BEP 52 pieces roots and builds hybrid torrents per §5. +4. Speaks BEP 10 and ignores unknown `t` values per §6.2. +5. Publishes and reads the BEP 44 pointer per §7.2. +6. Emits events per §8 where it emits events at all. + +Member conformance is defined per document; the checklist is collected in +[conformance.md](./conformance.md). + +## 11. Registered record types + +| Type | Signer | Document | +| --- | --- | --- | +| `openswarm.binding` | subject and pq | this document §4.6 | +| `ipfile.manifest` | file key (and publisher key) | ipfile | +| `ipfile.grant` | grantor | ipfile | +| `ippay.hub` | hub key | ippay | +| `ippay.pass` | hub key | ippay | +| `ippay.voucher` | payer key | ippay | +| `ippay.receipt` | hub key | ippay | +| `ipdb.entry` | feed key | ipdb | +| `ipdb.head` | feed key | ipdb | +| `ipdb.tombstone` | feed key | ipdb | +| `ipaudio.release`, `ipaudio.track` | publisher key | ipaudio | +| `ipvideo.title` | publisher key | ipvideo | +| `iplive.channel`, `iplive.head` | channel key | iplive | +| `ipname.pin` | publisher key | ipname | + +## 12. Version history + +| Version | Date | Change | +| --- | --- | --- | +| 0.1 | 2026-09-05 | Initial draft. Records, derived file keys, BEP 52 identifiers, BEP 10 extensions, BEP 44 pointers, CloudEvents. | diff --git a/packages/logicsrc-mcp/src/standards.test.ts b/packages/logicsrc-mcp/src/standards.test.ts index 50340e8..14ce0a8 100644 --- a/packages/logicsrc-mcp/src/standards.test.ts +++ b/packages/logicsrc-mcp/src/standards.test.ts @@ -203,7 +203,7 @@ describe("MCP: OpenPRD", () => { it("reports the next free id and the allowed lifecycle moves", async () => { const client = await connect(); // Asserted against the live prd/ directory, so this advances with every PRD added. - expect(toolText(await client.callTool({ name: "prd_next_id", arguments: {} }))).toBe("0005"); + expect(toolText(await client.callTool({ name: "prd_next_id", arguments: {} }))).toBe("0006"); const moves = await client.callTool({ name: "prd_next_statuses", arguments: { ref: "0001" } }); const payload = JSON.parse(toolText(moves)) as { status: string; allowedNext: string[] }; diff --git a/prd/0005-add-logicsrc-openswarm-spec.md b/prd/0005-add-logicsrc-openswarm-spec.md new file mode 100644 index 0000000..4ca0a49 --- /dev/null +++ b/prd/0005-add-logicsrc-openswarm-spec.md @@ -0,0 +1,184 @@ +--- +openprd: "0.2" +id: "0005" +title: "Add the LogicSRC OpenSwarm specification family" +status: Draft +authors: + - anthony@profullstack.com +created: 2026-09-05 +updated: 2026-09-05 +repo: profullstack/logicsrc +discussion: +implementation: +tags: + - openswarm + - ipfile + - ipdb + - ipaudio + - ipvideo + - iplive + - bittorrent + - x402 + - c0mpute +supersedes: +superseded-by: +--- + +## Problem + +BitTorrent distributes bytes better than any CDN and has never been a +product, for two reasons. Nobody is paid to seed, so swarms die when the +uploader loses interest. Nothing in it is private, so a publisher who wants +to charge for access cannot use it at all. + +Every Profullstack media property has answered that by putting a central +HTTP proxy in front of the bytes and bolting a pass system onto it: +bittorrented.com's seedbox, IPTV and radio rails, tipoffwatch and +genrewatch's shared playlists, p0dcasters' episodes. Each one re-implements +pay-per-pass grants, session caps and manifest sealing, and each one pays +for every byte it serves. The swarm the site is named after cannot carry +the payment, so it is not used for the thing that costs money. + +c0mpute.com has the other half: nodes that already store, transcode, +gateway and (by design) relay live segments, paid through a job auction. +They have no way to earn from serving media once a job is done, and no +protocol that tells them what to seed. + +What is missing is an open, versioned description of a paid, encrypted +swarm: how a file is encrypted so the tracker learns nothing, how a peer +that serves a verified piece is paid for it, how the key reaches a peer +that paid, and how a catalogue of such swarms is replicated without a +server. And the same primitives, reused, for audio, video and live. + +## Goals + +- A publisher adds a file once and it stays available as long as anyone is + paid to seed it, on c0mpute nodes or on strangers' machines. +- A seeder is paid per verified byte served, with exposure bounded to a + small credit window, and a leecher never pays for a byte it did not + verify. +- A public tracker, DHT node or crawler learns an infohash and a size, not + a title. +- One key model: a publisher seed from which every file, feed and channel + key derives and can be recovered; a fresh unrelated key only on request. +- Vanilla BitTorrent clients remain valid swarm members and browsers remain + first-class peers (WebTorrent, WebCrypto, MSE). +- Payment reuses x402 and CoinPay exactly as `@profullstack/x402-gateway` + sells crawl passes today; events reuse CloudEvents and Standard Webhooks + as `@profullstack/autoblog` emits them; post-quantum links reuse + `moshpit-transport`; names reuse the Moshpit registry. +- Every building block (chunking, manifests, keys, encryption, payment, + relay) is specified once and reused by `ipaudio`, `ipvideo`, `iplive` + and c0mpute; reuse is the default posture. +- c0mpute nodes have concrete workload types to bid on and a per-byte + income path that needs no auction. + +## Non-Goals + +- Not a new transport. The wire is BEP 3, the extension is BEP 10, hashing + is BEP 52, discovery is BEP 5, 44 and 46. +- Not a token or a chain. Passes are bought once in USDC; vouchers are + off-chain and redeemed at a hub; payouts are batched. +- Not trustless settlement. A hub is trusted as a payment processor is. +- Not DRM. A paying peer holds the key; the spec says so. +- Not a player, a recommender, or a moderation system. +- Not an implementation. This PRD adds specifications only. + +## Users + +- **A publisher** (label, studio, podcaster, vendor, company backing up) + who wants distribution paid for by the people who want the bytes. +- **A seeder or relay operator**, including every c0mpute node, who wants + to be paid for bandwidth and disk. +- **A listener or viewer** in a browser or an app, who wants to pay once + and stream without a server deciding whether they may. +- **An agent** with an x402 wallet that buys a dataset unattended. +- **An implementer** who wants to interoperate from the published + documents without reading Profullstack source. + +## Requirements + +- R1 [P0] Define the core: signed JSON records with JCS canonical form and + SHA-256 ids, Ed25519 signatures with domain prefixes, optional ML-DSA-65, + derived per-file, per-feed and per-channel keys from one seed, BEP 52 + identifiers, BEP 10 extension names, BEP 44 pointers, CloudEvents. +- R2 [P0] Define `ipfile`: the manifest, the grant, AES-256-CTR piece + encryption with two merkle roots, the `hello` and payment messages, the + credit window and voucher interval, key delivery by peer and by URL, + vanilla coexistence, publishing and retrieval procedures. +- R3 [P0] Define `ippay`: passes bought over x402 with CoinPay's v2 offer + and proof, payer binding, cumulative vouchers with exact integer value + arithmetic, the hub record and API, receipts and split arithmetic, payee + registration, standing, payout, HTTP presentation headers. +- R4 [P0] Define `ipdb`: signed hash-chained feeds, entries with put and + delete, heads on the DHT, replication by `ipfile` segments and by gossip, + fork detection, the query shape and result, playlists and ratings. +- R5 [P0] Define `ipaudio`: track and release records, renditions as + `ipfile`s, seek index, aligned anchors, gapless trim, royalty split, the + gateway bridge and RSS. +- R6 [P0] Define `ipvideo`: title record, CMAF renditions with segment + index, subtitles, thumbnails, chapters, MSE playback, byte-range HLS + bridge, transcode on c0mpute. +- R7 [P0] Define `iplive`: channel and head records, key epochs, the + `iplive` extension with `window` backpressure, roles and relay + economics, low-latency parts, recording to `ipvideo`. +- R8 [P1] Define `ipname`: the name record, the `openswarm` registry pin + kind, the DNS TXT form, clearnet-first, the `ip://` URL. +- R9 [P1] Define the c0mpute integration: node identity and payee + registration, seven workload types with `JobOffer` shapes, at least ten + use cases with CLI flows, the list of proposed changes to the node. +- R10 [P1] Propose the `ip` CLI as a contract: command tree, exit codes, + output fields, environment. +- R11 [P1] Publish conformance profiles and checklists, a security model, + and an FAQ. +- R12 [P1] Publish the family at `logicsrc.com/openswarm` with the same + site registration as OpenCreds (nav, docs registry, sitemap, landing). +- R13 [P2] Ship JSON Schemas under `@logicsrc/schemas` and a reference + implementation with the fixture suite described in conformance.md. + Deferred to a later PRD; this one is documents only. + +## UX Notes + +Adding a file is one command and prints everything a person needs to hand +out: the file key, both infohashes, a magnet link and an `ip://` URL. The +seed is created once and reused; the CLI never prints it without +`--reveal`. + +Fetching is one command that buys or reuses a pass, joins, pays, decrypts +and verifies, and reports what it paid and who granted the key. A pass +scoped to a publisher rather than a file is what a player wants, so a +listener buys once per label, not per track. + +Failure modes are named: `no-pass`, `pass-cap`, `unpaid`, `not-keeper`, +`voucher-stale`. A choke is explained by a `credit` message, not by +silence. + +## Success Metrics + +- A swarm seeded by one implementation, paid for by a leecher from a + second, with vouchers redeemed at a third party's hub, decrypts to + `plainRoot`. +- bittorrented.com's browser player streams an `ipaudio` track from peers + with no proxy in the path and the publisher's share arrives at the + address in the manifest. +- A c0mpute node with no jobs earns voucher income from a swarm it chose + to pin. +- Every spec document has zero em dashes and every JSON example is valid. + +## Risks & Open Questions + +- **Custodial hub.** CoinPay's x402 rail pays merchants directly and + collects no fee. A hub holds balances and pays out; that is a different + regulatory position and a product decision. +- **EIP-3009 cannot bind a resource.** Mitigated at the hub by recording + nonce to resource; inherent to x402 v2. +- **Key copying.** A grantee can leak the key. The family prices access + and does not prevent copying; re-encryption is the only remedy. +- **The Moshpit registry has no `openswarm` pin kind.** One-line change on + the registry side; until then `ipname` works over DNS TXT only. +- **Name.** The working name was IPMedia. OpenSwarm is proposed for the + reasons in the overview; the `ip*` protocol names stay either way. +- **Seed loss** freezes every derived file. Mitigated by storing the seed + in OpenCreds and by standalone keys for files that change hands. +- **Spec before code.** Every wire format here is unimplemented. The + fixture suite and the interop test are how this gets honest. diff --git a/prd/README.md b/prd/README.md index 057fda1..4ef71fd 100644 --- a/prd/README.md +++ b/prd/README.md @@ -15,3 +15,4 @@ Status lives in each file's front-matter and is the source of truth: | [0002](./0002-hourly-hire-us-rate.md) | Move Hire Us pricing from a weekly retainer to an hourly rate | Accepted | pricing, site, billing | | [0003](./0003-add-logicsrc-opencontext-spec.md) | Add the LogicSRC OpenContext specification | Draft | opencontext, context, agents, permissions, provenance, schemas | | [0004](./0004-add-logicsrc-opencreds-spec.md) | Add the LogicSRC OpenCreds specification | Draft | opencreds, credentials, vault, encryption, portability, schemas | +| [0005](./0005-add-logicsrc-openswarm-spec.md) | Add the LogicSRC OpenSwarm specification family | Draft | openswarm, ipfile, ipdb, ipaudio, ipvideo, iplive, bittorrent, x402, c0mpute |