feat: keel mesh create, the invite listener, join, accept and status (0048)

[marcos-mendez]

The second part of handbook decision 0048, "joining the mesh with one command", without etcd. The first part is keel#72.

Seams (written down before the tests)

1. The HTTP handler as a function, on both sides of the privilege split. Root side: keel.mesh.admit.Admitter.forward(path, signature, body, local, peer) -> Response (tests/test_mesh_admit.py). Unprivileged front: keel.mesh.listener.Listener.handle(method, path, signature, body, local, peer) (tests/test_mesh_listener.py). bound()/run() only put the front behind TLS (tests/test_mesh_channel.py, real TLS on the loopback). keel.mesh.bridge joins the two over a unix socket (tests/test_mesh_bridge.py, the whole path in one process).
2. The CLI. keel.cli.main → thin handlers in keel.mesh.commands (root check, token or line read, live Node built) → the flows joining.run, inviting.serve_invite, inviting.accept, create.create, status.lines, each taking its outside world as fields (post, sleep, route_dev, serve, run, clock) (tests/test_mesh_cli*.py, test_mesh_joining.py, test_mesh_inviting.py, test_mesh_create_status.py).
3. The spec writes through keel's writer. keel.mesh.node.Node: read the spec as keel reads it, validate the whole document, write it with keel.network.marker.write_private (temp file, fsync, rename, 0600), apply with apply_system(..., skip_uplink=True, defer_certificate=True), read back the marker apply armed, and confirm through keel.network.confirm. The flow tests use real Nodes on scratch roots, with only apply replaced by one that arms the marker, so the real confirmation logic decides.

What it does

• keel mesh create: random ULA /64, this node at ::1, identity, apply under the 0018 window, then self-confirmation (session.SELF) after the route check (gateways plus the operator's SSH client).
• keel mesh invite now starts keel-mesh-invite@<id> (systemd-run --collect -p RuntimeMaxSec=…, running keel mesh serve <id>), opens the port only through keel-firewall's new mesh_invites set (nft add element … timeout, expiry + 180 s like the helper) and otherwise says the fallback applies. One pending invite per port. It prints keel mesh join - <<< keel1:…, so the token is never in argv.
• Privilege split.
  • The root helper keel-mesh-invite@<id> (keel mesh serve) binds /run/keel/mesh/<id>.sock and starts keel-mesh-listen@<id> (keel mesh listen) with DynamicUser=yes, CapabilityBoundingSet=/AmbientCapabilities= empty, NoNewPrivileges, PrivateTmp, PrivateDevices, ProtectSystem=strict, ProtectHome, the kernel/clock/hostname/cgroup protections, RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX, RestrictNamespaces, SystemCallFilter=@system-service, MemoryMax=64M, TasksMax=32, LimitNOFILE=64.
  • The listener binds the bridge socket (0600) in its own RuntimeDirectory /run/keel/mesh-listen-<id> (0700, the dynamic user's), accepts only root, and removes the socket once connected. The helper connects, checks SO_PEERCRED against the unit's MainPID (systemctl show -p MainPID) and not root, and skips any other peer. It sends the parameters (no key) and the cert and TLS key as two memfds over SCM_RIGHTS.
  • The HMAC key never leaves the root side. The listener refuses to run with any capability; it does TLS, request parsing, and the shape, id, time and nonce checks, then forwards the raw bytes and signature. The root Admitter verifies the HMAC, counts forgeries (5 cancel the invite and tell the listener to stop), refuses a key it already knows, consumes the invite once, writes the spec and applies.
  • The confirm rests on root-side evidence: a wg show <if> latest-handshakes handshake from the joiner's key since the join (waited for up to 10 s), plus the HMAC-verified confirm request. The listener's local/peer report is logged and decides nothing. Tests include a lying listener: a forged join with a chosen key, and a confirm with the right addresses but no handshake; both are refused.
  • accept uses the same split, bound to the overlay address.
• Listener limits: a 15 s deadline on reading a request (TLS included; the apply is not under it), 5 s per read, 4 slots plus 2 kept for the overlay address (so a confirmation is never starved), one concurrent connection per source, headers capped at 8 KiB and the body at 16 KiB, and a per-source block after 5 refusals in 60 s. Only 5 requests naming this invite and failing its HMAC cancel it (counted root-side); scanners cancel nothing.
• keel mesh join - <<< TOKEN (the line invite prints; the token as an argument still works, with a warning): route check to the endpoint (otherwise names the 0024 rendezvous point, applies nothing), pin, request, then a signed answer that must echo nonce, id, key and address. It writes its own spec, applies, opens the mesh session over the overlay, and confirms both windows (session.MESH). On no TCP answer it applies its own side (window = time left, ≤15 min) and prints keel mesh accept keel1a:…. With no endpoint of its own it refuses (both behind NAT).
• keel mesh accept LINE|-: HMAC and reserved address checked before consuming. It stops the invite's helper, adds the peer with persistent_keepalive 25, applies, and runs the same helper and unprivileged listener on its overlay address in the foreground. A peer of the same key is refused, never replaced. The confirm answer's nonce and invite id are checked, as the join answer's are.
• keel mesh status: peers, endpoints, last handshakes (wg show public-key/endpoints/latest-handshakes only), pending invites; no secret.
• keel.network.confirm: the two 0048 sources, each confirming only the overlay change keel mesh made, identified by the marker's boot id and up time. They never confirm an uplink change.

Protocol messages

Signature header X-Keel-Mesh-Signature = hex HMAC-SHA256(invite HMAC key, "keel mesh 1\n" + method + "\n" + path + "\n" + body). Answers are signed with method ANSWER. Refusals are unsigned {"error": reason}.

• POST /v1/join (uplink): invite_id, public_key, endpoint ("[addr]:port" | null), address (reserved, with length), nonce (16 B hex), time
  • 200: invite_id, nonce, public_key, address (inviter's, with length), peers [{public_key, endpoint|null, address}], etcd ("none"), window
• POST /v1/confirm (overlay): invite_id, public_key, nonce, time
  • 200: invite_id, nonce, confirmed, detail
• keel1a: fallback line (binary, base64url): flags, public key, [endpoint + port], address + length, invite id, time, HMAC-SHA256, checksum.

Where I interpreted 0048

• "The machine's TLS key" is the invite's own key and self-signed cert (0048 step 2, and what keel#72's token fingerprint already pins), not /etc/ssl of the machine.
• RuntimeMaxSec = time to expiry + 180 s (one default window + 60), so a join accepted just before the expiry can still be confirmed. Joins after the expiry are refused by the store.
• The inviter's apply is apply --system-only --skip-uplink with --defer-certificate: an unattended listener should not move the uplink or ask Let's Encrypt.
• "Ignores any address but the one it reserved": a request naming another address is refused (403), not silently rewritten.
• The answer's peers are sent, but the joiner writes only the inviter. Announcing the new node to the other members ("cloud simple grows through the inviter") is not in this PR; the summary says so.
• Spec rewrite is YAML from the loaded document: comments are dropped and the mode becomes 0600, both documented in docs/mesh.md; keel spec set (keel#69) will unify the writers.
• IPv4 overlay (ipv4_address) is not carried, as in keel#72.

Not here (follow-ups)

• confconsole "Invite a node" / "Join a mesh" screens in Keel-Linux/confconsole, calling these commands (join -), after this is merged.
• etcd (CA, member certs, third-node quorum), announcements to existing peers, keel mesh remove (listing shared secrets, no rotation), keel mesh invites / invite --cancel, the boot unit for invites that expired while down, inspect reporting pending invites.

Known edges for the security review

• After the inviter confirms, if its confirm answer is lost in transit, the joiner retries against a listener that has stopped. Its window then reverts while the inviter keeps the peer. The peer is harmless (only the joiner's key, at a /128), and keel mesh status shows it.
• Reloading keel-firewall's table empties the mesh_invites set. A reload during an invite closes its port, and the fallback covers that.
• The listener cannot bind a --port under 1024: it has no capability.

Tests

• keel/mesh at 100% line and branch coverage; whole package 99% (gate 95).
• TLS loopback: pin checked before anything is sent; forged or unsigned answers, refusals and a non-TLS client handled; a slow client is cut at the deadline; slots are full but the overlay slot still answers; oversized headers are not read; a blocked source gets no answer.
• Bridge in one process: TLS → front → unix socket with memfds → Admitter; an untrusted peer, a listener that never connects, bad messages, and a listener with capabilities are all refused.
• Manually on Debian 13 (systemd 257, sudo): the real keel-mesh-listen@ unit ran as dynamic uid 62869 with CapEff=CapBnd=0, NoNewPrivs=1, Seccomp=2. Its RuntimeDirectory was 0700 and the dynamic user's, the MainPID check trusted it, the socket was removed after the helper connected, a TLS join went through the bridge, and the following requests were refused.
• tests/test_mesh_netns.py: three network namespaces joined by veths. Real wg-quick, real TLS, the real helper/bridge/listener (the listener a separate process under setpriv with all capabilities dropped, reporting CapEff 0), join, accept and confirm logic. B joins through the listener and C through the fallback; both ping A over the overlay with no keel network confirm typed. The spent invite is refused, and A's journal holds no secret. Runs twice: on clean links, and with netem at 250 ms ±25 ms and 2% loss on every link (about 17 s; passed 5/5 locally). The join's connect timeout was raised to 10 s so that two lost SYNs don't trigger the fallback. Runs in the hosted job (sudo -n unshare -n) and in a new mesh / trixie job on keel-lxc-1 (needs the wireguard module loaded on the runner host).
• On built Core images (0048's done-when), after this and the confconsole PR.