feat: keel mesh makes a full mesh, keeps one identity, finds its endpoint

[marcos-mendez]

Fixes the four problems a test of keel 0.17.0 found on three Keel Web nodes across two sites (decision 0048), plus the findings of the security review. keel 0.18.0. Not for merging before the security review.

1. A full mesh, on admission evidence (review: HIGH 1)

• Learning peers. The new node takes the join answer's peers it can verify. The inviter announces the confirmed join over the overlay. keel mesh sync, run at boot and every 15 min by keel-mesh-sync.timer, pulls rosters. A fallback join pulls from the inviter.
• Evidence. Each node has an Ed25519 signing key, made on the machine with openssl genpkey and kept in /var/lib/keel/mesh/node.key (0600). The inviter signs the admission: mesh identity, invite id, the new node's WireGuard and signing keys, its address and endpoint, and the time. The new node receives that evidence and the inviter's signing key in the HMAC-authenticated join (or confirm) answer.
  • The node's peer entry in every roster carries its evidence.
  • A member takes an entry only if its evidence verifies against a key it already trusts: its own, its inviter's, keys of members admitted the same way (chains are followed), or a root.
  • Entries without valid evidence are ignored.
• Bootstrap. For the hand-built web-1/web-2, keel mesh create --adopt or keel mesh sync --adopt (an explicit operator action) makes the spec's peers trust roots.
  • A root's signing key is bound only from a roster that pull fetched from the root's own overlay address, never from an announcement (2nd review: MEDIUM 1).
  • Evidence must name an invite, so no member, root or not, vouches for a node it did not admit (MEDIUM 2).
  • docs/mesh.md states plainly that any trusted member can vouch for a new node, and what a compromised member can then do.
• Applying changes. Everything a sync or the pending announcements bring is applied in one change under one 0018 window. It is confirmed by a handshake from a newly added key.
• Where trust lives. The trust store is /var/lib/keel/mesh/trust.json, not the spec: 0048 says there is no mesh section in the spec.
• docs/mesh.md "Admission evidence and trust" states the change to the threat model. The 0048 amendment is a separate handbook PR, text only.

2. Sandbox (HIGH 2) and limits (MEDIUM 3)

The design follows keel#75:

• The root helper keel-mesh-members starts keel-mesh-members-listen with DynamicUser, no capabilities and LISTENER_PROPERTIES.
• The two talk over a 0600 socket in a 0700 runtime directory. The helper checks SO_PEERCRED against the unit's MainPID.
• The listener serves TCP 51821 on the overlay address, with every socket bound to wg0. Its limits:
  • a 15 s deadline per request and a 5 s timeout per read;
  • 4 connection slots, 2 per source;
  • 8 KiB of headers and 256 KiB of body;
  • a per-source block after 5 refusals in a minute.

3. Queue (MEDIUM 4)

The queue keeps the latest announcement per sender, with at most 64 senders; a full queue answers 503. One worker applies everything pending in one window.

4. Tombstones (MEDIUM 5; 2nd review: MEDIUM 3, 4, LOW 5)

keel mesh remove KEY|ADDRESS removes a node from the mesh, as 0048 says:

1. It signs a tombstone.
2. It drops the peer under the 0018 window. A handshake from a peer it keeps confirms the change; with no peer left, the route check alone does.
3. It sends its roster to the others.

Each member takes the tombstone only if the signer is the node's admitter, a root, or the node itself. It then drops the peer in one window. If that change is not confirmed, the spec is put back and the next sync retries.

Tombstones are kept forever, at most 1024, and at most 64 per signer. Rosters carry them all, and sync never re-adds a tombstoned key.

Signing-key rotation is documented as follow-up work in handbook#44.

5. Identity (LOW 6, item 2)

Only a join or --adopt sets the identity. Sync without an identity is refused. invite refuses on a split, and when the node has peers but no identity.

Repair: first keel mesh create --adopt on web-1, then keel mesh sync --adopt <web-1> on web-2 and on web-3. The timers then converge.

6. Endpoint (LOW 7, item 3)

The endpoint is the uplink's stable global IPv6 address, plus a public IPv4. ULA, deprecated, temporary and wg* addresses are excluded; the order follows keel-core's banner picker, reimplemented in keel because the picker is shell in keel-core.

• With only a privacy or RFC 1918/CGNAT address, invite refuses and asks for --endpoint.
• join sends no endpoint in that case (the behind-NAT path), and says why.

7. Join message (item 4)

The join prints "confirming over the overlay…". Apply's revert warning is shown only if the node's own confirmation fails.

Test plan

• TDD at every seam:
  • real openssl signatures in test_mesh_signing and test_mesh_trust (chains, forgeries, untrusted signers, roots, tombstones);
  • the listener limits in test_mesh_memberlink;
  • the root helper and the listener in one process in test_mesh_memberd;
  • test_mesh_sync, test_mesh_adopt and test_mesh_remove.
• keel/mesh at 100% line and branch coverage (full suite in a trixie container).
• test_mesh_netns: A announces C to B, and B verifies the evidence A signed and confirms C. C pulls B from A, and B and C ping each other. The members listener runs under setpriv with CapEff 0. It passes on clean links and at 250 ms ±25 ms with 2% loss, three runs in a row.
• Re-run after the second review: 100% coverage, and the netns test passes three times in a row on clean links and at 250 ms ±25 ms with 2% loss.
• CI.