Skip to content

Engine: implement draft 2 (WireGuard tunnel, pluggable carriers, awp tunnel) - #30

Open
kylemclaren wants to merge 5 commits into
spec-draft-2from
draft-2-engine
Open

kylemclaren wants to merge 5 commits into
spec-draft-2from
draft-2-engine

Conversation

@kylemclaren

Copy link
Copy Markdown
Contributor

Stacked on #28 (the spec). Implements draft 2 across the reference implementation: engine, daemon, CLI, MCP, conformance runner, schema, plugin skill. No compatibility with draft 1 (protocol v0).

What changes

  • tunnel package (new, public): the connection layer.
    • Tailscale's wireguard-go keyed by the identity (Ed25519 → X25519, SPEC §4.2 vectors tested), a gVisor TCP/IP stack in userspace, AWP on port 1.
    • A conn.Bind over four carriers. udp. unix. ws, with a cloudflare option that runs cloudflared for a quick tunnel. tailcat, as a plain carrier: WireGuard datagrams length-prefixed on a tailcat stream.
    • Roaming via the fork's InitiationAwareEndpoint/PeerAwareEndpoint hooks. Paths prefer direct over relayed and fail over when one goes quiet. Carriers never block WireGuard.
    • awp1 addresses (CBOR, deterministic). Pre-shared keys become per pair once two peers meet, so awp address --rotate shuts out strangers only.
  • Node: no auth. Hello carries grants and addr, and hello.key must be the tunnel's key or the connection gets err auth. Admission refusals use the new refused code. Introductions hand on the merged awp1 address.
  • Schema v1 (schema/v1/, $id …/schema/v1/awp.schema.json). Every SPEC.md example validates, and every address in SPEC.md decodes.
  • awp conform dials through a tunnel of its own. The new hello-key and hello-own-key scenarios replace auth-bad-sig, and --listen takes a carrier.
  • awp tunnel (new): the helper the Python and TypeScript SDKs run. It holds the SDK's key, prints the address, dials on request over a control socket, and forwards accepted streams. It rejects a hello naming another key before the SDK sees it.
  • CLI/daemon: --listen takes carriers, and awp status shows the address, its endpoints and pending carriers. awp connect right after a bye no longer returns the closing connection.
  • Removed: the transport package, AWP_ALLOW_PLAINTEXT/tcp:, and the draft 1 single-file Python peer with its interop scripts. The Python SDK carries cross-implementation conformance now.

Also on #28: spec corrections found while implementing it (pair pre-shared keys, hello.addr and introduce.peer.address as addresses, tailcat and connection carrier framing, the helper as built).

Testing

  • go test -race ./... green. The Go node passes its own conformance suite in both modes.
  • make carriers runs the live carriers, and both pass here. Tailcat goes through the public DERP relays. Cloudflare goes through a real trycloudflare.com quick tunnel with no account. CI runs these in a non-blocking job.
  • End to end with the built binary:
    • Three daemons: A on tailcat and UDP, B on Cloudflare, C on a Unix socket.
    • B reached A with both paths offered and used UDP.
    • C reached B only through the Cloudflare quick tunnel.
    • B introduced C to A, and C connected by name.
    • A rotated its pre-shared key, and B still reconnected.
    • awp conform against A's live daemon passed 16 of 16 in about 4 seconds.
  • Found and fixed along the way:
    • Handshake responses were fanned out to every path.
    • Timestamp replays came from forced rehandshakes.
    • A slow relay blocked sends.
    • The responder never recorded a pair's pre-shared key.

Next: the SDK PRs (Go, Python, TypeScript) pin this branch's commit.

A new tunnel package is the connection layer: Tailscale's wireguard-go
keyed by the identity (Ed25519 converted to X25519), a gVisor TCP/IP
stack in userspace with AWP on port 1, and a conn.Bind over four
carriers (udp, unix, ws with a Cloudflare quick tunnel option, tailcat)
that roams with the peer. Addresses are awp1 CBOR blobs of key,
pre-shared key and endpoints; pre-shared keys become per pair once two
peers have met.

The node drops auth: hello carries grants and addr, and hello.key must
be the tunnel's key. The schema moves to v1, the conformance runner
dials through its own tunnel with hello-key scenarios, and `awp tunnel`
is the helper SDKs in other languages run for the tunnel. The transport
package, the plaintext binding and the draft 1 Python peer are gone.
…nitiation

With a listener on two carriers, three problems showed up under
conformance against a live daemon on tailcat and udp:

- A slow connection carrier (tailcat's rate-limited DERP relay) blocked
  WireGuard's send path. Connection carriers now have a bounded send
  queue that drops like a full UDP socket.
- The current path flipped to whichever carrier spoke last. It now moves
  to a less preferred kind (unix, udp, ws, tailcat in that order) only
  once the current one has been quiet for three seconds.
- A handshake response went down every path, and forcing a handshake on
  every dial replayed WireGuard's coarse timestamp. Responses now go
  back on the path the initiation came from, and a dial rides the
  existing session first, forcing a new handshake only if that goes
  nowhere for three seconds.

Tests: SPEC.md's addresses decode, the tunnel IP vector, repeated dials
over two paths, and failover when the preferred path dies mid-stream.
Tunnel.Listen adds a carrier after Start. The helper's control socket
takes {"listen": spec}, answered once the carrier is up, and
{"rotate": true}, so an SDK can start the helper once and listen when
its user asks.
A forwarded stream starts with one line of the helper's own naming the
peer's X25519 key, and a dial reply carries it too, so the SDK can pick
the grants to present in its hello before the peer's hello arrives.
A restarted peer can find its Unix socket or UDP port still held for a
moment by the process it replaces (an SDK's helper exits when its stdin
closes). Listen retries for up to three seconds before failing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant