#20081 describes an issue with the original hook point of disco public key advertisement over TSMP. In summary, TSMP disco key advertisement cannot be hooked in magicsock around path discovery, otherwise a packet scheduling loop is closed between magicsock and wireguard-go, leading to WireGuard sessions that never go idle in the absence of user-driven overlay traffic.
In response to #20081 we are moving to an advertisement model that is fixed around WireGuard session establishment. Handshake initiator sends when receiving handshake response, and responder sends when receiving first forward transport from initiator (likely the disco key), and sets its current keypair. wireguard-go changes supporting said model were implemented in tailscale/wireguard-go#80.
This issue tracks implementing the callback.
The second part of eventually consistent disco public key propagation is handling inbound advertisements. Currently there is a bias towards TSMP-learned keys, with some conditions to consider a control-learned key active around loose online/offline signals (from control's perspective). This is not eventually consistent, and can get into a wedged state, i.e. we can elect a disco key that is not current, and never recover.
We plan to move to an advertisement handling algorithm that is eventually consistent, which this issue also tracks. The algorithm is as follows:
- TSMP and control-learned disco keys are both held in memory when their values diverge
- When TSMP and control-learned disco key values are equal, storage is collapsed such that we only pay the cost of a nil pointer for the "2nd" slot
- We bias towards TSMP-learned keys at advertisement reception time. Control-learned keys are never set as active at advertisement reception time if there is a TSMP-learned key that differs.
- If we receive a disco message from a public key (which is in the disco message header) that is in the "inactive" slot, we flip the election
Implementation of this algorithm must consider DoS vectors.
Worth noting, lamport clocks were considered as an alternative, but we cannot guarantee durable/reliable storage on both ends, which would be required to persist timestamps around restarts. We also steered strongly away from any design leaning on assumption of synchronized clocks.
#20081 describes an issue with the original hook point of disco public key advertisement over TSMP. In summary, TSMP disco key advertisement cannot be hooked in magicsock around path discovery, otherwise a packet scheduling loop is closed between magicsock and wireguard-go, leading to WireGuard sessions that never go idle in the absence of user-driven overlay traffic.
In response to #20081 we are moving to an advertisement model that is fixed around WireGuard session establishment. Handshake initiator sends when receiving handshake response, and responder sends when receiving first forward transport from initiator (likely the disco key), and sets its current keypair. wireguard-go changes supporting said model were implemented in tailscale/wireguard-go#80.
This issue tracks implementing the callback.
The second part of eventually consistent disco public key propagation is handling inbound advertisements. Currently there is a bias towards TSMP-learned keys, with some conditions to consider a control-learned key active around loose online/offline signals (from control's perspective). This is not eventually consistent, and can get into a wedged state, i.e. we can elect a disco key that is not current, and never recover.
We plan to move to an advertisement handling algorithm that is eventually consistent, which this issue also tracks. The algorithm is as follows:
Implementation of this algorithm must consider DoS vectors.
Worth noting, lamport clocks were considered as an alternative, but we cannot guarantee durable/reliable storage on both ends, which would be required to persist timestamps around restarts. We also steered strongly away from any design leaning on assumption of synchronized clocks.