fix(dwn): port the enbox replication-feed Messages wire protocol - #270
Open
LiranCohen wants to merge 2 commits into
Open
fix(dwn): port the enbox replication-feed Messages wire protocol#270LiranCohen wants to merge 2 commits into
LiranCohen wants to merge 2 commits into
Conversation
Add the Messages interface (Query/Read/Subscribe) to the Go DWN client,
matching the enbox dwn-sdk-js wire shapes at HEAD:
- MessagesFilter with exact wire keys (interface, method, protocol,
protocolPathPrefix, contextIdPrefix, messageTimestamp{from,to}), empty
filters rejected, filters field always present on the wire.
- Plural permissionGrantIds invocation: canonical form is deduplicated and
sorted by UTF-16 code-unit order (JavaScript string order, NOT UTF-8 byte
order — astral characters sort before U+E000..U+FFFF); the same canonical
list is mirrored in descriptor and signature payload
{descriptorCid, permissionGrantIds}. Ambiguous/empty/not-canonical error
semantics mirror the SDK. The singular permissionGrantId path for
Records/Protocols is untouched; Messages ops never use
authorDelegatedGrant.
- Client.MessagesQuery with typed reply (entries incl. seq/messageCid/
isLatestBaseState/protocol/encodedData, ProgressToken cursor, drained,
fingerprint), 410 replies surfaced as ProgressGapError (now exported and
shared with the subscription machinery), 429 mapped to ErrRateLimited.
- Client.MessagesRead with the dwn-response header/body data split, 404
mapped to ErrMessagesEntryNotFound.
- SubscriptionManager.SubscribeMessages[WithLifecycle]: MessagesSubscribe
descriptor (filters + permissionGrantIds + cursor) over the existing
rpc.subscribe.dwn.processMessage / rpc.ack / rpc.subscribe.close
plumbing; subscribe-reply head + fingerprint surfaced on the established
lifecycle event; terminal error frames with code
MessagesSubscribeDeliveryAuthorizationFailed or
MessagesSubscribeDeliveryFailed wrap the new
ErrSubscriptionAuthorizationRevoked sentinel so callers can react to
grant revocation/expiry specifically.
- Shared permission scope matcher ported exactly from the SDK
(PermissionScopeMatcher.matches + boundary-aware matchesSubtree,
fail-closed combinations).
- Feed-grant selection (SelectFeedGrants) per the agent rules: active
Messages/Read grants only, full scope needs an unscoped grant,
protocol-set scope needs protocol-root coverage per protocol, invoked
set is all participating grants sorted by ID; typed
FeedGrantCoverageError names the uncovered protocol.
- Link identity hashing: ComputeProjectionID
(replication-log-feed-v1), ComputeOwner/DelegateAuthorizationEpoch
(messages-read-grants-v1) as base64url-no-pad SHA-256 over canonical
alphabetical-key JSON without HTML escaping, and BuildLinkKey joined
with '^'.
Refs #266
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… tests
Golden fixtures under internal/dwn/testdata were generated by executing the
ACTUAL enbox TypeScript implementation (dwn-sdk-js and agent sources at
~/src/enboxorg/enbox, main) with bun, using a fixed Ed25519 key (seed
0x01..0x20) and fixed message timestamps; each fixture file records the
generation command in its generatedBy field.
Parity assertions:
- Fully-signed MessagesQuery/MessagesRead/MessagesSubscribe fixtures: the Go
builders produce byte-identical descriptors and signature payloads and the
same descriptor CIDs; Go's dag-cbor CID discipline reproduces the TS
message CID over each TS-signed message. (Full-JWS bytes are not compared:
the SDK serializes the protected header as {"kid","alg"} while meshd emits
{"alg","kid"} — both valid, the server never compares header bytes.)
- permissionGrantIds canonicalization vectors incl. UTF-16 vs UTF-8 ordering
edge (astral vs U+FFFD) and empty/ambiguous error codes.
- projectionId/authorizationEpoch/linkKey vectors incl. unicode protocol
URIs, HTML-escaping hazards (& < >), and optional dateGranted omission.
Contract tests mirror the existing fake-server idiom: Messages subscribe
handshake (canonical grant IDs in descriptor AND payload, cursor in the
signed descriptor, reply head/fingerprint on the established lifecycle
event) and terminal delivery error-code mapping to
ErrSubscriptionAuthorizationRevoked. Client tests cover query reply parsing
(drained/fingerprint/cidsOnly), the 410 ProgressGap shape, rate limiting,
and the MessagesRead dwn-response data split. Table tests mirror the SDK's
scope matcher behavior and the agent's feed-grant selection rules.
Refs #266
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #266
What changed
enbox replaced its sync architecture: the speculative B1/B2 projection-root sync (
MessagesSync, merkle StateIndex) was deleted upstream (enbox #996/#1035) in favor of replication-log feed sync. This PR ports the read-side wire protocol to meshd's Go DWN client so the daemon can speak it:internal/dwn/messages.go):MessagesQuery(paged feed —filters[],cursorProgressToken,limit,cidsOnly; replyentries{seq, messageCid, isLatestBaseState, protocol, message?, encodedData?}+cursor/drained/fingerprint, 410ProgressGap),MessagesRead(by CID, data via thedwn-responseheader/body split), andMessagesSubscribethrough the existing WebSocket subscription manager — including the new replyhead+fingerprintsnapshot and EOSE/cursor-resume semantics.permissionGrantIds: canonical (deduped, UTF-16 code-unit sorted) grant lists mirrored in both descriptor and signature payload, with the upstream error semantics (ambiguous / empty / non-canonical). Records/Protocols keep the singular path untouched.permission_scope.go): protocol exact-match;protocolPathandcontextIdboth boundary-aware subtree matches; fail-closed on invalid combinations.SelectFeedGrants): activeMessages.Readgrants only, full scope requires an unscoped grant, protocol-set scope requires protocol-root coverage per protocol, typed error naming any uncovered protocol.link_identity.go):ComputeProjectionID(replication-log-feed-v1) and authorization-epoch hashing (messages-read-grants-v1) with byte-exact canonical JSON, plusBuildLinkKey.MessagesSubscribeDeliveryAuthorizationFailedmaps to an exportedErrSubscriptionAuthorizationRevokedsentinel so callers can react to grant revocation/expiry specifically.Out of scope by design:
dwn.applyReplicatedMessage/push replication and local fingerprint folding — meshd is a remote-reading client with no local DWN store.Parity testing
Golden fixtures in
internal/dwn/testdata/were generated by executing the actual TypeScript implementation (bun scripts importing enbox sources; fixed Ed25519 seed): fully-signed MessagesQuery/Read/Subscribe messages with byte-identical descriptors + signature payloads and identical CIDs, projection-ID/epoch hash vectors (incl. unicode edge cases that pinned UTF-16 vs bytewise ordering), and grant-ID canonicalization vectors. The independent review regenerated all fixtures from TS and confirmed byte-identical output. Existing Records subscribe contract/gap/hardening tests pass unchanged.Verification
go build ./...— cleango vet ./...— cleango test ./... -count=1 -race— all packages pass, no data races🤖 Generated with Claude Code