Repository navigation
mhda 1.2.0: derivation index range, stricter grammar and validation - #2
Merged
Merged
Conversation
The CIP-11 charge and the CIP-1852 role accept any level index, but ChargeType was uint8, so the parsed value was truncated to a byte: m/1852'/1815'/0'/256/0 parsed as role 0, re-serialised as m/1852'/1815'/0'/0/0 and returned role 0 in Levels(). Two URNs named the same key. ChargeType is now uint32 and keeps the full value in Charge(), Levels() and String(). Start CHANGELOG.md with the 1.2.0 entry.
A BIP-32 child number keeps the hardened flag in its top bit: n' is child number 2^31+n, so a level index must stay below 2^31. ParsePath accepted any 32-bit value, and a consumer computing hardened ? 0x80000000|index : index got an alias: m/44'/60'/2147483648'/0/0 derived the key of m/44'/60'/0'/0/0, and an unhardened .../0/2147483648 asked for a hardened child number through the public-key formula. Every level of every derivation type is now parsed with a 31-bit bound and refused with ErrInvalidDerivationPath otherwise, hardened or not. The ct component is not a path level and keeps its 32-bit range. Leading zeros are unchanged and now documented: a variable level accepts them and String() drops them; a fixed level (purpose, fixed coin, 0|1 charge) is a regex literal and refuses them. testdata/dp_conformance.txt is a table shared verbatim with the C++ port: every row is parsed standalone and inside a URN, so a path one implementation refuses, the other refuses too. The fuzz seed that pinned 4294967295 now sits at the new bound, and FuzzDerivationPath checks that no accepted level exceeds 2^31-1. SPEC.md and CHANGELOG.md describe the rule and the URNs that no longer parse.
The network type, the chain id and the derivation type are written verbatim into every URN and chain key, but nothing checked them when they were set programmatically. A chain id such as 1:dt:bip44:dp:m/44'/60'/0'/0/666 on a root address produced a URN that re-parsed as a bip44 path, so the text named a different key than the object. NewChain now requires a registered network type and a non-empty chain id of printable ASCII without ':', '?' or '#' (ASCII-trimmed, the rule of the free-form setters) and panics otherwise, like NewAddress. Chain.SetNetworkType and Chain.SetChainId return the error and keep the old value. NewDerivationPath and NewDerivationPathFromLevels panic with ErrInvalidDerivationType on an unregistered type. An Address whose path has no type serialises like a root address instead of emitting empty dt/dp components.
SetDerivationPath returned early for a root address, so a dp with no dt (which parses as root) or with dt:root was dropped silently: urn:mhda:nt:evm:ci:1:dp:m/44'/60'/0'/0/0 parsed as urn:mhda:nt:evm:ci:1, naming the root key instead of the path it spells out. ParsePath already refused a non-empty path for ROOT; the address setter now does the same and returns ErrInvalidDerivationPath. An empty dp under root stays a no-op. testdata/urn_conformance.txt is a URN-level table shared verbatim with the C++ port, run through ParseURN / ParseNSS; this change adds its first rows.
parseNSS skipped an unknown token on its own, so the token after it was read as a key: a value could become a key and a known key could become a value. A key with the wrong case was skipped with it, dropping the component silently. ...:ci:1:DT:bip44:DP:m/44'/60'/0'/0/5 parsed as the root address urn:mhda:nt:evm:ci:1, and ...:ci:1:ext:wi:dt:bip44:... took dt as the wallet id. The NSS is now split into pairs: an odd number of tokens is refused, an unknown component is skipped together with its value, and a key that differs from a known key only by case is refused with ErrInvalidNSS. Keys must be non-empty printable ASCII, like values. An empty NSS still has no components. ChainFromKey keeps reporting a dangling token as ErrInvalidChainKey.
ParseURN strips the RFC 8141 r/q/f components at the first '?' or '#',
but ParseNSS, ChainFromNSS and ChainFromKey took the NSS as given and
kept the byte in a value. The URN emitted from such an address was
truncated on the next parse:
ParseNSS("nt:evm:ci:1?=q:dt:bip44:dp:m/44'/60'/0'/0/0") returned a bip44
address whose String() re-parsed as the root address
urn:mhda:nt:evm:ci:1, and ChainFromKey("nt:evm:ci:1#a") was accepted.
parseNSS now refuses either byte with ErrInvalidNSS. FuzzParseNSS claimed
the idempotency contract without checking it; it now re-parses the
emitted URN and compares, with the truncating input as a seed.
BIP-32 serialises a key's depth in one byte, so a path deeper than 255 levels names a key no wallet can represent; ParsePath accepted any depth. A slip10 path with more than 255 levels is now refused with ErrInvalidDerivationPath. The shared derivation-path table gets the 255/256 boundary rows.
SPEC.md promised that a long form round-trips to itself, but an explicit dt:root is folded away: ...:ci:1:dt:root serialises as ...:ci:1. That is the intended behaviour (root is the default and has no path, so a root address keeps one canonical form and one hash, and consumers rely on it); the spec now says so, and the Algorand and TON notes no longer call dt:root the canonical form. zip32 parses, but no network registers it, so ParseURNStrict refuses it everywhere; the spec lists that as a known limitation. Tests pin both: dt:root rows in the shared URN table and a strict-refusal check over every network.
The hash tests only checked determinism and digest length. Add reference digests for str() of 55, 56, 63, 64, 119, 120, 128 and 1000 bytes (the NSS is 9 bytes shorter, so it covers 55 too), computed with Python's hashlib. The C++ port, whose SHA-1 and SHA-256 are hand-written, pins the same values, so both implementations must produce the same identifiers across every padding case.
The workflow ran plain `go test` on push only. It now also runs on pull requests with read-only permissions, vets and tests ./... on every Go version of the matrix (plus the current stable release), and a check job on stable verifies gofmt, runs the race detector and fuzzes each target for 30 seconds. checkout and setup-go move to v4 and v5.
ParsePath updated the fields of the path in place. Re-parsing a 4-level ZIP-32 path with a 3-level one kept the old index (m/32'/133'/1' read back as m/32'/133'/1'/5), a SLIP-10 path kept the coin, account and charge of a BIP-44 path parsed before it, and an error midway left a half-updated path. SetDerivationType changed the type and kept the old path: a bip44 address switched to zip32 serialised as dt:zip32:dp:m/32'/133'/3'/9, a valid URN naming a key nobody gave. ParsePath now parses into a fresh path and replaces the old one only on success. A new derivation type drops the old path; until a path is set, String() of the path is empty, the URN carries dt without dp (so it does not parse) and Validate / MarshalText return ErrInvalidDerivationPath. Address.SetDerivation(dt, dp) sets both at once and changes nothing on error; parseAddress uses it.
NewDerivationPathFromLevels kept the levels it was given while String() printed the type's template: a BIP-44 path built from 49'/60'/0/0/0 printed m/44'/60'/0'/0/0 but returned purpose 49' and an unhardened account from Levels(), which is what a wallet derives from. Too few levels or an index of 2^31 gave a URN that does not parse, and NewDerivationPath accepted a charge of 7 or an account of 2^31. Both constructors now return the path the parser gives back for their own String() and panic with ErrInvalidDerivationPath unless it has exactly the requested levels. Levels() returns a copy instead of the internal slice, and NewAddress keeps its own copies of the chain and the path, so addresses built from one path no longer change together. The fuzz targets check that a re-parsed path has the same levels.
Validate checked the algorithm, the format and the derivation type each against the network on its own. It accepted SLIP-10 ed25519 paths with soft levels (ed25519 has no soft derivation), bip44 with ed25519 on XRPL, NEAR and Aptos, Sui bip54 with ed25519 or secp256r1 (the purpose encodes the scheme), a Bitcoin bip84 path with af:p2pkh, and Cosmos bip44 with coin 118', which is the cip11 path under a second name. Validate now also refuses, with ErrIncompatible: an ed25519 path with an unhardened level, except cip1852 (BIP32-Ed25519); a derivation type with another curve than its own where the network binds them (Sui: slip10 ed25519, bip54 secp256k1, bip74 secp256r1; Aptos and NEAR: slip10 ed25519, bip44 secp256k1); an explicit Bitcoin format that does not match the purpose; and Cosmos bip44 with coin 118'. A Bitcoin URN without af stays valid, and SPEC.md no longer says strict mode needs it. The shared URN table gains strict rows for each rule. Tests that pinned now-refused pairs move to matching purposes (p2sh under bip49, p2wsh under bip32), and the XRPL ed25519 example becomes the root form its family-seed keys have.
String(), NSS(), the hashes, Algorithm() and Format() dereferenced the chain, so the zero Address or NewAddress(nil, ...) panicked, and SetDerivationPath dereferenced the path, so an address built without one panicked too. A missing chain now reads as the zero chain (urn:mhda:nt::ci:, the C++ port's default address) and a missing path as root; SetCoinType without a chain returns ErrUninitializedAddress.
MarshalText had a pointer receiver, so an Address held by value (a
struct field, a map value) was encoded by encoding/json and encoding/xml
as a plain struct: {"A":{}}, which then failed to decode. SPEC.md
promises the URN form for these codecs.
MarshalText now has a value receiver. The codecs still encode a nil
*Address themselves (JSON null); a direct MarshalText call on a nil
pointer now panics like any value method, so TestMarshalTextNilAddress
keeps only its zero-value case.
AlgorithmFromString, FormatFromString and NetworkTypeFromString returned plain errors.New values, so errors.Is could not discriminate them, although doc.go promises sentinel errors throughout and DerivationTypeFromString already wrapped ErrInvalidDerivationType. They now wrap ErrInvalidAlgorithm, ErrInvalidFormat and ErrInvalidNetworkType.
normalize and the Address setters used strings.TrimSpace and strings.ToLower, which are Unicode-aware: SetAddressAlgorithm accepted "secp256\u212a1" (Kelvin sign) as secp256k1, and NBSP or ideographic spaces around a derivation type, path, format or coin type were trimmed away. The URN parser and the C++ port refuse those bytes, so the two entry points and the two implementations disagreed. normalize now trims ASCII whitespace and folds ASCII case only, and the setters, parseChain and the case-insensitive URN prefix check use it.
parseNSS trimmed every value, so urn:mhda:nt:evm:ci: 1 :dt: bip44 :... parsed like the same URN without the spaces, although SPEC section 6.1 says values contain no whitespace and only the whole URN is trimmed. parseNSS no longer trims; whitespace in a key or value fails the charset check with ErrInvalidNSS. ParseURN trims the URN and the whitespace left before a stripped r/q/f component (ci:0 #frag still parses); ParseNSS, ChainFromNSS and ChainFromKey trim the NSS they are given as a whole.
Keys and values accepted any printable ASCII, so '"', '<', '>', '\',
'^', '`', '{', '|', '}', '[', ']' and a raw '%' passed and were emitted in
URNs that RFC 8141 does not allow, although SPEC section 1.5 requires the NSS
production (RFC 3986 pchar and "/").
One predicate, nssByte, now admits letters, digits and -._~!$&'()*+,;=@/
- pchar and "/" without ':' (the separator) and '%' (percent-encoding is
not supported, and %41 would be a second spelling of "A"). The parser
uses it for keys and values (ErrInvalidNSS), and validateFreeFormValue
for ap/as/wt/wi and the chain id (ErrInvalidValue). The shared URN table
gets a row per refused byte.
ci:1, ci:0x1 and ci:01 are three chain keys and hashes for one chain. MHDA keeps the chain id opaque; SPEC.md now says so and asks a producer to keep to one spelling (the EIP-155 decimal form is the usual one). The 1.2.0 changelog had grown entry by entry; it is regrouped into Changed (inputs refused now), Fixed and Documentation, with the intro pointing at the changes.
Two test inputs held the Kelvin sign, a no-break space and an ideographic space as literal characters, which look like a K and plain spaces in the source. They are now written as \u escapes; the bytes the tests use are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Version 1.2.0. URNs and values accepted by 1.1 are refused now; the full list is in CHANGELOG.md (new in this release).
What changes
ErrInvalidDerivationPath). Such an index collided with the hardened bit:m/44'/60'/2147483648'/0/0derived the key ofm/44'/60'/0'/0/0. Thectcomponent keeps its 32-bit range.ChargeTypeis 32 bits wide (wasuint8): the CIP-11 charge and the CIP-1852 role were truncated, so role 256 named the role-0 key.key:valuepairs (an unknown component is skipped with its value; a mis-cased key is refused); adpwithoutdtor underdt:rootis refused;?/#are refused in an NSS parsed on its own; no whitespace inside the NSS; keys and values use the RFC 3986 byte set; a SLIP-10 path has at most 255 levels.ParseURNStrict/Validate. ed25519 paths must be fully hardened (exceptcip1852); Sui, Aptos and NEAR bind each derivation type to its curve; an explicit Bitcoin format must match the purpose (a URN withoutafstays valid); Cosmosbip44with coin 118' is refused in favour ofcip11.NewChainpanics on an invalid network type or chain id (a:in a chain id injected components);Chain.SetNetworkType/SetChainIdnow return an error; constructed paths must be ones the parser would produce, soLevels()andString()always agree;NewAddresskeeps copies of its chain and path;Levels()returns a copy.ParsePathreplaces the whole path or nothing; a new derivation type drops the old path;Address.SetDerivation(dt, dp)sets both atomically.Address;MarshalTexthas a value receiver, so anAddressheld by value encodes as its URN; theFromStringhelpers wrap their sentinels; setters normalise ASCII only.dt:rootfolding, ZIP-32 without a network and the opaque EVM chain id. CI vets, checks gofmt, runs the race detector and fuzzes each target.Testing
go vet, gofmt,go test -raceclean; the three fuzz targets ran 60 s each with new invariants (re-parse idempotency ofParseNSS,Levels()equal after a round trip, no level index above 2^31-1).testdata/dp_conformance.txtandtestdata/urn_conformance.txt, are shared verbatim with the C++ port and run by both suites.ParseURN,ParseURNStrict,ParseNSSandChainFromKey, and for 600 000 derivation paths.go vetandgo teston its own.