Skip to content

mhda 1.2.0: derivation index range, stricter grammar and validation - #2

Merged
x88 merged 21 commits into
masterfrom
fix/review-followups
Oct 5, 2026
Merged

x88 merged 21 commits into
masterfrom
fix/review-followups

Conversation

@x88

@x88 x88 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Version 1.2.0. URNs and values accepted by 1.1 are refused now; the full list is in CHANGELOG.md (new in this release).

What changes

  • Derivation-path index range. A level index of 2^31 or more is refused at every level of every derivation type, hardened or not (ErrInvalidDerivationPath). Such an index collided with the hardened bit: m/44'/60'/2147483648'/0/0 derived the key of m/44'/60'/0'/0/0. The ct component keeps its 32-bit range.
  • ChargeType is 32 bits wide (was uint8): the CIP-11 charge and the CIP-1852 role were truncated, so role 256 named the role-0 key.
  • Stricter NSS grammar. The NSS is parsed as key:value pairs (an unknown component is skipped with its value; a mis-cased key is refused); a dp without dt or under dt:root is refused; ? / # are refused in an NSS parsed on its own; no whitespace inside the NSS; keys and values use the RFC 3986 byte set; a SLIP-10 path has at most 255 levels.
  • Stricter ParseURNStrict / Validate. ed25519 paths must be fully hardened (except cip1852); Sui, Aptos and NEAR bind each derivation type to its curve; an explicit Bitcoin format must match the purpose (a URN without af stays valid); Cosmos bip44 with coin 118' is refused in favour of cip11.
  • Programmatic values are validated. NewChain panics on an invalid network type or chain id (a : in a chain id injected components); Chain.SetNetworkType / SetChainId now return an error; constructed paths must be ones the parser would produce, so Levels() and String() always agree; NewAddress keeps copies of its chain and path; Levels() returns a copy.
  • No stale state. ParsePath replaces the whole path or nothing; a new derivation type drops the old path; Address.SetDerivation(dt, dp) sets both atomically.
  • Fixes. No nil dereference on a zero Address; MarshalText has a value receiver, so an Address held by value encodes as its URN; the FromString helpers wrap their sentinels; setters normalise ASCII only.
  • Docs and CI. SPEC.md describes every rule above, the dt:root folding, ZIP-32 without a network and the opaque EVM chain id. CI vets, checks gofmt, runs the race detector and fuzzes each target.

Testing

  • go vet, gofmt, go test -race clean; the three fuzz targets ran 60 s each with new invariants (re-parse idempotency of ParseNSS, Levels() equal after a round trip, no level index above 2^31-1).
  • Two conformance tables, testdata/dp_conformance.txt and testdata/urn_conformance.txt, are shared verbatim with the C++ port and run by both suites.
  • Differential run against the C++ port 1.2.0: byte-identical output for 500 000 mutated URNs through ParseURN, ParseURNStrict, ParseNSS and ChainFromKey, and for 600 000 derivation paths.
  • Every commit passes go vet and go test on its own.

x88 added 21 commits October 5, 2026 10:14
The CIP-11 charge and the CIP-1852 role accept any level index, but
ChargeType was uint8, so the parsed value was truncated to a byte:
m/1852'/1815'/0'/256/0 parsed as role 0, re-serialised as
m/1852'/1815'/0'/0/0 and returned role 0 in Levels(). Two URNs named the
same key. ChargeType is now uint32 and keeps the full value in Charge(),
Levels() and String().

Start CHANGELOG.md with the 1.2.0 entry.
A BIP-32 child number keeps the hardened flag in its top bit: n' is
child number 2^31+n, so a level index must stay below 2^31. ParsePath
accepted any 32-bit value, and a consumer computing
hardened ? 0x80000000|index : index got an alias:
m/44'/60'/2147483648'/0/0 derived the key of m/44'/60'/0'/0/0, and an
unhardened .../0/2147483648 asked for a hardened child number through
the public-key formula.

Every level of every derivation type is now parsed with a 31-bit bound
and refused with ErrInvalidDerivationPath otherwise, hardened or not.
The ct component is not a path level and keeps its 32-bit range.

Leading zeros are unchanged and now documented: a variable level accepts
them and String() drops them; a fixed level (purpose, fixed coin, 0|1
charge) is a regex literal and refuses them.

testdata/dp_conformance.txt is a table shared verbatim with the C++
port: every row is parsed standalone and inside a URN, so a path one
implementation refuses, the other refuses too. The fuzz seed that pinned
4294967295 now sits at the new bound, and FuzzDerivationPath checks that
no accepted level exceeds 2^31-1. SPEC.md and CHANGELOG.md describe the
rule and the URNs that no longer parse.
The network type, the chain id and the derivation type are written
verbatim into every URN and chain key, but nothing checked them when
they were set programmatically. A chain id such as
1:dt:bip44:dp:m/44'/60'/0'/0/666 on a root address produced a URN that
re-parsed as a bip44 path, so the text named a different key than the
object.

NewChain now requires a registered network type and a non-empty chain
id of printable ASCII without ':', '?' or '#' (ASCII-trimmed, the rule
of the free-form setters) and panics otherwise, like NewAddress.
Chain.SetNetworkType and Chain.SetChainId return the error and keep the
old value. NewDerivationPath and NewDerivationPathFromLevels panic with
ErrInvalidDerivationType on an unregistered type. An Address whose path
has no type serialises like a root address instead of emitting empty
dt/dp components.
SetDerivationPath returned early for a root address, so a dp with no dt
(which parses as root) or with dt:root was dropped silently:
urn:mhda:nt:evm:ci:1:dp:m/44'/60'/0'/0/0 parsed as urn:mhda:nt:evm:ci:1,
naming the root key instead of the path it spells out. ParsePath already
refused a non-empty path for ROOT; the address setter now does the same
and returns ErrInvalidDerivationPath. An empty dp under root stays a
no-op.

testdata/urn_conformance.txt is a URN-level table shared verbatim with
the C++ port, run through ParseURN / ParseNSS; this change adds its first
rows.
parseNSS skipped an unknown token on its own, so the token after it was
read as a key: a value could become a key and a known key could become a
value. A key with the wrong case was skipped with it, dropping the
component silently. ...:ci:1:DT:bip44:DP:m/44'/60'/0'/0/5 parsed as the
root address urn:mhda:nt:evm:ci:1, and ...:ci:1:ext:wi:dt:bip44:... took
dt as the wallet id.

The NSS is now split into pairs: an odd number of tokens is refused, an
unknown component is skipped together with its value, and a key that
differs from a known key only by case is refused with ErrInvalidNSS.
Keys must be non-empty printable ASCII, like values. An empty NSS still
has no components. ChainFromKey keeps reporting a dangling token as
ErrInvalidChainKey.
ParseURN strips the RFC 8141 r/q/f components at the first '?' or '#',
but ParseNSS, ChainFromNSS and ChainFromKey took the NSS as given and
kept the byte in a value. The URN emitted from such an address was
truncated on the next parse:
ParseNSS("nt:evm:ci:1?=q:dt:bip44:dp:m/44'/60'/0'/0/0") returned a bip44
address whose String() re-parsed as the root address
urn:mhda:nt:evm:ci:1, and ChainFromKey("nt:evm:ci:1#a") was accepted.

parseNSS now refuses either byte with ErrInvalidNSS. FuzzParseNSS claimed
the idempotency contract without checking it; it now re-parses the
emitted URN and compares, with the truncating input as a seed.
BIP-32 serialises a key's depth in one byte, so a path deeper than 255
levels names a key no wallet can represent; ParsePath accepted any
depth. A slip10 path with more than 255 levels is now refused with
ErrInvalidDerivationPath. The shared derivation-path table gets the
255/256 boundary rows.
SPEC.md promised that a long form round-trips to itself, but an explicit
dt:root is folded away: ...:ci:1:dt:root serialises as ...:ci:1. That is
the intended behaviour (root is the default and has no path, so a root
address keeps one canonical form and one hash, and consumers rely on
it); the spec now says so, and the Algorand and TON notes no longer call
dt:root the canonical form.

zip32 parses, but no network registers it, so ParseURNStrict refuses it
everywhere; the spec lists that as a known limitation. Tests pin both:
dt:root rows in the shared URN table and a strict-refusal check over
every network.
The hash tests only checked determinism and digest length. Add reference
digests for str() of 55, 56, 63, 64, 119, 120, 128 and 1000 bytes (the NSS
is 9 bytes shorter, so it covers 55 too), computed with Python's hashlib.
The C++ port, whose SHA-1 and SHA-256 are hand-written, pins the same
values, so both implementations must produce the same identifiers across
every padding case.
The workflow ran plain `go test` on push only. It now also runs on pull
requests with read-only permissions, vets and tests ./... on every Go
version of the matrix (plus the current stable release), and a check job
on stable verifies gofmt, runs the race detector and fuzzes each target
for 30 seconds. checkout and setup-go move to v4 and v5.
ParsePath updated the fields of the path in place. Re-parsing a 4-level
ZIP-32 path with a 3-level one kept the old index (m/32'/133'/1' read
back as m/32'/133'/1'/5), a SLIP-10 path kept the coin, account and
charge of a BIP-44 path parsed before it, and an error midway left a
half-updated path. SetDerivationType changed the type and kept the old
path: a bip44 address switched to zip32 serialised as
dt:zip32:dp:m/32'/133'/3'/9, a valid URN naming a key nobody gave.

ParsePath now parses into a fresh path and replaces the old one only on
success. A new derivation type drops the old path; until a path is set,
String() of the path is empty, the URN carries dt without dp (so it
does not parse) and Validate / MarshalText return
ErrInvalidDerivationPath. Address.SetDerivation(dt, dp) sets both at
once and changes nothing on error; parseAddress uses it.
NewDerivationPathFromLevels kept the levels it was given while String()
printed the type's template: a BIP-44 path built from 49'/60'/0/0/0
printed m/44'/60'/0'/0/0 but returned purpose 49' and an unhardened
account from Levels(), which is what a wallet derives from. Too few
levels or an index of 2^31 gave a URN that does not parse, and
NewDerivationPath accepted a charge of 7 or an account of 2^31.

Both constructors now return the path the parser gives back for their
own String() and panic with ErrInvalidDerivationPath unless it has
exactly the requested levels. Levels() returns a copy instead of the
internal slice, and NewAddress keeps its own copies of the chain and the
path, so addresses built from one path no longer change together. The
fuzz targets check that a re-parsed path has the same levels.
Validate checked the algorithm, the format and the derivation type each
against the network on its own. It accepted SLIP-10 ed25519 paths with
soft levels (ed25519 has no soft derivation), bip44 with ed25519 on
XRPL, NEAR and Aptos, Sui bip54 with ed25519 or secp256r1 (the purpose
encodes the scheme), a Bitcoin bip84 path with af:p2pkh, and Cosmos bip44
with coin 118', which is the cip11 path under a second name.

Validate now also refuses, with ErrIncompatible: an ed25519 path with an
unhardened level, except cip1852 (BIP32-Ed25519); a derivation type with
another curve than its own where the network binds them (Sui: slip10
ed25519, bip54 secp256k1, bip74 secp256r1; Aptos and NEAR: slip10
ed25519, bip44 secp256k1); an explicit Bitcoin format that does not
match the purpose; and Cosmos bip44 with coin 118'. A Bitcoin URN
without af stays valid, and SPEC.md no longer says strict mode needs it.

The shared URN table gains strict rows for each rule. Tests that pinned
now-refused pairs move to matching purposes (p2sh under bip49, p2wsh
under bip32), and the XRPL ed25519 example becomes the root form its
family-seed keys have.
String(), NSS(), the hashes, Algorithm() and Format() dereferenced the
chain, so the zero Address or NewAddress(nil, ...) panicked, and
SetDerivationPath dereferenced the path, so an address built without one
panicked too. A missing chain now reads as the zero chain
(urn:mhda:nt::ci:, the C++ port's default address) and a missing path as
root; SetCoinType without a chain returns ErrUninitializedAddress.
MarshalText had a pointer receiver, so an Address held by value (a
struct field, a map value) was encoded by encoding/json and encoding/xml
as a plain struct: {"A":{}}, which then failed to decode. SPEC.md
promises the URN form for these codecs.

MarshalText now has a value receiver. The codecs still encode a nil
*Address themselves (JSON null); a direct MarshalText call on a nil
pointer now panics like any value method, so TestMarshalTextNilAddress
keeps only its zero-value case.
AlgorithmFromString, FormatFromString and NetworkTypeFromString returned
plain errors.New values, so errors.Is could not discriminate them,
although doc.go promises sentinel errors throughout and
DerivationTypeFromString already wrapped ErrInvalidDerivationType. They
now wrap ErrInvalidAlgorithm, ErrInvalidFormat and ErrInvalidNetworkType.
normalize and the Address setters used strings.TrimSpace and
strings.ToLower, which are Unicode-aware: SetAddressAlgorithm accepted
"secp256\u212a1" (Kelvin sign) as secp256k1, and NBSP or ideographic
spaces around a derivation type, path, format or coin type were trimmed
away. The URN parser and the C++ port refuse those bytes, so the two
entry points and the two implementations disagreed.

normalize now trims ASCII whitespace and folds ASCII case only, and the
setters, parseChain and the case-insensitive URN prefix check use it.
parseNSS trimmed every value, so urn:mhda:nt:evm:ci: 1 :dt: bip44 :...
parsed like the same URN without the spaces, although SPEC section 6.1 says
values contain no whitespace and only the whole URN is trimmed.

parseNSS no longer trims; whitespace in a key or value fails the
charset check with ErrInvalidNSS. ParseURN trims the URN and the
whitespace left before a stripped r/q/f component (ci:0 #frag still
parses); ParseNSS, ChainFromNSS and ChainFromKey trim the NSS they are
given as a whole.
Keys and values accepted any printable ASCII, so '"', '<', '>', '\',
'^', '`', '{', '|', '}', '[', ']' and a raw '%' passed and were emitted in
URNs that RFC 8141 does not allow, although SPEC section 1.5 requires the NSS
production (RFC 3986 pchar and "/").

One predicate, nssByte, now admits letters, digits and -._~!$&'()*+,;=@/
- pchar and "/" without ':' (the separator) and '%' (percent-encoding is
not supported, and %41 would be a second spelling of "A"). The parser
uses it for keys and values (ErrInvalidNSS), and validateFreeFormValue
for ap/as/wt/wi and the chain id (ErrInvalidValue). The shared URN table
gets a row per refused byte.
ci:1, ci:0x1 and ci:01 are three chain keys and hashes for one chain.
MHDA keeps the chain id opaque; SPEC.md now says so and asks a producer
to keep to one spelling (the EIP-155 decimal form is the usual one).

The 1.2.0 changelog had grown entry by entry; it is regrouped into
Changed (inputs refused now), Fixed and Documentation, with the intro
pointing at the changes.
Two test inputs held the Kelvin sign, a no-break space and an
ideographic space as literal characters, which look like a K and plain
spaces in the source. They are now written as \u escapes; the bytes the
tests use are unchanged.
@x88
x88 merged commit 536973e into master Oct 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant