Skip to content

Record a collection's hash seed, asked for rather than defaulted - #135

Merged
maverox merged 1 commit into
mainfrom
work/hash-seed-explicit
Sep 14, 2026
Merged

maverox merged 1 commit into
mainfrom
work/hash-seed-explicit

Conversation

@maverox

@maverox maverox commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #134, which is stacked on #133. Supersedes #111 and #115.

What

A HashMap or HashSet iterates in a different order every process, because RandomState::new() draws fresh keys per collection. Any order that reaches the wire — a serialized object's key order, a list built by iterating a set — differs between a recording and its replay for a reason that has nothing to do with the candidate.

let seed = deja::hash_seed("routing::eligible_connectors");
let map: deja::SeededHashMap<K, V> = HashMap::with_hasher(seed);
let set: deja::SeededHashSet<T> = HashSet::with_hasher(seed);

BuildHasher is the seam, so one type covers every std hash collection — HashMap, HashSet, and anything generic over S: BuildHasher including IndexMap. Nothing per-structure is needed.

Why there is no Default

A collection could be seeded implicitly, by branching Default on ambient correlation state — HashMap::new() keeps working and every collection is covered. That is what #111/#115 did, and every cost they carried flowed from that one decision:

None of that is essential to recording a seed. It is the cost of not being asked. So: no Default impl, and the type will not construct without the caller naming a seed.

The honest trade: explicit does not touch fewer sites — both designs require typing the collection. It is more invasive per site and needs zero machinery. That is a good trade only because the set of collections whose order reaches the wire is small, and is found from order-only body diffs rather than guessed at.

Why per-collection

A correlation-wide seed was a way to hold event volume down when every collection was covered. With a handful of named ones that pressure is gone, and per-collection attributes an order difference to the collection that caused it. The name is a rank-1 explicit tag on the existing callsite ladder, not a new naming scheme.

Why a miss synthesizes — reversing an earlier position

I previously argued a seed miss must fail-stop: absorbing yields a fresh random seed and every downstream diff becomes noise. That argument was against nondeterminism, not against difference.

A content-addressed seed is stable run to run. Order-only differences are already absorbed by the comparator ("a permutation of an identical multiset is not a difference"), and an order difference that changes behaviour is deterministic and therefore attributable. The seed boundary goes from the strongest case for fail-stop to the best case for synthesis — which is why this could not land before #133.

The security argument, asserted rather than argued

A synthesized seed is derived from the collection's name and correlation, so anyone holding those can predict it. Randomized keys exist precisely to stop an attacker crafting colliding entries.

That is safe for a reason that does not depend on anyone being careful: synthesis is unreachable outside replay. Record and disabled modes never perform a lookup, so they never miss, so they never reach the miss arm. record_and_disabled_modes_never_reach_the_miss_arm fails loudly if that ever stops being true.

SipHash-1-3 from siphasher

DefaultHasher::new() is hardcoded to keys (0, 0) and std exposes no keyed constructor. Prefixing a DefaultHasher with the keys would compile, but std explicitly does not guarantee its algorithm across releases — and record and replay can run different toolchains, so a silent algorithm change would reintroduce the exact order divergence this removes, in a form nothing would attribute correctly. SipHash-1-3 is the same algorithm std uses, so the only thing that differs from stock RandomState is where the keys come from.

Cargo.lock moves by exactly the seven lines that dependency adds — no re-resolution, no downgrades.

The intended consumer: a std-mimicking facade in the vendor

This PR ships the machinery. The call-site ergonomics it is for are a vendor-side facade over it, so a site changes its import and little else:

// common_utils/src/collections.rs
pub type HashMap<K, V> = std::collections::HashMap<K, V, deja::DejaBuildHasher>;
pub trait New { fn new() -> Self; }   // HashMap::new() is inherent only for RandomState

That is not in this PR and does not change what is. But one measured finding about it belongs here, because it bears directly on the Default decision below:

.collect() will not compile against a hasher with no Default. FromIterator for HashMap<K, V, S> requires S: BuildHasher + Default. In hyperswitch origin/main: 426 constructions (new/with_capacity) swap cleanly, HashMap::from( has zero uses, but 47 turbofish collects plus an unknown share of 1,521 bare .collect() calls would fail to build.

So the no-Default decision has a real cost at the call site, and it is worth taking that cost deliberately rather than discovering it. Restoring Default is the option that reintroduces exactly the implicit path #111/#115 died on; giving the facade a collect_seeded() helper is the option that keeps this PR's guarantee and makes those sites explicit instead.

Full analysis: deja-handovers/results/collections-facade-feasibility.md.

Evidence

just verify green. 9 new tests. Four mutations, four kills, each by a distinct test:

mutation killed by
build_hasher ignores the keys the_keys_decide_the_hash, iteration_order_follows_the_seed
synthesized seed is a constant a_synthesized_seed_is_deterministic_and_name_sensitive
malformed image redraws instead of failing a_malformed_image_fails_rather_than_redrawing
both draw_keys tags identical drawn_pairs_differ_from_each_other

iteration_order_follows_the_seed searches a range of seeds rather than hoping one pair reorders, so it cannot pass by luck.

Open question for review

Ergonomics. HashMap::with_hasher(deja::hash_seed("name")) at each construction — acceptable, or does no-Default want a helper macro? The SeededHashMap/SeededHashSet aliases take the sting out of naming the third type parameter, but the construction site still has to be edited. This wants a vendor opinion before any large-scale adoption; nothing here forces one.

Not addressed

Serialized key order. The shipped router has serde_json's preserve_order transitively, so HashMap iteration order propagates into JSON object key order. Structural comparison is unaffected, but anything computing over serialized bytes — a signature or HMAC over a response body — would change with key order. Whether that path exists here is unknown; neither this seam nor the fan-out tolerance catches it.

🤖 Generated with Claude Code

https://claude.ai/code/session_019FmXkygUmueraF9oR4oqwS

…an defaulted

A `HashMap` or `HashSet` iterates in a different order every process, because
`RandomState::new()` draws fresh keys per collection. Any order that reaches
the wire differs between a recording and its replay for a reason that has
nothing to do with the candidate. `deja::hash_seed(name)` returns a
`BuildHasher` whose keys are recorded and replayed, so that whole class of
difference goes away.

`BuildHasher` is the seam, so ONE type covers every std hash collection --
HashMap, HashSet, and anything generic over `S: BuildHasher` including
IndexMap. Nothing per-structure is needed.

There is deliberately NO `Default` impl. A collection could be seeded
implicitly by branching `Default` on ambient correlation state, and every cost
that approach carried flowed from that one decision: `Default` runs on the hot
path (495 constructions in the vendor tree), so it needed a per-correlation
memo; the memo needed somewhere to live, which was a correlation-keyed
registry; and reading ambient state from inside `HashMap::new()` put a
thread-local borrow on a path that also runs during thread teardown, where it
aborts the process rather than panicking. None of that is essential to
recording a seed -- it is the cost of not being asked. Explicit is more
invasive per site and needs no machinery at all, which is a good trade only
because the set of collections whose order reaches the wire is small.

Per-collection rather than per-correlation. A correlation-wide seed was a way
to hold event volume down when every collection was covered; with a handful of
named ones that pressure is gone, and per-collection attributes an order
difference to the collection that caused it. The name is a rank-1 explicit tag
on the existing callsite ladder, not a new naming scheme.

A miss SYNTHESIZES rather than stopping, which is why this could not land
before the miss arm existed. An earlier position held that a seed miss must
fail-stop, because absorbing yields a fresh random seed and every downstream
diff becomes noise. That argument was against nondeterminism, not against
difference: a content-addressed seed is stable run to run, order-only
differences are already absorbed by the comparator, and an order difference
that changes behaviour is deterministic and therefore attributable. The seed
boundary goes from the strongest case for fail-stop to the best case for
synthesis.

Predictable keys are safe for a structural reason, now asserted rather than
argued: record and disabled modes never perform a lookup, so they never miss,
so they never reach the miss arm. Synthesized keys exist only inside a replay
harness. `record_and_disabled_modes_never_reach_the_miss_arm` fails loudly if
that ever stops being true.

SipHash-1-3 from `siphasher` rather than `DefaultHasher`, which is hardcoded to
keys (0, 0) and whose algorithm std explicitly does not guarantee across
releases -- record and replay can run different toolchains. Cargo.lock moves by
exactly the seven lines that dependency adds.

Four mutations, four kills, each by a distinct test: keys ignored by the
hasher, a constant synthesized seed, a malformed image redrawing instead of
failing, and both `draw_keys` tags made identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019FmXkygUmueraF9oR4oqwS
@maverox
maverox force-pushed the work/hash-seed-explicit branch from 541d901 to 2997274 Compare September 14, 2026 13:57
Base automatically changed from work/synth-helpers to main September 14, 2026 15:01
@maverox
maverox merged commit d587865 into main Sep 14, 2026
9 checks passed
@maverox
maverox deleted the work/hash-seed-explicit branch September 14, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant