fix: make masterfile reload safe under concurrent queries - #19
Conversation
Reloading the masterfile while queries run crashed consumers with "fatal error: concurrent map read and map write" (hit in production by Golbat, see UnownHash/Golbat#403): - LoadPokemonData unmarshaled into the live PokemonData struct; encoding/json reuses non-nil maps, writing into the very maps QueryPvPRank readers walk with no lock. - WatchPokemonData published a new PokemonData via an unsynchronized multi-word struct assignment (torn reads, no happens-before). - ClearCache overwrote the live sync.Map while queries were mid Load/Store on it. The masterfile snapshot and the rank cache computed from it now live in one immutable bundle behind an atomic.Pointer. Reloads build a fresh bundle (fresh maps, empty cache) and publish it with a single atomic store; readers pick up the bundle once per call. This also closes the window where a reload served new masterfile data with ranks cached from the old one (data was stored before the cache was cleared), and stops calculateAllRanksCompact from lazily writing RankingComparator on the query path. The exported PokemonData field is kept for backward compatibility: it still seeds the first bundle when populated directly, and reloads mirror into it for legacy readers, but internal code no longer reads it after publication. Regression test exercises LoadPokemonData and ClearCache concurrently with QueryPvPRank and CalculateCp under -race. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BMJMXHUiq6Pnce9KshgFTC
|
Heads-up: after opening this I found #18, which identified the same data races back in February (§1.6 of its I've left a comment on #18 comparing the two concurrency approaches and asking @lenisko how he'd prefer to resolve the overlap. Whichever vehicle wins, the Context for why this exists: the |
|
Feel free to merge @jfberry since Golbat is single consumer so far, I will close other one later. Please also bump https://github.com/UnownHash/gohbem/blob/master/ohbem.go#L18 to |
|
Golbat actually uses the current build in a race free way now; but I will take 18+19 and merge them together then to make gohbem reliable (and I guess we don't know for sure if there are other users or not!) |
Problem
Reloading the masterfile while queries are running is a data race that crashes consumers. Golbat hit this in production as
fatal error: concurrent map read and map writeinsideQueryPvPRank(ohbem.go:408) — root-caused and worked around app-side in UnownHash/Golbat#403, but the underlying issue is in gohbem and affects every consumer that reloads:LoadPokemonDatamutates live maps. It unmarshals into the existingo.PokemonData, andencoding/jsonreuses non-nil maps — writing keys into the veryPokemon/Costumesmaps concurrentQueryPvPRank/CalculateCpcalls are reading. This is the production crash.WatchPokemonDatapublishes via an unsynchronized struct assignment.o.PokemonData = pokemonDatais a multi-word write with no happens-before edge, so readers can see a torn mix of old/new fields, or a new map pointer before its contents are visibly published (arm64).ClearCacheoverwrites a livesync.Map(o.compactRankCache = sync.Map{}) while queries are mid-Load/Storeon it — clobbering its internal state.All three are flagged deterministically by
go test -racewith the included regression test.Fix
The masterfile snapshot and the rank cache computed from it now live in one immutable bundle behind an
atomic.Pointer:LoadPokemonData,FetchPokemonData, the watcher) build a fresh bundle — fresh maps, empty cache — and publish it with a single atomic store. Readers already holding the old bundle finish on it undisturbed.(cpCap, base stats)their values are masterfile-version-independent, so that window was benign for correctness — the atomic swap just removes it and reclaims the stale entries in one step.)ClearCacheswaps in a same-data/empty-cache bundle via CAS.calculateAllRanksCompactno longer lazily writeso.RankingComparatoron the query path (that write also raced concurrent queries).Compatibility
replacedirective.PokemonDatafield is kept: populating it directly before first use still works (it seeds the first snapshot), and reloads mirror into it for legacy readers. Its doc comment now spells out that it must not be read concurrently with a reload. If you'd rather unexport it in a future breaking release, happy to follow up.VERSIONleft untouched — assuming you prefer to bump on release.Verification
TestReloadConcurrentWithQueriesfails on master under-race(races 1 and 3 above) and passes here; ran with-count=3.-race; benchmarks confirm the cache path still hits (BenchmarkQueryPvPRankCached~25× faster than uncached).🤖 Generated with Claude Code
https://claude.ai/code/session_01BMJMXHUiq6Pnce9KshgFTC