Key the content caches by their input, not a 32-bit hash - #126
Merged
Merged
Conversation
ParsedYamlCache and ProcessedFrontmatterCache both keyed on FNV-1a-32 and never compared the input, so a hash collision silently served one document's frontmatter for another. Key on the text itself (with the option flags, for the processed cache) so a hit means the input really is identical. HashUtil existed only to produce those keys, so it and the HashDepot dependency go with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XD56Kg65nsraeL7vTL9MgU
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Fixes #125.
The bug
ParsedYamlCache(Frontmatter/YamlSerializer.cs) andProcessedFrontmatterCache(Frontmatter/Frontmatter.cs) were bothConcurrentDictionary<uint, …>keyed solely onHashUtil.ComputeHash/CreateCacheKey, which wrapHashDepot.Fnv1a.Hash32. Neither cache kept the input, so a hash match was taken as a content match — and both caches live for the lifetime of the process.Reproduced on
main, with two genuine FNV-1a-32 collisions found by brute force:Both pairs hash to
0x27196CB9and0x12657B7Erespectively. One document's frontmatter is silently emitted into an unrelated document, with no exception and no warning — the one failure mode a round-tripping frontmatter library must not have.The fix
The issue offers two routes: widen the key to a 64/128-bit hash, or store the input and verify equality on a hit. This takes the second, in its simplest form — key on the input itself:
ParsedYamlCachebecomesConcurrentDictionary<string, Dictionary<string, object>>built withStringComparer.Ordinal, keyed on the YAML text.ProcessedFrontmatterCachebecomesConcurrentDictionary<(string Content, uint Options), string>. The option flags still pack into the sameuint, so the two documents differ by content or by options, never by a digest of either.A wider hash would have made the collision rarer rather than impossible; since the values these caches already hold are the full parsed dictionary and the full processed document, comparing the key exactly is not a meaningfully different memory profile from carrying the input alongside the value, which is what the issue's second option asks for.
HashUtilexisted only to produce these two keys — nothing else in the library or the tests referenced it — so it is removed, and with it theHashDepotpackage reference and itsDirectory.Packages.propspin.Tests
Four tests, two of them regressions that fail against the previous implementation (verified by restoring the source files and re-running: 2 failed, 131 passed):
TryParseYamlObject_WithHashCollidingInputs_ParsesEachIndependently— fails onmainwithAssert.AreEqual("Release Notes 386342", secondResult["title"]).CombineFrontmatter_WithHashCollidingDocuments_ProcessesEachIndependently— fails onmainwithAssert.Contains("Release Notes 1084130", secondResult).Plus two guards that pass both before and after by design, so a later change cannot quietly make the caches incoherent in the other direction:
TryParseYamlObject_WithTheSameInputTwice_ReturnsEqualResultsCombineFrontmatter_WithTheSameDocumentTwice_ReturnsEqualResultsThe collision documents are written with
"\n"escapes rather thanEnvironment.NewLine, so the bytes hashed are identical on every platform;DetectNewLinereads each document's own convention, so they still parse on Windows.Full suite: 133 passed, 0 failed (129 pre-existing, unchanged). Release build clean across all five target frameworks (net8.0, net9.0, net10.0, netstandard2.0, netstandard2.1), 0 warnings.
Scope
No public API change. One thing deliberately not touched: both caches are still unbounded and process-lifetime, which is the reason the collision was reachable at all. Bounding or evicting them is a separate design decision and a separate change.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XD56Kg65nsraeL7vTL9MgU
Generated by Claude Code