Skip to content

Key the content caches by their input, not a 32-bit hash - #126

Merged
matt-edmondson merged 1 commit into
mainfrom
claude/verify-cache-keys-exactly
Sep 24, 2026
Merged

matt-edmondson merged 1 commit into
mainfrom
claude/verify-cache-keys-exactly

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Fixes #125.

The bug

ParsedYamlCache (Frontmatter/YamlSerializer.cs) and ProcessedFrontmatterCache (Frontmatter/Frontmatter.cs) were both ConcurrentDictionary<uint, …> keyed solely on HashUtil.ComputeHash / CreateCacheKey, which wrap HashDepot.Fnv1a.Hash32. Neither cache kept the input, so a hash match was taken as a content match — and both caches live for the lifetime of the process.

Reproduced on main, with two genuine FNV-1a-32 collisions found by brute force:

TryParseYamlObject("title: Release Notes 19")      -> { title = "Release Notes 19" }
TryParseYamlObject("title: Release Notes 386342")  -> { title = "Release Notes 19" }   // wrong
CombineFrontmatter("---\ntitle: Release Notes 281277\n---\n")   -> …Release Notes 281277…
CombineFrontmatter("---\ntitle: Release Notes 1084130\n---\n")  -> …Release Notes 281277…   // wrong

Both pairs hash to 0x27196CB9 and 0x12657B7E respectively. One document's frontmatter is silently emitted into an unrelated document, with no exception and no warning — the one failure mode a round-tripping frontmatter library must not have.

The fix

The issue offers two routes: widen the key to a 64/128-bit hash, or store the input and verify equality on a hit. This takes the second, in its simplest form — key on the input itself:

  • ParsedYamlCache becomes ConcurrentDictionary<string, Dictionary<string, object>> built with StringComparer.Ordinal, keyed on the YAML text.
  • ProcessedFrontmatterCache becomes ConcurrentDictionary<(string Content, uint Options), string>. The option flags still pack into the same uint, so the two documents differ by content or by options, never by a digest of either.

A wider hash would have made the collision rarer rather than impossible; since the values these caches already hold are the full parsed dictionary and the full processed document, comparing the key exactly is not a meaningfully different memory profile from carrying the input alongside the value, which is what the issue's second option asks for.

HashUtil existed only to produce these two keys — nothing else in the library or the tests referenced it — so it is removed, and with it the HashDepot package reference and its Directory.Packages.props pin.

Tests

Four tests, two of them regressions that fail against the previous implementation (verified by restoring the source files and re-running: 2 failed, 131 passed):

  • TryParseYamlObject_WithHashCollidingInputs_ParsesEachIndependently — fails on main with Assert.AreEqual("Release Notes 386342", secondResult["title"]).
  • CombineFrontmatter_WithHashCollidingDocuments_ProcessesEachIndependently — fails on main with Assert.Contains("Release Notes 1084130", secondResult).

Plus two guards that pass both before and after by design, so a later change cannot quietly make the caches incoherent in the other direction:

  • TryParseYamlObject_WithTheSameInputTwice_ReturnsEqualResults
  • CombineFrontmatter_WithTheSameDocumentTwice_ReturnsEqualResults

The collision documents are written with "\n" escapes rather than Environment.NewLine, so the bytes hashed are identical on every platform; DetectNewLine reads each document's own convention, so they still parse on Windows.

Full suite: 133 passed, 0 failed (129 pre-existing, unchanged). Release build clean across all five target frameworks (net8.0, net9.0, net10.0, netstandard2.0, netstandard2.1), 0 warnings.

Scope

No public API change. One thing deliberately not touched: both caches are still unbounded and process-lifetime, which is the reason the collision was reachable at all. Bounding or evicting them is a separate design decision and a separate change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XD56Kg65nsraeL7vTL9MgU


Generated by Claude Code

ParsedYamlCache and ProcessedFrontmatterCache both keyed on FNV-1a-32 and
never compared the input, so a hash collision silently served one document's
frontmatter for another. Key on the text itself (with the option flags, for
the processed cache) so a hit means the input really is identical.

HashUtil existed only to produce those keys, so it and the HashDepot
dependency go with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XD56Kg65nsraeL7vTL9MgU
@sonarqubecloud

Copy link
Copy Markdown

@matt-edmondson
matt-edmondson merged commit b282443 into main Sep 24, 2026
12 checks passed
@matt-edmondson
matt-edmondson deleted the claude/verify-cache-keys-exactly branch September 24, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Content caches keyed only by a 32-bit hash can return another document's cached frontmatter

2 participants