Delegate property-name fuzzy matching to ktsu.FuzzySearch

[matt-edmondson]

What's hand-rolled

Two independent, hand-rolled approximate string-matching implementations exist in this repo for the same underlying problem — matching a non-standard frontmatter property name against a set of known/existing names:

1. NameStandardizer.FindStandardPropertyMatch (used by StandardizePropertyNames, the NameStandardizer step of CombineFrontmatter):

https://github.com/ktsu-dev/Frontmatter/blob/main/Frontmatter/NameStandardizer.cs#L91-L152

if (normalizedKey.Contains(normalizedStandard) || normalizedStandard.Contains(normalizedKey))
{
    return standardProperty;
}

(NameStandardizer.cs#L105) — a plain substring check, returning the first standard property tested, not the best.

2. PropertyMerger.FindSemanticCanonicalName/CalculateWordMatchScore (used by the PropertyMerger step, a separate merge pass):

https://github.com/ktsu-dev/Frontmatter/blob/main/Frontmatter/PropertyMerger.cs#L251-L329

private static int CalculateWordMatchScore(string[] words1, string[] words2)
{
    int score = 0;
    foreach (string word1 in words1)
    {
        foreach (string word2 in words2)
        {
            if (word1 == word2) { score += 2; }
            else if (word1.Contains(word2) || word2.Contains(word1)) { score += 1; }
        }
    }
    return score;
}

(PropertyMerger.cs#L313-L329) — its own word-tokenized scoring loop, feeding an OrderByDescending(match => match.Score) pick of the best existing key (PropertyMerger.cs#L262-L273).

Both files also carry their own separate, near-identical NormalizePropertyName helper (NameStandardizer.cs#L114-L152 and PropertyMerger.cs#L280-L310).

The class doc comment on NameStandardizer says "using fuzzy matching" (NameStandardizer.cs#L18) and PropertyNameCache is documented as a "Cache for fuzzy matched property names" (NameStandardizer.cs#L15) — but neither implementation is a scored approximate match in the usual sense; they're substring containment and word-overlap counting.

Notably, this repo already declares ktsu.FuzzySearch as a package reference:

https://github.com/ktsu-dev/Frontmatter/blob/main/Frontmatter/Frontmatter.csproj#L9

but no .cs file in the repo calls into it — it's an unused dependency. The repo's own CLAUDE.md already flags this: "ktsu.FuzzySearch - Fuzzy matching (referenced but main matching uses PropertyMappings)".

What ktsu.FuzzySearch provides

ktsu.FuzzySearch.Fuzzy.Contains(ReadOnlySpan<char> subject, ReadOnlySpan<char> pattern, out int outScore):

https://github.com/ktsu-dev/FuzzySearch/blob/main/FuzzySearch/Fuzzy.cs#L84-L88

It reports whether every character of pattern occurs in subject in sequence, and produces a score that rewards consecutive matches, matches after _/space separators, and camelCase-boundary matches, while penalizing unmatched characters. Either hand-rolled matcher above could rank candidates by this score instead of by containment or raw word-overlap count.

Why it's worth it

Both current implementations have the same correctness gap: they don't rank candidates by similarity, so the result depends on iteration order (NameStandardizer, StandardOrder.PropertyNames's order) or on a coarse, unweighted count (PropertyMerger's +2/+1 scoring, which e.g. scores "date" vs ["updated", "at"] the same as "date" vs ["created", "date"] in some cases since it sums per-word-pair hits rather than accounting for word position or adjacency). Fuzzy.Contains's scoring — consecutive-match bonus, separator-boundary bonus, camelCase bonus, unmatched-character penalty — is exactly the kind of deliberate ranking both call sites currently approximate by hand, and it already ships in this repo's dependency graph, so this isn't a new package to add — just wiring up code that's already referenced.

Compatibility

• Subject (ktsu.Frontmatter) targets: the ktsu.Sdk-provided default multi-target set (no <TargetFrameworks> override in Frontmatter.csproj; the project conditions packages on netstandard2.0 and netstandard2.1, so both are in the set it builds).
• ktsu.FuzzySearch targets: the same ktsu.Sdk default set (its .csproj likewise has no override and conditions packages on netstandard2.0/netstandard2.1).
• No gap in practice: ktsu.FuzzySearch is already a working PackageReference of Frontmatter.csproj, so it already compiles against every framework Frontmatter targets.
• Dependency direction: ktsu.FuzzySearch has zero ktsu.* dependencies (its Directory.Packages.props lists only Polyfill, System.Memory, System.Threading.Tasks.Extensions), so there's no cycle.

Sketch

// NameStandardizer.FindStandardPropertyMatch — before
foreach (string standardProperty in standardProperties)
{
    string normalizedStandard = NormalizePropertyName(standardProperty);
    if (normalizedKey.Contains(normalizedStandard) || normalizedStandard.Contains(normalizedKey))
    {
        return standardProperty;
    }
}
return null;

// after
string? best = null;
int bestScore = int.MinValue;
foreach (string standardProperty in standardProperties)
{
    string normalizedStandard = NormalizePropertyName(standardProperty);
    if (Fuzzy.Contains(normalizedKey, normalizedStandard, out int score) && score > bestScore)
    {
        best = standardProperty;
        bestScore = score;
    }
}
return best;

// PropertyMerger.FindSemanticCanonicalName — before
(string Key, int Score)? bestMatch = existingKeys
    .Select(existingKey => (Key: existingKey, Score: CalculateWordMatchScore(keyWords, ...)))
    .Where(match => match.Score > 0)
    .OrderByDescending(match => match.Score)
    .Cast<(string Key, int Score)?>()
    .FirstOrDefault();

// after
(string Key, int Score)? bestMatch = existingKeys
    .Select(existingKey => (Key: existingKey, Matched: Fuzzy.Contains(NormalizePropertyName(key), NormalizePropertyName(existingKey), out int score), Score: score))
    .Where(match => match.Matched)
    .OrderByDescending(match => match.Score)
    .Select(match => (match.Key, match.Score))
    .Cast<(string Key, int Score)?>()
    .FirstOrDefault();

Caveats

• Fuzzy.Contains requires every pattern character to appear in sequence in the subject; NameStandardizer's current check accepts a match in either direction (key-contains-standard or standard-contains-key), and PropertyMerger's scores word-by-word rather than whole-string. Adapting either call site needs a decision on which span is pattern vs. subject (and whether to check both directions and keep the better score), since it changes which non-standard keys match successfully.
• Both call sites are internal (NameStandardizer, PropertyMerger, and their methods are all internal), so there's no public API to break — but the change in matching behavior could change which standard/existing property some non-standard frontmatter keys get mapped to, which is an observable behavior change for consumers of CombineFrontmatter.
• Neither Frontmatter.Test/NameStandardizerTests.cs nor Frontmatter.Test/PropertyMergerTests.cs appears to pin the exact fallback-match behavior beyond what exact/known mappings cover (based on the test file list), so this is worth double-checking before changing either scoring.
• Adopting Fuzzy.Contains in both places would also be a chance to fold the two near-duplicate NormalizePropertyName helpers into one, though that's a separate cleanup from the library swap itself.

[matt-edmondson]

Triage

Category: Improvement
Priority: Low
Area: Frontmatter — property-name fuzzy matching

Low: nothing broken, consolidation onto ktsu.FuzzySearch.

Suggested assignment: none specific.

Related: tracked under ktsu-dev/.github#15 with the rest of this audit rotation.

Check this before starting: ktsu-dev/FuzzySearch#74 is an open documentation bug reporting that FuzzySearch's README documents a Match/Filter/FuzzyOptions/FuzzyResult API that does not exist in the library. Anyone picking this up who works from that README will write code against a fictional surface. Read the actual public API, not the docs, until #74 is resolved — that issue is the single most useful thing to know here and is easy to trip over.

Open PR covering this: none.

Suggested next step: beyond the API caveat above, establish what the current matching does for near-miss property names and pin it with tests first. Fuzzy matching is a behaviour, not just an implementation: swapping the algorithm changes which typo'd frontmatter keys resolve and which don't, and for a parser that silently changes how documents are interpreted. Low priority, but it deserves characterisation tests before the swap rather than after.

---

Generated by Claude Code

[matt-edmondson]

Progress

#120 wires ktsu.FuzzySearch into the NameStandardizer call site. The PropertyMerger half is still open, and the reason is worth recording.

What landed

FindStandardPropertyMatch returned the first standard property that passed the containment test, so the winner was decided by StandardOrder.PropertyNames' listing order. Two real cases:

key	before	after
subtitles_track	title	subtitle
review_status_flag	status	review_status

Both now rank by Fuzzy.Contains's score. Ties still fall back to the standard order.

On the gate, measured rather than assumed

The sketch swaps containment for Fuzzy.Contains. I measured that against the real StandardOrder.PropertyNames list first, and it is safe but pointless:

video_subtitles        substring: title, subtitle, video
                       fuzzy:     subtitle=33, video=20, title=5, doi=-14
author_editorial_notes substring: author, editor
                       fuzzy:     authors=21, author=20, editors=15, editor=14, audio=3, area=2
my_custom_thing / wordcount / notes / xyzzy
                       substring: (none)     fuzzy: (none)

The extra admissions are all low- or negative-scoring and every one loses on ranking. So the gate widening changes no outcome I could find, while the downside if it ever does bite is a preserved key getting silently renamed — data loss, in a library whose job is preserving frontmatter. I left the gate as-is and used Fuzzy purely to rank. Happy to widen it if you'd rather, it's a two-line change.

Why PropertyMerger is still open

This one is genuinely different, and the issue's caveat about "which span is pattern vs subject" understates it.

FindSemanticCanonicalName's candidates only have to share a word with the key. Related pairs like review_notes / notes_reviewed are neither a substring nor a subsequence of one another in either direction, so Fuzzy.Contains returns false for both directions and the pair gets no score at all. Swapping CalculateWordMatchScore for a whole-name fuzzy score would therefore tie every such candidate and hand the decision back to iteration order — strictly worse than the coarse +2/+1 it replaces.

Scoring per word pair does work (sum each key word's best fuzzy score against the candidate's words), and demoting CalculateWordMatchScore to a boolean SharesRelatedWord gate keeps the admitted set bit-identical. I implemented that and the full suite stayed green — but green only says nothing regressed, and I could not construct a case through the public API where the new ranking actually picks a different key from the old one. Most candidate pairs I tried never reach FindSemanticCanonicalName at all, because FindBasicCanonicalName resolves them first.

So it would have landed as an unpinned behaviour change on a merge path, which is not something to do unattended. I reverted it. The code is small and the approach is sound if you want it — what it needs is a FrontmatterMergeStrategy.Maximum case that reaches the semantic matcher and discriminates between the two scorers, which is a question about the merger's real-world inputs rather than about the library swap.

Also still open

The two near-duplicate NormalizePropertyName helpers are untouched, as the issue suggests. Note they are not quite identical — PropertyMerger's trims the input first and NameStandardizer's does not — so folding them is a small behaviour change in its own right, not a pure move.

---

Generated by Claude Code

[matt-edmondson]

Progress

The NormalizePropertyName fold is in #124. The PropertyMerger ranking half is unchanged and still sits behind the call recorded above. Unassigning myself, since what remains is that decision.

What landed

The last item on the previous note — "the two near-duplicate NormalizePropertyName helpers are untouched" — plus a bug that doing it exposed.

The two copies had drifted further than "not quite identical" suggested. The standardizer's lower-cased without trimming, compared its affixes culture-sensitively, and replaced only the space character. Over a 30-key corpus the two disagreed on 15, including every tab- or newline-padded key. Both now call one PropertyNameNormalizer with the merger's behaviour, which is the more robust on each point of difference.

The part worth flagging

Folding them surfaced a data-loss bug already reachable on main, with no padding involved. A key made only of decoration — page_, _value, _, page__value — normalizes to the empty string, and every string contains the empty string, so the containment tests in both classes admit it against the whole candidate set:

StandardizePropertyNames({"page_": v})  ->  {"title": v}
MergeSimilarProperties({"page_", "description"}, Maximum)  ->  merged into "description"

The value is silently reattributed to an unrelated property — the failure mode this issue's earlier note was careful to avoid ("a preserved key getting silently renamed — data loss, in a library whose job is preserving frontmatter"), arriving by a different route.

Padding was masking part of it by accident: " page_ " survived only because the missing trim blocked the prefix strip, while bare "page_" did not. So folding the helpers without fixing this would have made the failure uniform rather than removed it — the two changes had to go together. Both call sites now treat an empty normalized form as "nothing to match on".

Three regression tests, all three failing against the previous implementation. Suite is 137 green, with the 129 pre-existing tests unchanged.

Still open here

The PropertyMerger half, exactly as recorded: per-word fuzzy scoring works and keeps the admitted set bit-identical, but there is no case reachable through the public API where it picks a different key from CalculateWordMatchScore, so it would land as an unpinned behaviour change on a merge path. That still needs a FrontmatterMergeStrategy.Maximum case that reaches FindSemanticCanonicalName and discriminates between the two scorers — a question about the merger's real inputs, not about the library swap.

One adjacent looseness noticed and deliberately left alone: a key normalizing to a single character still containment-matches any standard property containing it, so meta_x_field → x → next. That is main's behaviour today; #124 only makes the padded form agree with the unpadded one. Narrowing it is a separate behaviour judgement and probably wants its own issue.

---

Generated by Claude Code

[matt-edmondson]

Triage

Category: Improvement (cross-library reuse) — partly in progress, remainder blocked on a decision
Priority: Medium
Area: PropertyNameNormalizer, PropertyMerger.FindSemanticCanonicalName

Re-triaging because half of this issue is now in an open PR and the halves have different dispositions, and the issue is open and unassigned.

Medium rather than Low, which is a change in emphasis from a pure reuse cleanup: the work above turned up a data-loss bug reachable on main today, no padding required. A decoration-only key (page_, _value, _) normalizes to the empty string, every string contains the empty string, and both containment tests therefore admit it against the entire candidate set — so {"page_": v} silently becomes {"title": v}. In a library whose job is preserving frontmatter, a value being reattributed to an unrelated property is the worst failure mode available, and it is exactly the one this issue's earlier note was trying to avoid by a different route.

That is also why the two changes could not be separated: folding the helpers without the fix would have made the failure uniform across both call sites rather than removed it.

Suggested assignment: the PropertyMerger remainder needs whoever knows the merger's real inputs. The landed half needs a reviewer, not an owner.

Duplicates: none. Part of the ktsu-reuse rotation tracked in ktsu-dev/.github#15.

Already in progress — open PR #124 covers the NormalizePropertyName fold and the empty-normalization fix: one PropertyNameNormalizer, three regression tests (all failing against the previous implementation), 137 green. It does not close this issue — the ktsu.FuzzySearch adoption in PropertyMerger is untouched by it. The repo's other open PR, #126 (keying content caches by input rather than a 32-bit hash), is unrelated.

Suggested next step: get #124 reviewed and merged on its own merits — the data-loss fix should not wait on the reuse decision it arrived alongside. Then the remaining question is narrow and answerable: per-word fuzzy scoring keeps the admitted set bit-identical, and there is no input reachable through the public API where it picks a different key from CalculateWordMatchScore, so adopting it today would land as an unpinned behaviour change on a merge path. Either produce a FrontmatterMergeStrategy.Maximum case that reaches FindSemanticCanonicalName and discriminates between the two scorers — which pins the change and makes it safe — or close the reuse half as won't-do, because a swap no test can distinguish is all risk and no benefit.

The single-character looseness noted and deliberately left alone (meta_x_field → x → next) is the same class of bug as the empty-string one, one step less severe. Worth its own issue rather than leaving it in a comment — it is main's behaviour today and will outlive this thread.

---

Generated by Claude Code

[matt-edmondson]

Decision (maintainer, 2026-09-28)

• Adopt ktsu.FuzzySearch in PropertyMerger too. The previous triage offered "close the reuse half as won't-do", and that option is rejected. This issue stays open until the PropertyMerger half lands.
• This follows the org-wide reuse rule "prefer reuse, accept costs; fix gaps upstream" (Cross-library reuse audit log .github#15). The earlier concern was a swap that no existing test can distinguish, and the rule accepts that cost. The change still gets pinned by a test, per below.
• Use the approach already prototyped and reverted: per-word fuzzy scoring, with CalculateWordMatchScore demoted to a boolean SharesRelatedWord gate so the admitted set stays bit-identical.

Next reader: re-apply per-word Fuzzy.Contains scoring in FindSemanticCanonicalName. Add a FrontmatterMergeStrategy.Maximum test that reaches the semantic matcher and pins the new ranking. If ktsu.FuzzySearch itself has a gap, fix it there rather than locally. #124 (the normalizer fold and the data-loss fix) should still merge on its own.

---

Generated by Claude Code

[matt-edmondson]

The PropertyMerger half of this is covered by #180 (Maximum adopts per-word FuzzySearch scoring there). Implement that half through #180.

---

Generated by Claude Code