Wire the batched Levenshtein kernel into the diff engine

[ESultanik]

graphtage/batch_distance.py has been able to price a whole cross product of strings in one vectorized batch
since #195, and nothing called it. This wires it into the engine.

After #197 made StringEdit a ConstantCostEdit, levenshtein_distance was 71 to 76 percent of the remaining
time of a diff of 300 strings, reached through the single seam of exact_string_distance. That function prices
one pair, so a batch has to be filled before it is called. Two places know a cross product is coming and can fill
it: MultiSetEdit.__init__ and EditDistance.__init__.

Results

Documents are JSON lists and dictionaries of random lowercase words of 6 to 24 characters, each element mutated by
one substitution, deletion, or insertion. The unordered shapes are built with ignore_list_order, which routes
them through MultiSetEdit and the bipartite matcher. Measured on an Apple M3 Max, Python 3.14.0.

shape	before	after	speedup	total edit cost
ordered list, 300 elements	1.994 s	0.508 s	3.9x	8837 both
unordered, 300 elements	1.870 s	0.352 s	5.3x	1737 both
dictionary, 300 keys and values mutated	2.976 s	0.984 s	3.0x	1776 both
ordered list, 400 elements	3.508 s	0.998 s	3.5x
unordered, 400 elements	3.287 s	0.647 s	5.1x
ordered list, 500 elements	5.410 s	1.624 s	3.3x
unordered, 500 elements	5.299 s	1.015 s	5.2x
ordered list, 100 elements	0.209 s	0.061 s	3.4x	2679 both
unordered, 100 elements	0.204 s	0.150 s	1.4x	578 both
ordered list, 30 elements	15.9 ms	6.2 ms	2.6x	724 both
ordered list, 10 elements	2.0 ms	1.0 ms	2.0x	212 both
ordered list, 60 strings of 200 to 400 characters	30.42 s	1.23 s	24.8x	120 both
8 lists of 40 strings	2.632 s	0.811 s	3.2x	945 both
12 lists of 30 strings	3.324 s	1.346 s	2.5x	1068 both
16 lists of 20 strings	2.638 s	1.476 s	1.8x	945 both

The long-string shape is the largest win because its pairs are big enough that the scan beats the scalar dynamic
program on the count of array passes rather than only on interpreter overhead. The 100 element unordered shape is
the smallest: 9,000 pairs of short strings is not much for a batch to amortize over, and what remains is the
matcher rather than the distances.

In the profile of the 300 element ordered shape, levenshtein_distance no longer appears at all, where it was 76
percent before. What is left is spread across the edit machinery itself: the largest single entry is
EditDistance._best_match at 11 percent, followed by _exact_cost, Range arithmetic, and isinstance.

Design

A cost oracle in batch_distance. cost(a, b) answers one pair, from a pre-priced block when one holds it
and by computing it otherwise. preprice_product(S, T) and preprice(pairs) fill blocks.

A block is a row index, a column index, and a numpy.int32 matrix, not a dict keyed on pairs of strings. For a
400 by 400 product that is about 640 KB against tens of megabytes of tuples and hashes. Cells that no pair filled hold
a sentinel, so a sparse block reports an unasked pair as absent rather than as a distance of zero.

The stack holds at most 8 blocks and is searched newest first. Evicting the oldest is always safe because the
values are deterministic, so an eviction costs a recomputation. Misses are deliberately not recorded: recording
them would grow the cache with the number of distinct pairs a diff asks about, which is the whole cross product,
and nothing would ever evict it. Installing a block replaces the stack rather than mutating it, so a reader never
sees a partial update and a lost race costs only the block it was installing.

The seam. exact_string_distance calls batch_distance.cost through a function-local import, because
batch_distance imports levenshtein_distance; bounds.make_distinct already defers an import the same way. The
miss path keeps the shared prefix and suffix strip and then chooses between the scalar dynamic program and a
one-pair vectorized scan by the size of the matrix.

Populator 1, MultiSetEdit.__init__. Projects each candidate node pair onto the strings its edit will compare:
two leaves give one pair, two key/value pairs give their keys and their values, and two containers give nothing.
The auto_match_keys loop interleaved deciding which key/value pairs share a key with constructing their edits,
which put the first cost before the batch; it is split into _match_by_key and a second pass that builds the
edits, preserving the multiplicity of a pair that repeats.

Populator 2, EditDistance.__init__. Collects the leaves of both sequences after the shared prefix and suffix
strip and prices the product. A new keyword-only preprice argument defaults to True, and
string_edit_distance passes False: the character-level lattice is itself an EditDistance, and any implicit
guard would still cost a pass over every cell of every string a diff renders.

WeightedBipartiteMatcher._make_edges_distinct. StringEdit knows its cost at construction, so
make_distinct has nothing to tighten, but it built an interval tree over every edge before finding that out.
On the 300 element unordered shape that was 0.87 seconds of a 4.62 second profile. The shortcut is exactly
equivalent: make_distinct tightens a bound only when it is not finite, and a definitive bound is finite.

Cell edits are not replaced with Match objects anywhere. StringNode.edits still returns a StringEdit, so a
substituted element still renders a character-level diff.

Thresholds

PREPRICE_MIN_PAIRS = 32. Diffing an ordered list of n strings, minimum of 15 runs, in milliseconds:

elements	pairs	not pre-priced	pre-priced
3	9	0.210	0.269
5	25	0.458	0.543
6	36	0.619	0.518
7	49	0.866	0.637
8	64	1.270	0.727
16	240	4.538	2.004
20	360	6.936	2.999
30	810	15.323	6.055

The crossing is between 25 and 36 pairs, which is where NUMPY_MIN_PAIRS = 32 already sits: below it a batch is
answered by the python backend, so a block buys the cost of vectorizing without the vectorization. The sweep
over the threshold itself agrees, in milliseconds:

PREPRICE_MIN_PAIRS	9 pairs	25 pairs	64 pairs	240 pairs	360 pairs	810 pairs
0	0.272	0.541	0.723	2.063	2.999	6.055
16	0.210	0.540	0.726	2.068	2.981	6.027
32	0.209	0.455	0.730	2.055	3.006	6.032
64	0.210	0.456	0.728	2.015	2.948	5.996
128	0.210	0.452	1.323	2.061	3.005	6.032
256	0.209	0.453	1.253	4.544	3.015	6.032
512	0.215	0.489	1.340	4.627	6.974	5.961
1024	0.209	0.483	1.317	4.570	6.936	15.323

VECTORIZED_MIN_CELLS = 4096, which is 64 by 64 characters. Microseconds per pair, 20 random pairs per row:

length	cells	scalar	scan	ratio
8	64	6.0	43.3	0.14
16	256	21.7	75.4	0.29
24	576	47.3	107.8	0.44
32	1,024	82.5	138.0	0.60
48	2,304	185.4	205.6	0.90
64	4,096	327.2	269.7	1.21
96	9,216	741.0	414.1	1.79
128	16,384	1,347.7	565.9	2.38
192	36,864	3,028.2	875.8	3.46
256	65,536	5,382.9	1,234.1	4.36
400	160,000	15,112.8	2,019.2	7.48

The crossing is near 52 by 52 characters; 64 by 64 sits just above it.

PREPRICE_MAX_CELLS = 4 * 1024 * 1024 is a safety valve rather than a tuned number. The pairs a caller offers
need not fill the rectangle their distinct sides span, and a batch whose rectangle would need more than 16 MB is
left to the scalar path.

NUMPY_MAX_CHUNK_BYTES is unchanged at 256 KiB, which is inside the measured optimal band.

MATCHING_SIZE_WARNING_THRESHOLD goes from 400 to 250,000. Four hundred pairs is now a few milliseconds. Diffing
unordered collections after this change:

elements	pairs	seconds
300	81,000	0.35
400	144,000	0.65
500	225,000	1.02
600	324,000	1.51

250,000 pairs is about 1.1 seconds, which is where a warning about a long wait earns its place.

Differential

Not committed. 100 document pairs, being 10 shapes rendered into all 10 formats (JSON, JSON5, XML, HTML, YAML,
TOML, INI, CSV, plist, pickle), diffed in three modes each: default, --only-edits, and --html. The shapes are
ordered lists, shuffled lists, ragged lists, dictionaries with and without shared keys, nested containers,
non-ASCII text including combining marks and astral characters, multi-line strings, long strings of 180 to 320
characters, and a list mixing strings with integers, floats, booleans, and null. Bytes reach the tree through the
plist and pickle documents.

stdout was captured and hashed, with stderr discarded so that the progress bar's timing noise does not enter the
comparison.

run	comparisons	mismatches
default backend selection	300	0
GRAPHTAGE_BATCH_BACKEND=python	300	0
GRAPHTAGE_BATCH_BACKEND=numpy	300	0

900 comparisons, zero differences. No edit cost changed. The oracle is exact and exact_string_distance was
already exact, so the costs are identical rather than merely non-increasing, and the rendered output is
byte-identical rather than merely equal in cost.

Tests

pytest -q: 272 passed, 1770 subtests. ruff check graphtage test docs bindist is clean, and the documentation
still builds under -W --keep-going.

The #196 contract tests pass unchanged: test_edit_script_tie_break_is_stable,
test_shared_prefix_biases_the_alignment, test_edit_script_realizes_the_reported_cost, and TestRenderedDiffs.

Every new test was written against a deliberate break in the code it covers, and each was confirmed to fail before
it was confirmed to pass. Thirteen breaks, thirteen caught:

break	caught by
block lookup transposes its axes	test_cost_matches_levenshtein_distance_from_a_block
unfilled block cells read as zero	test_pre_pricing_a_flat_list_leaves_unasked_pairs_absent
the block stack is never trimmed	test_eviction_does_not_change_answers
a miss is memoized	test_misses_are_not_memoized
a mixed str and bytes pair reaches a backend	test_pairs_that_mix_str_and_bytes_are_left_to_the_scalar_path
a single pair is never vectorized	test_one_big_pair_goes_to_a_vectorized_backend
an oversized rectangle is pre-priced anyway	test_an_oversized_rectangle_is_not_pre_priced
make_distinct is skipped for indefinite edges too	test_indefinite_edges_still_reach_make_distinct
key matching loses its multiplicity	test_repeated_key_value_pairs_match_as_many_times_as_they_repeat
key/value pairs contribute no strings to the batch	test_pre_pricing_does_not_change_a_diff
the character lattice is pre-priced too	test_a_character_lattice_is_not_pre_priced
the Levenshtein matrix pre-prices nothing	test_a_sequence_of_leaves_is_pre_priced
exact_string_distance bypasses the oracle	test_a_pre_priced_diff_reads_its_costs_out_of_the_block

One thing here cannot be tested by its effect. A block is keyed by string values and holds only the distance
between them, so a projection that does not match what an edit later asks for produces a cache miss, never a wrong
answer. test_pre_pricing_does_not_change_a_diff asserts that each shape installs a block at all, which catches a
projection that collects nothing, but a projection that collects the wrong strings is a lost optimization that no
assertion about output can see. The same applies to collecting before rather than after the shared prefix and
suffix strip: it prices pairs nobody asks for, and costs time rather than correctness.

Surprises

Pre-pricing pays off from about 36 pairs, an order of magnitude lower than expected. The batch does not have to be
large; it only has to be large enough for the numpy backend to accept it.

The interval tree in make_distinct was a bigger share of the unordered path than expected: 0.87 seconds of a
4.62 second profile, spent entirely on reaching a loop that breaks on its first iteration.

One caveat worth recording: pre-pricing in a constructor is eager, while the matrix it serves is built lazily. If
a parent stops tightening an EditDistance after a few diagonals, the pre-priced product is partly wasted. The
batch is roughly 15 times faster per pair, so the break-even is a parent that builds under a fifteenth of the
matrix. The nested shapes in the table are the case where that could bite, because each cell of the outer matrix
is an inner EditDistance that pre-prices its own product; they are 1.8 to 3.2 times faster, so it does not bite
there. PREPRICE_MIN_PAIRS keeps smaller nested sequences out of the question entirely.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F2sHz5c5TvMs9tFn2HhwaC