Hash make_distinct intervals on data identity

[ESultanik]

make_distinct in graphtage/bounds.py builds one intervaltree.Interval per bounded object and
puts them all in an IntervalTree. intervaltree.Interval hashes on (begin, end) alone, while its
equality test also compares data, and IntervalTree keeps its intervals in set objects. At the
start of a bipartite match every edge reports near-identical initial bounds, so hundreds of intervals
collapse into a handful of hash buckets and each set.add and membership test degenerates into a
linear scan of equality tests.

Profile evidence

A 24-key dict-versus-dict JSON diff, profiled with cProfile before the change:

   ncalls  tottime  cumtime  filename:lineno(function)
 44018706    6.384    6.384  intervaltree/interval.py:159(__eq__)
  1042569    2.267    7.731  {method 'add' of 'set' objects}
  1042667    0.112    0.161  intervaltree/interval.py:151(__hash__)

44.0 M equality tests against only 1.04 M hash computations is about 42 equality probes per lookup,
and accounts for 6.38 s of the 9.96 s profiled run. After the change, Interval.__eq__ no longer
appears in the top 15 entries and the same profiled run takes 1.10 s.

The change

graphtage/bounds.py gains an IdentityInterval subclass whose __hash__ is
hash((self.begin, self.end, id(self.data))), and make_distinct builds its intervals from it.
Graphtage's edits use identity equality — AbstractEdit.__eq__ and __hash__ are the ones inherited
from object — so folding id(data) into the hash agrees with the equality test that
intervaltree.Interval already performs. It only spreads the buckets.

The subclass is the route taken here rather than replacing intervaltree with a sorted list and
bisect. The risk with a subclass is that IntervalTree might build a plain Interval internally
for a lookup: a plain instance and a subclass instance hash differently, so such a lookup would miss
and results would be wrong without any error. That was checked empirically rather than by reading.
Instrumenting Interval.__new__ and exercising the add, remove, overlap, overlaps, __len__,
and __iter__ paths that make_distinct uses shows IntervalTree constructs no intervals of its
own on any of them. Range queries, membership tests, and removals were also compared against a
brute-force scan and agree. The one method that does build a plain Interval, Interval.copy, is on
no path make_distinct touches. The constraint is recorded in the class docstring.

Measurements

Random dicts of n keys against n keys with all keys differing, built with
graphtage.json.build_tree, diffed with t1.diff(t2), and rendered with JSONFormatter into a
quiet, uncolored Printer:

n	before	after	speedup	rendered output identical
16	0.559 s	0.101 s	5.5x	yes
24	5.936 s	0.332 s	17.9x	yes
32	34.421 s	0.895 s	38.4x	yes
40	126.885 s	1.985 s	63.9x	yes

The rendered diff is byte-for-byte identical before and after at every size. The change affects only
how the interval tree distributes its entries across hash buckets, not which intervals it stores or
which ones a query returns.

Tests

Two tests in test/test_bounds.py, both driven by a new CollidingRange helper whose instances all
report the same initial bounds:

• test_make_distinct_with_identical_initial_bounds asserts the postcondition callers rely on: after
  make_distinct, the sorted ranges are either definitive or non-overlapping. It asserts the outcome,
  not the data structure, so it also covers a future rewrite of make_distinct. Verified by dropping
  the re-add of the second-biggest interval, which makes the test fail with
  Range(204, 209) was followed by Range(209, 214).
• test_make_distinct_does_not_probe_quadratically counts data equality comparisons during a
  64-element run and requires them to stay linear in the input size. Verified by reverting
  IdentityInterval.__hash__ to the inherited (begin, end) hash, which takes the run from 0
  comparisons to 800,996.

pytest -q passes in full: 193 tests and 23 subtests. ruff check graphtage test docs bindist is
clean.

🤖 Generated with Claude Code