Skip to content

python-math #26: feat(math): Q11 — replicate vectorz distance cancellation in legacy kmeans - #2654

Draft
jucor wants to merge 1 commit into
spr/edge/00d8bc52from
spr/edge/7ae9c688
Draft

python-math #26: feat(math): Q11 — replicate vectorz distance cancellation in legacy kmeans#2654
jucor wants to merge 1 commit into
spr/edge/00d8bc52from
spr/edge/7ae9c688

Conversation

@jucor

@jucor jucor commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Why

Quirk Q11 (delphi/docs/CLOJURE_QUIRKS.md): Clojure's kmeans distances go through vectorz's matrix/distance on ArraySubVector row views, which computes d^2 = |a|^2 + |b|^2 - 2ab; the cancellation in that formula floors true distances below ~1e-8 to EXACTLY 0.0, so near-coincident points TIE against multiple clusters and min-key's last-wins tie-break merges them into the LATER cluster (the emptied cluster is dropped). Verified in-process on the vw every-vote step-57 pair (true distance 4.66e-15 → both cluster distances 0.0 → merge), via the new cache-safe diagnostic math/dev/proj_probe.clj, which also reproduces the recording chain bit-exactly.

What

_euclidean in legacy_kmeans.py now uses the same formula (the module runs only in clojure-legacy mode — gated by construction). numpy reproduces the 0.0 collapse bit-for-bit on the real pair. The pre-Q11 all-singletons expectation in test_base_cluster_lineage.py pinned Python's old difference-based distance, not Clojure — updated to the verified contract (merges only ever happen at Q11-distance 0.0).

Testing

TDD: RED on the real step-57 pair (unit + warm-start merge integration).

commit-id:7ae9c688


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

This was referenced Jul 22, 2026
@jucor jucor changed the title feat(math): Q11 — replicate vectorz distance cancellation in legacy kmeans feat(math): Q11 — replicate vectorz distance cancellation in legacy k… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/00d8bc52 to edge July 22, 2026 09:58
@jucor
jucor force-pushed the spr/edge/7ae9c688 branch from 20e58ae to 2ac4c32 Compare July 22, 2026 09:58
This was referenced Jul 25, 2026
@jucor
jucor force-pushed the spr/edge/00d8bc52 branch from 6fac726 to 97c8fa1 Compare July 28, 2026 00:11
@jucor
jucor force-pushed the spr/edge/7ae9c688 branch from 2ac4c32 to 3861330 Compare July 28, 2026 00:11
@jucor jucor changed the title feat(math): Q11 — replicate vectorz distance cancellation in legacy kmeans python-math #26: feat(math): Q11 — replicate vectorz distance cancellation in legacy kmeans Jul 28, 2026
…ation in legacy kmeans

## Why

Quirk Q11 (`delphi/docs/CLOJURE_QUIRKS.md`): Clojure's kmeans distances go through vectorz's `matrix/distance` on ArraySubVector row views, which computes d^2 = |a|^2 + |b|^2 - 2ab; the cancellation in that formula floors true distances below ~1e-8 to EXACTLY 0.0, so near-coincident points TIE against multiple clusters and `min-key`'s last-wins tie-break merges them into the LATER cluster (the emptied cluster is dropped). Verified in-process on the vw every-vote step-57 pair (true distance 4.66e-15 → both cluster distances 0.0 → merge), via the new cache-safe diagnostic `math/dev/proj_probe.clj`, which also reproduces the recording chain bit-exactly.

## What

`_euclidean` in `legacy_kmeans.py` now uses the same formula (the module runs only in clojure-legacy mode — gated by construction). numpy reproduces the 0.0 collapse bit-for-bit on the real pair. The pre-Q11 all-singletons expectation in `test_base_cluster_lineage.py` pinned Python's old difference-based distance, not Clojure — updated to the verified contract (merges only ever happen at Q11-distance 0.0).

## Testing

TDD: RED on the real step-57 pair (unit + warm-start merge integration).

commit-id:7ae9c688
@jucor
jucor force-pushed the spr/edge/7ae9c688 branch from 3861330 to 4f4cefa Compare July 28, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants