Increment 16b-0: the noder's verifier by sort and sweep (+ the 16b design)

ROADMAP.md

@@ -45,7 +45,7 @@ increment that most needs a picture to check against
 | — | The serial phase: profile refine's serial insert-and-flip phase, then parallelise what the profile blames. Scaling tops out at about 2.0-2.2×. Profiled 2026-09-27: serial part about a third of 1-thread refine, mostly Lawson legalisation; the scan stops near 5× from load imbalance | profiled; designed as increment 21 (Ola's rulings 2026-09-27: L1 determinism, one path, at most 2 % more triangles). **21a shipped with branch `increment21a-quick-wins`'s PR**: dynamic scan blocks, active merge, reused flip stack; mesh bit-identical; on AC refine -10 to -15 % at 8 threads, ceiling 2.1x -> 2.3x (`docs/benchmarks/2026-09-27/21a-acceptance.md`). **21b shipped with branch `increment21b-lattice-incircle`'s PR**: an int64 lattice incircle answers 99.97-100 % of refine's incircle tests; mesh bit-identical; on battery refine -15 % at 8 threads, ceiling 2.3x -> 2.5x (`docs/benchmarks/2026-09-27/21b-acceptance.md`). **21c measured** (`docs/benchmarks/2026-09-27/21c/README.md`): option C costs 6.5-10 % more triangles; A1 about 1 %; A0 is bit-identical (on these two inputs, not proven), and with evaluate-once is modelled at only 7-11 % faster refine at 8 threads and slower at 4. **21d is deferred** (Ola, 2026-09-27: "Review and push 21c, then basin-work"), behind the work in "Order of work" below; domain decomposition parallelises scan and split together | `docs/increments/21-parallel-refine.md`, `docs/benchmarks/2026-09-27/serial-profile/README.md` |
 | — | Release hardening: measure libc++'s `_LIBCPP_HARDENING_MODE_FAST` (and libstdc++'s assertions on the GCC leg), which bounds-check `std::vector`, `span` and the like in Release, on the 1 m benchmark; switch it on if the cost is small. Today an out-of-range index in shipped code the tests miss is undefined behaviour and crashes Python (Ola, 2026-09-27: "I'm surprised we don't have proper memory control"; CI's ASan/UBSan/TSan cover what the tests reach) | to measure (Ola, 2026-09-27); 21d, which it was placed after, is deferred behind the basin work | none yet |
 | 15 | A DEM in several tiles, and the domain in its own CRS: 15a Norway, many tiles in one projected CRS (Ola's 254 DTM10 UTM33 tiles; only the selected tiles must share a lattice, since 8 of them sit half a cell off); 15b the domain polygon reprojected into the DEM's CRS; 15c and 15d the São Francisco basin (geographic DEMs meshed in the DEM's own lattice frame, not resampled; window decoding and memory) | **15a shipped with branch `increment15-dem-mosaic`'s PR**: `--dem DIR` or several files, `--bbox`, tiles selected and stitched on one lattice, Q2-Q5 refusals; overlaps that disagree (real DTM10 tiles exported on different dates, up to 52 m) are split down the middle and reported per seam (Ola's Q1 revised, 2026-09-28). The design's acceptance box, 9 tiles and 10,051² nodes, meshes: 11.05 M triangles in 31 s on battery. **15b built with branch `increment15b-domain-crs`'s PR**: `--domain` in its own CRS (e.g. EPSG:4326) over a UTM33 mosaic, reprojected vertex by vertex. Then 16b (Ola, 2026-09-27: Norway first) | `docs/increments/15-dem-mosaic.md` |
-| 16b | Interior polygons and polylines as constraints ("terrain polygons": lakes, land cover, roads, rivers): `--features PATH`, a GeoJSON `FeatureCollection`, each feature naming a vocabulary property; closed or open `Breakline`s, crossings noded, off-node vertices with bilinear z. **Its working example is real data** (Ola, 2026-09-27): CORINE Land Cover 2018 over the benchmark tile `7908_3_10m_z33.tif`. The source is Ola's local copy, `rasputin_data/corine_sql/.../U2018_CLC2018_V2020_20u1.gpkg` (8.2 GB, EPSG:3035, a sibling of this repository, which also holds the 254-tile DTM10 archive for gap 6). It reads without GDAL: sqlite3 over its R-tree, the GeoPackage blob header stripped, `shapely.wkb`, then `pyproj` to 25833, so CRS stops in Python as before. Probed 2026-09-27 in 0.2 s: 60 polygons in 8 classes (heath, bare rock, sparse vegetation, bogs, intertidal flats, water, sea, urban), 11 068 vertices clipped to the tile, median segment 54 m against 10 m cells. The EEA's public ArcGIS service (`image.discomap.eea.europa.eu`, `Corine/CLC2018_WM`) returns the same 11 068 clipped vertices and is the route for anyone without the file. What it forces on 16b's design: neighbouring polygons share their boundaries, so each shared edge arrives twice; the polygons run past the domain and must be clipped; the extract is committed as a fixture with the Copernicus attribution. Placed before 20c because 20c may split constraint segments and should be designed and measured on inputs that have interior ones | designed in 16's R6, no increment file yet; after the serial phase, before 20c | `docs/increments/16-domain-polygon.md` (R6) |
+| 16b | Interior polygons and polylines as constraints ("terrain polygons": lakes, land cover, roads, rivers): `--features PATH`, a GeoJSON `FeatureCollection`, each feature naming a vocabulary property; closed or open `Breakline`s, crossings noded, off-node vertices with bilinear z. **Its working example is real data** (Ola, 2026-09-27): CORINE Land Cover 2018 over the benchmark tile `7908_3_10m_z33.tif`. The source is Ola's local copy, `rasputin_data/corine_sql/.../U2018_CLC2018_V2020_20u1.gpkg` (8.2 GB, EPSG:3035, a sibling of this repository, which also holds the 254-tile DTM10 archive for gap 6). It reads without GDAL: sqlite3 over its R-tree, the GeoPackage blob header stripped, `shapely.wkb`, then `pyproj` to 25833, so CRS stops in Python as before. Probed 2026-09-27 in 0.2 s: 60 polygons in 8 classes (heath, bare rock, sparse vegetation, bogs, intertidal flats, water, sea, urban), 11 068 vertices clipped to the tile, median segment 54 m against 10 m cells. The EEA's public ArcGIS service (`image.discomap.eea.europa.eu`, `Corine/CLC2018_WM`) returns the same 11 068 clipped vertices and is the route for anyone without the file. What it forces on 16b's design: neighbouring polygons share their boundaries, so each shared edge arrives twice; the polygons run past the domain and must be clipped; the extract is committed as a fixture with the Copernicus attribution. Placed before 20c because 20c may split constraint segments and should be designed and measured on inputs that have interior ones | **designed** (`@architect`, 2026-09-28): two PRs, 16b-0 (the noder's verifier indexed: it is quadratic, 25 s of a 30 s CORINE run) and 16b-1/2 (GeoJSON and GeoPackage features, class maps, clip as linework, the noder merges shared edges); Q1-Q6 ruled by Ola 2026-09-28 (Q6: the legacy CORINE GML stays and is read). **16b-0 shipped with branch `increment16b0-verifier-sweep`'s PR**: the verifier's pair search by sort and sweep, always on; `node` on a 48 km CORINE square 24.8 s → 0.067 s. 16b-1/2 next; before 20c | `docs/increments/16b-terrain-polygons.md`, `docs/increments/16-domain-polygon.md` (R6) |
 | 20c | Soft quality criterion: a penalty that each Steiner node or constraint split must pay for in angle gained, instead of 20's hard 25°; applied at the start and during DEM refinement; may split constraint segments when that improves the mesh. Ola's rulings on 20's C1-C3 | to design after 16b (`@architect` measures cost against 20 first) | `docs/increments/20-start-quality.md` (Ola's rulings) |
 | — | Auto-catchment: the watershed upstream of a coordinate, computed from the DEM and handed to `--domain`, so a catchment no longer has to be supplied as a file (Ola, 2026-09-27: "not far into the future"). The textbook route is depression handling (Priority-Flood, Barnes, Lehman and Mulla 2014), D8 flow directions (O'Callaghan and Mark 1984) and accumulation, the pour point snapped to the strongest flow nearby, the upstream cells traced and their outline turned into a polygon; the literature check is `@architect`'s. Open for its design: whether it runs in the C++ core (a 10 m tile is 25 M cells); how a stair-stepped cell outline becomes a domain polygon, which meets input coarsening; and that a real catchment crosses tile edges, so it needs gap 6 (a DEM in several tiles) first. Legacy has nothing on it (`grep -rliE "watershed|flow.?acc|flow.?dir|pour.?point|catchment" legacy` returns no files) | to design; after gap 6, which it needs; placed after 20c, can move ahead of it on Ola's word | none yet |
 | — | `raster/`: grid-to-world geometry and bilinear sampling | shipped (`7785fea`), **no record** | none — predates the protocol |

docs/benchmarks/2026-09-28/16b0-acceptance.md

@@ -0,0 +1,34 @@
+# Increment 16b-0 acceptance (@perf, 2026-09-28): summary
+
+**Verdict: ACCEPTED.** Measured on battery against a battery baseline, back
+to back. The branch is `4e83389`; the base is master `dc372a5`. Method,
+tables and raw data: `16b0-acceptance/README.md`.
+
+- **1 m benchmark and thread sweep** (`tools/bench.py`, three pairs): no
+  change. Pooled over all 42 cells (each cell's median over the three base runs
+  against its median over the three 16b-0 runs), the change is a median of
+  -0.28 % (range -5.9 % to +2.8 %). bench.py said ACCEPTED on pairs 1 and 3. On pair 2 it
+  flagged two tile cells (+6.9 % at 10 threads, +8.0 % at 16); those cells
+  were +0.5 % to +2.1 % in the other pairs. The same build moves by up to
+  +13.5 % between runs, and `refine` does not compile the changed header.
+  The mesh sha256 is equal on both domains in all six runs. Ceiling:
+  2.5-2.9x, as the base.
+- **`node` on M3's CORINE squares.** Layout B at 48 km: **84.05 s -> 0.1435 s
+  (586x)**, against the Acceptance section's 83.1 s. Layout A at
+  12/24/36/48 km: 0.0041/0.0174/0.0364/0.0668 s (reference
+  0.0041/0.0167/0.0364/0.0670). From 12 to 48 km the segments grow 13.8x;
+  `node` grows 16-18x, against 185x on master. A 144 km block (870 k segments,
+  layout B) nodes in 2.2 s. The noded output is identical between layouts A
+  and B and between master and 16b-0.
+- **The admitted worst case.** The ladder at 300 rungs takes 0.262 s
+  (master: 210 s); at 1 200 rungs (2.9 M noded edges) it takes 17.3 s. The
+  comb of parallel east-west lines is quadratic, as R8 admits: 4x the lines
+  costs 14.4-15.6x, and 64 000 lines take 21.4 s. The split between the
+  driver and the verifier has not been isolated.
+- **The first CORINE baseline** (48 km square in `6603_4`, layout A, 16b-0).
+  At 1 m: 6.75 M triangles, `node` 0.068 s, refine 5.48 s, peak RSS 2.35 GB.
+  At 10 m, **the quality start makes the mesh 3.61x the featureless one**
+  (480 961 triangles, against 212 255 with the quality start off and 133 379
+  without features). That is 20c's input. `node` is no longer the cost.
+- **Not measured**: a verifier-only split on the ladder and comb; master at
+  96/144 km and on the large synthetic cases; any AC run.

docs/benchmarks/2026-09-28/16b0-acceptance/16b0-r2/README.md

@@ -0,0 +1,80 @@
+# Benchmark run `16b0-acceptance/16b0-r2`
+
+Generated by `tools/bench.py`.
+
+## Verdict
+
+- REGRESSION: tile refine_s[t=10] 0.1860 -> 0.1989 (+6.9 %)
+- REGRESSION: tile refine_s[t=16] 0.1906 -> 0.2059 (+8.0 %)
+
+Time threshold: 5.0 %.
+
+## Method
+
+- Started 2026-09-28T15:28:37.286233+02:00; tree `4e833893377430683ac1dce967783ead5ff9a2de` (dirty); bench.py blob `77765b181714c31714da476c2335c799c6a84021`. Build: Release, AppleClang 21.0.0.21000101, `-O3 -DNDEBUG`, _core sha256 `190587e7ba0ea96a74a8621767d7d20257c3968126a710db1364e1195afb27d2`.
+- Apple M1 Max, 8 P + 2 E cores, 32 GiB, macOS 27.0, Python 3.14.7, numpy 2.5.3.
+- Power **battery** (72%), `pmset -g batt` before and after (in run.json).
+- DEM `/Users/skavhaug/projects/rasputin/tests/fixtures/dem_archive/7908_3_10m_z33.tif` (sha256 `aabd0cbc28471ce8593e4c278381811bec3fbf4e1194058b3b4888d00c3af575`), tolerance 1.0, extra mesh args ``; domains: `quarter` (/Users/skavhaug/projects/rasputin/docs/benchmarks/2026-09-26/quarter.geojson), `tile` (the whole tile).
+- One child per sample, `/Users/skavhaug/projects/rasputin/.venv/bin/python /Users/skavhaug/projects/rasputin/tools/bench.py _child --pkg /Users/skavhaug/projects/rasputin/build-bench/pkg --threads <N> -- mesh --dem /Users/skavhaug/projects/rasputin/tests/fixtures/dem_archive/7908_3_10m_z33.tif --tolerance 1 --out <path> --binary`, repeats interleaved over thread counts; t=0 is the CLI's default, other counts are forced into `refine`. Quality: one `--ascii` run per domain at t=0, kept out of the repository; rerun the child with `--ascii --out PATH` to regenerate it.
+
+## `quarter`
+
+| threads | n | median s | min s | max s |
+|---:|---:|---:|---:|---:|
+| 0 | 5 | 0.1645 | 0.1634 | 0.1955 |
+| 1 | 5 | 0.4241 | 0.4132 | 0.4549 |
+| 2 | 5 | 0.2765 | 0.2663 | 0.3091 |
+| 3 | 5 | 0.2262 | 0.2212 | 0.2617 |
+| 4 | 5 | 0.2013 | 0.1978 | 0.2109 |
+| 5 | 5 | 0.1881 | 0.1847 | 0.1912 |
+| 6 | 5 | 0.1753 | 0.1733 | 0.1833 |
+| 7 | 5 | 0.1704 | 0.1669 | 0.2283 |
+| 8 | 5 | 0.1634 | 0.1624 | 0.1708 |
+| 9 | 5 | 0.1660 | 0.1641 | 0.1819 |
+| 10 | 5 | 0.1644 | 0.1626 | 0.1710 |
+| 11 | 5 | 0.1635 | 0.1602 | 0.1774 |
+| 12 | 5 | 0.1652 | 0.1628 | 0.1880 |
+| 13 | 5 | 0.1691 | 0.1643 | 0.1740 |
+| 14 | 5 | 0.1673 | 0.1603 | 0.1754 |
+| 15 | 5 | 0.1638 | 0.1620 | 0.1718 |
+| 16 | 5 | 0.1703 | 0.1644 | 0.1737 |
+| 17 | 5 | 0.1659 | 0.1631 | 0.2303 |
+| 18 | 5 | 0.1657 | 0.1647 | 0.1892 |
+| 19 | 5 | 0.1679 | 0.1650 | 0.1710 |
+| 20 | 5 | 0.1691 | 0.1631 | 0.1808 |
+
+Ceiling: 2.51x at 20 threads over 1, best 2.59x at 8; 2026-09-26: about 2.2x, flat from about 7.
+
+Quality: worst angle 0.3955 deg, median 45.00, share under 1 deg 0.00003, max degree 18, within tolerance True, Delaunay 0 violations of 641791 edges (80136 decided exactly), mesh sha256 `ccebf96a86c6c5e244e4a0281919de4e866fcfe789b66024a290ac2af33771a1`.
+
+## `tile`
+
+| threads | n | median s | min s | max s |
+|---:|---:|---:|---:|---:|
+| 0 | 5 | 0.1923 | 0.1851 | 0.2074 |
+| 1 | 5 | 0.4771 | 0.4620 | 0.5637 |
+| 2 | 5 | 0.3092 | 0.3050 | 0.3685 |
+| 3 | 5 | 0.2618 | 0.2510 | 0.3227 |
+| 4 | 5 | 0.2296 | 0.2250 | 0.2526 |
+| 5 | 5 | 0.2175 | 0.2092 | 0.2440 |
+| 6 | 5 | 0.2071 | 0.1982 | 0.2148 |
+| 7 | 5 | 0.1965 | 0.1940 | 0.2545 |
+| 8 | 5 | 0.1872 | 0.1838 | 0.1938 |
+| 9 | 5 | 0.1923 | 0.1879 | 0.1940 |
+| 10 | 5 | 0.1989 | 0.1834 | 0.2022 |
+| 11 | 5 | 0.1952 | 0.1865 | 0.2134 |
+| 12 | 5 | 0.1915 | 0.1863 | 0.2067 |
+| 13 | 5 | 0.1904 | 0.1832 | 0.2016 |
+| 14 | 5 | 0.1924 | 0.1810 | 0.1956 |
+| 15 | 5 | 0.1893 | 0.1822 | 0.2128 |
+| 16 | 5 | 0.2059 | 0.1891 | 0.2099 |
+| 17 | 5 | 0.1879 | 0.1855 | 0.2008 |
+| 18 | 5 | 0.1915 | 0.1860 | 0.2001 |
+| 19 | 5 | 0.1936 | 0.1872 | 0.2199 |
+| 20 | 5 | 0.1892 | 0.1846 | 0.2069 |
+
+Ceiling: 2.52x at 20 threads over 1, best 2.55x at 8; 2026-09-26: about 2.2x, flat from about 7.
+
+Quality: worst angle 0.6296 deg, median 45.00, share under 1 deg 0.00000, max degree 74, within tolerance True, Delaunay 0 violations of 692056 edges (94469 decided exactly), mesh sha256 `11741a81adfa17b34a1ea56875ec9e791414d17625eb1c58305e7c0cf10e005c`.
+
+<!-- bench.py: generated above this line; hand-written prose below is kept -->