Build immutable offline global tileset pipeline

.gitignore

@@ -1,4 +1,5 @@
 node_modules/
+node_modules
 dist/
 .generated/
 public/style/occumed-open.json
@@ -11,3 +12,8 @@ public/sprites/occumed@2x.png
 .env.*
 !.env.example
 .DS_Store
+build/offline-global/
+config/immutable-owner-plan.json
+immutable-inputs/
+.build-tools/
+visual-validation/immutable-*/

README.md

@@ -19,9 +19,7 @@ The goal is a close visual replica without a Mapbox token or Mapbox-hosted runti
 
 - MapLibre GL JS
 - one permanent MapLibre vector source at `/tiles/{z}/{x}/{y}.pbf`
-- a server-side virtual worldwide tileset backed by the 754 PMTiles storage archives
-- one consolidated zoom 0–5 overview and one generalized worldwide physical surface
-- open elevation data used only for hillshade, not as a second basemap
+- one immutable worldwide foundation plus deterministic non-overlapping PMTiles owners
 - locally compiled sprites
 - browser-local glyph rendering
 - generated runtime style at `/style/occumed-open.json`
@@ -32,17 +30,19 @@ No `VITE_MAPBOX_ACCESS_TOKEN`, `mapbox-gl`, `mapbox://` URL, Mapbox API endpoint
 
 MapLibre sees only `occumed-open`, whose URL is permanent from zoom 0 through 16. The browser does not load the world manifest, select an archive, register a PMTiles protocol, or replace a source while the map moves.
 
-The Node tile gateway resolves each Z/X/Y request on the server:
+The existing worldwide and regional PMTiles archives are offline inputs only.
+The offline builder assigns one authority per layer family, clips and normalizes
+geometry, removes duplicates and contained overlaps, rejects malformed,
+oversized, and tile-shaped surface polygons, and writes each final Z/X/Y once.
 
-- zoom 0–5 comes from a consolidated overview built from the same regional schema;
-- zoom 6–16 is resolved against every storage shard intersecting the requested tile;
-- boundary tiles are decoded, deduplicated by stable feature ID, and re-encoded as one MVT;
-- the worldwide `land` layer is merged into the same response at every zoom;
-- nested Natural Earth bathymetry bands are served as the `depth` layer at globe
-  and regional zooms, then fade before detailed navigation zooms;
-- completed virtual tiles are held in a bounded in-memory cache and exposed with CDN cache headers.
+Production loads a versioned ownership manifest, performs one deterministic
+owner lookup, and returns the selected archive's stored MVT bytes unchanged.
+It does not connect to Neon, merge shards, synthesize landcover, create
+geometry, or stretch parent/child tiles. The browser never sees the owner
+inventory and never switches its single source.
 
-The PMTiles archives and routing manifest are storage implementation details. Their URLs never appear in the MapLibre style.
+See [the immutable tileset build guide](docs/offline-global-tileset.md) for
+the complete offline build and mandatory visual validation workflow.
 
 ## Source of truth
 
@@ -73,6 +73,12 @@ Optional Render variable:
 PUBLIC_ORIGIN=https://map-yxjb.onrender.com
 ```
 
+Required tileset location (unless deployed at `dist/immutable-world/manifest.json`):
+
+```text
+OCCUMED_IMMUTABLE_TILESET_MANIFEST=/absolute/path/to/immutable-world/manifest.json
+```
+
 ## Reuse
 
 Install the repository in the consuming application:
@@ -105,10 +111,11 @@ The build verifies:
 - the original export remains intact;
 - the generated style passes the MapLibre style specification;
 - no active source, sprite, or glyph URL points to Mapbox;
-- globe, terrain, landcover, water, labels, and viewer-quality settings remain calibrated to the screenshot reference set;
+- globe, landcover, water, labels, and viewer-quality settings remain calibrated to the screenshot reference set;
 - only one permanent vector source and one same-origin Z/X/Y template exist in the style;
 - browser-side PMTiles routing, `source.setUrl()`, fallback URLs, and OpenFreeMap are absent;
-- the routing index includes every intersecting shard, including antimeridian segments;
-- duplicate features are removed while boundary geometry is preserved;
-- the overview, physical surface, regional merge, in-memory cache, and virtual release workflow remain wired;
+- the immutable manifest is complete, non-overlapping, and fail-closed;
+- every split-prefix ancestor and descendant tile has one deterministic prebuilt owner;
+- production contains no Neon tile cache, runtime merge, geometry creation, landcover synthesis, or parent/child stretch path;
+- mandatory static and exact-camera motion captures reject seams, tile footprints, stretched polygons, inconsistent neighbors, blank frames, and source switching;
 - no application-specific overlay data is included.

docs/offline-global-tileset-recovery.md

@@ -0,0 +1,118 @@
+# Offline Global Tileset Recovery Handoff
+
+Status captured after Codex usage credits were exhausted on 2026-07-28/29.
+
+## Critical state
+
+- Target branch: `rebuild/offline-global-tileset`
+- Remote branch was still identical to `main` at `ba835de833b7c62d26f7767a0cf06ed8dec287ee` when this handoff was written.
+- The implementation described below had not yet been committed or pushed by Codex.
+- The Codex workspace is therefore the only known location of the uncommitted source changes and generated artifacts.
+- Do not delete or reset that Codex workspace before exporting/committing its changes.
+
+## Intended architecture
+
+- One browser vector source.
+- Immutable worldwide PMTiles output.
+- Deterministic non-overlapping owner partitions when a single physical file is too large.
+- Every `z/x/y` tile belongs to exactly one owner.
+- No Neon tile cache in the active production path.
+- No runtime shard merging.
+- No runtime landcover synthesis.
+- No runtime parent/child stretching or overscaling.
+- No production geometry creation.
+- Offline-only clipping, schema normalization, deduplication, ancestor materialization, and malformed-feature rejection.
+
+## Input inventory discovered
+
+- 754 regional PMTiles archives plus worldwide overview and surface archives.
+- Published regional input total reported by Codex: approximately 258.0 GB.
+- Inputs were resolved from the existing GitHub Release and SHA-256 locked.
+- Local `dist/virtual-assets` PMTiles files were only 325-byte fixtures and were correctly rejected as production inputs.
+
+## Representative artifact results
+
+Codex completed a representative validation build covering 17,901 exact `z/x/y` tiles plus three deterministic owner partitions for Fresno and both sides of the antimeridian.
+
+### Foundation
+
+- Final stable size: `250,938,363` bytes.
+- Reported SHA-256 prefix: `b76dfa...`.
+- Independent PMTiles verification passed after staged write, fsync, pending-file verification, and atomic rename.
+
+### Representative manifest
+
+- Total size across foundation and three non-overlapping owners: `272,198,460` bytes.
+- A later manifest/artifact version after Fresno landcover correction was reported as `729e166...`.
+
+## Builder defects already found and corrected locally
+
+1. Node `Buffer` was returned where the PMTiles reader required an exact-range `ArrayBuffer`.
+2. Polygon bounds used `Math.min(...points)` / `Math.max(...points)` and exceeded the JavaScript argument limit on detailed polygons; replaced with bounded linear scans.
+3. Same-property polygon containment/deduplication was quadratic on dense tiles; replaced with deterministic spatial bucket indexing.
+4. Legitimate buffered overview polygons exceeded the strict tile box; offline normalization was changed to clip to the exact tile boundary and revalidate encoded output.
+5. The inherited runtime overscale helper validated intermediate MVT too early; ancestor materialization was moved into the offline builder and changed to feature-granular transform, clip, reject, then encode.
+6. PMTiles owner metadata inherited an invalid center zoom; metadata was changed to derive center and zoom bounds from the owner’s actual addressed tiles.
+7. Initial foundation output changed after the success report; writer was hardened to staging-path generation, verification, fsync, pending-copy verification, descriptor close, and atomic promotion.
+8. Fresno z16 tiles lacked landcover because the overview z6 parent had no Fresno landcover. Authority was corrected so overview owns landcover at z0-6 and exactly one deterministic regional archive owns landcover above z6 for each partition.
+9. Legacy AWS terrain/hillshade remained as a second browser source and caused readiness/network failures; Codex removed that source and dependent hillshade layer from the generated runtime style to enforce one browser source.
+
+## Validation state reached before credit exhaustion
+
+### Static views
+
+Codex reported all nine static views passing after the Fresno landcover rebuild:
+
+- Global
+- North America
+- South America
+- Europe
+- Pacific
+- Antimeridian
+- Fresno regional
+- Fresno city
+- Fresno street
+
+Reported results:
+
+- Required land rendered.
+- Regional landcover rendered at Fresno street level.
+- Transportation rendered.
+- Exactly one style source.
+- Zero network failures.
+- Zero page errors.
+- Zero detected seams.
+- Zero rectangular tile footprints.
+- Zero stretching detections.
+- Zero blank-frame detections.
+- Zero source changes.
+- Zero neighboring-tile inconsistency detections.
+
+### Motion validation
+
+Motion validation was not finished.
+
+The last reported change corrected only the first motion setup predicate: global starting views should require immutable-source readiness at the target camera, not visible landcover, because the global contract is land plus ocean depth. Motion frames were still intended to undergo every pixel/source gate.
+
+## Immediate recovery instruction for the next Codex turn
+
+Do not rebuild first. Preserve the workspace immediately:
+
+1. Confirm the workspace still contains the uncommitted changes and generated reports.
+2. Confirm the current branch is `rebuild/offline-global-tileset`.
+3. Review `git status --short` and `git diff --stat`.
+4. Exclude `node_modules`, downloaded PMTiles inputs, browser binaries, temporary build directories, and generated PMTiles artifacts from Git.
+5. Commit all source, configuration, workflow, manifest, validator, report, and tracked screenshot changes.
+6. Push the branch.
+7. Open a draft PR.
+8. Report the commit SHA and PR number before resuming any build or validation.
+
+## Work still required after preservation
+
+- Finish all continuous motion validations.
+- Inspect saved screenshots manually, not only numeric gates.
+- Prove complete deterministic worldwide ownership for every intended tile.
+- Execute the production-scale immutable partition build, not only the 17,901-tile representative artifact.
+- Keep every final PMTiles release asset below GitHub’s per-asset limit.
+- Upload only final immutable partitions and the versioned ownership manifest.
+- Keep the PR draft until the actual worldwide artifact set exists and is visually verified.

docs/offline-global-tileset.md

@@ -0,0 +1,137 @@
+# Immutable worldwide PMTiles build
+
+The production map uses one browser source and an immutable PMTiles ownership
+manifest. Existing worldwide and regional archives are build inputs only.
+Production never decodes MVT geometry, merges shards, queries Neon, synthesizes
+landcover, or substitutes parent/child tiles.
+
+## Authority and ownership
+
+| Output layers | Sole offline authority |
+|---|---|
+| `land`, `depth` | `occumed-world-surface.pmtiles` |
+| `landcover` | `occumed-world-overview.pmtiles` |
+| roads, buildings, boundaries, labels, other cartography | one regional owner |
+
+The deterministic plan starts with z6 logical cells. A cell whose locked
+candidate input set is too large is split through z7 or z8. Leaf prefixes never
+overlap. When a prefix is split, its own ancestor tile is assigned explicitly
+to the first descendant owner in stable child order. Consequently every
+addressed z0–z16 tile resolves to exactly one foundation, exact-tile, or prefix
+owner.
+
+Missing entries are immutable empty results from that owner; production never
+tries a second archive.
+
+## Full offline build
+
+The complete published input inventory is about 270 GB. Use a build host with
+enough space for the locked input cache, work files, and final owners. All
+outputs are created at staging paths, verified as PMTiles, fsynced, copied to a
+pending file, verified again, and atomically promoted.
+
+Create the SHA-locked owner plan and worldwide foundation target list:
+
+```bash
+npm run tiles:plan-world -- \
+  --output config/immutable-owner-plan.json
+
+npm run tiles:targets-foundation -- \
+  --plan config/immutable-owner-plan.json \
+  --output build/offline-global/foundation-targets.json
+```
+
+Localize the two worldwide inputs and build the z0–z6 foundation:
+
+```bash
+npm run tiles:localize -- \
+  --plan config/immutable-owner-plan.json \
+  --targets build/offline-global/foundation-targets.json \
+  --output-dir immutable-inputs \
+  --report build/offline-global/foundation-inputs.json
+
+npm run tiles:build-foundation -- \
+  --plan config/immutable-owner-plan.json \
+  --targets build/offline-global/foundation-targets.json \
+  --input-report build/offline-global/foundation-inputs.json \
+  --output-dir build/offline-global/artifact
+```
+
+For every `owners[].id` in the plan, localize its locked candidates, enumerate
+the exact addresses in their PMTiles directories, and build that owner. The
+input directory is a shared digest-verified cache, so repeated candidates are
+reused.
+
+```bash
+OWNER_ID=z6-10-24
+
+npm run tiles:localize -- \
+  --plan config/immutable-owner-plan.json \
+  --owner-id "$OWNER_ID" \
+  --output-dir immutable-inputs \
+  --report "build/offline-global/$OWNER_ID-inputs.json"
+
+npm run tiles:targets-owner -- \
+  --plan config/immutable-owner-plan.json \
+  --owner-id "$OWNER_ID" \
+  --input-report "build/offline-global/$OWNER_ID-inputs.json" \
+  --output "build/offline-global/$OWNER_ID-targets.json"
+
+npm run tiles:build-owner -- \
+  --plan config/immutable-owner-plan.json \
+  --owner-id "$OWNER_ID" \
+  --targets "build/offline-global/$OWNER_ID-targets.json" \
+  --input-report "build/offline-global/$OWNER_ID-inputs.json" \
+  --output-dir build/offline-global/artifact
+```
+
+Finalize only after every planned owner report exists:
+
+```bash
+npm run tiles:finalize -- \
+  --plan config/immutable-owner-plan.json \
+  --foundation build/offline-global/artifact/reports/foundation.json \
+  --owner-dir build/offline-global/artifact/reports/owners \
+  --output build/offline-global/artifact/manifest.json
+```
+
+The finalizer marks an incomplete inventory as `validationFixture: true`.
+Production rejects it. `OCCUMED_ALLOW_PARTIAL_TILESET_FIXTURE=true` exists only
+for the bounded local visual fixture and must not be set in deployment.
+
+## Production
+
+Set `OCCUMED_IMMUTABLE_TILESET_MANIFEST` to the local manifest path or an HTTPS
+manifest URL. Local assets are resolved beneath the manifest directory. Remote
+assets use the manifest's optional `assetBaseUrl`.
+
+The server validates completeness, artifact identity, SHA metadata, safe asset
+paths, non-overlapping prefixes, and unique exact-tile assignments before
+serving tiles. Each request performs a bounded index lookup and a single
+PMTiles byte-range read. Stored gzip MVT bytes are returned unchanged.
+
+## Mandatory validation
+
+Run the structural and production checks:
+
+```bash
+npm run build
+```
+
+Run the browser gate against the finalized manifest and validation target
+document:
+
+```bash
+PLAYWRIGHT_BROWSERS_PATH=build/offline-global/playwright \
+npm run check:immutable-visuals -- \
+  --manifest build/offline-global/artifact/manifest.json \
+  --targets build/offline-global/representative-targets.json \
+  --output-dir visual-validation/immutable-final
+```
+
+The target document must cover global, North America, South America, Europe,
+Pacific, antimeridian, regional, city, and street views, plus 30 exact-camera
+motion checkpoints. Validation fails for rectangular footprints, vertical or
+horizontal seams, stretched polygons, inconsistent neighbors, blank frames,
+source switching, page errors, or tile delivery failures. Saved screenshots
+must also be inspected at full size before an artifact is accepted.