Greatly improve web performance by providing a manifest of changed tiles

[Headline]

The web interface refetched every tile in the viewport on a fixed interval, whether or not anything about them had changed. It now refetches only the tiles the server reports as rewritten.

I'll go through the changes and the motivations for the fixes individually.
AI Disclosure: I used claude code throughout these code changes. I'm submitting these changes to upstream in good faith because users would probably appreciate the snappier map.

Tile manifest

The server writes a small manifest into each world's tile directory listing the tiles it has recently rewritten and when. The web interface polls that and refetches only what it names now.

Tiles are requested with the version the server last published for them, so a rewritten tile gets a new URL while an unchanged one keeps the URL the browser already has cached. That lets versioned requests be cached indefinitely, while a request without a version has to revalidate, as the img behind it can change.

The manifest keeps two minutes of history and records the newest entry it dropped. A client that polled inside that window can be told what changed.

Manifest serving

The manifest is served from the in-memory JSON cache, so an unchanged poll comes back as a not-modified response with no body. That needed a cache-control header the cache wasn't sending before — without it browsers never revalidate, so they never get the cheap response. Marker data gets the same benefit.

Tile encoding

Zoom levels below the most detailed are shared between regions- at the default maximum zoom, the most zoomed-out tile covers an 8x8 block of them. Every region save used to decode that shared tile, paint into it, and re-encode it, so the same file was rewritten once for every region that touched it.

Those shared levels are now held in memory and written out when they fall out of the cache or on a short timer, so a run of regions in the same area encodes them once rather than once each. The most detailed zoom has one tile per region and nothing to coalesce, so it still goes straight to disk as soon as its region is saved. A flush that finds nothing to write drops the cache, so an idle world holds no images. The manifest is recorded from the same path that writes the files, so it reflects what is on disk.

Pixels are written into the image's backing array directly instead of one call at a time through the image API, which was doing a color conversion per pixel on any tile that already existed on disk.

Dropped chunks

A chunk whose data can't be read draws nothing, but the background render treated it as done and took it off the retry list, so it was never looked at again. Chunks now report whether they actually drew, and only those are cleared, the rest retry for a few cycles before giving up so chunks that will never be readable don't pile up.

Notes

• If you serve the web directory through nginx rather than the built-in server, it needs equivalent cache rules or versioned tiles lose their caching and the manifest can never answer not-modified. In the http context:

  map $arg_v $squaremap_tile_cache {
      ""      "no-cache";
      default "public, max-age=31536000, immutable";
  }

  Then add_header Cache-Control $squaremap_tile_cache; on the tile location, and add_header Cache-Control "no-cache"; on .json.

• The web interface's tile poll interval is derived from the background render interval, so the two are always the same value. With the caching above, it is safer to drop the interval to around 5 seconds, which makes the map feels considerably more live. So give it a go with background-render: interval-seconds: 5

What's needed

This needs testing at larger player counts, where the benefits should be obvious. I've been testing on my small server with my friends and has done well with no errors.
Worth being clear that the tradeoff moves from being encoding-bound to the image-writing thread and manifest traffic that can't be cached.

[Headline]

I'd provide a jar for folks to test if they would like, but I'll leave that to someone this project trusts a bit more than me, a stranger :)