Buoys, surf forecasts and the aircraft worth a row

[ralyodio]

Three sources, all keyless, all verified live before they were written.

BUOYS AND SURF, INTO WATER

NDBC publishes the newest observation from every buoy it is listening to in one
hundred-kilobyte file: 853 reports, 184 of them carrying a wave height. Wave
height, dominant period and mean direction are what a surf report is actually
made of, and the period is the number people leave out -- two metres at 18
seconds and two metres at 5 seconds are completely different days in the water.
So a wave row is published as "8.9 ft at 8 s, mixed swell out of the W" as well
as stored as numbers, in feet as well as metres, because every US surf forecast
is in feet and converting at read time is how a nine-foot day becomes a
three-foot day.

Beside it, the National Weather Service Surf Zone Forecast from every coastal
office, in full. A buoy off Oahu reading 2.4 m at 16 seconds is a fact;
"advisory level surf along south facing shores through today" is what it means
to anyone standing on the beach, and no arithmetic over the buoy produces it.
The high surf advisory and the rip current risk are lifted from the
forecaster's own wording into tags so a coast under an advisory can be found
without reading 851 products, and the full text is stored as the authority.

AIRCRAFT, INTO AVIATION

There is a version of this that stores every aeroplane in the sky every few
minutes. It would be tens of thousands of rows a day, each a position that was
true for four seconds, and nobody would read one. A position is not news; an
aircraft squawking 7700 is. So this reads the event surfaces: the three
emergency transponder codes and the military list. At the time of writing the
emergency codes returned zero aircraft between them and the military query
returned 422, which is the shape you want -- the rare thing is rare, so its
appearance means something.

An emergency is an episode, not a ping, and it is held the same way
faa-nas-status holds a ground stop: open episodes live in the cursor keyed on
the ICAO hex, the row is keyed on when the aircraft was first seen, and when it
leaves the list it is written once more with a duration. "N123AB squawked 7700
for 22 minutes" is the row. The forty positions in between are not.

FOUR THINGS THE LIVE DATA DECIDED

• NDBC writes a missing reading as MM, in every column, in a fixed-width
  table. Number('MM') || 0 turns "this buoy has no anemometer" into a flat
  calm and "no wave sensor" into a dead-flat sea, and 669 of 853 stations
  report no wave height at all. One guard, and null means the buoy did not say.
• The station register really does publish name="" -- 15009 in the Atlantic
  array is one of several. An empty string is not a name, and left as one it
  beats the fallback and titles the row with nothing at all. That was visible
  in the first smoke run as rows titled ": marine observation".
• Some stations report a dominant period of 0 or 1 second. That is a sensor
  saying nothing, and classifying it as "windswell" states something about the
  water nobody measured. Under two seconds, the period is left out.
• adsb.lol rate limits /v2/mil far harder than the squawk lists: it answered
  429 while /sqk/7700 beside it answered 200. A limit is not a failure, so it
  returns no items and asks to be called back, which also keeps the cursor
  intact -- a run that threw there would leave every open episode untouched and
  then report them all as ended on the run after.

ALSO CORRECTED

The comment added in #38 said the NTSB reader slices its output because
upsertItems sends every row in one statement. That is wrong: runSource
already chunks writes into batches of 200. The slicing is still right, for the
reasons now stated -- 140 MB of narrative text held in memory, and the
four-minute ingest deadline -- but the stated reason was not the real one.

Verified live: 853 buoy observations with 184 reporting waves and one station
over 2.5 m; 40 Pacific and 23 Hawaii surf forecasts with the Honolulu high surf
advisory correctly tagged; all three emergency squawk lists answering with zero
aircraft, which is the correct answer. 596 tests pass, biome clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CjTtJPEpyJbvPeQcVfYPPs