Skip to content

feat(observe): accept batched observations (backwards-compatible) - #1

Merged
jfberry merged 1 commit into
mainfrom
feat/batched-observe
Aug 5, 2026
Merged

feat(observe): accept batched observations (backwards-compatible)#1
jfberry merged 1 commit into
mainfrom
feat/batched-observe

Conversation

@jfberry

@jfberry jfberry commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

POST /observe/{hour} now also accepts a batch envelope, folding one client session's fingerprinted stops at a location in a single request instead of one POST per stop:

{
  "protocol": 2,
  "observations": [
    { "location_id": 1, "stop_id": "fort-a", "layer": "no_ar", "fp": "a1b2c3d4e5f60718", "nk": false },
    { "location_id": 1, "stop_id": "fort-b", "layer": "no_ar", "fp": "1122334455667788", "nk": false }
  ]
}

Why

Dragonite currently POSTs one observation per stop. At each UTC-hour rollover the coordinator's per-hour state resets, so every session re-establishes values from scratch — a front-loaded burst of POSTs that (together with the idle-GET herd, already jittered on the client) makes the rollover spike. Batching collapses a session's multi-stop POSTs into one, cutting that burst exactly where it's worst.

Backwards-compatible by construction

  • The route sniffs the body shape: an observations array ⇒ batch; anything else ⇒ the existing single-observation path, untouched.
  • A single-observation body still folds and echoes exactly as before, so old and new Dragonite instances interoperate during a rollout.
  • No protocol bump. The fingerprint recipe is unchanged, so MIN_PROTOCOL stays 2 — bumping it would 426-reject old instances mid-rollout.

⚠️ Deploy order matters: the coordinator must be deployed before any batch-sending Dragonite ships. An old coordinator would 400 an array body (no top-level location_id) and silently lose those observations. This PR is purely additive (new array path; single path untouched), so it's safe to deploy while every existing instance is still single-posting.

Changes

  • validate.ts — extracted shared validateProtocol/validateFields; new validateBatch (protocol once on the envelope; a malformed item rejects the whole batch, since the client builds it — a bad item is a client bug worth surfacing).
  • http.ts — body-shape sniff; a batch folds every item in one hour transaction and skips per-item semantic outcomes (unknown_stop/capped/killed) best-effort so one dead stop can't drop the rest of the session; MAX_OBSERVE_BODY_BYTES 4096 → 8192.
  • types.tsMAX_BATCH_OBSERVATIONS = 24 (a location has ≤10 expected stops; generous defensive bound).
  • README.md — documents the batch body, the 8KB cap, and the deploy-order note.

Tests

6 new cases in test/http.test.ts: folds all items + echoes once; envelope-level protocol; below-min ⇒ 426; empty / malformed / too-many ⇒ 400; best-effort skip of an unknown stop (good item still folds, 200); and the single (non-batch) body still works unchanged. Full suite 34 passing, typecheck + lint clean.

🤖 Generated with Claude Code

POST /observe/{hour} now also accepts a batch envelope:

  { "protocol": 2, "observations": [ {location_id, stop_id, layer, fp, nk}, ... ] }

folding one session's fingerprinted stops at a location in a single request
(protocol carried once on the envelope, at most MAX_BATCH_OBSERVATIONS=24).
This lets Dragonite collapse its per-stop POSTs into one per session, cutting
the per-UTC-hour observe volume that drives the coordinator's rollover spike.

Backwards-compatible by construction: a single-observation body (no
`observations` array) is still accepted unchanged, so old and new Dragonite
instances interoperate. The route sniffs the body shape — an `observations`
array => batch, anything else => single. This is why the coordinator MUST be
deployed before any batch-sending client ships: an old coordinator would 400 an
array body and lose those observations.

- validate.ts: shared validateProtocol/validateFields; new validateBatch
  (whole-batch reject on a malformed item — the client builds the batch, so a
  bad item is a client bug).
- http.ts: body-shape sniff; batch folds every item in one hour transaction and
  skips per-item semantic outcomes (unknown_stop/capped/killed) best-effort so
  one dead stop can't drop the rest; MAX_OBSERVE_BODY_BYTES 4096 -> 8192.
- No protocol bump: the fingerprint recipe is unchanged, so MIN_PROTOCOL stays
  2; bumping it would 426-reject old instances mid-rollout.

Tests: 6 new batch cases (folds all / envelope protocol / below-min 426 /
empty+malformed+too-many 400 / best-effort skip of unknown stop / single body
still works). Full suite 34 passing, typecheck + lint clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jfberry
jfberry merged commit 0e249eb into main Aug 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant