Repository navigation

feat(mq): a Broker over an external NATS cluster - #636

Merged

EricAndrechek merged 90 commits into

feat/mq-nats-topologyfrom

feat/mq-external-broker

Sep 29, 2026

Merged

feat(mq): a Broker over an external NATS cluster#636
EricAndrechek merged 90 commits into
feat/mq-nats-topologyfrom
feat/mq-external-broker

Commits on Sep 25, 2026
- feat(config): choose each layer's implementation at boot
  
  EricAndrechek
  and
  claude
  committed
  3022b92
- docs(config): say coord.backend is reserved; sync the boot-config lists
  
  EricAndrechek
  and
  claude
  committed
  fde17ba
- feat(coord): leases, in-process implementation
  
  EricAndrechek
  and
  claude
  committed
  13422da
- docs(coord): state what an unfenced sweeper overlap can cost
  
  EricAndrechek
  and
  claude
  committed
  a82f74b
- docs(config): no backend has a sub-block yet; index backends.go in AGENTS.md
  
  EricAndrechek
  and
  claude
  committed
  f129d57
- test(mq): one conformance suite for every Broker
  
  EricAndrechek
  and
  claude
  committed
  bb027da
- docs(coord): name ErrClosed as RunElected's other exit; changelog files
  
  EricAndrechek
  and
  claude
  committed
  d913a18
- test(mq): run the embedded conformance in a test binary of its own
  
  EricAndrechek
  and
  claude
  committed
  1adc286
- docs(mq): keep the per-tenant no-queue case in ErrQueueFull's contract
  
  EricAndrechek
  and
  claude
  committed
  80d6c22
- test(mq): pin the exactly-once failed report through durable deletion
  
  EricAndrechek
  and
  claude
  committed
  bf11ecc
- docs(api): list the unavailable broker among the request aborts
  
  EricAndrechek
  and
  claude
  committed
  ef4153c
- test(mq): end delivery only once the pulls are live
  
  EricAndrechek
  and
  claude
  committed
  8792d89
- Merge origin/feat/boot-backends into feat/coord-leases
  
  EricAndrechek
  and
  claude
  committed
  5a9d1f1
- feat(coord): coord.backend selects the coordinator
  
  EricAndrechek
  and
  claude
  committed
  e0705f2
- Merge remote-tracking branch 'origin/mq-tenant-streams' into feat/mq-conformance
  EricAndrechek
  committed
  3425243
- feat(app): process roles
  
  EricAndrechek
  and
  claude
  committed
  beab0fd
- fix(mq): a replay whose connection closed is not caught up
  
  EricAndrechek
  and
  claude
  committed
  e2434d6
- docs(app): scope instance_id and sweeper claims to what ships today
  
  EricAndrechek
  and
  claude
  committed
  5de4fd0
- Merge remote-tracking branch 'origin/feat/mq-conformance' into feat/mq-external-broker
  EricAndrechek
  committed
  aa9a655
- feat(mq): a Broker over an external NATS cluster
  
  EricAndrechek
  and
  claude
  committed
  0aaeddb
- fix(mq): count a replay down, and size the duplicate window to retries
  
  EricAndrechek
  and
  claude
  committed
  c452cb6
- fix(mq): keep a replay consumer through a slow client's batch
  
  EricAndrechek
  and
  claude
  committed
  c8ad3f0
- fix(test): retry removing an embedded broker store in testutil
  
  EricAndrechek
  and
  claude
  committed
  2757e64
- Merge remote-tracking branch 'origin/feat/process-roles' into feat/mq-nats-wiring
  EricAndrechek
  committed
  dbd4661
- feat(app): mq.backend selects embedded or external NATS
  
  EricAndrechek
  and
  claude
  committed
  83ef8d0
- fix(config): refuse credentials in mq.nats.urls; review fixes to docs
  
  EricAndrechek
  and
  claude
  committed
  2bfc9ee
- fix(mq): drain partitions a lower N leaves; quiet close
  
  EricAndrechek
  and
  claude
  committed
  18d47de
- fix(mq): cover a publishing old-N process; name the stream delete
  
  EricAndrechek
  and
  claude
  committed
  6d42f5a
- feat(mq): coord leases on a NATS KV bucket
  
  EricAndrechek
  and
  claude
  committed
  d4f7450
- fix(mq): time lease step-down from the renewal sent; review fixes
  
  EricAndrechek
  and
  claude
  committed
  71fdc07
- fix(mq): a per-term id in the lease value; lease tests off the unit budget
  
  EricAndrechek
  and
  claude
  committed
  b29929e
- test(mq): the unit verifier cases skip the lease bucket they do not check
  
  EricAndrechek
  and
  claude
  committed
  8a9af4b
- feat(mq): give every tenant a queue of its own (#612)
  taitelee
  authored
  2a2b886
- Merge remote-tracking branch 'origin/main' into feat/mq-nats-topology
  EricAndrechek
  committed
  8e8cc73
- Merge remote-tracking branch 'origin/main' into feat/mq-conformance
  EricAndrechek
  committed
  5570296
- Merge remote-tracking branch 'origin/main' into feat/boot-backends
  
  EricAndrechek
  and
  claude
  committed
  6f2ddb6
- fix(mq): refuse an embedded store it cannot create at once
  
  EricAndrechek
  and
  claude
  committed
  184864e
- test(mq): run the embedded broker's tests in parallel, without fsync
  
  EricAndrechek
  and
  claude
  committed
  a3381d2
- test(app): boot without the embedded broker's fsync per write
  
  EricAndrechek
  and
  claude
  committed
  c566f55
- test(app): a 1ms topology_wait where the outcome cannot change
  
  EricAndrechek
  and
  claude
  committed
  a48ce1f
- fix(mq): create the embedded store at 0700; changelog the fail-fast
  
  EricAndrechek
  and
  claude
  committed
  61b643f
- docs(changelog): scope the fail-fast store entry to what mkdir catches
  
  EricAndrechek
  and
  claude
  committed
  0e04e3e
- test(mq): keep the streams directory occupied through a failed open
  
  taitelee
  authored and
  EricAndrechek
  committed
  5ba2bc1
- docs(mq): an external NATS ack rests on replicas, not an fsync
  
  EricAndrechek
  and
  claude
  committed
  5de3e98
- fix(mq): refuse async persist mode on an ingest partition
  
  EricAndrechek
  and
  claude
  committed
  9ed7119
- feat(mq): generate a 15-minute history stream
  
  EricAndrechek
  and
  claude
  committed
  f00b0f4
- fix(mq): do not warn on a gap window equal to the history's max_age
  
  EricAndrechek
  and
  claude
  committed
  85d5a17
- docs(changelog): an equal gap window read as longer, not short
  
  EricAndrechek
  and
  claude
  committed
  d563fa9
- Merge origin/feat/boot-backends into feat/coord-leases
  
  EricAndrechek
  and
  claude
  committed
  b4bd984
- Merge origin/feat/coord-leases into feat/process-roles
  
  EricAndrechek
  and
  claude
  committed
  eaeb52f
- test(mq,app): fit the unit budget; fail fast on an uncreatable store (#647)
  
  EricAndrechek
  and
  claude
  authored
  0cbb386
- fix(config): keep an explicit false/0/"" from config.yaml (#632)
  
  EricAndrechek
  and
  claude
  authored
  cb78f78
- fix(discovery): jitter the refresh retry backoff (#616)
  
  EricAndrechek
  and
  claude
  authored
  73c75ee
- fix(mq): count dead letters per table on both brokers
  
  EricAndrechek
  and
  claude
  committed
  7847c14
Commits on Sep 26, 2026
- docs(changelog): list deadletter.go in the external-broker entry
  
  EricAndrechek
  and
  claude
  committed
  50a1170
- refactor(keyenc): one key escaping, keep '-', DLQ counts per table (#655)
  
  EricAndrechek
  and
  claude
  authored
  77cf4a7
- Merge remote-tracking branch 'origin/main' into feat/mq-conformance
  
  EricAndrechek
  and
  claude
  committed
  22d30b0
- test(mq): nothing parked is an empty Tables map, never nil
  
  EricAndrechek
  and
  claude
  committed
  6ebfc6f
- fix(ingest): retry ClickHouse outages instead of dead-lettering (#619)
  
  EricAndrechek
  and
  claude
  authored
  9aacb9c
- feat(config): choose each layer's implementation at boot (#618)
  
  authored
  b20ae6e
- Merge origin/main into feat/coord-leases
  
  EricAndrechek
  and
  claude
  committed
  5329eb1
- Merge feat/coord-leases (synced with main) into feat/process-roles
  
  EricAndrechek
  and
  claude
  committed
  b643ba1
- docs(app): a bearer token on a worker's ops listener gets 401
  
  EricAndrechek
  and
  claude
  committed
  01cd262
- feat(coord): leases, in-process implementation (#615)
  
  authored
  a1774fb
- Merge origin/main into feat/process-roles
  
  EricAndrechek
  and
  claude
  committed
  1c26d11
- fix(api): map ClickHouse query failures by class, not HTTP status (#627)
  
  EricAndrechek
  and
  claude
  authored
  d7420f8
- fix(app): ops-auth warning names the nested case; own store in test
  
  EricAndrechek
  and
  claude
  committed
  dafea4c
- Merge origin/main into feat/process-roles
  
  EricAndrechek
  and
  claude
  committed
  23ac730
- test(integration): name the roles in TestQueryErrors_ClickHouseDown
  
  EricAndrechek
  and
  claude
  committed
  ada4bdd
- feat(app): process roles (#622)
  
  authored
  5b6efe0
- Merge remote-tracking branch 'origin/main' into feat/mq-conformance
  
  EricAndrechek
  and
  claude
  committed
  ff67ab5
- test(mq): one conformance suite for every Broker (#623)
  
  authored
  5004cd2
- feat(cache): shared redis cache, snapshot keys, uncached write pipes (#614)
  
  authored
  9db8185
- fix(dedupe)!: reserve/commit ids, windowed ingest, retention, dynamodb (#625)
  
  authored
  4ff5074
Commits on Sep 29, 2026
- fix(mq): keep one tenant's failed queue join from ending ingest for all (#680)
  taitelee
  authored
  0de28ad
- Merge feat/coord-nats-kv and fix/mq-external-close-and-shrink into feat/mq-nats-wiring
  
  EricAndrechek
  and
  claude
  committed
  704176f
- Merge feat/mq-external-broker into feat/mq-nats-wiring
  
  EricAndrechek
  and
  claude
  committed
  f91ab6f
- Merge feat/mq-external-broker into feat/mq-nats-topology
  
  EricAndrechek
  and
  claude
  committed
  bd1113f
- Merge the final head of #622 (feat/process-roles) into feat/mq-nats-topology
  
  EricAndrechek
  and
  claude
  committed
  eab1830
- Merge the final head of #623 (feat/mq-conformance) into feat/mq-nats-topology
  
  EricAndrechek
  and
  claude
  committed
  e395743
- Merge origin/main into feat/mq-nats-topology
  
  EricAndrechek
  and
  claude
  committed
  873c39f
- fix(config): give the mq.nats defaults to defaults()
  
  EricAndrechek
  and
  claude
  committed
  d896450
- fix(mq): ExternalNATS honours the idempotency key
  
  EricAndrechek
  and
  claude
  committed
  492fde8
- feat(mq): nats duplicate_window covers dedupe.lease; warn on short retention
  
  EricAndrechek
  and
  claude
  committed
  cbd7464
- docs: the combined backends' claims main's merge left stale
  
  EricAndrechek
  and
  claude
  committed
  9229f15
- fix(mq): manifests cover dedupe.lease; review fixes to docs
  
  EricAndrechek
  and
  claude
  committed
  5bb6f2d
- docs(changelog): list the manifests files in the mq.backend: nats entry
  
  EricAndrechek
  and
  claude
  committed
  c7311e5
- test(integration): give the NATS tests' hand-built configs a dedupe lease
  
  EricAndrechek
  and
  claude
  committed
  a058c6c
- refactor(config): move the nats blocks and rules into mq_nats.go
  
  EricAndrechek
  and
  claude
  committed
  9fefe7c
- refactor(app): move the ops-only listener's wiring into wire_ops.go
  
  EricAndrechek
  and
  claude
  committed
  7fa5e46

Commits

Commits on Sep 25, 2026

Commits on Sep 26, 2026

Commits on Sep 29, 2026