Repository navigation

test(e2e): api, ingest and sweeper in separate processes - #645

Closed

EricAndrechek wants to merge 142 commits into

mainfrom

integration/distributed

Commits on Sep 25, 2026

fix(mq): a failed resize restores the ingest stream's own cap; review fixes
taitelee
committed
db2d20a
fix(app): a rejected tenant keeps its replay history; review fixes
taitelee
committed
07c6a91
fix(mq): share the hub bridge's fetch-ahead across tenants; review fixes
taitelee
committed
04aad9f
fix(mq): publish only into a queue the broker recorded open; review fixes
taitelee
committed
57870c4
docs(mq): a consumer that cannot join a queue opened at runtime; review fixes
taitelee
committed
7cdb794
fix(mq): pace publish-side retries of a queue that cannot open; review fixes
taitelee
committed
e199a03
test(mq): one tenant's failed purge stops no other; review fixes
taitelee
committed
55137d8
docs(mq): size a dead-letter stream the shrink guard kept; review fixes
taitelee
committed
060ca9e
feat(config): choose each layer's implementation at boot

EricAndrechek
and
claude
committed
3022b92
fix(discovery): jitter the refresh retry backoff

EricAndrechek
and
claude
committed
651956a
test(mq): pin interest partitions with a sourced history (S1)

EricAndrechek
and
claude
committed
77ef65d
docs(config): say coord.backend is reserved; sync the boot-config lists

EricAndrechek
and
claude
committed
fde17ba
fix(cache): snapshot versions at lookup; tenant token on every key

EricAndrechek
and
claude
committed
431084e
feat(coord): leases, in-process implementation

EricAndrechek
and
claude
committed
13422da
docs(coord): state what an unfenced sweeper overlap can cost

EricAndrechek
and
claude
committed
a82f74b
docs(config): no backend has a sub-block yet; index backends.go in AGENTS.md

EricAndrechek
and
claude
committed
f129d57
test(mq): one conformance suite for every Broker

EricAndrechek
and
claude
committed
bb027da
docs(coord): name ErrClosed as RunElected's other exit; changelog files

EricAndrechek
and
claude
committed
d913a18
fix(ingest): retry ClickHouse outages instead of dead-lettering

EricAndrechek
and
claude
committed
a131fb9
test(mq): run the embedded conformance in a test binary of its own

EricAndrechek
and
claude
committed
1adc286
docs(mq): keep the per-tenant no-queue case in ErrQueueFull's contract

EricAndrechek
and
claude
committed
80d6c22
fix(dedupe)!: reserve, commit or release ids keyed by tenant and table

EricAndrechek
and
claude
committed
58d9e77
feat(mq): topology spec and verifier for operator-owned NATS

EricAndrechek
and
claude
committed
735fd78
fix(ingest): back off a failing table alone; floor the probe-out delay

EricAndrechek
and
claude
committed
8ae9614
test(mq): pin the exactly-once failed report through durable deletion

EricAndrechek
and
claude
committed
bf11ecc
fix(dedupe): state what Managed guarantees a backend; default a zero lease

EricAndrechek
and
claude
committed
0ec037e
docs(discovery): sweep the remaining unjittered backoff claims

EricAndrechek
and
claude
committed
2a55fb1
fix(ingest): back off a table denied a grant alone; fix password docs

EricAndrechek
and
claude
committed
356125f
docs(api): list the unavailable broker among the request aborts

EricAndrechek
and
claude
committed
ef4153c
fix(mq): pace a park's reopen, and warn only on a missing consumer; review fixes
taitelee
committed
9344f1b
docs: say the DLQ takes rejected rows, not every failed insert

EricAndrechek
and
claude
committed
e97edc8
fix(dedupe): release only claimed claims on a wrong-count answer

EricAndrechek
and
claude
committed
f484511
fix(mq): refuse per-subject eviction, unattached sources, shared partitions

EricAndrechek
and
claude
committed
68b74ff
perf(cache): flat version index, pruned per tenant

EricAndrechek
and
claude
committed
67df317
test(mq): end delivery only once the pulls are live

EricAndrechek
and
claude
committed
8792d89
Merge origin/feat/boot-backends into feat/coord-leases

EricAndrechek
and
claude
committed
5a9d1f1
feat(coord): coord.backend selects the coordinator

EricAndrechek
and
claude
committed
e0705f2
Merge remote-tracking branch 'origin/mq-tenant-streams' into feat/dedupe-reserve
EricAndrechek
committed
20c28d1
docs(dedupe): link the old-key sweep to #220 rather than promise it

EricAndrechek
and
claude
committed
386ce74
docs(cache): name the cache prune hook; pin LocalCache as a pruner

EricAndrechek
and
claude
committed
ed8022b
feat(cache): Redis-compatible shared cache backend

EricAndrechek
and
claude
committed
50b7e0c
Merge remote-tracking branch 'origin/mq-tenant-streams' into feat/mq-conformance
EricAndrechek
committed
3425243
feat(app): process roles

EricAndrechek
and
claude
committed
beab0fd
fix(api): map ClickHouse query failures by class, not HTTP status

EricAndrechek
and
claude
committed
01ea690
fix(cache): count only transport failures against the breaker

EricAndrechek
and
claude
committed
e346426
fix(mq): a replay whose connection closed is not caught up

EricAndrechek
and
claude
committed
e2434d6
test(mq): keep the unit suite inside its per-package timeout

EricAndrechek
and
claude
committed
e78ea2a
docs(changelog): leave the other entries' spelling as it was

EricAndrechek
and
claude
committed
a6fdcfd
Merge remote-tracking branch 'origin/mq-tenant-streams' into feat/mq-nats-topology
EricAndrechek
committed
5b5f988
fix(cache): replace token and value keys of the wrong type

EricAndrechek
and
claude
committed
d2a87ab
docs(app): scope instance_id and sweeper claims to what ships today

EricAndrechek
and
claude
committed
5de4fd0
test(mq): give the S1 tests a package of their own

EricAndrechek
and
claude
committed
0fb7637
ci(cov): keep the external-NATS topology out of the e2e suite's gate

EricAndrechek
and
claude
committed
c63ec9a
fix(api): let ClickHouse report a role time cap overrun itself

EricAndrechek
and
claude
committed
4ff30f3
Merge remote-tracking branch 'origin/feat/mq-conformance' into feat/mq-external-broker
EricAndrechek
committed
aa9a655
test(cov): keep the unreachable Redis backend out of the e2e gate

EricAndrechek
and
claude
committed
d67a46a
test(app): tell a verified token by the ClickHouse error, not the status

EricAndrechek
and
claude
committed
148a160
test(app): tell a verified token by the ClickHouse error, not the status

EricAndrechek
and
claude
committed
3a314af
Merge remote-tracking branch 'origin/feat/boot-backends' into feat/cache-redis-wiring
EricAndrechek
committed
44cec4d
test: a role cap overrun is 400 limit_exceeded in the limits test

EricAndrechek
and
claude
committed
f485505
feat(dedupe): DynamoDB backend, conformance-tested on dynamodb-local

EricAndrechek
and
claude
committed
108499f
fix(ingest): windowed reserve/publish/commit; 503 when dedupe is unavailable

EricAndrechek
and
claude
committed
37ef757
Merge remote-tracking branch 'origin/feat/cache-flat-versions' into feat/cache-redis-wiring
EricAndrechek
committed
85803f0
fix(ingest): qualify the duplicate-window claims; steadier tests

EricAndrechek
and
claude
committed
71ab824
fix(dedupe): attempt every Commit and Release chunk; keep the breaker honest

EricAndrechek
and
claude
committed
6f7944a
test(dedupe): make the Dynamo failure tests fail without their fix

EricAndrechek
and
claude
committed
ff047d2
feat(mq): a Broker over an external NATS cluster

EricAndrechek
and
claude
committed
0aaeddb
docs(ingest): Release gives back definite failures only; tighten wording

EricAndrechek
and
claude
committed
9824663
docs(ingest): say the Pebble benchmark stubs the queue; finish the rename

EricAndrechek
and
claude
committed
99a5eee
feat(config): cache.backend=redis selects the shared cache

EricAndrechek
and
claude
committed
c068084
Merge origin/feat/boot-backends into feat/dedupe-dynamodb-wiring

EricAndrechek
and
claude
committed
83a6949
fix(mq): count a replay down, and size the duplicate window to retries

EricAndrechek
and
claude
committed
c452cb6
fix(mq): keep a replay consumer through a slow client's batch

EricAndrechek
and
claude
committed
c8ad3f0
feat(app): choose the DynamoDB dedupe backend at boot

EricAndrechek
and
claude
committed
ced118c
fix(cache): review round — e2e gate, trust boundary, #386, addr spaces

EricAndrechek
and
claude
committed
13c6cb3
feat(dedupe): retention per tenant and table, and an expiry sweep

EricAndrechek
and
claude
committed
b83e670
fix(app): retry a failed DynamoDB table check; refuse no region

EricAndrechek
and
claude
committed
a8e43bd
test(dedupe): pin that a commit mid-sweep-chunk survives; docs wording

EricAndrechek
and
claude
committed
6c74b65
fix(pipes): run write pipes every call, uncached and uncoalesced

EricAndrechek
and
claude
committed
1134d5c
docs(dedupe): name every region source; untangle the dynamodb check clause

EricAndrechek
and
claude
committed
0b60451
fix(config): keep an explicit false/0/"" from config.yaml

EricAndrechek
and
claude
committed
252ef7b
fix(pipes): no-store on write pipes; reconcile the mutation-path docs

EricAndrechek
and
claude
committed
74a7bf4
docs(pipes): name the operator as a write pipe's author; no-store in api.md

EricAndrechek
and
claude
committed
7bb8c27
fix(test): retry removing an embedded broker store in testutil

EricAndrechek
and
claude
committed
2757e64
Merge remote-tracking branch 'origin/feat/process-roles' into feat/mq-nats-wiring
EricAndrechek
committed
dbd4661
test(mq): keep the streams directory occupied through a failed open
taitelee
committed
f5d8f48
feat(app): mq.backend selects embedded or external NATS

EricAndrechek
and
claude
committed
83ef8d0
fix(config): refuse credentials in mq.nats.urls; review fixes to docs

EricAndrechek
and
claude
committed
2bfc9ee
fix(mq): drain partitions a lower N leaves; quiet close

EricAndrechek
and
claude
committed
18d47de
fix(mq): cover a publishing old-N process; name the stream delete

EricAndrechek
and
claude
committed
6d42f5a
feat(mq): coord leases on a NATS KV bucket

EricAndrechek
and
claude
committed
d4f7450
Merge remote-tracking branch 'origin/fix/config-yaml-zero' into integration/distributed
EricAndrechek
committed
a1aedc8
Merge remote-tracking branch 'origin/fix/discovery-retry-jitter' into integration/distributed
EricAndrechek
committed
fd9b341
fix(mq): time lease step-down from the renewal sent; review fixes

EricAndrechek
and
claude
committed
71fdc07
Merge remote-tracking branch 'origin/fix/query-error-classes' into integration/distributed
EricAndrechek
committed
6339f47
test(integration): open the outage test's queue on #612's broker API

EricAndrechek
and
claude
committed
ddf0101
Merge remote-tracking branch 'origin/fix/mq-external-close-and-shrink' into integration/distributed
EricAndrechek
committed
5bc481f
fix(config): give the backend, roles and mq.nats defaults to defaults()

EricAndrechek
and
claude
committed
a08a6d6
fix(mq): a per-term id in the lease value; lease tests off the unit budget

EricAndrechek
and
claude
committed
b29929e
Merge remote-tracking branch 'origin/feat/cache-redis-wiring' into integration/distributed
EricAndrechek
committed
0a64b9e
fix(config): cache.redis defaults in defaults(); compress_min_bytes 0 is never

EricAndrechek
and
claude
committed
eaf69ef
Merge remote-tracking branch 'origin/fix/mutation-pipes-uncached' into integration/distributed
EricAndrechek
committed
aea078e
docs(deployment): the shared cache no longer caches write pipes

EricAndrechek
and
claude
committed
00e165c
test(mq): the unit verifier cases skip the lease bucket they do not check

EricAndrechek
and
claude
committed
8a9af4b
Merge remote-tracking branch 'origin/feat/dedupe-retention' into integration/distributed
EricAndrechek
committed
605a6c6
test(api): an unreachable broker mid-window is uncertain, a 503

EricAndrechek
and
claude
committed
d2ce776
fix(mq): ExternalNATS honours the idempotency key; conformance pins it

EricAndrechek
and
claude
committed
6a69c03
Merge remote-tracking branch 'origin/feat/dedupe-dynamodb-wiring' into integration/distributed
EricAndrechek
committed
aa8a83a
fix(config): dedupe defaults in defaults(); pin the lease cap to mq's window

EricAndrechek
and
claude
committed
cc53a6a
feat(mq): nats duplicate_window covers dedupe.lease; warn on short retention

EricAndrechek
and
claude
committed
09f2d3d
docs(deployment): say what nats and dynamodb share; retention reaches TTL

EricAndrechek
and
claude
committed
c696b2c
docs(changelog): one list for #613's Unreleased entries

EricAndrechek
and
claude
committed
5666252
Merge branch 'feat/coord-nats-kv' into integration/distributed
EricAndrechek
committed
b89ee1f
docs(deployment): a split by role has four shared backends to choose from

EricAndrechek
and
claude
committed
713b782
test(e2e): api, ingest and sweeper in separate processes

EricAndrechek
and
claude
committed
9c1d252
fix(mq): refuse an embedded store it cannot create at once

EricAndrechek
and
claude
committed
2a19d7e
test(mq): run the embedded broker's tests in parallel, without fsync

EricAndrechek
and
claude
committed
96e2112
test(app): boot without the embedded broker's fsync per write

EricAndrechek
and
claude
committed
bd059f2
test(app): retry the DynamoDB table check sooner under test

EricAndrechek
and
claude
committed
8776b4d
test(app): a 1ms topology_wait where the outcome cannot change

EricAndrechek
and
claude
committed
09d5c14
test(integration): name roles and backends in hand-built configs

EricAndrechek
and
claude
committed
e830f34
fix(mq): create the embedded store at 0700; changelog the fail-fast

EricAndrechek
and
claude
committed
3d959b9
docs(changelog): scope the fail-fast store entry to what mkdir catches

EricAndrechek
and
claude
committed
3c8a93b
Merge branch 'perf/unit-test-budget' into integration/distributed
EricAndrechek
committed
e3d83c7
refactor(app): move the shared-backend wiring out of wire.go

EricAndrechek
and
claude
committed
d002146
refactor(app): wireCoord's nats case becomes wireNATSCoord in wire_nats.go

EricAndrechek
and
claude
committed
f10b9de
test(integration): remove the binary C2 builds when the suite ends

EricAndrechek
and
claude
committed
468b4fe
docs: claims the combined backends made false, round two

EricAndrechek
and
claude
committed
8c6057e
docs: retention's refusal names 2m; a split needs a shared cache only when api and ingest part

EricAndrechek
and
claude
committed
d6b4d4b
Merge remote-tracking branch 'origin/main' into integration/distributed

EricAndrechek
and
claude
committed
49a3b01
test(integration): run own-stack tests in parallel; build C2's binary first

EricAndrechek
and
claude
committed
6789325
test(integration): parallelize the last own-stack tests; name C2's binary

EricAndrechek
and
claude
committed
e954536
docs(changelog): the integration timing measured on the tree it ships

EricAndrechek
and
claude
committed
48dc144
test(integration): wait for NATS's and Redis's host ports, not only the log

EricAndrechek
and
claude
committed
0e776c6
test(integration): cite #613 in the roles tests' comments

EricAndrechek
and
claude
committed
079fa9a
test(cache): seed the breaker tests' fill outside their 100ms budget

EricAndrechek
and
claude
committed
e08367c
test(cache): name the setup timeout, not the breaker, as the failure

EricAndrechek
and
claude
committed
5f392e7
docs: neutral tags in the DynamoDB example; neutral benchmark wording

EricAndrechek
and
claude
committed
1914eac

Commits

Commits on Sep 24, 2026

Commits on Sep 25, 2026