You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(j5): shared conversations no longer replay the whole event log at startup - #1008
Since nightly 0.0.52-nightly.20261110.16, the openclaw server crash-loops: it exits on a V8 heap OOM (4 GB) about every 2.5 minutes, and every client linked to it drops and reconnects each time. The cause is the prompt-settle live tail in SharedConversation. The v0.1.7 merge (dd89865d44) moved it from threads.streamDomainEvents (which starts at the newest event) to threads.streamStoredEvents, which is eventSink.stream() with no start, i.e. a replay from sequence 0. So every server start reads and decodes the whole event log. openclaw's log has 1.1M events (about 2.5 GB of payload), which doesn't fit in the heap. Smaller installs survive, but still pay for a full replay on every start.
What changed
The tail starts at the event store's high-water mark (readEventStoreHighWater, the start point every other J5 stream daemon uses). It reads through streamStoredEventsFrom with eventType: "runtime-request.updated", so the query filters to settle events, and it resumes from the last handled sequence after a stream failure. Settles from before the start are still marked by LifecycleService from its cursor, as decision 2 intended.
SharedConversation.layer now requires EventSinkV2. The J5 runtime layer already provides it, and the two integration tests provide their harness's own orchestrator layer.
The unit harness's upstream log now behaves like upstream's event sink: streamStoredEvents replays its history from 0, and streamStoredEventsFrom replays what comes after the given sequence, then live. Its old live-only PubSub mock matched the wrong assumption and hid this bug. The new test seeds history and asserts the tail reads none of it. It fails on j5/main (expected 1 to equal +0) and passes here.
Live testing
Server only, no visible surface. On a stood-down migrate-dev-db copy of real data (124,232 orchestration events), I ran vp run dev:server for 60 s and counted the OrchestrationEventStore.rowToV2StoredEvent spans in the trace:
Build
Events decoded in the first 60 s
j5/main
124,232 (124,231 under one parent span: the full replay)
This branch
7
Not exercised live: a real prompt settled through a shared conversation. The unit test covers the live path (a settle logged after start moves the thread's authors).
Checklist
One concern: the description has no "also"
Tests cover the changed behavior (backend changes ship with focused tests)
UI changes: none
Live testing: the section above says what ran live and what did not
Upstream-owned files: none edited (all paths are under apps/server/src/j5)
Upstream product: unchanged
Surfaces: server only; no contracts, clients, or providers change
Docs: none affected
Local note: vp run check:changed and the pre-commit hook flag apps/mobile/src/j5/agents/PersonaSheet.tsx as unused (knip). That's a macOS case-insensitivity false positive: ./PersonaSheet and ./personaSheet both resolve to personaSheet.ts. j5/main's Linux CI is green, and this branch doesn't touch mobile, so I committed with --no-verify after format and lint passed.
… startup
The prompt-settle live tail in SharedConversation subscribed to
`threads.streamStoredEvents`, which is the event sink's stream from
sequence 0. On every start it read and decoded the entire orchestration
event log; on a server with 1.1M events that exhausted the 4 GB heap
within ~2.5 minutes, so the server crash-looped.
The tail now starts at the event store's high-water mark, as the other
J5 daemons do, filters to settle events in the query, and resumes from
the last handled sequence. Settles from before the start remain
LifecycleService's, from its cursor.
The test harness's log now behaves like upstream's sink (history from 0,
then live), so a tail that reads history fails the new test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No actionable comments were generated in the recent review.
Walkthrough
The prompt-settlement listener now starts from the event-store high-water mark and consumes matching events after its cursor. The test harness models seeded history and live events. Sharing tests verify live settlement handling and provide the orchestrator service.
Changes: Prompt Settlement Event Stream
Layer / File(s)
Summary
History and live event test harness apps/server/src/j5/sharing/test-support/conversationHarness.ts
The harness now models seeded event history, counts emitted history entries, and streams live events after stored history. It also provides an EventSinkV2 layer and a conversationLayerWith(history) constructor.
The listener reads PROMPT_SETTLED_EVENT events after the event-store high-water mark, processes each event before advancing its cursor, and retries after stream completion or failure. The layer now requires EventSinkV2. Tests provide the orchestrator and verify live settlement handling without reading upstream history.
sequenceDiagram
participant SharedConversation
participant EventSinkV2
participant ThreadService
participant SharedThreads
SharedConversation->>EventSinkV2: Read event-store high-water mark
SharedConversation->>ThreadService: Stream PROMPT_SETTLED_EVENT after cursor
ThreadService-->>SharedConversation: Return stored and live events
SharedConversation->>SharedThreads: Mark event settled and touch its thread
SharedConversation->>ThreadService: Request events after advanced cursor
Loading
Suggested labels:size:M
Suggested reviewers:bryantderosier
Merge Risk: Minimal · up to 925d6
This change stops the shared-conversation listener from replaying the whole event log at startup, which removes the reported memory exhaustion and restart loop. No concrete merge-blocking risk was identified in the reviewed changes.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title clearly summarizes the main change: preventing SharedConversation from replaying the full event log at startup.
Description check
The description explains the problem, the fix, testing, and scope. It does not link a tracking issue as requested by the template, and several checklist items for non-applicable changes remain uncheck…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Description check
Explanation
The description explains the problem, the fix, testing, and scope. It does not link a tracking issue as requested by the template, and several checklist items for non-applicable changes remain unchecked.
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Included review availability: This review used your included allowance. 2 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
size:M30-99 effective changed lines (test files excluded in mixed PRs).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
github-actions
Bot
added
the
vouch:trusted
Problem
Since nightly
0.0.52-nightly.20261110.16, the openclaw server crash-loops: it exits on a V8 heap OOM (4 GB) about every 2.5 minutes, and every client linked to it drops and reconnects each time. The cause is the prompt-settle live tail inSharedConversation. The v0.1.7 merge (dd89865d44) moved it fromthreads.streamDomainEvents(which starts at the newest event) tothreads.streamStoredEvents, which iseventSink.stream()with no start, i.e. a replay from sequence 0. So every server start reads and decodes the whole event log. openclaw's log has 1.1M events (about 2.5 GB of payload), which doesn't fit in the heap. Smaller installs survive, but still pay for a full replay on every start.What changed
readEventStoreHighWater, the start point every other J5 stream daemon uses). It reads throughstreamStoredEventsFromwitheventType: "runtime-request.updated", so the query filters to settle events, and it resumes from the last handled sequence after a stream failure. Settles from before the start are still marked byLifecycleServicefrom its cursor, as decision 2 intended.SharedConversation.layernow requiresEventSinkV2. The J5 runtime layer already provides it, and the two integration tests provide their harness's own orchestrator layer.streamStoredEventsreplays its history from 0, andstreamStoredEventsFromreplays what comes after the given sequence, then live. Its old live-onlyPubSubmock matched the wrong assumption and hid this bug. The new test seeds history and asserts the tail reads none of it. It fails onj5/main(expected 1 to equal +0) and passes here.Live testing
Server only, no visible surface. On a stood-down
migrate-dev-dbcopy of real data (124,232 orchestration events), I ranvp run dev:serverfor 60 s and counted theOrchestrationEventStore.rowToV2StoredEventspans in the trace:j5/mainNot exercised live: a real prompt settled through a shared conversation. The unit test covers the live path (a settle logged after start moves the thread's authors).
Checklist
apps/server/src/j5)Local note:
vp run check:changedand the pre-commit hook flagapps/mobile/src/j5/agents/PersonaSheet.tsxas unused (knip). That's a macOS case-insensitivity false positive:./PersonaSheetand./personaSheetboth resolve topersonaSheet.ts.j5/main's Linux CI is green, and this branch doesn't touch mobile, so I committed with--no-verifyafter format and lint passed.Claude Opus 5.5 in Claude Code (J5 Code).
🤖 Generated with Claude Code