fix: stop a single streamed token invalidating the whole conversation

.agents/notes/implemented/bug-fix/2026-09-16-streaming-cost-scales-with-conversation-length.md

@@ -0,0 +1,153 @@
+# Streaming cost scaled with conversation length, not with what changed
+
+Status: implemented
+Translation: current
+
+[中文](2026-09-16-streaming-cost-scales-with-conversation-length.zh.md)
+
+## Abstract
+
+After windowed reads shipped, users reported the renderer becoming unresponsive on
+long conversations. The window was not the problem: a content notification reported
+the span between the lowest and highest position it touched and named every turn id
+inside it, so one synced batch carrying an early status write alongside the streaming
+tail told the display cache that the whole conversation had changed — measured at 996
+directory rows and 997 turn bodies re-read, about 1.2 s of main-thread CPU, for a
+single token delta on a 1,000-turn conversation. Notifications now carry the exact
+touched positions, the cache invalidates only the ids the reader named, and four
+whole-conversation costs that ran per frame or per open were made incremental or
+deferred. The same delta now re-reads two rows and two bodies. Total work for the
+shared fact table is unchanged — it still materializes every turn once per session —
+so a first open on a very long conversation is improved but not solved; that needs
+facts to move into write-time metadata.
+
+## The reported symptom and what it was not
+
+The report was "the renderer freezes easily on the latest version, 0.93.3 was fine".
+0.93.3 and 0.94.0 differ by the windowed conversation read. Two plausible causes were
+ruled out first: Electron is unchanged across the two releases (`^39.2.6`), and a
+pure tail delta — a token arriving with nothing else in the batch — already cost 7-9 ms
+and still does. The regression only appears when a batch carries more than the tail.
+
+## Root cause: a sparse change reported as a dense range
+
+`changeRangeOf` in the Loro session adapter reduced a batch to `[min(position),
+max(position) + 1)` and `observe` then read every id in that range. A batch that
+touched positions 4 and 999 named 996 turns. The display cache merged its refresh
+targets the same way, so two independent notifications arriving before one flush
+produced the same span.
+
+The cache then re-read the whole directory for that span and, for every hydrated row
+in it, re-materialized the body. That second amplification had its own cause:
+`rowChanged` compared `itemCount` and `planCount`, which a container-backed directory
+row deliberately never carries, against the real counts a hydrated row holds. The
+comparison was `42 !== undefined` on every refresh, so every hydrated row reported a
+change, bumped its content epoch and was re-read. Of the 140 bodies re-read in the
+smaller reproduction, 2 had changed.
+
+Bumping those epochs also cancelled in-flight reads for turns nothing had touched.
+A 300-turn window lease taken while such batches arrived every 5 ms took 1,890 ms and
+performed 3.5 body reads per turn; the 5 ms interval fired 4 times in that window,
+which is the unresponsiveness the report describes.
+
+## What changed
+
+- A content batch reports its exact positions. Structural batches keep the
+  shifted-suffix range they need, because later positions really did move.
+- The view tracks content targets as the reported ids, resolves their positions at
+  flush time, and refreshes them in contiguous runs. A content refresh that finds the
+  length changed escalates to a structural re-key rather than splicing rows from a
+  sparse read.
+- Body invalidation follows the reported ids, never the directory diff: a directory
+  row cannot tell whether a body changed, since a grown text item moves no scalar.
+- `rowChanged` compares counts only after the refresh has carried forward the ones it
+  cannot supply, and an unchanged row keeps its object — placeholder items and Virtua
+  rows key on that identity, and replacing it defeated their caches.
+
+Four costs that scale with turn count were addressed alongside it:
+
+- A user turn's send configuration projected eagerly on every directory row. The
+  projection is a schema parse per turn while `resolveSessionConversationConfig` reads
+  the newest source and walks older ones only until it finds an explicit Role, so it
+  is now deferred and memoized at each hop that used to force it. Full directory read:
+  4,000 turns 728 ms to 264 ms.
+- The shared fact table discarded its facts on the last consumer release, so reopening
+  a session tab re-materialized every body to rebuild them. It is now held rather than
+  disposed, keeps its view subscription, and resumes. Its background pass yields to
+  real idle time instead of back-to-back macrotasks.
+- `use-session-diff-summary` serialized every turn's file diffs each frame to decide
+  whether anything changed (144 KiB per frame at 4,000 turns). Identical entries are
+  settled by reference; only a replaced entry is serialized.
+- `normalizeTexMathDelimiters` and the Mermaid fence test re-scanned the whole
+  accumulated answer on every delta, which is quadratic in answer length. Both now
+  settle the common case with a substring test: a 120 KiB math-free answer went from
+  2.49 ms to 0.06 ms per delta.
+
+The index-row list, the chat stream items, the conversation config sources and the
+ordered fact list hand back their previous array when nothing changed, and
+`ConversationView` gained `structureVersion` so the accepted-history projection stops
+rebuilding a whole-conversation slot array at token rate.
+
+## Alternatives considered
+
+Reporting the dense range but letting the cache filter it was rejected: the cache
+would still read the whole directory to discover that nothing else moved, which is the
+larger of the two costs at 4,000 turns.
+
+Making the directory row carry `itemCount`/`planCount` would also have removed the
+`rowChanged` false positive, at one extra container crossing per turn on every
+directory read. Carrying the previous row's counts forward costs nothing and keeps the
+adapter's existing "omit rather than guess" rule.
+
+Keeping the fact table alive after the last release trades memory for the reopen cost.
+The table is keyed by the view, so the session store's existing eviction bounds it; a
+per-consumer timeout was rejected as a second lifetime to reason about.
+
+## Two corrections found in review
+
+Both were introduced by this work and are fixed in the same branch, each pinned by
+a test that fails on the intermediate implementation.
+
+Holding the fact table instead of disposing it stopped the background pass but left
+the view subscription in place. That is correct when the table is keyed on the
+conversation's own view, and wrong when it is keyed on a projection wrapper: the
+wrapper is rebuilt whenever an optimistic entry appears or resolves, so the base
+view's listener set accumulated one released wrapper — and one live fact table — per
+message sent, each still deriving on every token. Tables are now keyed and
+subscribed on `factSource`, the underlying view. That also collapses a duplicate
+that predates this branch: the diff summary acquired on the base view while the
+turn-fact readers acquired on the wrapper, so an unconfirmed entry meant two full
+fact tables for one conversation.
+
+Keeping the index row object when `rowChanged` reported no change assumed that
+`rowChanged` sees everything a row carries. It does not see `inputConfig`, which is
+a deferred projection that cannot be diffed without forcing it — exactly the cost
+this branch removed. A user turn whose send configuration changed while outside
+every hydrated window therefore kept its old model, Role and MCP selection in the
+index, which is what the sticky-configuration resolver reads. A reported turn now
+always takes the fresh row; identity is preserved only for turns the notification
+did not name, which is where the churn this optimization targets came from.
+
+## Verification and limits
+
+Regression coverage pins the invalidation contract: a batch carrying an early edit and
+the streaming tail reads 2 directory rows and 2 bodies, and fails at 23 rows on the
+previous implementation. The fact-table test pins that facts survive the last release
+and that the background pass is held until someone re-acquires.
+
+Numbers above come from synthetic Loro fixtures in Node on one machine and are library
+measurements, not device acceptance. They are useful as ratios, not as budgets. Real
+turns carry far more content than the fixture's.
+
+Two known costs are unchanged. The shared fact table still materializes every turn once
+per session, because deriving a goal, scheduled task, proposed plan or file diff needs
+the body; removing that needs those facts written alongside history, which
+[versioned history hashes and primitive metadata](../architecture/2026-09-14-versioned-history-hashes-and-primitive-metadata.md)
+opens the way for. Snapshot import still decodes on the renderer thread, as
+[windowed conversation reads](../architecture/2026-09-10-windowed-reader-integration.md)
+already recorded; moving it would mean moving the document off the main thread.
+
+That note's evidence boundary listed streaming frame time as separate acceptance work
+that was never done. This note is the correction: the work was necessary, and the cost
+it would have found was a correctness-shaped defect in change scoping rather than a
+tuning problem.

.agents/notes/implemented/bug-fix/2026-09-16-streaming-cost-scales-with-conversation-length.zh.md

@@ -0,0 +1,118 @@
+# 流式开销随对话长度增长，而非随实际变更量增长
+
+Status: implemented
+Translation: current
+
+[English](2026-09-16-streaming-cost-scales-with-conversation-length.md)
+
+## 摘要
+
+窗口化读取上线后，用户反馈长对话会让渲染进程失去响应。问题不在窗口化：内容变更通知
+上报的是它触及的最低与最高位置之间的整个区间，并把区间内每一个 turn id 都列为已变更，
+因此一个同步批次只要同时带上一条早期 turn 的状态写入和正在流式输出的尾部，就等于告诉
+显示缓存「整个对话都变了」——实测在 1000 turn 的对话上，单个 token delta 触发 996 行
+目录重读、997 个 turn body 重新物化，约 1.2 秒主线程 CPU。现在通知只携带真正触及的位置，
+缓存只让 reader 点名的 id 失效，另有四处按帧或按打开执行的全对话开销改为增量或惰性。
+同一个 delta 现在只重读 2 行和 2 个 body。共享 fact table 的总工作量未变——每个会话仍会
+把每个 turn 物化一次——所以超长对话的首次打开有所改善但未解决，那需要把 facts 落到写入时
+的元数据里。
+
+## 报告的现象，以及排除了什么
+
+反馈是「最新版很容易卡死，0.93.3 没遇到过」。0.93.3 与 0.94.0 之间的差异是窗口化对话读取。
+先排除了两种可能：两个版本的 Electron 完全一致（`^39.2.6`）；纯尾部 delta——批次里只有一个
+token 到达——此前就是 7–9 ms，现在仍是。回归只在批次里不止有尾部时出现。
+
+## 根因：稀疏变更被上报为稠密区间
+
+Loro session 适配器的 `changeRangeOf` 把一个批次归约成 `[min(position), max(position)+1)`，
+`observe` 随后读出该区间内的每一个 id。一个只触及位置 4 和 999 的批次会点名 996 个 turn。
+显示缓存用同样的方式合并刷新目标，所以两条独立通知只要在同一次 flush 之前到达，也会产生
+同样的区间。
+
+缓存随后重读整个区间的目录，并把区间内每一个已 hydrate 的行的 body 重新物化。第二重放大
+另有原因：`rowChanged` 会比较 `itemCount` 与 `planCount`，而容器形态的目录行刻意从不携带
+这两个字段，已 hydrate 的行却持有真实值。于是每次刷新的比较都是 `42 !== undefined`，每个
+已 hydrate 的行都被判定为已变更、content epoch 被 bump、body 被重读。在较小的复现里重读的
+140 个 body 中，真正变了的是 2 个。
+
+bump 这些 epoch 还会作废那些与变更无关的 turn 的在途读取。在这类批次每 5 ms 到达一次的情况下
+获取一个 300 turn 的窗口 lease 耗时 1890 ms、每个 turn 平均读取 3.5 次 body；那个 5 ms 的
+定时器在此期间只触发了 4 次——这正是反馈里描述的无响应。
+
+## 改动内容
+
+- 内容批次上报确切位置。结构性批次保留它需要的「位移后缀」区间，因为后续位置确实移动了。
+- 视图以上报的 id 记录内容刷新目标，在 flush 时解析位置，并按连续段刷新。内容刷新若发现
+  长度变化，会升级为结构性重建索引，而不是从稀疏读取里拼接行。
+- body 失效只跟随上报的 id，不跟随目录 diff：目录行无法判断 body 是否变化，因为一条变长的
+  文本条目不会移动任何标量。
+- `rowChanged` 只在刷新已把它无法提供的计数继承过来之后才比较它们；未变更的行保留原对象——
+  placeholder 条目与 Virtua 行以该身份为键，替换它会让这些缓存失效。
+
+同时处理了四处随 turn 数增长的开销：
+
+- user turn 的发送配置在每个目录行上被立即投影。该投影是每个 turn 一次 schema 解析，而
+  `resolveSessionConversationConfig` 只读最新的来源、并且只在找到显式 Role 之前向前少量回溯，
+  因此改为在每个原本会强制求值的环节上惰性求值并记忆化。完整目录读取：4000 turn 从 728 ms
+  降到 264 ms。
+- 共享 fact table 在最后一个消费者释放时丢弃 facts，导致重新打开会话标签时要把每个 body 重新
+  物化。现在改为保持而非销毁，保留对视图的订阅并可续跑。其后台 pass 让出到真正的空闲时间，
+  而不是连续的宏任务。
+- `use-session-diff-summary` 每帧序列化每个 turn 的文件改动来判断是否有变化（4000 turn 时每帧
+  144 KiB）。现在相同条目按引用判定，只有被替换的条目才会被序列化。
+- `normalizeTexMathDelimiters` 与 Mermaid 围栏检测在每个 delta 上重扫整段累计回答，对回答长度
+  是平方级。两者现在都用子串检测先行了结常见情况：120 KiB 且不含公式的回答从每个 delta
+  2.49 ms 降到 0.06 ms。
+
+索引行列表、聊天流条目、对话配置来源与有序 fact 列表在无变化时都返回上一次的数组；
+`ConversationView` 新增 `structureVersion`，使已接受历史的投影不再以 token 频率重建整对话的
+槽位数组。
+
+## 考虑过的替代方案
+
+「仍上报稠密区间、由缓存自行过滤」被否决：缓存仍需读完整个目录才能发现其余部分没有变化，
+而在 4000 turn 时这正是两项开销中较大的一项。
+
+让目录行携带 `itemCount`/`planCount` 同样能消除 `rowChanged` 的假阳性，代价是每次目录读取里
+每个 turn 多一次容器跨越。继承上一行的计数没有额外成本，也保留了适配器现有的「宁可省略，
+不做猜测」规则。
+
+在最后一次释放后保留 fact table 是用内存换重新打开的开销。该表以视图为键，因此受会话 store
+现有的淘汰机制约束；按消费者设置超时被否决，因为那会引入第二套需要推理的生命周期。
+
+## 评审中发现的两处修正
+
+两处都是本轮工作引入的，已在同一分支修复，各自由一个在中间实现上失败的用例锁定。
+
+把 fact table 改为保持而非销毁时，只停掉了后台扫描，却保留了对视图的订阅。当表以对话
+自身的视图为键时这是对的；以投影包装器为键时则是错的：包装器会在乐观条目出现与确认时
+被重建，于是每发送一条消息，底层视图的监听集合就多积累一个已释放的包装器和一张仍在
+工作的 fact table，每个 token 都会各推导一次。现在表以 `factSource`（底层视图）为键并
+在其上订阅。这同时消除了一个早于本分支的重复：diff summary 在底层视图上获取，而 turn
+fact 读取方在包装器上获取，因此只要存在未确认条目，一个对话就会有两张完整的 fact table。
+
+在 `rowChanged` 判定无变化时保留索引行对象，前提是 `rowChanged` 能看到行携带的一切。
+它看不到 `inputConfig`——那是一个惰性投影，不强制求值就无法比较，而强制求值正是本分支
+要消除的开销。于是一个处于所有已 hydrate 窗口之外的 user turn，其发送配置变更后，索引
+里仍保留旧的模型、Role 与 MCP 选择，而这正是粘性配置解析器读取的内容。现在被点名的
+turn 一律采用新行；只有通知未点名的 turn 才保留身份，而那正是该优化针对的抖动来源。
+
+## 验证与局限
+
+回归用例锁定了失效契约：同时携带早期编辑与流式尾部的批次只读 2 行目录、2 个 body，在改动前的
+实现上会在 23 行处失败。fact table 的用例锁定了 facts 在最后一次释放后存活，且后台 pass 在
+有人重新获取之前保持暂停。
+
+以上数字来自单机 Node 环境下的合成 Loro fixture，属于库级测量，不是设备验收。它们作为比例有
+参考价值，不能当作预算。真实 turn 携带的内容远多于 fixture。
+
+有两项已知开销未改变。共享 fact table 仍然每个会话把每个 turn 物化一次，因为推导 goal、定时
+任务、提案计划或文件改动都需要 body；要去掉它，需要把这些 facts 与历史一起写入，而
+[版本化历史哈希与原始元数据](../architecture/2026-09-14-versioned-history-hashes-and-primitive-metadata.md)
+为此铺好了路。快照 import 仍在渲染线程解码，这一点
+[窗口化对话读取](../architecture/2026-09-10-windowed-reader-integration.md)
+已有记录；要移走它意味着把文档移出主线程。
+
+那篇笔记的证据边界把流式帧时间列为「另行验收的工作」，而这项工作从未进行。本笔记即是对它的
+更正：这项工作是必要的，且它本应发现的开销是变更范围界定上的正确性缺陷，而不是调优问题。