Skip to content

MarkdownStreamController.append incorrectly drops repeated boundary characters when streaming delta chunks #189

@ominus3

Description

@ominus3

MarkdownStreamController.append(chunk) currently calls mergeStreamingText(prev, next) to support both delta-mode and accumulated-mode producers. This can incorrectly remove valid
repeated characters when a delta chunk starts with the same text that the previous accumulated content ends with.

Reproduction:

await controller.append('共 3');
await controller.append('3 条');

Expected rendered content:

共 33 条

Actual rendered content:

共 3 条

Root cause appears to be the overlap merge logic:

if (prev.endsWith(next.substring(0, len))) {
return prev + next.substring(len);
}

For prev = '共 3' and next = '3 条', the longest overlap is '3', so the second 3 is dropped. This is unsafe for true delta streams because repeated boundary characters can be legitimate
content, especially numbers, markdown punctuation, CJK text, or tokenized model output.

This caused a production-visible issue where the LLM and tool result both returned 33 rows, but the Feishu streaming card displayed 3 rows.

Suggested fix options:

  • Make append(chunk) append deltas verbatim without overlap merging.
  • Add a separate API for accumulated/full-content producers.
  • Or make overlap merging opt-in, not the default behavior of append.

Workaround currently used by downstream code:

let content = '';

async function appendMarkdown(delta: string) {
content += delta;
await controller.setContent(content);
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions