Skip to content

Never cut text mid-emoji before agents or notifications - #7796

Merged
ymichael merged 7 commits into
mainfrom
bb/never-cut-text-mid-emoji-before-agents-or-notifi-thr_nk6q4nbknq
Nov 18, 2026
Merged

ymichael merged 7 commits into
mainfrom
bb/never-cut-text-mid-emoji-before-agents-or-notifi-thr_nk6q4nbknq

Conversation

@ymichael

@ymichael ymichael commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Human comments

What was wrong

bb capped text with raw UTF-16 .slice(0, n) / .slice(-n) / .substring in dozens of places. When an emoji's surrogate pair straddled the cap, the cut kept half of it, a lone surrogate. Much of that text goes to agents: child-thread output reported to the parent (child-thread-notifications.ts), the restart-resume note (restart-resume.ts), plugin instructions (thread-runtime-config.ts, host-policy.ts), MCP/browser/annotation tool results, and more. It also goes to push notifications and terminal titles. JSON.stringify escapes a lone surrogate as \ud83d. Codex's serde-based app-server rejects that escape and drops the whole message without answering. bb then waits on thread/start until it times out. Nothing made text well-formed before it left for a provider, so any one bad cut could break a turn.

T3 Code fixed the same class: pingdotgg/t3code#30369, #29168, #29653, #29906 (#30369 pointed us at it).

What changed

  • Safe cuts where text reaches an agent or a person. Caps on user or agent text now use sliceUtf16Head / sliceUtf16Tail from @bb/text-utils (added in Centralize Unicode-safe text handling across core #3995, but Allow plugin dynamic instructions up to 32768 characters #7050's plugin-instructions cap and older cuts such as child-thread output never used them): server, daemon, app, desktop, CLI, web, packages, and first-party plugins. Plugins get the same functions through the SDK (below) instead of local copies. Cuts on text that cannot hold a surrogate pair (ASCII slugs, handles, hostnames, URLs, tag names) and log-only or dev-tooling output are left alone. The page script in agent-annotations is stringified into the browser, so its copy lives inside the injected functions. boundedMarkdownPreview now cuts with sliceUtf16Head and checks for a split token against the original text.
  • Well-formed at the provider boundary. serializeJsonRpcLine in bridge-kit/runtime-json-rpc.ts replaces lone surrogates with U+FFFD in every string the runtime sends to a provider bridge (requests, results, errors), so every provider gets well-formed turn input and instructions. It serializes with plain JSON.stringify and only re-serializes with the replacer when the line holds an escaped lone surrogate (\ud), so normal traffic pays nothing extra (20k-string message: 1.3 ms plain, 4.6 ms always-replacer, 1.5 ms with the check). The ACP message stream and the Codex bridge's app-server writer use the same function, which also covers text a bridge builds itself. New toWellFormedUtf16 in @bb/text-utils.
  • Plugin SDK exports. Core's own functions, re-exported with experimental_ names (entries in docs/api_to_audit.md, symbols in the Plugin Guide):
    • experimental_sliceUtf16Head from @get-bb/plugin-sdk (server code) and @get-bb/plugin-sdk/app (wired in plugin-sdk-app-impl.tsx).
    • experimental_sliceUtf16Head / experimental_sliceUtf16Tail from @get-bb/plugin-sdk/host, because the root SDK runtime in host bundles is a stub. Browser Automation's host code and the Claude Code and Codex providers use it.
    • experimental_serializeJsonRpcLine from @get-bb/plugin-sdk/provider-bridge, used by the Codex app-server writer.
    • Agent Annotations keeps an inline copy because its page script is stringified into the browser. The 10 plugins that use the runtime-provided root or /app export require bbPluginSdk >=0.6.161, as Use the shared host-path parser across bundled plugins #6760 did for the host-path helpers.
  • No lint guard: the provider boundary makes any future raw cut harmless to providers, so a name-based heuristic rule was not worth its false positives.
  • No server/daemon wire change, so HOST_DAEMON_PROTOCOL_VERSION stays the same. The JSON-RPC change is between the daemon and its own bridge processes. @get-bb/plugin-sdk is bumped to 0.6.161 for the new exports and the changed bridge code. @bb/text-utils was added as a dependency to the packages that newly import it.

How you verified

Repro on origin/main (502cf5a), real dev instance (pnpm start:worktree) with real Codex: a local plugin contributed instructions "x".repeat(32767) + "😀 …"; the 32,768-char cap cut the emoji in half.

  • Codex thread: Command thread.start failed — codex app-server did not answer thread/start within 60000ms (status error).
  • Control with the plugin disabled: the thread replies ok.
  • Talking to codex app-server (codex-cli 0.169.0) directly with the same bytes: stderr Failed to deserialize JSONRPCMessage: unexpected end of hex escape at line 1 column 32921, and no response. With a safe cut, both thread/start and turn/start succeed.
  • Claude Code accepted the same instructions, so this broke Codex turns but not Claude Code turns.

After the fix, same dev data dir and plugin enabled: the Codex thread starts and replies ok.

Tests added:

  • runtime JSON-RPC: a lone surrogate never reaches the provider's stdin, while real emoji pairs survive.
  • Cut boundaries for child-thread output, plugin instructions, restart-resume command names, and push-notification truncate.
  • toWellFormedUtf16.

Commands:

  • pnpm exec turbo run typecheck --filter='...[origin/main]': 112/112 passed.
  • pnpm exec turbo run test lint --filter='[origin/main]': everything passed except @bb/app CommandPalette.test.tsx and @bb/server install-machine-script.test.ts. Both were timeouts while a dev instance was running and pass on rerun; neither touches changed code.
  • pnpm exec turbo run prepare:bundled: 70/70 passed.

After trimming the change (811d979): pnpm exec turbo run typecheck lint --filter='...[HEAD]' 151/151, pnpm exec turbo run test --filter='[HEAD]' 29/29, prepare:bundled for the touched plugins 12/12. On a rebuilt pnpm start:worktree with real Codex, a plugin whose instructions put an emoji at the 32,768 cap and one that sends a raw lone surrogate both get ok.

After moving the helpers into the SDK (2d6fa48): pnpm exec turbo run typecheck lint --filter='...[HEAD]' 129/129, pnpm exec turbo run test --filter='[HEAD]' 23/23 (including @bb/app, @get-bb/plugin-sdk and the Plugin Guide), prepare:bundled for the 12 touched plugins 18/18. The built bundles show the Codex host bundle inlining serializeJsonRpcLine, the Docs server importing experimental_sliceUtf16Head from the runtime root, and Browser Automation's app reading it from the app runtime. Both live Codex checks still reply ok on a rebuilt pnpm start:worktree.

🤖 Generated with Claude Code

AGENT GENERATED

@ymichael
ymichael force-pushed the bb/never-cut-text-mid-emoji-before-agents-or-notifi-thr_nk6q4nbknq branch from c8ad59d to 57db3b1 Compare November 18, 2026 21:36
ymichael and others added 3 commits November 18, 2026 13:41
Raw UTF-16 .slice/.substring caps left a lone surrogate whenever an emoji
straddled the cut. bb then sent that text into agent turns and instructions:
Codex's JSON-RPC parser rejects the escaped lone surrogate and drops the
message, so thread/start never answers and the thread fails after 60s.

- Cut text with sliceUtf16Head/sliceUtf16Tail at every length cap (child
  output, plugin instructions, restart-resume notes, push notifications,
  terminal titles, tool results, status messages, ...). Forkable plugins
  carry a local copy since they cannot import @bb/text-utils.
- Replace lone surrogates with U+FFFD where JSON-RPC leaves the runtime for
  a provider bridge, the Codex app-server, and ACP agents.
- Add bb/no-raw-text-truncation to oxlint so new raw text caps fail lint.
- Bump @get-bb/plugin-sdk to 0.6.159 (bundled bridge code changed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ymichael
ymichael force-pushed the bb/never-cut-text-mid-emoji-before-agents-or-notifi-thr_nk6q4nbknq branch from 57db3b1 to dfe6917 Compare November 18, 2026 21:42
ymichael and others added 4 commits November 18, 2026 14:22
The provider JSON-RPC boundary already makes every string well-formed, so
the name-based lint rule only forced rewrites of cuts on text that cannot
hold a surrogate pair: ASCII slugs, handles, hostnames and URLs, plus
log-only and dev-tooling output. Revert those, keep safe cuts on text that
reaches an agent or a person, and serialize JSON-RPC lines with plain
JSON.stringify unless the output holds an escaped lone surrogate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-emoji-before-agents-or-notifi-thr_nk6q4nbknq

# Conflicts:
#	apps/app/src/components/provider-cli/provider-cli-install-failure.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plugins carried their own copies of sliceUtf16Head/Tail and the Codex bridge
its own well-formed JSON writer. Export core's functions instead:
experimental_sliceUtf16Head from the SDK root and /app,
experimental_sliceUtf16Head/Tail from /host (the host root runtime is a stub),
and experimental_serializeJsonRpcLine from /provider-bridge. Plugins that use
the runtime-provided root or /app export require SDK 0.6.161.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ymichael
ymichael merged commit 343b5e8 into main Nov 18, 2026
97 of 98 checks passed
@ymichael
ymichael deleted the bb/never-cut-text-mid-emoji-before-agents-or-notifi-thr_nk6q4nbknq branch November 18, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant