feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235
feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235blackwell-systems wants to merge 1 commit into
Conversation
zyzyzyryxy
left a comment
There was a problem hiding this comment.
Thank you for the contribution and detailed benchmarking!
I left some comments to be addressed, but overall GCF idea looks good.
| 'Whether to format structured data in text response using Token-Oriented Object Notation. Defaults to false which represents the embedded content as formatted JSON instead.', | ||
| hidden: true, | ||
| }, | ||
| experimentalGcfFormat: { |
There was a problem hiding this comment.
TOON and GCF experiments are mutually exclusive, we should be explicit about this. Either replace both experimental...Format flags with a single one with enum values or at least add checks and/or appropriate message handling the case both are used together.
| type ChromeReleaseChannel as BrowsersChromeReleaseChannel, | ||
| } from '@puppeteer/browsers'; | ||
| export {encode as toonEncode} from '@toon-format/toon'; | ||
| export {encodeGeneric as gcfEncode} from '@blackwell-systems/gcf'; |
There was a problem hiding this comment.
I just moved TOON dependency to be optional in #2260, I think this PR should follow suit - we should not force on all users a dependency that's only needed for an experiment.
| toolName: string, | ||
| context: McpContext, | ||
| useToon = false, | ||
| useGcf = false, |
There was a problem hiding this comment.
Regardless if two flags will stay or you merge them, we should resolve them into a single enum choice on the top level.
| "@rollup/plugin-json": "^6.1.0", | ||
| "@rollup/plugin-node-resolve": "^16.0.3", | ||
| "@stylistic/eslint-plugin": "^5.4.0", | ||
| "@blackwell-systems/gcf": "2.1.2", |
There was a problem hiding this comment.
You need to regenerate package-lock.json after this change.
Address review feedback from ChromeDevTools#2235: 1. Replace two boolean flags (experimentalToonFormat, experimentalGcfFormat) with a single --experimentalDataFormat=json|toon|gcf enum. The legacy --experimentalToonFormat flag still works via fallback for backward compatibility. 2. Make @blackwell-systems/gcf an optional peer dependency following the same pattern established for @toon-format/toon in ChromeDevTools#2260. Uses dynamic import() so the package is only loaded when the format is requested. 3. Resolve format choice into a single compactEncode function at the top of format(), eliminating the dual-boolean checks throughout. 4. Add @blackwell-systems/gcf to rollup externals. 5. Regenerate package-lock.json.
|
Thanks for the detailed review @zyzyzyryxy! I've addressed all four points. Rebased on top of #2260 (TOON optional dep) and reworked the approach:
Force-pushed with the reworked changes. |
1296b19 to
1ad2d7a
Compare
Summary
Add
--experimentalGcfFormatas a hidden experimental flag alongside the existing--experimentalToonFormat. When enabled, tool responses encode structured data using GCF (Graph Compact Format) instead of formatted JSON. No changes to existing behavior or default output.Why
Benchmarked on Chrome DevTools data shapes
Data shapes verified from source code and test snapshots (
ConsoleMessageConcise,NetworkRequestConcise,HeapSnapshotFormatter.toJSON(),SnapshotFormatter.toJSON()):TOON increases token count on DOM snapshots (-4.6% to -17.2% vs JSON). GCF saves tokens on every data type.
Console message corruption
TOON's decoder crashes on console messages containing bracket-colon patterns, which are standard browser output:
10 of 20 console messages fail TOON round-trip. GCF: zero failures. While Chrome DevTools MCP encodes only (no decode), the structurally ambiguous output can affect downstream consumers.
LLM comprehension
GCF scores 100% on general structured data and 90.7% on adversarial payloads across GPT-4o, GPT-5.5, Claude, and Gemini. JSON scores 53.6%. TOON scores 68.5%. (1,700+ evaluations.)
Full eval data: GCF benchmarks
Data integrity
GCF verified lossless across 43 billion+ round-trips in 5 formats and 6 languages. Zero failures. Zero runtime dependencies.
Implementation
Mirrors the existing
experimentalToonFormatpattern exactly:useGcfparameter threaded throughhandle()→format()compactEncodehelper selects GCF, TOON, or null (formatted JSON)Changes
src/bin/chrome-devtools-mcp-cli-options.tsexperimentalGcfFormatflagsrc/third_party/index.tsgcfEncodefrom@blackwell-systems/gcfsrc/McpResponse.tsuseGcfparameter,compactEncodehelpersrc/ToolHandler.tsexperimentalGcfFormattohandle()package.json@blackwell-systems/gcf(pinned 2.1.2, zero deps)Links