Skip to content

feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235

Open
blackwell-systems wants to merge 1 commit into
ChromeDevTools:mainfrom
blackwell-systems:experimental-gcf-format
Open

feat: add experimentalGcfFormat flag for GCF-encoded tool responses#2235
blackwell-systems wants to merge 1 commit into
ChromeDevTools:mainfrom
blackwell-systems:experimental-gcf-format

Conversation

@blackwell-systems

Copy link
Copy Markdown

Summary

Add --experimentalGcfFormat as a hidden experimental flag alongside the existing --experimentalToonFormat. When enabled, tool responses encode structured data using GCF (Graph Compact Format) instead of formatted JSON. No changes to existing behavior or default output.

Why

Benchmarked on Chrome DevTools data shapes

Data shapes verified from source code and test snapshots (ConsoleMessageConcise, NetworkRequestConcise, HeapSnapshotFormatter.toJSON(), SnapshotFormatter.toJSON()):

Dataset JSON TOON GCF TOON vs JSON GCF vs JSON GCF vs TOON
Console (100) 2,534 1,716 1,663 32.3% 34.4% +3.1%
Console (500) 12,879 8,761 8,475 32.0% 34.2% +3.3%
Network (100) 3,287 1,776 1,682 46.0% 48.8% +5.3%
Network (500) 16,547 8,936 8,442 46.0% 49.0% +5.5%
Heap (100) 2,210 797 753 63.9% 65.9% +5.5%
DOM (3x3) 282 295 244 -4.6% 13.5% +17.3%
DOM (4x3) 396 464 407 -17.2% -2.8% +12.3%

TOON increases token count on DOM snapshots (-4.6% to -17.2% vs JSON). GCF saves tokens on every data type.

Console message corruption

TOON's decoder crashes on console messages containing bracket-colon patterns, which are standard browser output:

[Error]: net::ERR_CONNECTION_REFUSED
[React DevTools]: Component rendered 3 times
[Violation]: Forced reflow while executing JavaScript took 42ms
[Performance]: Long task detected (duration: 234ms)

10 of 20 console messages fail TOON round-trip. GCF: zero failures. While Chrome DevTools MCP encodes only (no decode), the structurally ambiguous output can affect downstream consumers.

LLM comprehension

GCF scores 100% on general structured data and 90.7% on adversarial payloads across GPT-4o, GPT-5.5, Claude, and Gemini. JSON scores 53.6%. TOON scores 68.5%. (1,700+ evaluations.)

Full eval data: GCF benchmarks

Data integrity

GCF verified lossless across 43 billion+ round-trips in 5 formats and 6 languages. Zero failures. Zero runtime dependencies.

Implementation

Mirrors the existing experimentalToonFormat pattern exactly:

  • Hidden boolean CLI flag, defaults to false
  • useGcf parameter threaded through handle()format()
  • compactEncode helper selects GCF, TOON, or null (formatted JSON)
  • GCF takes precedence if both flags are set

Changes

File Change
src/bin/chrome-devtools-mcp-cli-options.ts Add experimentalGcfFormat flag
src/third_party/index.ts Export gcfEncode from @blackwell-systems/gcf
src/McpResponse.ts Add useGcf parameter, compactEncode helper
src/ToolHandler.ts Pass experimentalGcfFormat to handle()
package.json Add @blackwell-systems/gcf (pinned 2.1.2, zero deps)

Links

@zyzyzyryxy zyzyzyryxy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution and detailed benchmarking!

I left some comments to be addressed, but overall GCF idea looks good.

'Whether to format structured data in text response using Token-Oriented Object Notation. Defaults to false which represents the embedded content as formatted JSON instead.',
hidden: true,
},
experimentalGcfFormat: {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TOON and GCF experiments are mutually exclusive, we should be explicit about this. Either replace both experimental...Format flags with a single one with enum values or at least add checks and/or appropriate message handling the case both are used together.

Comment thread src/third_party/index.ts Outdated
type ChromeReleaseChannel as BrowsersChromeReleaseChannel,
} from '@puppeteer/browsers';
export {encode as toonEncode} from '@toon-format/toon';
export {encodeGeneric as gcfEncode} from '@blackwell-systems/gcf';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved TOON dependency to be optional in #2260, I think this PR should follow suit - we should not force on all users a dependency that's only needed for an experiment.

Comment thread src/McpResponse.ts Outdated
toolName: string,
context: McpContext,
useToon = false,
useGcf = false,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless if two flags will stay or you merge them, we should resolve them into a single enum choice on the top level.

Comment thread package.json Outdated
"@rollup/plugin-json": "^6.1.0",
"@rollup/plugin-node-resolve": "^16.0.3",
"@stylistic/eslint-plugin": "^5.4.0",
"@blackwell-systems/gcf": "2.1.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to regenerate package-lock.json after this change.

Address review feedback from ChromeDevTools#2235:

1. Replace two boolean flags (experimentalToonFormat, experimentalGcfFormat)
   with a single --experimentalDataFormat=json|toon|gcf enum. The legacy
   --experimentalToonFormat flag still works via fallback for backward
   compatibility.

2. Make @blackwell-systems/gcf an optional peer dependency following the
   same pattern established for @toon-format/toon in ChromeDevTools#2260. Uses dynamic
   import() so the package is only loaded when the format is requested.

3. Resolve format choice into a single compactEncode function at the top
   of format(), eliminating the dual-boolean checks throughout.

4. Add @blackwell-systems/gcf to rollup externals.

5. Regenerate package-lock.json.
@blackwell-systems

blackwell-systems commented Jun 26, 2026

Copy link
Copy Markdown
Author

Thanks for the detailed review @zyzyzyryxy! I've addressed all four points.

Rebased on top of #2260 (TOON optional dep) and reworked the approach:

  1. Single enum: Replaced both experimentalToonFormat and experimentalGcfFormat boolean flags with --experimentalDataFormat=json|toon|gcf. The legacy --experimentalToonFormat still works via fallback for backward compatibility.

  2. Optional peer dependency: @blackwell-systems/gcf is now an optional peer dep using dynamic import() via getGcfEncode(), following the exact pattern from chore: make TOON dependency optional #2260.

  3. Single encoder resolution: ToolHandler resolves the format choice into one dataFormat string, McpResponse.format() resolves it into one compactEncode function at the top. No more dual-boolean checks.

  4. package-lock.json: Regenerated.

  5. Rollup externals: Added @blackwell-systems/gcf.

Force-pushed with the reworked changes.

@blackwell-systems blackwell-systems force-pushed the experimental-gcf-format branch from 1296b19 to 1ad2d7a Compare June 26, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants