Skip to content

feat: add Gemini vision optimization, custom canvas geometry, 2-column layout, detailed layout reporting, fix Windows build path bug - #252

Open
iG8R wants to merge 2 commits into
teamchong:mainfrom
iG8R:main
Open

feat: add Gemini vision optimization, custom canvas geometry, 2-column layout, detailed layout reporting, fix Windows build path bug#252
iG8R wants to merge 2 commits into
teamchong:mainfrom
iG8R:main

Conversation

@iG8R

@iG8R iG8R commented Aug 26, 2026

Copy link
Copy Markdown

📌 Motivation & Background

pxpipe was originally built around Anthropic’s vision architecture ($28\text{px}$ patches, $1568 \times 728\text{px}$ long-edge caps). When targeting Google AI Studio / Gemini models, this caused several inefficiencies:

  1. Sub-optimal Tile Alignment: Gemini uses a discrete $768 \times 768\text{ px}$ tiling system with a standard high-resolution container budget (~1,080–1,120 tokens up to $1536 \times 1536\text{ px}$ or $2048 \times 2048\text{ px}$). The hardcoded $728\text{px}$ height split files across multiple images unnecessarily, multiplying token costs per image container upload.
  2. Fixed Padding & Slack Waste: Canvas widths did not dynamically adapt to different font metrics or custom target resolutions, leaving unused pixel margins.
  3. No Multi-Column Support: Wide images (e.g., $1536\text{px}$ or $1912\text{px}$) were limited to single wide rows, causing shorter lines of prose or code to leave empty space on the right.
  4. Windows Build Path Bug: scripts/build.mjs failed on Windows due to an invalid file:// URL interpolation on drive-letter paths (resulting in Cannot find module 'F:\F:\...').

🚀 Key Changes & Features

1. Dedicated Gemini Vision Profiles (src/core/gemini-model-profiles.ts)

  • Added explicit model profiles and resolvers for gemini-2.0-flash, gemini-1.5-flash, gemini-1.5-pro, gemini-3.6-flash, and gemini-3.7-flash.
  • Configured baseline canvas geometry to $1536 \times 1536\text{ px}$ ($2 \times 2$ tile grid = ~1,081 tokens) for 1:1 pixel sharpness with zero downscaling.

2. Universal Dynamic Geometry Adapter (src/core/render.ts)

  • Implemented computeCanvasGeometry(targetWidth, targetHeight, cellW, cellH):
    • Automatically calculates the maximum integer cols and lines for any arbitrary dimensions (e.g., $1536 \times 1536$, $1912 \times 1912$, $768 \times 768$, $1024 \times 1024$).
    • Adapts dynamically to any active font atlas (spleen-5x8, jbmono10, jbmono14).
    • Allocates odd remainder pixels asymmetrically, weighting the extra pixel into the left margin (padXLeft = Math.ceil(slack / 2)).

3. Two-Column Layout Engine (--columns 2)

  • Added parallel 2-column rendering support in renderChunkToPng and renderTextToPngsUncached.
  • Text wraps into two balanced side-by-side columns with a crisp vertical dividing rule ().
  • Effectively doubles the vertical line capacity per image (e.g., up to 476 lines on a $1912 \times 1912\text{ px}$ canvas).

4. Dynamic Character Budget Scaling (src/core/library.ts)

  • Scaled dynamicCharBudget with maxLines * maxCols, eliminating premature page breaks caused by the legacy 728px character-cap ceiling.
  • Pages now fill their full vertical line capacity (e.g. all 191 lines for 1536px height) without dropping trailing lines onto a split page.

5. CLI Geometry Overrides (src/core/export.ts & src/node.ts)

  • Added CLI argument support for --width <px>, --height <px>, and --columns <1|2>.
  • Values inherit from the active --model profile by default, with CLI arguments selectively overriding specific layout properties.

6. Detailed, Column-Aligned Terminal Reporting (src/node.ts)

  • Updated printExportReport to display global layout configuration (font, cell pitch, columns, margins).
  • Added a formatted per-page tree output with aligned | vertical dividers displaying dimensions, line counts, character counts, file sizes, tile counts, and estimated vision tokens.

7. Cross-Platform Build Fix (scripts/build.mjs)

  • Replaced flawed new URL(..., 'file://${tsPkgPath}').pathname with standard path.resolve(path.dirname(tsPkgPath), tscRelative).
  • Resolves Windows double-drive-letter concatenation bugs during pnpm run build.

🧪 Verification & Results

Tested on a 153,532 character codebase export:

Configuration Output Layout Tokens in AI Studio Savings
Default Claude ($1568 \times 728\text{px}$) 6 pages ($1568 \times 728\text{px}$) ~8,008 tokens 80.7%
Gemini ($1536 \times 1536\text{px}$) 3 pages ($1536 \times 1536\text{px}$) ~3,270 tokens (1,090/page) 92.1%
Gemini ($1912 \times 1912\text{px}$) 2 pages ($1912 \times 1912\text{px}$) ~2,171 tokens (1,085/page) 94.6%

CLI Output Preview:

pxpipe export
  out:            f:\pxpipe-export-FBlfku
  model:          gemini-2.0-flash [font: spleen-5x8 @ 5x8px | grid: 306 cols | padX: 3L/3R | padY: 4T/4B]
  files:          1
  source chars:   153,532
  pages:          3 (749,682 bytes)
    ├─ page-001.png:   1536x1536px | 191 lines |  58,642 chars | 290,484 B | 4 tiles (~1,120 tokens)
    ├─ page-002.png:   1536x1536px | 191 lines |  58,642 chars | 272,959 B | 4 tiles (~1,120 tokens)
    └─ page-003.png:   1536x984px  | 121 lines |  36,248 chars | 186,239 B | 4 tiles (~1,120 tokens)
  text tokens:    ~41,495
  image tokens:   ~3,360  (91.9% saved)
  factsheet:      96 items (337 dropped)

📂 Modified Files

  • src/core/gemini-model-profiles.ts
  • src/core/render.ts
  • src/core/library.ts
  • src/core/export.ts
  • src/node.ts
  • scripts/build.mjs

…out, Detailed Layout Reporting, fix Windows Build Path Bug
@iG8R iG8R changed the title Add Gemini Vision Optimization, Dynamic Canvas Geometry, 2-Column Layout, Detailed Layout Reporting, fix Windows Build Path Bug feat: add Gemini vision optimization, custom canvas geometry, 2-column layout, detailed layout reporting, fix Windows build path bug Aug 26, 2026
Fix Gemini Default Width Fallback (src/core/library.ts): Ensure default Gemini exports resolve directly to 1536px instead of evaluating opts.cols * cellW + 8 (1538px), preventing a 2px spillover into an accidental 6th tile.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant