Skip to content

Send Droid screenshots as attachment paths over worker IPC - #1179

Merged
arul28 merged 2 commits into
mainfrom
ade/droid-path-image-ipc
Aug 29, 2026
Merged

Send Droid screenshots as attachment paths over worker IPC#1179
arul28 merged 2 commits into
mainfrom
ade/droid-path-image-ipc

Conversation

@arul28

@arul28 arul28 commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Droid forked-worker screenshot IPC now sends { path, mimeType, rootPath } instead of inlined base64 on child.send, matching Cursor and Pi.
  • The worker re-opens the attachment through readFileWithinRootSecure (10 MB cap). Remote image URLs become prompt text because Droid's stream API has no URL image form.
  • Follow-up to Fix Cursor hook.sock recycle and forked-worker screenshot IPC #1177, which left Droid on the stalling inline-bytes path.

Test plan

  • droidSdkPool.test.ts asserts send payloads carry paths, not data
  • agentChatService.test.ts sendMessage with a PNG attachment does not inline PNG magic bytes
  • Desktop typecheck on the prior tree; protocol/pool/image unit tests green after the unused type-guard drop

Note

Medium Risk
Changes how user images reach Droid across process boundaries; wrong path/sandbox reads would break vision turns but avoids IPC stalls from large payloads.

Overview
Droid chat turns no longer inline screenshot base64 on the worker JSON pipe. Main process now builds prompts with the same path-only attachment shape as Pi (buildPathOnlyWorkerPrompt / buildDroidWorkerPrompt): text without embedded image bytes, plus { path, mimeType, rootPath } for on-disk attachments. The Droid worker materializes those paths locally via materializeWorkerImages before calling @factory/droid-sdk.

DroidSdkUserImage is updated to document and type path refs (with optional tiny inline data for tests). Pool and integration tests assert IPC carries paths, not data, and orchestration tests use vi.waitFor where awaitDispatch resolves before sendPrompt. Docs note Pi/Droid URL images become prompt text, not IPC URLs.

Reviewed by Cursor Bugbot for commit 71dd320. Configure here.

@vercel

vercel Bot commented Aug 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
ade Ignored Ignored Preview Aug 29, 2026 8:08am

@arul28 arul28 changed the title Cursor SDK Hook Socket Errors -> main Send Droid screenshots as attachment paths over worker IPC Aug 29, 2026
@coderabbitai

coderabbitai Bot commented Aug 29, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 13 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7d6d7a1d-e7aa-43a1-9cf6-b3324a4eed41

📥 Commits

Reviewing files that changed from the base of the PR and between 714e3c3 and 0e00a0e.

⛔ Files ignored due to path filters (2)
  • docs/features/chat/README.md is excluded by !docs/**
  • docs/features/chat/agent-routing.md is excluded by !docs/**
📒 Files selected for processing (6)
  • apps/desktop/src/main/services/chat/agentChatService.test.ts
  • apps/desktop/src/main/services/chat/agentChatService.ts
  • apps/desktop/src/main/services/chat/droidSdkPool.test.ts
  • apps/desktop/src/main/services/chat/droidSdkProtocol.ts
  • apps/desktop/src/main/services/chat/droidSdkWorker.ts
  • apps/desktop/src/main/services/chat/workerAttachmentImages.ts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

arul28 and others added 2 commits August 29, 2026 03:58
Inlining screenshot base64 on child.send can stall the Factory turn the same way it stalled Cursor and Pi. The worker now re-opens the attachment file itself.

Co-authored-by: Cursor <cursoragent@cursor.com>
The path IPC contract lives on DroidSdkUserImage and the send/pool tests; a production-uncalled validator was extra surface.

Co-authored-by: Cursor <cursoragent@cursor.com>
@arul28
arul28 force-pushed the ade/droid-path-image-ipc branch from 71dd320 to 0e00a0e Compare August 29, 2026 08:08
@arul28
arul28 merged commit aee9720 into main Aug 29, 2026
37 checks passed
@arul28
arul28 deleted the ade/droid-path-image-ipc branch August 29, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant