Skip to content

fix(e2big): one spawn facade + copilot prompts on stdin end the recurring E2BIG (#2640)#2718

Merged
rysweet merged 1 commit into
mainfrom
feat/issue-2640-comprehensively-eliminate-the-recurring-e2big-argu
Jul 6, 2026
Merged

fix(e2big): one spawn facade + copilot prompts on stdin end the recurring E2BIG (#2640)#2718
rysweet merged 1 commit into
mainfrom
feat/issue-2640-comprehensively-eliminate-the-recurring-e2big-argu

Conversation

@rysweet

@rysweet rysweet commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Problem

"Argument list too long" (E2BIG / os error 7) kept recurring across Simard's agent launch sites even after two piecemeal fixes (#2660 decision-cycle, #2700 journal) and resurfaced on the SIGNAL channel — because there was no single chokepoint every large-payload launch was forced through. The root pattern: a large prompt/context inlined into argv (or expanded via -p "$(cat FILE)") overflows ARG_MAX and execve fails with E2BIG before the child runs.

Fix: one spawn facade

New simard::spawn_payload — the single chokepoint that composes the two already-shipping transports under one policy (a payload ≥ ARGV_PAYLOAD_MAX_BYTES = 8 KiB never rides argv/envp):

  • copilot prompts → stdin via attach_prompt_std/attach_prompt_tokio (forces PromptDelivery::Stdin; copilot reads its prompt from stdin when no -p is given, and its -- positional would be misparsed as a subcommand — so a prompt of any size stays off argv).
  • recipe context → file path via recipe_context-c <key>_path=<abs> (composes recipe_context_file::ContextFile).
  • pre-exec spawn errorsrecord_spawn_failure → errno-keyed classify_spawn_failure into overseer::failure_sink (no silent swallow).

Copilot prompt sites routed through the facade (the real recurring failure)

Site Before After
base_type_copilot meeting turn (backs the SIGNAL channel) apply_std(Stdin) routed through attach_prompt_std
engineer_loop::run_engineer_subprocess (Copilot) inlined -p <prompt> in argv, stdin(null) prompt-less argv (copilot --subprocess-safe --allow-all-tools --allow-all-paths) + prompt streamed on stdin from a feeder thread

The #1717 permission contract (both flags in order + COPILOT_ALLOW_ALL=1) is preserved; the RustyClawd path is untouched (not production-wired).

Anti-regression guard

tests/e2big_argv_guard.rs fails CI if a $(cat argv-expansion, an inline unbounded recipe key (day_context/draft/episodes/pr_body), or removal of the spawn_payload facade is introduced.

Whole-repo launch-site audit

Tier Site Disposition
A base_type_copilot meeting/signal stdin via facade — ships
A engineer_loop/agent_spawn.rs copilot stdin via facade; prompt-less argv — ships
A ooda_actions/session.rs stdin (cat 'PATH' | amplihack copilot, #2660)
A journal / distillation / merge-judge file <key>_path — in-repo recipes (#2692/#2700)
B overseer task_description, self-improve proposal, engineer-loop objective bounded-inline sanitize(…,8000) (E2BIG-safe: 8000 chars ≪ ARG_MAX). File channel blocked on external recipe _path supportsmart-orchestrator.yaml / simard-self-improve-cycle.yaml / simard-engineer-loop.yaml are amplihack-bundle assets (not this repo) that read {{key}} inline; filing a large value would leave the agent with an empty var. recipe_context is ready for them once those assets gain a _path read.
C short IDs / paths / slugs (target_repo, pr_number, state_root, …) unchanged (safe)

Tests (hermetic, > 256 KiB payloads)

tests/spawn_payload_facade.rs, e2big_argv_guard.rs, and per-path transport tests for meeting, signal, engineer (+ a no-PTY stdin round-trip), overseer-launch, self-improve. The engineer permissions test now verifies the prompt lands on stdin (not argv) while #1717 flags are preserved.

Local verification: full suite green (cargo test --tests), cargo fmt --check, cargo clippy --all-targets --all-features --locked -D warnings, mkdocs build --strict.

Docs

  • Concept: docs/concepts/e2big-elimination.md
  • Reference: docs/reference/large-payload-spawn-api.md
  • How-to: docs/howto/add-a-safe-agent-spawn-site.md

Builds on #2660 and #2700 without regressing them.

Closes #2640

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…ring E2BIG (#2640)

The "Argument list too long" (E2BIG / os error 7) spawn failure kept
recurring across Simard's agent launch sites even after two piecemeal fixes
(#2660 decision-cycle, #2700 journal) and resurfaced on the SIGNAL channel,
because there was no single chokepoint every large-payload launch was forced
through.

This introduces `simard::spawn_payload` — the single facade that composes the
two already-shipping transports under one policy (payload >= 8 KiB never rides
argv/envp): copilot prompts on stdin (via prompt_delivery), recipe context on
a file path (via recipe_context_file::ContextFile), and pre-exec spawn errors
classified into overseer::failure_sink.

Copilot prompt sites routed through the facade's stdin transport:
- base_type_copilot meeting turn (and the SIGNAL channel it backs)
- engineer subprocess `run_engineer_subprocess` (Copilot) — was inlining the
  whole prompt as `-p <prompt>`; now the argv is prompt-less
  (`copilot --subprocess-safe --allow-all-tools --allow-all-paths`) and the
  prompt streams on stdin from a feeder thread. #1717 permission contract
  (both flags + COPILOT_ALLOW_ALL=1) preserved.

Anti-regression: `tests/e2big_argv_guard.rs` fails CI if a `$(cat` argv
expansion, an inline unbounded recipe key, or removal of the facade is
introduced. Per-path hermetic > 256 KiB transport tests for meeting, signal,
engineer (incl. a no-PTY stdin test), overseer-launch and self-improve.

Recipe `-c` sites whose assets are EXTERNAL (smart-orchestrator,
simard-self-improve-cycle, simard-engineer-loop) stay bounded-inline
(sanitize 8000 — already E2BIG-safe, 8000 chars << ARG_MAX): those recipes
read `{{key}}` inline with no `{{key_path}}` support, so filing a large value
would leave the agent with an empty var. The facade's `recipe_context` file
channel is ready for them once the external assets gain a `_path` read.

Docs: concept, large-payload spawn API reference, and add-a-safe-spawn-site
how-to, with a whole-repo launch-site audit table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module Lines Covered Coverage
Total 146341 122010 83.4%

Coverage data from CI run. Test files matching tests?/ are excluded from line counts.

@rysweet rysweet merged commit 69cca79 into main Jul 6, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant