fix(e2big): one spawn facade + copilot prompts on stdin end the recurring E2BIG (#2640)#2718
Merged
rysweet merged 1 commit intoJul 6, 2026
Conversation
…ring E2BIG (#2640) The "Argument list too long" (E2BIG / os error 7) spawn failure kept recurring across Simard's agent launch sites even after two piecemeal fixes (#2660 decision-cycle, #2700 journal) and resurfaced on the SIGNAL channel, because there was no single chokepoint every large-payload launch was forced through. This introduces `simard::spawn_payload` — the single facade that composes the two already-shipping transports under one policy (payload >= 8 KiB never rides argv/envp): copilot prompts on stdin (via prompt_delivery), recipe context on a file path (via recipe_context_file::ContextFile), and pre-exec spawn errors classified into overseer::failure_sink. Copilot prompt sites routed through the facade's stdin transport: - base_type_copilot meeting turn (and the SIGNAL channel it backs) - engineer subprocess `run_engineer_subprocess` (Copilot) — was inlining the whole prompt as `-p <prompt>`; now the argv is prompt-less (`copilot --subprocess-safe --allow-all-tools --allow-all-paths`) and the prompt streams on stdin from a feeder thread. #1717 permission contract (both flags + COPILOT_ALLOW_ALL=1) preserved. Anti-regression: `tests/e2big_argv_guard.rs` fails CI if a `$(cat` argv expansion, an inline unbounded recipe key, or removal of the facade is introduced. Per-path hermetic > 256 KiB transport tests for meeting, signal, engineer (incl. a no-PTY stdin test), overseer-launch and self-improve. Recipe `-c` sites whose assets are EXTERNAL (smart-orchestrator, simard-self-improve-cycle, simard-engineer-loop) stay bounded-inline (sanitize 8000 — already E2BIG-safe, 8000 chars << ARG_MAX): those recipes read `{{key}}` inline with no `{{key_path}}` support, so filing a large value would leave the agent with an empty var. The facade's `recipe_context` file channel is ready for them once the external assets gain a `_path` read. Docs: concept, large-payload spawn API reference, and add-a-safe-spawn-site how-to, with a whole-repo launch-site audit table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📊 Coverage Summary
Coverage data from CI run. Test files matching |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
"Argument list too long" (E2BIG /
os error 7) kept recurring across Simard's agent launch sites even after two piecemeal fixes (#2660 decision-cycle, #2700 journal) and resurfaced on the SIGNAL channel — because there was no single chokepoint every large-payload launch was forced through. The root pattern: a large prompt/context inlined intoargv(or expanded via-p "$(cat FILE)") overflowsARG_MAXandexecvefails withE2BIGbefore the child runs.Fix: one spawn facade
New
simard::spawn_payload— the single chokepoint that composes the two already-shipping transports under one policy (a payload ≥ARGV_PAYLOAD_MAX_BYTES= 8 KiB never ridesargv/envp):attach_prompt_std/attach_prompt_tokio(forcesPromptDelivery::Stdin; copilot reads its prompt from stdin when no-pis given, and its--positional would be misparsed as a subcommand — so a prompt of any size stays off argv).recipe_context→-c <key>_path=<abs>(composesrecipe_context_file::ContextFile).record_spawn_failure→ errno-keyedclassify_spawn_failureintooverseer::failure_sink(no silent swallow).Copilot prompt sites routed through the facade (the real recurring failure)
base_type_copilotmeeting turn (backs the SIGNAL channel)apply_std(Stdin)attach_prompt_stdengineer_loop::run_engineer_subprocess(Copilot)-p <prompt>in argv,stdin(null)copilot --subprocess-safe --allow-all-tools --allow-all-paths) + prompt streamed on stdin from a feeder threadThe #1717 permission contract (both flags in order +
COPILOT_ALLOW_ALL=1) is preserved; the RustyClawd path is untouched (not production-wired).Anti-regression guard
tests/e2big_argv_guard.rsfails CI if a$(catargv-expansion, an inline unbounded recipe key (day_context/draft/episodes/pr_body), or removal of thespawn_payloadfacade is introduced.Whole-repo launch-site audit
base_type_copilotmeeting/signalengineer_loop/agent_spawn.rscopilotooda_actions/session.rscat 'PATH' | amplihack copilot, #2660)<key>_path— in-repo recipes (#2692/#2700)task_description, self-improveproposal, engineer-loopobjectivesanitize(…,8000)(E2BIG-safe: 8000 chars ≪ ARG_MAX). File channel blocked on external recipe_pathsupport —smart-orchestrator.yaml/simard-self-improve-cycle.yaml/simard-engineer-loop.yamlare amplihack-bundle assets (not this repo) that read{{key}}inline; filing a large value would leave the agent with an empty var.recipe_contextis ready for them once those assets gain a_pathread.target_repo,pr_number,state_root, …)Tests (hermetic, > 256 KiB payloads)
tests/spawn_payload_facade.rs,e2big_argv_guard.rs, and per-path transport tests for meeting, signal, engineer (+ a no-PTY stdin round-trip), overseer-launch, self-improve. The engineer permissions test now verifies the prompt lands on stdin (not argv) while #1717 flags are preserved.Local verification: full suite green (
cargo test --tests),cargo fmt --check,cargo clippy --all-targets --all-features --locked -D warnings,mkdocs build --strict.Docs
docs/concepts/e2big-elimination.mddocs/reference/large-payload-spawn-api.mddocs/howto/add-a-safe-agent-spawn-site.mdBuilds on #2660 and #2700 without regressing them.
Closes #2640
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com