Idea: run agents inside Azure Container Apps sandboxes via a remote `aca` provider (Agent-in-Sandbox)

[jrob5756]

Summary

Add an experimental aca provider that runs an agent's entire runtime — the
agentic loop and its file/shell tool execution — inside an Azure Container Apps
dynamic session (custom‑container session pool), Hyper‑V isolated off the host, while
LLM inference still goes to the remote Copilot/Anthropic API. Conductor stays on the host
as the orchestrator (routing, context, checkpoints, event bus) and delegates each
execute() to an in‑container "agent runner" it drives over HTTP.

This is the "Agent‑in‑Sandbox" pattern (Pattern 1 in the taxonomy below). It emerged
from a design exploration of how Conductor could use ACA sandboxes; the two lighter
alternatives (sandbox as a type: code/script backend, and sandbox‑as‑MCP‑tool) are
captured under Alternatives considered and are worth separate issues.

Status: idea — speculative, not yet committed. This issue captures the research so we
can decide whether/when to build it and in what order.

Motivation

Today every side effect an agent produces runs on the host, as the user:

• The Copilot/Claude SDK executes its built‑in file/shell tools locally against
  os.getcwd() (src/conductor/providers/copilot.py:803 sets
  working_directory=os.getcwd(); the Claude Agent SDK spawns the claude CLI with the
  process cwd and inherited os.environ).
• type: script steps run via local asyncio.create_subprocess_exec
  (src/conductor/executor/script.py), and command: is Jinja2‑templated — so a
  command: "{{ planner.output.cmd }}" runs LLM‑generated shell on your box.

For workflows that run untrusted or model‑generated code, coding agents that clone → edit →
run → commit, or multi‑tenant scenarios, we want off‑host, isolated, ephemeral
execution. ACA dynamic sessions are purpose‑built for this — Microsoft explicitly names
"AI agents" and "development environments" running user‑provided code in Hyper‑V isolated
(optionally network‑isolated) sandboxes as a target use case, and ships an official sample
(Azure‑Samples/dynamic-sessions-custom-container,
Microsoft Agent Framework).

Background: what the research established

1. The SDKs fuse the loop and its tools (this constrains the design)

Both providers run the agentic loop client‑side (a CLI subprocess), with remote
inference but all tool execution local, in that subprocess, against cwd. Neither SDK
exposes a seam to relocate just the tools, and neither has a native HTTP server mode.

Consequence:

Approach	Reachable with Conductor's SDKs?
Loop on host / tools in sandbox (what Anthropic Cowork + the official Azure sample do)	❌ Not with built‑in SDK tools — loop+tools are one unit. Only via MCP tools that execute in the sandbox (= Alternative B / Pattern 2).
Whole SDK in the sandbox (loop and tools inside)	✅ Yes — this is the proposal here (Pattern 1).

Both SDKs are cleanly containerizable: bake the CLI into the image, inject auth via env
var (GITHUB_TOKEN / GH_TOKEN for Copilot; ANTHROPIC_API_KEY / CLAUDE_CODE_OAUTH_TOKEN
for Claude), and wrap provider.execute() in a thin FastAPI/aiohttp server. Copilot's SDK
also supports a TCP/URI transport (RuntimeConnection.for_tcp/for_uri) for a sidecar CLI.

2. ACA custom‑container sessions fit the agent‑runtime model — with caveats

Confirmed against MS Learn (sources at bottom):

• One session = a long‑lived HTTP server serving many sequential requests keyed by a
  free‑form identifier (4–128 chars); file/process/memory state persists for the
  session's lifetime. Requests to <POOL_ENDPOINT>/<path>?identifier=<id> are forwarded to
  the container's <TARGET_PORT>/<path>; /.management/* is reserved.
• identifier = the isolation/persistence knob: existing id → routed to that session;
  new id → session auto‑allocated. Maps directly onto per‑run / per‑agent / per‑for‑each‑item
  scoping.
• Lifecycle: Timed (cooldown 300–3600 s, reset by every request — so tool‑call traffic
  keeps a session alive) or OnContainerExit + maxAlivePeriodInSeconds ("run until
  done / hard TTL").
• EgressEnabled (off by default) lets the in‑container agent reach the Copilot/Anthropic
  API. Prewarming via readySessionInstances gives subsecond allocation. Caller auth is the
  Azure ContainerApps Session Executor role (token audience https://dynamicsessions.io,
  DefaultAzureCredential).

Caveats / risks (see dedicated sections): undocumented per‑request timeout; per‑session
secret injection unsupported; no native egress allowlist; no GPU; no volume mount; Dedicated
E16 billing.

3. Prior art: the leaders run the loop outside, and credentials are the crux

• LangChain names the exact axis — Pattern 1 "Agent‑in‑Sandbox" (loop inside) vs
  Pattern 2 "Sandbox‑as‑Tool" (loop outside, sandbox called for execution).
• Anthropic Cowork originally ran the whole loop inside the VM, then deliberately moved the
  loop out (keeping only code execution inside) because "any VM failure made Cowork
  unusable." The official Azure sample is also loop‑outside / exec‑inside. Two independent
  sophisticated sources chose the hybrid.
• Credential consensus: the sandbox should hold no long‑lived credentials; an egress
  proxy holds the token outside and injects a scoped one (Cloudflare Outbound Workers,
  Anthropic Cowork), or OIDC/workload‑identity token exchange. Baking tokens into images is
  the named anti‑pattern. Nuno Campos: "no part of your agent can have more privileges than
  the bash tool does." Anthropic observed real exfiltration through an allow‑listed
  domain.

Proposed architecture — a remote AgentProvider

┌───────────────── Conductor host (orchestrator) ─────────────────┐
│  WorkflowEngine: routing · context · checkpoints · event bus     │
│  AcaRuntimeProvider(AgentProvider)   ← new, experimental tier     │
│    execute(agent, ctx, prompt, tools, event_callback):           │
│      id = identifier_for(scope)          # run | agent | item     │
│      POST {pool}/execute?identifier=id   # Bearer (Session Exec)  │
│      stream NDJSON events → event_callback   # dashboard / JSONL  │
│      return AgentOutput                                            │
└───────────────────────────┬──────────────────────────────────────┘
                            │ HTTPS · aud=https://dynamicsessions.io
              ┌─────────────▼──────────────┐  auto-allocate/reuse by identifier
              │  ACA custom-container pool  │
              │   session (Hyper-V isolated)│
              │  ┌───────────────────────┐  │
              │  │ conductor-agent-runner│  │  ← HTTP server (baked into image)
              │  │  wraps CopilotProvider│  │
              │  │  SDK loop + CLI tools │──┼─▶ edits/exec on CONTAINER fs
              │  │  egress proxy ────────┼──┼─▶ Copilot/Anthropic API
              │  └───────────────────────┘  │
              └─────────────────────────────┘

• Host stays the harness — engine, routing, WorkflowContext, checkpoints, and the event
  pub/sub remain on the host (exactly the shape Anthropic retreated to). Only per‑agent
  execute() is delegated.
• New provider slots into existing seams: register in
  src/conductor/providers/factory.py (create_provider match arm), declare a class‑level
  CAPABILITIES: ProviderCapabilities (src/conductor/providers/capabilities.py), and map
  the runner's streamed events to the same agent_message / agent_tool_* / agent_reasoning
  callbacks so the dashboard, JSONL logger, and console subscriber render identically.
• In‑container conductor-agent-runner wraps the real CopilotProvider/ClaudeProvider
  (the SDK + CLI), baked into the image. Its HTTP API is async/streaming (see Transport).

Config shape (sketch)

Reuse the structured runtime.provider object (src/conductor/config/schema.py
ProviderSettings, RuntimeConfig), which already gives us SecretStr redaction and
env‑var fallback:

runtime:
  provider:
    name: aca                                  # new provider type
    pool_endpoint: ${ACA_SESSION_POOL_ENDPOINT}
    api_version: "2025-07-01"
    inner_provider: copilot                    # SDK to run *inside* the sandbox
    identifier_scope: agent                    # run | agent | item | step
    egress: enabled                            # maps to sessionNetworkConfiguration
    lifecycle: timed                           # timed | on_container_exit
    # credentials: see next section — NOT baked into the pool
    auth: azure_default                         # DefaultAzureCredential for the Session Executor role

agents:
  - name: coder
    # inherits the aca provider; optionally override the isolation scope per agent
    sandbox:
      identifier_scope: run                    # this agent shares the run-wide workspace
    prompt: "Implement {{ workflow.input.task }} and run the tests."

The credential boundary (the crux)

To run the SDK inside the session, the model API key must be inside the session — where the
model‑driven shell can read it. ACA makes this harder than best‑in‑class: per‑session
secret injection is unsupported (pool‑level secrets are identical for every session),
everything in a session is readable by the session's own code, and there is no native
per‑destination egress allowlist.

Strategy	Exposure	Verdict
Bake token as pool secret / env	Whole pool, indefinitely	❌ anti‑pattern
Short‑lived scoped token per request (in body, not a baked secret)	Only during that agent's run, still readable by its shell	⚠️ acceptable stopgap
Route the in‑sandbox SDK's inference through a Conductor‑hosted gateway that injects the real upstream key; sandbox holds only a short‑lived gateway token	Real key never enters the sandbox	✅ recommended — reuses Conductor's existing Copilot custom routing (COPILOT_PROVIDER_BASE_URL / COPILOT_PROVIDER_BEARER_TOKEN)

Recommendation: make the egress gateway the sandbox's only route out (the runner points
its SDK at the gateway via custom routing), so the credential boundary lives on the host.
This directly reuses machinery Conductor already has. Anthropic's caution applies — "the
weakest layer is the one you built yourself" — so the proxy must be minimal and the image
must enforce "proxy is the only egress" (ACA won't filter egress for us).

Transport constraints (must‑validate before building)

MS publishes no max forwarded‑request duration; general ACA ingress idle‑timeout defaults
to 4 min (premium max 30 min). An agent step runs for minutes. Therefore the runner API
cannot be one blocking request per turn — it must be async submit → stream NDJSON
events → poll/stream final result, each chunk resetting the Timed cooldown. This maps
onto Conductor's existing event stream and the resumable‑SSE prior art (AG‑UI;
Last-Event-ID + replayable event log). Platform‑managed MCP over streamable‑HTTP is a
strong signal chunked streaming survives the endpoint, but this is the #1 item to prove
empirically (Phase 0 spike).

Capabilities / carve‑outs (experimental tier)

Per docs/providers/experimental.md, declared ProviderCapabilities:

Capability	Value	Rationale
mcp_tools, workflow_tools_passthrough	✅ True	the in‑sandbox Copilot SDK supports MCP + tools natively — higher parity than claude-agent-sdk
concurrent_safe	✅ True	sessions isolated by identifier; parallel/for‑each get distinct ids
streaming_events	⚠️ True iff chunked streaming passes the endpoint (validate)
max_session_seconds	✅ via maxAlivePeriodInSeconds
interrupt	⚠️ hard‑abort via stopSession; mid‑call partial depends on streaming design
checkpoint_resume	❌ False	sessions ephemeral, no volume mount — in‑sandbox session state dies on cooldown; resume re‑runs the agent

Cost, file staging, provisioning

• Cost is a separate dimension from tokens: custom‑container pools run on Dedicated E16
  nodes and the warm pool (readySessionInstances) bills even when idle. Right‑size warm
  capacity and pack sessions with modest --cpu/--memory. Surface sandbox time as its own
  usage row (don't fold into token cost).
• File staging: the sandbox FS is the workspace and is ephemeral (no documented
  volume mount). Seed inputs (e.g. git clone) at session start; push artifacts out (git
  push / blob) before cooldown. Natural unit: one session = one agent's (or one run's)
  workspace.
• Provisioning: BYO‑pool first (user runs az containerapp sessionpool create --container-type CustomContainer …, passes the endpoint) — matches the BYO‑endpoint
  philosophy of custom routing. Two‑step deploy (build/push image to ACR, then create pool),
  per the official sample.

Alternatives considered

• A — Sandbox as a type: code/script backend. Route type: script (and a new
  type: code) through the sandbox instead of local subprocess. Smallest blast radius,
  reuses the cleanest seam (ScriptExecutor), orthogonal to the LLM provider. Doesn't isolate
  the agent's built‑in tool calls. (Separate issue.)
• B — Sandbox‑as‑MCP‑tool (Pattern 2). Ship an MCP server (fits runtime.mcp_servers)
  exposing run_code/run_command/file ops that execute in the sandbox; agents opt in via
  tools:. Zero engine change; keeps credentials on the host (safer). This is the hybrid the
  leaders chose, and the only way to get "loop outside / exec inside" with Conductor's SDKs.
  Trades away "all built‑in tools isolated." (Separate issue — arguably do this first.)
• D — Whole‑workflow isolation. Run conductor run itself inside a session/Job. Coarse;
  a hosting story, not an integration.

Framing: if the goal is max isolation of every built‑in tool side effect off‑host, this
proposal (C) delivers it. If the goal is safe containment of untrusted execution, B is
architecturally safer and simpler — recommend sequencing B before C.

Phased plan / de‑risking

• Phase 0 — transport spike (blocking). Stand up an ACA custom‑container pool; run a
  container that streams NDJSON for 10+ minutes through the pool endpoint. Confirm the real
  max request/idle duration and that chunked streaming survives. Everything else is buildable;
  this is the true unknown.
• Phase 1 — runner image + provider. conductor-agent-runner (CLI + FastAPI /execute
  wrapping CopilotProvider.execute()), auth via GITHUB_TOKEN; host‑side AcaRuntimeProvider
  with identifier‑scope + event remap; ship as experimental behind an extra.
• Phase 2 — credential gateway. Route inference through a host gateway; prove the sandbox
  never holds the real key (attempt to read it from a tool call).
• Phase 3 — file staging + cost surfacing + example workflow.

Open questions

• Runner API: NDJSON stream vs submit+poll vs both? How do we resume a dropped stream
  mid‑agent (Last‑Event‑ID + server event log)?
• Identifier scoping defaults — per‑agent vs per‑run? How do we name ids to stay parallel‑safe
  and within 4–128 chars / allowed charset?
• Do we own the pool image (publish conductor-agent-runner) or document a contract and let
  users bring their own?
• Egress: accept open egress + a self‑hosted proxy, or require a VNet + NSG for allowlisting?
• How does this interact with conductor resume? (Declared checkpoint_resume=False, but can
  we persist the workspace to blob to make resume meaningful?)
• Cost model surfacing in the CLI summary / dashboard (session‑seconds as a distinct row).
• Should the same remote‑runner mechanism generalize to non‑ACA sandboxes (E2B/Modal/Daytona)
  behind one interface?

Acceptance criteria

Phase 0 (spike):

• Documented empirical max request/idle duration for the sessions endpoint + whether
  chunked HTTP streaming passes through.
• Decision record: streaming vs polling runner API.

Phase 1 (MVP):

• runtime.provider: { name: aca, ... } accepted by the schema (validator rejects
  incomplete combos, mirroring the existing ProviderSettings guardrails).
• AcaRuntimeProvider implements the AgentProvider lifecycle and declares accurate
  ProviderCapabilities (experimental tier, checkpoint_resume=False).
• conductor-agent-runner image drives CopilotProvider inside a session and streams
  events back; host remaps them to standard Conductor events.
• Model API credential never present inside the sandbox (verified).
• Smoke test against a mocked runner (construct + execute), per experimental‑provider
  rules.
• Experimental banner + docs page; one runnable examples/ workflow.

References

ACA (MS Learn):
session‑pool ·
custom‑container sessions ·
sessions usage (identifiers/auth/egress/MI) ·
dynamic sessions concepts/regions ·
billing ·
sessionpool CLI ·
ARM template 2025‑07‑01 ·
premium ingress (timeout context) ·
official sample

Prior art:
LangChain — two patterns ·
Cloudflare — Sandbox auth (Outbound Workers) ·
Anthropic "How We Contain Claude" (analysis) ·
Claude Code sandbox environments ·
GitHub Copilot coding‑agent firewall ·
AG‑UI protocol

Conductor seams:
providers/copilot.py:803 (working_directory=os.getcwd()) ·
providers/factory.py (create_provider) ·
providers/capabilities.py (ProviderCapabilities) ·
config/schema.py (ProviderSettings / RuntimeConfig) ·
docs/providers/experimental.md ·
executor/script.py (ScriptExecutor)

[jrob5756]

Picking this back up: the blocking Phase 0 transport spike (#312) is resolved — Branch S (streaming) confirmed viable, results at #312 (comment). Committing the solution design doc and folding the spike outcome into DD3 + the capabilities table next.