diff --git a/sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx b/sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx index 1045907240..9e8bfa7c86 100644 --- a/sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx +++ b/sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx @@ -187,4 +187,6 @@ agent: - **args**: a fixed argument list. No shell or option substitution is applied. - **imageBackend**: `docker` (default) or `none`. It picks the Devsy-side image backend, not a build capability in the plugin. With `none`, prebuild publishing is unavailable. -Each platform needs a SHA-256 checksum, and the runtime must already be in the agent binary directory. Devsy verifies the checksum before every operation and never downloads or removes the file. The plugin is trusted provider code and runs with the same access as Devsy. +Every platform declaration for the selected `agent.external.binary` key requires a SHA-256 checksum. Agent setup prepares the executable; the external host verifies its bytes before launching each operation and does not download or remove it. The selected path can be relative to the prepared agent-binaries directory or an absolute local path. It must resolve to a regular file, with effective execute access on Unix. This verification applies to the selected external runtime, not every arbitrary provider installation binary. The supervisor is Devsy's own executable. The plugin is trusted provider code and runs with the agent user's access. + +See the [runtime rollout decision](./external-runtime-rollout.mdx) for MicroSandbox migration and compatibility limits. diff --git a/sites/docs-devsy-sh/content/docs/developing-providers/external-runtime-rollout.mdx b/sites/docs-devsy-sh/content/docs/developing-providers/external-runtime-rollout.mdx new file mode 100644 index 0000000000..e8944f14e5 --- /dev/null +++ b/sites/docs-devsy-sh/content/docs/developing-providers/external-runtime-rollout.mdx @@ -0,0 +1,37 @@ +--- +title: External MicroSandbox Runtime Rollout Decision +sidebar_label: Runtime Rollout Decision +description: Evidence and conditions for a future fresh-install source change to the external MicroSandbox provider. +--- + +**Status: proposed and gated.** The external provider is validated for the covered Linux/KVM scenarios, but the fresh-install choice has not changed. Existing internal providers continue to use the embedded MicroSandbox driver. A reviewed Desktop preset change can select the published source for new installations after these documented compatibility and evidence boundaries are accepted. + +## Current host and runtime contract + +Devsy starts a supervised provider process for each runtime operation. It selects the executable from the selected `Agent.Binaries` entry, requires its SHA-256 declaration, verifies the resolved file and its bytes, then launches it through the Devsy runtime supervisor. The SDK transports protocol messages; it does not download or establish trust in the provider executable. The published provider is trusted host code and runs with the local agent user's authority. + +The external provider inherits the host process environment, then receives Devsy's resolved provider/workspace values; this path supplies no separate extra-environment map. This preserves existing provider behavior for settings such as `PATH`, `HOME`, proxy configuration, Docker credentials, and temporary directories. The current evidence does not justify an environment allowlist. Provider installation downloads and runtime image downloads are separate from the host's verification of the selected `Agent.Binaries` executable. + +Runtime Protocol v1 uses Info API 1.3 to negotiate recreation-on-run support. Negotiation does not grant permission: Devsy forwards the current operation's explicit recreation permission, and the runtime validates effective configuration before replacing a workspace. This is replacement, not in-place reprovisioning. Failures after replacement starts do not have a general rollback guarantee. + +## Verified behavior + +PR #1483 established the Linux/KVM parity foundation and is merged. The source-migration acceptance is [PR #1487](https://github.com/devsy-org/devsy/pull/1487), tested head [`0aeea936`](https://github.com/devsy-org/devsy/commit/0aeea936aba901a4d2a1622458cda1dfb9ac0cdf), implementation run [38628263920](https://github.com/devsy-org/devsy/actions/runs/38628263920). Its exact-head CI completed 88 jobs successfully. The Linux/KVM native catalogue passed 80 cases once: 37 built-in and 43 external. It includes three external-only registry-consumer cases, one remote-agent delivery case, and two existing-workspace source-migration cases. The change is merged at [`e4286137`](https://github.com/devsy-org/devsy/commit/e42861373711f28aea7262baa5561a5e97f5a17a) after completed Greptile and full CodeRabbit review, including supplemental security/architecture analysis. + +The registry cases exercise the published provider against an owned TLS registry and credential helper. They cover successful authenticated access, missing CA refusal, and missing credentials refusal. The remote-agent case checks provider runtime download and workspace-copy reuse when its runtime endpoint returns 404. Its separate Devsy-agent endpoint stays available; this is not an offline-agent guarantee. The source migration cases use the actual `provider set-source` command on an existing provider named `microsandbox`, once while its workspace is running and once while stopped. They check provider source and options, workspace association, VM generation, and guest root, volume, and host-bind markers. The running VM keeps its boot ID. The stopped VM remains stopped until ordinary `up`, then resumes with preserved markers. + +The separate Windows provider-update job exercises Go archive and provider-update tests; it is not evidence of a MicroSandbox VM running on Windows. Native provider/VM evidence is Linux/amd64 with KVM only. + +## Proposed fresh-install change + +Change only the Desktop Microsandbox preset to install the pinned source [`github.com/devsy-org/devsy-provider-microsandbox@v0.1.8`](https://github.com/devsy-org/devsy-provider-microsandbox/releases/tag/v0.1.8), while retaining the provider name `microsandbox`. The existing Desktop setup path uses `provider add ... --use=false`; this preserves its current default-provider behavior. The explicit embedded source alias `microsandbox` remains available, as does the embedded manifest for parity tests and compatibility with existing internal provider records. + +This proposal is fresh-install only. Existing records change source only through the supported `provider set-source` command. Changing the embedded manifest does not migrate those records. If initialization fails after a source change is committed, the new source remains selected and the provider records `InitError`; the command does not promise rollback. The running and stopped migration cases provide evidence for the tested successful transition, not for rollback after arbitrary failures. + +## Limits and decision conditions + +The performance artifact from run 38628263920 records one Linux/amd64 hosted run with SDK v1.6.0 and provider v0.1.8. It observed fresh and reused `up` wall times of 13.504s and 6.335s. Four direct/supervised/supervised/direct probe runs collected one first sample and 100 warm `Info` and `Find` samples each; all 808 launches were reaped. The artifact marks `cold_cache_measured=false`. These are observations, not thresholds, cold-start results, or evidence that supervision improves performance. No process pool is proposed. + +Other provider-supported host profiles remain untested here. Upstream requires Apple Silicon on macOS, KVM on Linux, and WHP on Windows; this campaign does not claim runtime parity on those other hosts. The registry tests do not cover every proxy, Docker daemon/context/socket, credential-store, or temp-directory configuration. Neither the built-in driver nor the external provider guarantees removal of every effect after a failed or interrupted create; use a fresh `Find` and explicit cleanup where recovery is needed. The exact native VM artifacts surviving a particular interruption remain unproven. + +The source-migration review and merge gates are complete. A fresh-install preset change remains a separate reviewed implementation and must preserve these platform, performance, environment, and recovery limits. Keep the embedded provider and existing records intact until the explicit migration path and compatibility window are accepted. diff --git a/sites/docs-devsy-sh/content/docs/developing-providers/meta.json b/sites/docs-devsy-sh/content/docs/developing-providers/meta.json index 425287ff60..9d81d7db7d 100644 --- a/sites/docs-devsy-sh/content/docs/developing-providers/meta.json +++ b/sites/docs-devsy-sh/content/docs/developing-providers/meta.json @@ -6,6 +6,7 @@ "binaries", "agent", "driver", - "runtime-protocol" + "runtime-protocol", + "external-runtime-rollout" ] } diff --git a/sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx b/sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx index a479fed6d8..59379d6cce 100644 --- a/sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx +++ b/sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx @@ -39,7 +39,7 @@ For MicroSandbox, explicit replacement requires v0.7.7 or newer even with worksp The first client frame is exactly one ExecStart containing argv. Later frames contain stdin bytes or exactly one CloseStdin; the client then closes its send side. Data after CloseStdin, repeated Start, unset payloads and empty stdin data frames, and unexpected EOF before CloseStdin are InvalidArgument. v1 Devsy callers use `tty=false`; runtimes reject unsupported TTY requests. -Each side uses one send pump and one receive loop. Data chunks should be at most 32 KiB. Empty input is represented by CloseStdin with no preceding data frames. Stdout/stderr are separate byte streams, with no text decoding or PTY. The receiver drains output before exactly one terminal ExecExit. Command success requires `exit_code == 0` and an empty `signal`. A nonempty `signal` means command failure regardless of `exit_code`, including its protobuf default of zero. Ordinary nonzero or signal-terminated command exit is carried in ExecExit and the RPC succeeds. Setup/transport/backend failures are RPC errors; stream EOF without an exit is not command success. Context cancellation/deadlines terminate the operation and release its resources. Plugins that launch children must ensure child cleanup; the SDK bootstrap does not implement an OS process-tree manager. +Each side uses one send pump and one receive loop. Data chunks should be at most 32 KiB. Empty input is represented by CloseStdin with no preceding data frames. Stdout/stderr are separate byte streams, with no text decoding or PTY. The receiver drains output before exactly one terminal ExecExit. Command success requires `exit_code == 0` and an empty `signal`. A nonempty `signal` means command failure regardless of `exit_code`, including its protobuf default of zero. Ordinary nonzero or signal-terminated command exit is carried in ExecExit and the RPC succeeds. Setup/transport/backend failures are RPC errors; stream EOF without an exit is not command success. Context cancellation/deadlines terminate the operation and release its resources. Plugins that launch children must ensure child cleanup; server bootstrap and host-supervised process ownership are separate responsibilities. Logs uses merged binary OutputChunk frames. Output buffering must remain bounded. Do not call Send concurrently from stdout and stderr copiers. @@ -47,7 +47,17 @@ Logs uses merged binary OutputChunk frames. Output buffering must remain bounded Use canonical gRPC status and attach RuntimeError details for stable categories, actionable messages, optional backend diagnostics, retryability, and structured context. Raw backend diagnostics must be redacted before display. Unknown detail fields remain forward-compatible; callers must not parse messages to classify errors. -The plugin binary is trusted provider code. The host resolves it from checksum-verified `Agent.Binaries`, rather than PATH discovery, and runs it through the runtime supervisor. The magic cookie is not a security boundary. +The plugin binary is trusted provider code. Devsy resolves the selected external runtime from its `Agent.Binaries` declaration and requires a SHA-256 checksum before launch, including cached and local absolute paths. The resolved executable must be a regular file; Unix hosts also check effective execute access. The runtime supervisor is the running Devsy executable's internal `runtime-supervisor` entry point, not another provider-distributed binary. The SDK transport runner does not download executables or establish checksum trust. A generic provider download or an `Agent.Binaries` path alone does not establish this trust, and the magic cookie is not authentication or sandboxing. + +## Plugin process environment + +Devsy inherits the agent host's environment, then appends resolved provider/workspace values. This external-driver path does not supply a separate extra-environment map. This is the environment of the trusted plugin and its runtime CLI children, separate from the guest environment carried in RunImage and ExecStart. The external driver preserves this behavior; it does not introduce an environment allowlist. + +`PATH` and home directories can select runtime CLIs and configuration. Proxy and CA settings, Docker configuration and credential helpers, and temporary-directory settings can affect the provider's image clients. Preserving those inputs does not prove every proxy topology, credential store or host profile works. The native consumer cases below cover their controlled registry/TLS/proxy/credential paths. Never log the full environment or secret-bearing runtime arguments; redact backend diagnostics before display. + +The host owns each supervised plugin session. The SDK server bootstrap does not itself own backend children; runtimes remain responsible for independently managed services and children outside the supervisor's process ownership. Cancellation does not promise rollback of backend effects that already committed. + +See the [rollout decision](./external-runtime-rollout.mdx) for the tested migration path, platform applicability and compatibility boundary. For SDK usage and development commands, see the [SDK README](https://github.com/devsy-org/devsy-runtime-sdk#readme).