feat(providers): serve sandbox config files on demand

[drew]

Summary

Serve non-secret provider configuration files to sandbox workloads on demand. The profile files API is experimental and may change or be removed. The sandbox intercepts read-only opens for managed absolute paths and injects sealed, memory-backed file descriptors; provider updates replace the snapshot seen by later opens.

Related Issue

Closes #3830

Changes

• Add declarative files templates to provider profiles, rendered only from config.KEY values, with bounded sizes and safe file names. Mark the public contract and SDK types as experimental.
• Carry revisioned file snapshots through the gateway, supervisor, and sandbox boundary. Older supervisors ignore the additive file field and still receive their provider credentials; file path variables may point to unavailable files until the supervisor is upgraded.
• Serve managed paths through seccomp open/openat/openat2 interception. Updates affect later opens, while existing descriptors keep their original content; detach returns ENOENT.
• Add the profile fixture in examples/provider-managed-files/, Go SDK profile conversion, and a Docker e2e scenario. Published feature docs are deferred while this interface is under review.

Example

Save this profile as acme-config.yaml:

id: acme-config
display_name: Acme client configuration
description: Non-secret settings installed as a managed sandbox file
category: other
files:
  - path: client.toml
    env_var: ACME_CONFIG_FILE
    content: |
      endpoint = "{{config.endpoint}}"
      project = "{{config.project}}"

With a running gateway, import it and attach a provider:

openshell profile lint -f acme-config.yaml
openshell profile import -f acme-config.yaml

openshell provider create \
  --name acme-prod \
  --type acme-config \
  --config endpoint=https://api.acme.example \
  --config project=production

openshell sandbox create \
  --name acme-demo \
  --from ubuntu:24.04 \
  --provider acme-prod \
  --no-tty \
  --detach \
  -- sleep infinity

openshell sandbox exec -n acme-demo -- cat /run/openshell/providers/acme-prod/client.toml
openshell sandbox exec -n acme-demo -- printenv ACME_CONFIG_FILE

Update the provider and reopen the path to see the new content; detach removes it for later opens:

openshell provider update acme-prod --config project=staging --wait
openshell sandbox exec -n acme-demo -- cat /run/openshell/providers/acme-prod/client.toml
openshell sandbox provider detach acme-demo acme-prod --wait

This open-only version supports direct absolute-path reads. Directory listing, metadata lookup, and inotify do not expose the virtual files.

Testing

• mise run pre-commit passes
• mise run test — provider-file compatibility and public schema inventory tests passed. The concurrent openshell-supervisor-network MCP forwarding test failed while expecting HTTP 200; it passed when rerun alone.
• OPENSHELL_E2E_DOCKER_TEST=provider_files mise run e2e:rust (shared conformance and provider-file scenario)
• Ran the CLI example in examples/provider-managed-files/ against an ephemeral Docker gateway: import, create, read, update, reopen, detach, and confirmed ENOENT after detach.
• Unit tests added/updated
• E2E tests added/updated
• mise run go:proto:gen and mise run go:docs:check
• mise run go:ci — unrelated gateway tests assume no system gateway, but this host has /etc/openshell/gateways/default; converter tests and other Go checks passed.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

Label test:e2e applied for 246e09d. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-3832.docs.buildwithfern.com/openshell

[mrunalp]

Potential concerns

• Older supervisors can start without an attached provider’s credentials in the local-policy path. The new gateway rejects the entire provider response (https://github.com/NVIDIA/
OpenShell/blob/b195e7ac1735dedf3146b94f9efbeb9c2cb6cbae/crates/openshell-server/src/grpc/policy.rs#L3353) when a supervisor lacks file support. In the local-policy startup path, a failed
fetch falls back to an empty provider environment and continues (https://github.com/NVIDIA/OpenShell/blob/b195e7ac1735dedf3146b94f9efbeb9c2cb6cbae/crates/openshell-supervisor/src/
lib.rs#L762). Before this PR, that supervisor could receive the provider’s existing credentials; once its profile declares a file, the workload can start without either the credentials
or the file. The capability mismatch needs an explicit startup failure on this path.

• The checked-in example does not meet issue #3830’s (#3830) documentation criterion. examples/provider-managed-files/ contains only the YAML
profile (

OpenShell/examples/provider-managed-files/acme-config.yaml

Line 4 in b195e7a

id: acme-config

). The CLI workflow and open-only limits
appear in the PR description, but not in a runnable example README or the provider profile docs. The related openshell-cli skill also has no companion update. An operator using the
repository or published docs cannot discover the complete workflow and its metadata/inotify limits.

Design and verification

The core choice is a complete file-map replacement (

OpenShell/crates/openshell-sandbox/src/provider_files.rs

Line 53 in b195e7a

pub(crate) fn replace(&self, desired: HashMap<String, String>) -> io::Result<()> {

):
an open takes content from one snapshot, so an existing descriptor keeps its bytes while later opens see the update. The PR also probes the sandbox boundary before sending a file-bearing
snapshot.

Rust and Go branch checks and Docker E2E pass. Podman E2E fails (https://github.com/NVIDIA/OpenShell/actions/runs/36625502817/job/109606066878) in the existing sandbox stop/start scenario:
the restarted sandbox remains in Error. The log does not establish whether this PR caused that failure, so it needs investigation before merge.

[mrunalp]

Non-blocking / informational. This provides a useful subset of Kubernetes ConfigMap volume behavior for clients that open a known absolute path. The --wait acknowledgement and reopen semantics are particularly useful.

Capability	Kubernetes ConfigMap volume	PR 3832
File access	A read-only directory with normal metadata and listing.	Known absolute paths served through open, openat, and openat2; stat, access, listing, and inotify do not expose the virtual files.
Updates	Mounted files update eventually; environment variables require a restart, and subPath mounts do not update. ConfigMap docs	--wait confirms publication; later opens get new bytes while existing descriptors retain their content.
Layout	Select keys, map them to paths, and set file modes. Projected volumes	One safe filename per entry under /run/openshell/providers/<provider>/, with an optional environment variable containing its path.
Data	UTF-8 data and binary binaryData, up to 1 MiB per ConfigMap. ConfigMap docs	UTF-8 templates rendered from config.KEY, capped at 64 KiB per file and 256 KiB per sandbox.

The most consequential gap for native clients is filesystem compatibility: a client that checks existence before opening, scans the directory, or watches for updates will not observe the virtual file. Kubernetes' atomic writer exposes updates through its ..data symlink; applications need to watch that change rather than assume a watch on the individual file will persist. Calling out the open-only boundary in the checked-in example and testing a representative native client would help users judge compatibility.

If broader compatibility is needed later, a read-only projected volume mode could provide metadata, listing, and watch behavior. Smaller possible additions are explicit escaping for template values used in TOML or JSON, binary content, and mapped or nested paths. These are future-scope suggestions, not merge blockers.

[drew]

On the older-supervisor concern: commit dc36e06 removes the supports_provider_files request field and the gateway's whole-response rejection. The files response field is additive, so older supervisors ignore it while continuing to receive their existing provider environment and credentials. This avoids the local-policy startup path treating a file capability mismatch as an empty credential snapshot.

The tradeoff is that an older supervisor cannot serve managed files, even though a profile's env_var may still point to the virtual path. File-dependent workloads therefore need an upgraded supervisor. We chose to preserve existing provider behavior during a rolling upgrade rather than fail startup for older supervisors. The new gateway regression test covers a response containing both credentials and a managed file.