Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,20 @@ title: Continuous Integration
sidebar_label: Continuous Integration
---

`devsy ci` starts a devcontainer workspace and runs a command in it. It deletes a newly created workspace afterward unless you pass `--keep`, but never deletes a workspace that existed before the run. Use it to run tests inside the container and check the build of a newly created workspace.
`devsy ci` starts a devcontainer workspace and runs a command in it. Use it to run tests inside the container and check the build of a newly created workspace.

<Callout type="warn">
If you omit the workspace argument and select an existing workspace interactively, Devsy can delete that workspace after the run, including its workspace data. Pass `--keep` when selecting a workspace interactively to prevent cleanup.
</Callout>

## How it works

1. Devsy resolves a workspace from the current directory or a given source.
1. Devsy resolves a workspace from the given source. With no source, it prompts for a workspace in an interactive terminal.
2. Devsy builds and starts the devcontainer like `devsy workspace up`, without IDE or SSH config changes.
3. Devsy runs the command inside the container.
4. Devsy deletes the workspace, even when the command fails, unless you pass `--keep`. Devsy never deletes a workspace that existed before the run.
4. Unless you pass `--keep`, Devsy attempts cleanup when its initial lookup finds no existing workspace, even when the command fails. This can delete an existing workspace selected interactively after that lookup.

The command's exit code becomes the exit code of `devsy ci`, so a failing test fails the CI job.
A failing command fails the CI job. Workspace setup or cleanup errors also cause `devsy ci` to fail.

## Usage

Expand All @@ -23,26 +27,26 @@ devsy ci [flags] [workspace-path|workspace-name] -- <cmd> [args...]
Run the test suite inside the devcontainer for the current project:

```sh
devsy ci -- make test
devsy ci . -- make test
```

Target a specific project folder and pass environment variables:

```sh
devsy ci ./service -- npm test
devsy ci --remote-env CI=true --remote-env NODE_ENV=test -- npm test
devsy ci --remote-env CI=true --remote-env NODE_ENV=test . -- npm test
```

Select a named config, such as a `.devcontainer/<name>` profile:

```sh
devsy ci --devcontainer id:ci -- make test
devsy ci --devcontainer id:ci . -- make test
```

Keep the workspace after the run, for debugging:

```sh
devsy ci --keep -- ./run-integration-tests.sh
devsy ci --keep . -- ./run-integration-tests.sh
```

### Common flags
Expand Down Expand Up @@ -87,7 +91,7 @@ structured result.

## Setup history after cleanup

`devsy ci` deletes the workspace, but Devsy keeps the build and lifecycle hook history in the local store. Read it by run ID after the job ends, or from an artifact step on the same machine:
Devsy keeps the build and lifecycle hook history in the local store after workspace cleanup. Read it by run ID after the job ends, or from an artifact step on the same machine:

```sh
devsy workspace logs --source setup --list
Expand All @@ -104,7 +108,7 @@ Desktop launches, `--detach`, JSON output, and management or platform tasks also

Pass values to the workspace in one of these ways:

- Environment variables: `devsy ci --workspace-env NAME=value -- make test`. Use `--workspace-env` for values that hooks need. The `--remote-env` flag applies only to the command that CI runs.
- Environment variables: `devsy ci --workspace-env NAME=value . -- make test`. Use `--workspace-env` for values that hooks need. The `--remote-env` flag applies only to the command that CI runs.
- Secrets: `--secret` injects a stored Devsy secret. See [Secrets](./secrets.mdx).
- Files: write the value to a file that the hook reads.
- Arguments: put a value that is not secret in the hook command, such as `"postStartCommand": ["./start.sh", "example-name"]`.
Expand Down Expand Up @@ -144,5 +148,5 @@ jobs:
- name: Run CI in devcontainer
run: |
devsy provider add docker && devsy provider use docker
devsy ci -- make test
devsy ci . -- make test
```
Original file line number Diff line number Diff line change
Expand Up @@ -176,17 +176,21 @@ This prints the configuration and live state. Add `--result-format json` for scr

Recreating applies changes to `devcontainer.json` or its Dockerfile. If a prebuild repository is configured, Devsy looks there for the new image first.

```sh
devsy workspace up my-workspace --recreate
```

<Callout type="warn">
Recreating keeps only the project folder and mounted volumes. Anything else in the container is lost.
</Callout>

```sh
devsy workspace up my-workspace --recreate
```

## Reset a workspace

A reset pulls the latest Git changes, or uploads your local folder again. For local-folder workspaces, Devsy removes and re-seeds its managed named volumes. Bind mounts and unmanaged volumes stay.
A reset recreates the container and prepares the source again. For a Git workspace, Devsy removes the existing project folder and clones the configured ref again. For a local-folder workspace, Devsy re-seeds its managed named volumes from the local folder. Bind mounts and unmanaged volumes stay.

<Callout type="warn">
Back up changes in the workspace before resetting. Removing a Git workspace's project folder deletes uncommitted work and other files in that folder. Resetting managed named volumes deletes their existing contents.
</Callout>

```sh
devsy workspace up my-workspace --reset
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,16 @@ To send HTTP headers when Devsy downloads a feature archive, set them under `cus

After you edit the configuration, apply it without deleting the workspace:

<Callout type="warn" title="Recreating">
Anything outside volumes is lost. The project folder and other mounted paths stay.
</Callout>

```sh
devsy workspace up my-workspace --recreate
```

Devsy builds and validates the new image before stopping or replacing the existing container. If the build fails, the existing workspace stays in its previous state, whether running or stopped. A later failure can leave the old container stopped or removed, depending on the provider. Pass `--recovery` to start a recovery container with features and lifecycle commands disabled if the build fails.

<Callout type="warn" title="Recreating">
Anything outside volumes is lost. The project folder and other mounted paths stay.
</Callout>

## postStartCommand and waitFor

`waitFor` names the lifecycle hook that `devsy workspace up` waits for before it returns. The default is `updateContentCommand`, so `postStartCommand` can finish after `up` returns.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ Defaults when the option is unset:

**Docker, Podman, Apple, and Microsandbox:** A daemon in the container watches for activity. For Microsandbox, Devsy also passes the timeout to the microsandbox CLI as `--idle-timeout`.

<Callout type="warn" title="Docker and Podman">
The detached container daemon exits when its timeout expires, but that exit does not stop a Docker or Podman container. Do not rely on `INACTIVITY_TIMEOUT` to stop these containers. Stop them explicitly with `devsy workspace stop`.
</Callout>

<Callout type="warn" title="Apple provider">
The Apple provider option describes the timeout as deleting the container instead of stopping it. Deleting a container loses files outside mounted volumes.
</Callout>
Expand All @@ -35,9 +39,9 @@ A workspace is active while either of these holds:
- A tracked `devsy workspace ssh` session is connected. Devsy records activity when the session connects, periodically while it stays open, and when it ends. The idle period starts at disconnect.
- Setup is running: the synchronous `devsy workspace up` setup, deferred lifecycle hooks, and `postAttachCommand`.

CPU use and processes you start yourself do not count. For example, `nohup make &` in a closed session does not keep the workspace running.
CPU use and processes you start yourself do not count as activity. For example, `nohup make &` in a closed session does not reset the idle timer.

The daemon checks idle time about once a second, so a workspace stops a few seconds after the timeout passes.
The daemon checks idle time about once a second. A managed Kubernetes pod ends a few seconds after the timeout passes.

On Kubernetes, SSH sessions and hooks register with the daemon before they do work and stay active while their connection lasts. If the daemon connection is lost, Devsy cancels the protected work. Failed activity-file writes or a full disk cannot hide live work.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,25 @@ title: Workspace Snapshots
sidebar_label: Workspace Snapshots
---

A snapshot saves a workspace at one point in time: the container filesystem (installed packages, config changes) and the contents of the bind-mounted volume, including uncommitted code. Devsy stores snapshots as OCI artifacts in a registry you control. Restore a snapshot later or use it to move a workspace to another provider.
A snapshot saves the container filesystem (installed packages and config changes) and the contents of a host bind-mounted workspace, including uncommitted code. Devsy stores snapshots as OCI artifacts in a registry you control. Restore a snapshot later or use it to move a workspace to another provider.

<Callout title="Registry required">
Devsy pushes snapshots to an OCI registry and pulls them from it when needed. Creating a snapshot also leaves the committed container image locally after the push, but the registry is still required for the snapshot manifest and volume. You need push and pull access.
</Callout>

## Limitations

- The workspace mount must be the only bind mount. Snapshot creation fails if `devcontainer.json` adds more bind mounts.
- The workspace must use a host bind mount, and it must be the only bind mount. Snapshot creation fails if `devcontainer.json` adds more bind mounts.
- Named workspace volumes are not reliably captured. Snapshot creation can fail or capture an unrelated host directory instead. Do not rely on a snapshot to back up a named workspace volume.
- Other named volumes are not included, even when the workspace bind mount is the only bind mount. Back up and restore their contents separately.
- Only local providers, such as Docker and Podman, can create snapshots. Machine-provider workspaces cannot.

## Create a snapshot

The workspace must exist and be running.

Stop applications that write to the container filesystem or workspace mount before creating the snapshot. Devsy captures them separately while the workspace is running, so changes made during capture can leave the snapshot inconsistent.

```sh
devsy snapshot create my-workspace --registry ghcr.io/my-org/snapshots
```
Expand All @@ -44,7 +48,7 @@ The output lists every snapshot of the workspace, newest first.
devsy snapshot restore <snapshot-ref>
```

Restore creates a workspace from the saved filesystem and volume. It skips the devcontainer build and reuses the original workspace ID. Pass `--workspace-id` to use another ID or `--target-provider` to restore onto another provider.
Restore creates a workspace from the saved container filesystem and workspace mount. It skips the devcontainer build and reuses the original workspace ID. Pass `--workspace-id` to use another ID or `--target-provider` to restore onto another provider.

`devsy workspace up` can also restore a snapshot. Do not combine it with a source argument or `--source`.

Expand All @@ -62,7 +66,7 @@ devsy snapshot delete <snapshot-ref>

## Export and import

`devsy workspace export` and `devsy workspace import` keep the snapshot reference of a restored workspace. The import restores the filesystem and volume only when the manifest still exists and the importing machine can pull from the registry. Otherwise it restores only the workspace metadata.
`devsy workspace export` and `devsy workspace import` keep the snapshot reference of a restored workspace. The import restores the container filesystem and workspace mount only when the manifest still exists and the importing machine can pull from the registry. Otherwise it restores only the workspace metadata.

## What the snapshot includes

Expand Down
32 changes: 9 additions & 23 deletions sites/docs-devsy-sh/content/docs/developing-providers/agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,40 +3,26 @@ title: Provider Agent
sidebar_label: Provider Agent
---

Devsy injects itself into an environment when it connects through a provider. That copy is the Devsy agent. It is the same binary, run as `devsy agent`. The agent deploys the container, forwards credentials, runs the SSH server, and stops idle environments.
Devsy runs a copy of its CLI as an agent in the provider environment. For a remote provider, it places the binary on the remote machine. A local provider uses the host's binary. The agent deploys the workspace, forwards credentials, runs the SSH server, and manages inactivity shutdown.

The `agent` section of `provider.yaml` configures the agent.
Configure it in the `agent` section of `provider.yaml`. This example uses a local Docker daemon:

```yaml
agent:
path: ${DEVSY}
local: true
driver: docker
inactivityTimeout: 10m
containerInactivityTimeout: 10m
injectGitCredentials: ${INJECT_GIT_CREDENTIALS}
injectDockerCredentials: ${INJECT_DOCKER_CREDENTIALS}
binaries:
MY_BINARY:
- os: linux
arch: amd64
path: https://example.com/my-binary
checksum: <sha256 of the file>
exec:
shutdown: |-
${MY_BINARY} stop
```

This section can use option values.

- **path**: where the agent is placed on the remote machine.
- **local**: set to `true` when Devsy runs on the same machine instead of a remote one. The built-in providers set it.
- **path**: path to the agent binary in the provider environment. If omitted, a local agent uses the current executable; a remote agent uses Devsy's default install path.
- **local**: set to `true` when the agent runs on the user's machine. Leave it unset or `false` for a remote machine.
- **driver**: how the container is deployed. One of `docker` (default), `kubernetes`, `apple`, `microsandbox`, `custom` or `external`. See [Drivers](./driver.mdx).
- **inactivityTimeout**: how long an idle machine runs before Devsy stops it. Machine providers only.
- **containerInactivityTimeout**: how long the container may stay idle before it stops. A Go duration such as `10m`. Empty or `0` disables it. Non-machine providers only.
- **injectGitCredentials**, **injectDockerCredentials**: whether to copy those credentials into the environment.
- **containerInactivityTimeout**: idle period monitored by the container daemon, as a Go duration such as `10m`. For Kubernetes, an empty value falls back to the deprecated `agent.kubernetes.inactivityTimeout`. If both fields are set, their durations must match. Clear both fields or set both to `0` to disable the timeout. For Kubernetes, expiry ends the pod. A detached Docker or Podman daemon does not stop its container when the timer expires.
- **injectGitCredentials**, **injectDockerCredentials**: whether to forward those credentials into the environment. The context's credential-forwarding settings override these provider values. See [Credentials](../developing-in-workspaces/credentials.mdx).
- **exec.shutdown**: the command that stops the machine once it is idle. Option values are available in it.
- **binaries**: helper binaries for `exec.shutdown`. Same format as [provider binaries](./binaries.mdx).
- **binaries**: helper binaries for `exec.shutdown`. Declare them under `agent.binaries`, using the format in [Provider binaries](./binaries.mdx).

## Docker management policy

Expand All @@ -58,7 +44,7 @@ Set `configureDaemon: false` when the provider owns the Docker daemon, including

## Stopping idle environments

Devsy tracks connections and stops the environment after the timeout. It starts the environment again when the user returns. State survives only if the provider keeps the storage. A shutdown command that deletes the machine must keep workspace data elsewhere, for example on a persistent volume.
Inactivity shutdown depends on the provider and driver. Start a stopped workspace with `devsy workspace up`. State survives only if the provider keeps the storage. A shutdown command that deletes the machine must keep workspace data elsewhere, for example on a persistent volume.

### Non-machine providers

Expand All @@ -70,4 +56,4 @@ Any UID in the same container can block inactivity through a live activity conne

### Machine providers

Stopping only the container leaves the VM running and billed. Define `agent.exec.shutdown` with a command that stops or deletes the machine on the cloud side. Set the timeout with `inactivityTimeout`. Each official cloud provider does this differently. One calls a cloud API with a token saved in an option.
Stopping only the container leaves the VM running and billed. Define `agent.exec.shutdown` with a command that stops or deletes the machine on the cloud side, and set `inactivityTimeout`. If the command needs a helper binary, declare it under `agent.binaries`. Check the provider's API and storage policy before choosing between stopping and deleting the machine.
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ binaries:
MY_BINARY:
- os: linux
arch: amd64
path: https://example.com/my-binary-linux-amd64 # URL or local path
path: https://example.com/my-binary-linux-amd64
checksum: <sha256 of the file>
name: my-binary # optional, defaults to the file name
name: my-binary
- os: darwin
arch: arm64
path: https://example.com/my-binary-darwin-arm64
Expand All @@ -23,6 +23,8 @@ binaries:

Supported archives are `.tgz`, `.tar`, `.tar.gz` and `.zip`. When a user adds the provider, Devsy downloads the item that matches their OS and architecture and verifies `checksum` if set.

`path` can also point to a local executable. `name` sets the installed file name for a direct download and defaults to the file name in `path`.

See the official providers, such as [devsy-provider-aws](https://github.com/devsy-org/devsy-provider-aws), for complete examples.

<Callout>
Expand All @@ -39,7 +41,7 @@ binaries:
- os: linux
arch: amd64
path: https://example.com/my-binary-linux-amd64.tar.gz
archivePath: bin/my-binary # executable inside the archive
archivePath: bin/my-binary
checksum: <sha256 of bin/my-binary after extraction>
```

Expand All @@ -49,7 +51,7 @@ Devsy extracts the whole archive into a folder for the binary and keeps its layo

`checksum` is optional, except for the `agent.binaries` entry selected by `agent.external.binary`, which needs one for every platform. See [External Driver](./driver.mdx#external-driver).

When set, Devsy computes the SHA256 of the file it runs and compares it case-insensitively. For an HTTP(S) archive, that file is the one extracted at `archivePath`, never the archive. A mismatch deletes the file and fails the download without retrying.
When set, Devsy computes the SHA-256 of the executable and compares it case-insensitively. For an HTTP(S) archive, it checks the extracted file at `archivePath`, not the archive. A downloaded file with a checksum mismatch is removed, and the download fails without retrying. A local source file with a mismatch is left in place. Verify the binary against a trusted source before changing its checksum. Update the manifest checksum only for an intentional, verified binary change; otherwise, restore the expected file before trying again.

## Use a binary

Expand Down
Loading
Loading