Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4,264 changes: 3,379 additions & 885 deletions public/openapi.json

Large diffs are not rendered by default.

Binary file modified src/assets/shots/snapshots.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified src/assets/shots/workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified src/assets/shots/workspaces.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 21 additions & 6 deletions src/content/docs/admin/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: Installation settings
description: "How an installation is configured: the file that seeds it, the dashboard that edits it, and how a changed file becomes a proposal."
reads:
- "spin:configs/spin-example.yaml@v20261110.01"
- "spin:internal/controlplane/installconfig/apply.go@v20261110.01"
- "spin:configs/spin-example.yaml@v20261114.01"
- "spin:internal/controlplane/installconfig/apply.go@v20261114.01"
---

An installation's settings live in its database, and an administrator edits all of them in the
Expand Down Expand Up @@ -51,13 +51,28 @@ control plane's own test reads.
On AWS the module adds what it knows itself: the domain, which machines join as runners, and the
autoscaling settings.

## Settings only the dashboard has
## Changing a setting

Some settings are not in the file. In **Admin → Security** and **Admin → Settings**:
Every setting has one definition, and the file, the dashboard and the terminal all read it.
[`spin admin settings get` lists every setting as it is in force, with where it is changed, and
`spin admin settings set` changes one](<feature:Every installation setting read and changed from the terminal, by the definitions the dashboard and the file use>):

```bash
spin admin settings get
spin admin settings set session_duration_hours 72
```

Most settings can be seeded by the file too. A few are the file's alone, such as the base domain,
and a few have a page of their own that checks them before they are written, such as the object
store and the mail server. Among the ones an administrator changes:

- **Sign-in providers**: GitHub and OIDC. See [Users and teams](/admin/users/).
- **Session length**: seven days by default, at most fourteen.
- **Sharing**: whether snapshots may be shared (on by default), and whether ports and snapshots
- **Sharing**: whether workspaces may be shared (on by default), and whether ports and snapshots
may be made public (off by default).
- **Rate limits** on sign-in and on the API.
- **Telemetry**: which traces and metrics are kept. See [Observability](/admin/observability/).
- **A root shell on a host**: whether another administrator approves each one, and how long it
may stay open. See [Operations](/admin/operations/).

Sign-in providers, GitHub and OIDC, are not settings: they are in **Admin → Security**. See
[Users and teams](/admin/users/).
10 changes: 8 additions & 2 deletions src/content/docs/admin/hosts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: Operating hosts
description: Drain, activate and revoke runners, see what runs where, replace a runner's machine, and add a runner outside the autoscaling group.
reads:
- "spin:docs/runners.md@v20261110.01"
- "spin:cmd/cli/cmd/admin.go@v20261110.01"
- "spin:docs/runners.md@v20261114.01"
- "spin:cmd/cli/cmd/admin.go@v20261114.01"
- "spin-terraform-aws:modules/runners/main.tf@v20261027.02"
---

Expand Down Expand Up @@ -56,8 +56,14 @@ disk is refused, naming them:

```bash
spin admin runner remove <runner-id>
spin admin runner remove <runner-id> --taken
```

`--taken` is for a machine that may be compromised: its credentials are deleted and it is cut off
before its group is asked anything, and every disk whose key it was given is marked for re-keying,
as a revoke does. With `--leave-workspaces` as well, it goes even where workspaces' disks are only
there; each of those runs again only once it is recovered onto another runner.

After a revoke, or when a runner is wedged, terminate its instance without lowering the group's
size. The group starts another, which joins by its role:

Expand Down
8 changes: 5 additions & 3 deletions src/content/docs/admin/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Observability
description: The logs, traces and metrics an installation keeps about itself and its workspaces, where they are kept, how long, and how administrators read them.
reads:
- "spin:internal/metricstore/metricstore.go@v20261108.01"
- "spin:internal/metricstore/metricstore.go@v20261114.01"
- "spin-terraform-aws:modules/controlplane/telemetry.tf@v20261027.02"
- "spin-terraform-aws:modules/controlplane/logging.tf@v20261027.02"
---
Expand Down Expand Up @@ -68,8 +68,10 @@ minute.

## What is not there

- **Nothing pages the operator.** Alert rules fire in the installation's Grafana, and no contact
point tells anyone. A workspace's owner is emailed when it waits for them.
- **No pager.** Alert rules fire in the installation's Grafana, whose one contact point is the
control plane: it records each firing and resolution once, and emails every administrator who
has not turned that kind off, with the rule and its runbook. An alert is an email, nothing
louder. A workspace's owner is emailed when it waits for them.
- The collector accepts OpenTelemetry from whatever its security group lets in: the proxy and the
runners.
- AWS keeps its own records outside the installation: the VPC flow log and every machine's boot
Expand Down
51 changes: 41 additions & 10 deletions src/content/docs/admin/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
title: Jobs, the bill and the store
description: What an administrator runs and reads beyond runners, from the control plane's jobs and the month's bill to the store, disk keys and a root shell.
reads:
- "spin:cmd/cli/cmd/admin_jobs.go@v20261110.01"
- "spin:cmd/cli/cmd/admin_costs.go@v20261110.01"
- "spin:cmd/cli/cmd/admin_storage.go@v20261110.01"
- "spin:cmd/cli/cmd/admin_volume.go@v20261110.01"
- "spin:cmd/cli/cmd/admin_shell.go@v20261110.01"
- "spin:cmd/cli/cmd/admin_jobs.go@v20261114.01"
- "spin:cmd/cli/cmd/admin_costs.go@v20261114.01"
- "spin:cmd/cli/cmd/admin_storage.go@v20261114.01"
- "spin:cmd/cli/cmd/admin_volume.go@v20261114.01"
- "spin:cmd/cli/cmd/admin_shell.go@v20261114.01"
---

These are the `spin admin` commands that are not about a runner's state. For drain, activate and
Expand Down Expand Up @@ -38,15 +38,20 @@ spin admin costs by team
spin admin costs by owner
```

The first form is the cloud's bill by service, what the month is forecast to end at, and its
budget. It is the bill as the control plane last read it, never the cloud itself. The control
plane reads only the costs that `bill_scope` names in the
[configuration file](/reference/configuration/file/): without a scope it reads none.
The first form starts with the standby: the installation's daily cost with nobody working,
estimated from the last billed day. Then the cloud's bill by service, what the month is forecast
to end at, and its budget. It is the bill as the control plane last read it, never the cloud
itself. The control plane reads only the costs that `bill_scope` names in the
[configuration file](/reference/configuration/file/): without a scope it reads none, and says so.

`by` shares the month out. The runners are shared by the cores and memory each workspace held,
and the volumes' bucket by what its volumes kept. What nobody caused is said apart and never
spread: runner capacity nobody held, bytes no workspace keeps, and the platform's own cost by
service.
role and service.

The same estimate reaches the people it concerns:
[a workspace's owner sees its cost this month and its rate on its page, a team's admins their
workspaces summed, and a project's admins its workspaces with its prebuilds' storage](<feature:A workspace's cost this month and its rate, as an estimate with its day, for its owner; a team's workspaces summed for its admins, and a project's with its prebuilds' storage for its admins>).

## The store

Expand Down Expand Up @@ -82,6 +87,32 @@ the old disk. That destroys the old key, and every snapshot of the disk with it.
is anybody's, named by its id, and must be stopped or quarantined. A disk something was cloned from is refused. `rekeys` lists the
disks being re-keyed, and the ones a revoked host's keys marked for it.

## A workspace that is not yours

An administrator can
[stop a workspace that is not theirs, move it off a host that is gone, read where it has sent, and
list the disks hosts refuse](<feature:An administrator stops a workspace that is not theirs, moves it off a lost host with a reason its owner reads on its history, reads where it sends, and lists the disks hosts refuse: each audited>).
Each is audited.

```bash
spin admin workspace stop <workspace-id> --reason "<why>"
spin admin workspace recover <workspace-id> --reason "<why>"
spin admin workspace egress <workspace-id> --reason "<why>"
spin admin volume refused
```

`stop` does what its owner's stop does; `--force` gives up what only its host has rather than
publish it first. `recover` needs a reason, which its owner reads, with who asked, on the
workspace's history; run without `--accept-data-loss` it shows the work the old host had not
published, and changes nothing. `refused` lists every disk its host refuses to serve, why, and
whose workspace cannot run for it.

## What happened

[The timeline](<feature:A timeline: what happened to a workspace, a runner, a team, a project or the installation, by whom, in one read>),
in **Admin → Timeline**, reads what happened to a workspace, a runner, a team, a project or the
installation, and who did it, in one place.

## A root shell on a runner

An administrator can open
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/admin/recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Backups and recovery
description: What an installation's durable state is, how each part is backed up, and what can be recovered after losing a runner, a machine, or the database.
reads:
- "spin:internal/pgserver/server.go@v20261108.01"
- "spin:internal/pgserver/server.go@v20261114.01"
- "spin-terraform-aws:modules/controlplane/database_bucket.tf@v20261027.02"
- "spin-terraform-aws:modules/controlplane/attested_key.tf@v20261027.02"
- "spin-terraform-aws:modules/controlplane/bucket.tf@v20261027.02"
Expand Down
6 changes: 3 additions & 3 deletions src/content/docs/admin/users.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
title: Users and teams
description: How people sign in, who may register, administrators, teams and their roles, API keys, and the audit log.
reads:
- "spin:internal/controlplane/settings/security.go@v20261108.01"
- "spin:cmd/cli/cmd/team.go@v20261108.01"
- "spin:cmd/cli/cmd/apikey.go@v20261108.01"
- "spin:internal/controlplane/settings/registry.go@v20261114.01"
- "spin:cmd/cli/cmd/team.go@v20261114.01"
- "spin:cmd/cli/cmd/apikey.go@v20261114.01"
---

## Signing in
Expand Down
4 changes: 2 additions & 2 deletions src/content/docs/aws/network.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: What the internet can reach in an installation, which machine talks
reads:
- "spin-terraform-aws:modules/controlplane/network.tf@v20261027.02"
- "spin-terraform-aws:modules/controlplane/logging.tf@v20261027.02"
- "spin:docs/security/traffic.md@v20261108.01"
- "spin:docs/security/traffic.md@v20261114.01"
---

import Figure from '../../../components/Figure.astro';
Expand All @@ -26,7 +26,7 @@ public address is only a way out. A runner has no inbound rule at all. No machin

| From | To | Port | Who the other end is |
|---|---|---|---|
| Proxy | Control plane | 8080 | TLS under the installation's CA |
| Proxy | Control plane | 8080 | TLS under the installation's CA; the request's own credential, and for the calls only a proxy makes, the proxy's token, which it proves again regularly |
| Runners | Control plane | 8080 | TLS under the installation's CA, and the runner's token |
| Runners | Proxy (relay) | 443 | The relay's certificate, and a one-time token per session |
| Proxy, runners | Collector on the control plane | 4317 | Only the security group |
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/aws/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ These are made for you, and each one has a reason:
|---|---|
| Every machine is replaced to update, never patched | Its root filesystem is read-only and verified; a machine runs exactly what the release signed. |
| The proxy is the only machine the internet reaches | The control plane, the runners and the database take no connection from outside the VPC. |
| No SSH on any machine | A shell on the control plane or the proxy is AWS Session Manager, decided by IAM and logged by AWS. A runner takes none. |
| No SSH on any machine | A shell on the control plane or the proxy is AWS Session Manager, decided by IAM and logged by AWS. A runner takes no Session Manager shell; an administrator's root shell on one goes through the control plane, the runner emptied first and the session recorded ([Operations](/admin/operations/#a-root-shell-on-a-runner)). |
| No NAT gateway | It would cost more than the machines that run all the time. |
| Runners are spot and start from zero | They are the largest cost, and they only cost while a workspace runs. |
| Runners join by their IAM role | There is no join token to create, store, rotate or leak. |
Expand Down
23 changes: 18 additions & 5 deletions src/content/docs/aws/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ description: Where to look when an installation does not come up, a workspace wa
reads:
- "spin-terraform-aws:outputs.tf@v20261027.02"
- "spin-terraform-aws:modules/runners/main.tf@v20261027.02"
- "spin:cmd/cli/cmd/status.go@v20261108.01"
- "spin:docs/runbook.md@v20261108.01"
- "spin:cmd/cli/cmd/status.go@v20261114.01"
- "spin:docs/runbook.md@v20261114.01"
---

## A machine does not come up
Expand All @@ -25,9 +25,11 @@ means the image and `spin_version` disagree: the images the module names are tho
## A shell on a machine

No machine runs SSH. The control plane's and the proxy's machines take a shell with Session
Manager, on the one in service. A runner takes none: its image carries no agent and its role
grants none, so its serial console, its boot log and its metrics say what it does, and one that
fails is replaced.
Manager, on the one in service. A runner takes no Session Manager shell: its image carries no
agent and its role grants none, so its serial console, its boot log and its metrics say what it
does, and one that fails is replaced. When a runner has to be looked at from inside, an
administrator opens [a recorded root shell on it](/admin/operations/#a-root-shell-on-a-runner)
through the control plane.

```bash
aws autoscaling describe-auto-scaling-groups \
Expand All @@ -36,6 +38,17 @@ aws autoscaling describe-auto-scaling-groups \
aws ssm start-session --target <instance-id>
```

On a runner, when its metrics, logs and console have not answered:

```bash
spin admin runner list
spin admin shell open <runner-id> --reason "<why>" --incident
```

`--incident` opens the shell on the runner as it is, its workspaces still running; without it the
runner is emptied first. Either way the session is recorded and the runner is retired once the
shell ends, so it costs the runner.

Spin OS has the tools to look with — `tcpdump`, `strace`, `ss`, `nft`, `conntrack`, `dig`,
`bpftool`, `htop`, `iostat` — because nothing can be installed on a machine.

Expand Down
4 changes: 2 additions & 2 deletions src/content/docs/concepts/images.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: Images
description: What an image is in Spinbox, the default image, how an administrator publishes and retires images, and how to build your own starting point from a snapshot.
reads:
- "spin:docs/storage/README.md@v20261110.01"
- "spin:cmd/cli/cmd/admin_storage.go@v20261110.01"
- "spin:docs/storage/README.md@v20261114.01"
- "spin:cmd/cli/cmd/admin_storage.go@v20261114.01"
---

An image is a disk a workspace can start from. In Spinbox an image **is** a published snapshot
Expand Down
6 changes: 3 additions & 3 deletions src/content/docs/concepts/network.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: Network and egress
description: How network policies control where a workspace connects, what is always blocked, how DNS works, and how to see what it reached.
reads:
- "spin:docs/network/README.md@v20261110.01"
- "spin:configs/spin-example.yaml@v20261110.01"
- "spin:docs/network/README.md@v20261114.01"
- "spin:configs/spin-example.yaml@v20261114.01"
---

import Figure from '../../../components/Figure.astro';
Expand All @@ -27,7 +27,7 @@ default declares a policy that says so, for example `allow: ["0.0.0.0/0"]`, and

## Always blocked

Whatever its policies say, a workspace never reaches:
Whatever its policies say, the host drops a workspace's traffic to:

- private, loopback, link-local and reserved addresses, including the cloud's instance
metadata service,
Expand Down
6 changes: 3 additions & 3 deletions src/content/docs/concepts/ports.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
title: Ports and sharing
description: How to reach a service running in a workspace over HTTPS, who can reach it, and how to share it with other users or make it public.
reads:
- "spin:cmd/cli/cmd/port.go@v20261108.01"
- "spin:cmd/cli/cmd/portaccess.go@v20261108.01"
- "spin:cmd/cli/cmd/share.go@v20261108.01"
- "spin:cmd/cli/cmd/port.go@v20261114.01"
- "spin:cmd/cli/cmd/portaccess.go@v20261114.01"
- "spin:cmd/cli/cmd/share.go@v20261114.01"
---

A service listening on a TCP port in a workspace [can be reached through the installation's
Expand Down
12 changes: 6 additions & 6 deletions src/content/docs/concepts/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
title: Projects and repositories
description: How a project ties a repository to the shape of its workspace, and how you open one from a URL, prebuild its refs and connect your GitHub account.
reads:
- "spin:cmd/cli/cmd/project.go@v20261110.01"
- "spin:cmd/cli/cmd/project_prebuild.go@v20261110.01"
- "spin:cmd/cli/cmd/open.go@v20261110.01"
- "spin:cmd/cli/cmd/git.go@v20261110.01"
- "spin:cmd/cli/cmd/project.go@v20261114.01"
- "spin:cmd/cli/cmd/project_prebuild.go@v20261114.01"
- "spin:cmd/cli/cmd/open.go@v20261114.01"
- "spin:cmd/cli/cmd/git.go@v20261114.01"
---

A project is [a repository and the shape of the workspace that opens it](<feature:Projects of the installation: a repository and the shape of the workspace that opens it, shared with everyone or some>).
Expand Down Expand Up @@ -56,8 +56,8 @@ A plain repository URL opens at the project's ref. A workspace can also be
[opened at a commit or at a pull request, which is checked out as `pr-<n>`](<feature:A workspace opened at a commit or a pull request, checked out as pr-%3Cn%3E>).

The workspace is made in `--team`, or in your personal team, and sized by the project's preset
unless `--preset` names another. `spin open` ends in `spin ssh`,
unless you pass `--no-connect`.
unless `--preset` names another. `spin open` ends in `spin ssh`, which logs in with a key you
added (`spin ssh-key add`), unless you pass `--no-connect`.

To open a project by its slug instead of a URL:

Expand Down
17 changes: 10 additions & 7 deletions src/content/docs/concepts/runners.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
title: Runners and hosts
description: What a runner is, how it joins, how workspaces are placed on it, how runners scale from zero, and what happens when one is lost.
reads:
- "spin:docs/runners.md@v20261110.01"
- "spin:internal/controlplane/domain/autoscale/autoscale.go@v20261110.01"
- "spin:internal/controlplane/domain/hostclass/hostclass.go@v20261110.01"
- "spin:docs/runners.md@v20261114.01"
- "spin:internal/controlplane/domain/autoscale/autoscale.go@v20261114.01"
- "spin:internal/controlplane/domain/hostclass/hostclass.go@v20261114.01"
- "spin-terraform-aws:modules/runners/main.tf@v20261027.02"
---

Expand Down Expand Up @@ -47,7 +47,9 @@ workspaces until it is back under 65%.

A preset may name a **host class**, a kind of runner, and its workspaces are placed only on
runners of that class. An installation the module's current release makes has one class,
`default`, and every runner is of it.
`default`, and every runner is of it. A spot class can be set to fall back: when AWS refuses spot
capacity for its group, the control plane runs the group on demand for 30 minutes, then tries spot
again.

## Scaling from zero

Expand All @@ -56,13 +58,14 @@ a workspace is waiting, and when the last one stopped.

- With no runner up, the first workspace to start waits in *Waiting for a host* while the control
plane asks for one. It is placed on the runner as soon as it registers.
- If a workspace has waited longer than a runner takes to start, the control plane adds one.
- If a workspace has waited ten minutes, and no runner was added in that time, the control plane
adds one.
- When nothing has run for [`runner_idle_minutes`](/reference/terraform/root/#runner_idle_minutes), it scales the group back to
none. During the installation's `quiet_hours`, it does so as soon as nothing is running.
- It never asks for more than `max_runners`.

It never chooses one runner of several to remove: it only goes from none to one, adds one, and
goes back to none.
While workspaces run and none waits, a runner that holds nothing only it has, for as long as
`runner_idle_minutes`, is emptied and given back to the group, one at a time.

## When a runner goes away

Expand Down
Loading
Loading