Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/test-release-canary/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ Loopback registration auto-derives the gateway name to `openshell` if `--name` i
| `macos`/`ubuntu-deb`/`fedora` job fails on `install.sh` | Dev release missing an asset, checksum mismatch, or `install.sh` regression on this branch. | Job log around the `curl … install.sh \| sh` step. |
| Sandbox create or exec fails | Published sandbox and supervisor artifacts are missing, incompatible, or cannot establish the protected runtime channel. | Gateway logs plus Docker, Podman, VM, Snap, or Kubernetes runtime diagnostics for the job. |
| `macos`/`ubuntu-deb`/`fedora` job fails on `openshell status` | Local gateway service did not start (systemd/brew/podman). Often a driver issue. | Service logs in the job log; `OPENSHELL_COMPUTE_DRIVER` env in the "Ensure …" step. |
| `ubuntu-snap-system-docker` fails during `install.sh` | System Docker was unavailable, the edge revision or automatic interfaces were unavailable, or the gateway did not become reachable. | Failure diagnostics dump system Docker, snap service/connection/change state, gateway and snapd journals, snap logs, and port 17670 listeners. |
| `ubuntu-snap-system-docker` fails during `install.sh` | System Docker was unavailable to the runner user, the edge revision or automatic interfaces were unavailable, or the user gateway did not become reachable. | Failure diagnostics dump system Docker, snap service/connection/change state, user and legacy gateway journals, snap logs, and port 17670 listeners. |
| `ubuntu-snap-system-docker` fails during the prover checks | The prover artifact is missing or packaged for the wrong architecture, `openshell.prover` is not exposed or confined to read the test policies, or its solver linkage is not runnable. | The `Verify Snap installation` and `Check a policy boundary with the Snap prover` steps, plus `snap info openshell` and `snap connections openshell`. |
| `ubuntu-snap-docker-preflight` unexpectedly succeeds | The installer no longer fails before installing the OpenShell snap when Docker is absent or supplied by the Docker snap. | Inspect `install.log`, `docker-snap.log`, `snap list`, and snapd changes. |
| `kubernetes` job fails on `helm install --wait` | Chart did not deploy in 5 min — usually image pull failure or readiness probe failing. | "Diagnostics on failure" step dumps `helm status`, manifest, pod describe, pod logs. |
Expand Down
18 changes: 12 additions & 6 deletions .github/workflows/release-canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ jobs:
- name: Install and check status
run: |
set -euo pipefail
sudo systemctl set-environment \
systemctl --user set-environment \
"OPENSHELL_TELEMETRY_ENABLED=${OPENSHELL_TELEMETRY_ENABLED}"
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh | sh
sudo snap list openshell
Expand All @@ -234,8 +234,12 @@ jobs:
sudo snap connections openshell | grep -E '^docker +openshell:docker +:docker +'
openshell --version
openshell.prover --version
sudo snap services openshell
sudo journalctl -b -u snap.openshell.gateway.service --no-pager |
snap services openshell
systemctl --user is-enabled --quiet snap.openshell.user-gateway.service
systemctl --user is-active snap.openshell.user-gateway.service
! systemctl is-enabled --quiet snap.openshell.system-gateway.service
! systemctl is-active --quiet snap.openshell.system-gateway.service
journalctl --user -u snap.openshell.user-gateway.service --no-pager |
grep -F "mTLS user authentication enabled"
openshell gateway list | grep -F "https://127.0.0.1:17670"
openshell status
Expand Down Expand Up @@ -280,10 +284,12 @@ jobs:
sudo snap services openshell
sudo snap connections openshell
sudo snap changes
sudo systemctl status snap.openshell.gateway.service --no-pager
sudo journalctl -b -u snap.openshell.gateway.service --no-pager -n 300
systemctl --user status snap.openshell.user-gateway.service --no-pager
journalctl --user -u snap.openshell.user-gateway.service --no-pager -n 300
sudo systemctl status snap.openshell.system-gateway.service --no-pager
sudo journalctl -b -u snap.openshell.system-gateway.service --no-pager -n 300
sudo journalctl -b -u snapd.service --no-pager -n 300
sudo snap logs openshell.gateway -n=300
snap logs openshell.user-gateway -n=300
sudo ss -ltnp '( sport = :17670 )'

ubuntu-snap-docker-preflight:
Expand Down
2 changes: 1 addition & 1 deletion CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -491,7 +491,7 @@ These workflows run after merge to publish dev/tagged artifacts and verify them.
|---|---|
| `.github/workflows/release-dev.yml` | Publishes the rolling `dev` build on every push to `main`. Builds gateway, sandbox, and supervisor images and binaries, packages, wheels, and pushes the Helm chart as `oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev` (plus an immutable `0.0.0-dev.<sha>` pin). Also dispatchable manually. |
| `.github/workflows/release-tag.yml` | Publishes tagged stable releases and manually dispatched pre-releases. Its automatic tag trigger excludes `-pre.*`. Protobuf, security, and integration failures do not block pre-release artifact publication. Stable publication requires the currently implemented qualification profile to pass; the summary identifies the remaining RFC 0014 coverage. |
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts in the `macos`, `ubuntu-deb`, `ubuntu-snap-system-docker`, `fedora`, and `kubernetes` (kind + Helm) jobs. Each job reaches its gateway and creates, exercises, and deletes a sandbox. The Snap lanes verify a compatible system Docker lifecycle and `ubuntu-snap-docker-preflight` tests fail-fast behavior when Docker is absent or supplied by the Docker snap. The positive Snap lane also runs a local policy containment check with the packaged prover. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts in the `macos`, `ubuntu-deb`, `ubuntu-snap-system-docker`, `fedora`, and `kubernetes` (kind + Helm) jobs. Each job reaches its gateway and creates, exercises, and deletes a sandbox. The `ubuntu-snap-system-docker` job verifies a compatible system Docker lifecycle, while `ubuntu-snap-docker-preflight` tests fail-fast behavior when Docker is absent or supplied by the Docker snap. The positive Snap lane also runs a local policy containment check with the packaged prover. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |

## Required status contexts

Expand Down
4 changes: 3 additions & 1 deletion deploy/man/openshell-gateway.8.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,9 @@ The Debian and Ubuntu systemd user unit runs preflight before certificate
generation, while retaining its EnvironmentFile and bare ExecStart behavior. The
Snap wrapper replays its effective daemon arguments through preflight. It first
uses a nonempty OPENSHELL_GATEWAY_CONFIG. Otherwise it passes the canonical
SNAP_COMMON/gateway.toml path whenever it exists or is a symlink. A broken symlink
service-specific gateway.toml path whenever it exists or is a symlink. The legacy
system service uses SNAP_COMMON/gateway.toml; the user service uses
SNAP_USER_COMMON/.config/openshell/gateway.toml. A broken symlink
fails preflight before the gateway is started. Correct or manually migrate an
operator-owned v1 file, then run preflight again before restarting the service.

Expand Down
36 changes: 28 additions & 8 deletions docs/about/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -117,31 +117,51 @@ The snap installs the standalone policy prover as `openshell.prover`. The
The prover reads local policy files through the `home` interface and does not
connect to the gateway.

The gateway runs as a system service at `https://127.0.0.1:17670` and reads `/var/snap/openshell/common/gateway.toml`. It requires a client certificate. The install script copies that certificate to the installing user's Snap state and registers the gateway automatically. If you installed with `sudo snap install openshell`, give each trusted user the certificate and register the gateway from that user's account:
The gateway runs at `https://127.0.0.1:17670`. It reads `~/snap/openshell/common/.config/openshell/gateway.toml` when that file exists. Its database defaults to `~/snap/openshell/common/.local/state/openshell/gateway/openshell.db`, and its TLS material lives under `~/snap/openshell/common/.local/state/openshell/tls`. Register it from the same user account:

```shell
snap services openshell.user-gateway
openshell gateway add https://127.0.0.1:17670 --local --name openshell
openshell status
```

Existing installations upgraded from older revisions retain their state under `/var/snap/openshell/common` through the compatibility service `openshell.system-gateway`. The installer preserves that service model and updates the installing user's registration when required.

An automatic refresh or direct `snap refresh` cannot enroll a particular local user. If `snap get openshell gateway-mode` reports `system`, wait for the compatibility service to be active, then copy its client credentials into each trusted user's Snap state and replace that user's old HTTP registration:

```shell
snap services openshell.system-gateway
d=~/snap/openshell/common/.local/state/openshell/tls
mkdir -p -m 700 "$d" "$d/client"
sudo install -o "$USER" -m 600 /var/snap/openshell/common/tls/ca.crt "$d/"
sudo install -o "$USER" -m 600 -t "$d/client" \
/var/snap/openshell/common/tls/client/tls.crt /var/snap/openshell/common/tls/client/tls.key
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/ca.crt "$d/"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/client/tls.crt "$d/client/"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/client/tls.key "$d/client/"
openshell gateway list
openshell gateway remove <old-http-registration>
openshell gateway add https://127.0.0.1:17670 --local --name openshell
openshell status
```

Keep the client key private.
Use the name shown by `openshell gateway list` for `<old-http-registration>`. Older direct Snap instructions may have used `openshell-gateway`, while the install script uses `openshell`.

To install a locally built snap, connect its interfaces manually. The gateway may reach systemd's start limit before Docker is connected, so reset the failed unit and restart the gateway after connecting the interfaces:
Switch between the user-owned and compatibility services with `sudo snap set openshell gateway-mode=user` or `sudo snap set openshell gateway-mode=system`.

To install a locally built snap, connect its interfaces manually. The user gateway may reach systemd's start limit before Docker is connected, so reset the failed unit and restart it from the affected user session after connecting the interfaces:

```shell
sudo snap install ./openshell_*.snap --dangerous
sudo snap connect openshell:log-observe
sudo snap connect openshell:system-observe
sudo snap connect openshell:docker :docker
sudo systemctl reset-failed snap.openshell.gateway.service
sudo snap restart openshell.gateway
systemctl --user reset-failed snap.openshell.user-gateway.service
snap start openshell.user-gateway
```

For an installation upgraded from a legacy system gateway, use `sudo systemctl reset-failed snap.openshell.system-gateway.service` and `sudo snap restart openshell.system-gateway` instead.

## Kubernetes

Deploy the gateway to a cluster with the OpenShell Helm chart. See [Kubernetes Setup](/kubernetes/setup).
Expand Down
8 changes: 5 additions & 3 deletions docs/how-it-works/gateways/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Package-managed gateways use either built-in defaults or a package-seeded TOML f
| Homebrew | `$XDG_CONFIG_HOME/openshell/gateway.toml` when it exists, otherwise the Homebrew prefix config such as `/opt/homebrew/var/openshell/gateway.toml`. |
| Debian/Ubuntu | `$XDG_CONFIG_HOME/openshell/gateway.toml`, usually `~/.config/openshell/gateway.toml` for the systemd user service. |
| Fedora/RHEL RPM | `$XDG_CONFIG_HOME/openshell/gateway.toml`, usually `~/.config/openshell/gateway.toml`; the systemd user service seeds this file from the packaged template on first start. |
| Snap | `$SNAP_COMMON/gateway.toml`, usually `/var/snap/openshell/common/gateway.toml`. |
| Snap | New installations use `$SNAP_USER_COMMON/.config/openshell/gateway.toml`, usually `~/snap/openshell/common/.config/openshell/gateway.toml`. Upgraded legacy installations retain `$SNAP_COMMON/gateway.toml`, usually `/var/snap/openshell/common/gateway.toml`. |

The Fedora/RHEL RPM template leaves `[openshell.gateway].bind_address` unset. The gateway therefore uses its built-in `127.0.0.1:17670` primary listener. Host-networked Podman supervisors use that same loopback endpoint, so the primary listener does not need a wildcard address. Set `bind_address` explicitly only when clients must reach the primary multiplexed API through another interface.

Expand Down Expand Up @@ -1268,8 +1268,10 @@ Debian and Ubuntu run preflight from the systemd user unit before local certific
generation. The unit still loads the `gateway.env` environment file and starts the
gateway with no configuration arguments. Snap replays the exact effective daemon
arguments through preflight. It gives a nonempty `OPENSHELL_GATEWAY_CONFIG`
precedence; otherwise it validates and passes its canonical
`SNAP_COMMON/gateway.toml` only when that path exists in the filesystem. A broken
precedence; otherwise it validates and passes its canonical service-specific
path only when that path exists in the filesystem. The legacy system service
uses `$SNAP_COMMON/gateway.toml`; the user service uses
`$SNAP_USER_COMMON/.config/openshell/gateway.toml`. A broken
symlink is therefore rejected instead of being treated as absent.

Package startup does not modify an operator-owned v1 file. Back it up, follow
Expand Down
Loading
Loading