Skip to content

fix(snap): restart the gateway on refresh only when it is running - #3731

Open
drew wants to merge 12 commits into
mainfrom
fix/snap-refresh-single-restart
Open

drew wants to merge 12 commits into
mainfrom
fix/snap-refresh-single-restart

Conversation

@drew

@drew drew commented Sep 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #3726. Its post-refresh hook restarts the gateway unconditionally, which double-restarts it on every refresh after the first mTLS release. This restarts the gateway only when it is still running when the hook fires.

Stacked on #3726: base is fix/snap-gateway-mtls. Retarget to main after #3726 merges.

Related Issue

No issue required: follow-up fix to #3726.

Changes

  • snap/hooks/post-refresh: run snapctl restart only when snapctl services reports the gateway active.
    • Refresh from a revision with refresh-mode: endure (the currently published snap): the old plaintext gateway is still running, so the hook restarts it into the migrated mTLS config, as before.
    • Refresh from a revision with refresh-mode: restart (every later refresh): snapd has already stopped the gateway and starts it after the hook, so the hook no longer restarts it a second time. Previously the queued restart stopped the fresh gateway ~7s after it started, interrupting sandboxes twice and repeating the startup image refresh.
  • tasks/scripts/test-snap-install-hook.sh: the post-refresh test now covers an active gateway (restart) and an inactive gateway (no restart). It fails against the unconditional restart.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated: test-snap-install-hook.sh passes, and fails against the previous hook.
  • E2E (tmachine ubuntu-docker-rootful VM, published edge snap repacked with this branch's hooks):
    • mTLS-to-mTLS refresh, 3/3 runs: refresh Done, the gateway started exactly once after the refresh (journal), and the client was authenticated again within 2-3s. Before this change the journal showed two starts per refresh, and one of five runs saw the gateway refuse connections for 60s after the refresh.
    • Upgrade from the store latest/stable snap (refresh-mode: endure) with an existing HTTP registration: the old plaintext gateway was still restarted into mTLS, the HTTP registration stopped working, and rerunning install.sh switched to HTTPS with mTLS authentication.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable): not applicable

🤖 Generated with Claude Code

drew added 12 commits September 25, 2026 16:25
Replace the installer opt-in with an authenticated snap gateway. The wrapper
no longer forces plaintext, so the gateway serves TLS from the bundle it
already generates in $SNAP_COMMON/tls. The install hook writes a config that
enables mTLS user auth instead of unauthenticated access, and a new
post-refresh hook migrates the exact legacy default on existing installs.

install.sh waits for the gateway, detects whether it serves TLS, copies the
client bundle into the target user's snap state directory, and registers the
gateway over HTTPS. Older plaintext snap revisions still register over HTTP
with a warning. The release canary asserts mTLS auth and HTTPS registration.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
An explicit [openshell.gateway.mtls_auth] table fails config preflight,
which validates mTLS auth before the local TLS bundle supplies the client
CA. Write a default that pins the Docker driver instead; with the wrapper's
TLS bundle the gateway requires client certificates and enables mTLS user
auth automatically, as the native packages do.

The mTLS gateway rejects TLS handshakes without a client certificate, and
it still answers plaintext loopback HTTP for sandbox service routing, so the
installer could misdetect it as a legacy plaintext gateway. Probe HTTPS with
the root-owned client bundle, and treat a gateway as legacy only when a
plaintext gRPC Health call succeeds.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Detect the mTLS snap from the installed revision's post-refresh hook instead
of probing plaintext gRPC, and drop the scheme global. Remove the installer's
pre-hook config fallback, which is dead now that every channel ships the
install hook and which wrote the insecure default. Give the install hook a
single write path with a simple backup name, and shorten the manual client
certificate steps.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Stop selecting the OpenShell snap just because the snap command exists. Linux
installs default to the Debian or RPM package; OPENSHELL_INSTALL_METHOD=snap
(or deb, rpm) selects the package explicitly. Hosts that already have the
OpenShell snap keep refreshing it rather than gaining a second gateway on the
same port. The release canary and snap repro script opt in explicitly.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Published revisions use refresh-mode: endure, and snapd honors the old
revision's setting during a refresh, so the plaintext gateway kept running
with the migrated config unused until a manual restart. Restart the gateway
from the post-refresh hook so the mTLS config takes effect immediately.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
The post-refresh hook restarted the gateway unconditionally. Refreshing from
a revision with refresh-mode: restart already stops the gateway before the
hook and starts it afterwards, so the queued restart stopped the fresh
gateway and started it again: a second sandbox interruption and a second
startup image refresh on every refresh.

Restart only when the gateway is still running, which happens when the
previous revision used refresh-mode: endure and kept its plaintext gateway
up.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Base automatically changed from fix/snap-gateway-mtls to main September 26, 2026 01:59

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant