Repository navigation
Description
Before submitting
- I have searched existing issues to confirm this is not a duplicate
- I am running the latest version of Devsy
What happened?
When a Kubernetes workspace's pod terminates (PID 1 exits, for example an OOM kill or a
crash), the pod goes Completed and the PVC stays bound. devsy workspace status
correctly reports Stopped, but devsy workspace up fails after about 2 minutes instead
of bringing the workspace back. Deleting the pod by hand makes up work again, and the
PVC contents survive.
Likely cause: FindDevContainer (pkg/driver/kubernetes/find.go) treats any pod not in
phase Running as exited, so status is right. But reconcileExistingPod
(pkg/driver/kubernetes/run.go) finds the existing pod, sees the provider options are
unchanged, logs "Pod already exists and nothing changed, skipping update" and returns
without recreating it. It never checks the pod's phase, and only deletes the pod when
options changed. Agent injection then tries to exec into the terminated container.
What did you expect to happen instead?
workspace up recreates the pod (reusing the existing PVC) and the workspace starts.
Steps to reproduce
devsy provider add kubernetesdevsy provider set kubernetes -o KUBERNETES_CONTEXT=docker-desktop -o KUBERNETES_NAMESPACE=devsy-p3test -o DISK_SIZE=1Gidevsy workspace up ./p3test --id p3test --provider kubernetes --ide nonekubectl -n devsy-p3test exec <pod> -- kill 1(simulates PID 1 exiting)kubectl -n devsy-p3test get pods,pvcshows the podCompletedand the PVCBound.devsy workspace status p3testreports"state":"Stopped".- Run the
workspace upcommand from step 3 again. It fails.
devcontainer.json
{ "image": "alpine:3.20" }Error output / logs
platform-native agent delivery failed: deliver agent (post-start): in-container download failed:
download in container: Internal error occurred: unable to upgrade connection: container not found ("devsy")
exec-stream delivery failed: Internal error occurred: unable to upgrade connection: container not found ("devsy")
legacy agent injection failed: inject agent: [inject] read ping: EOF
# afterwards
NAME READY STATUS RESTARTS AGE
devsy-default-p3-30abe 0/1 Completed 0 2m39sHow often does this happen?
Every time
Operating system
macOS
Linux distribution (if applicable)
N/A (the pod image is alpine:3.20)
Architecture
ARM64
Desktop app or CLI?
CLI only
Devsy version
v1.24.0
Devsy provider
Kubernetes
Provider version
v1.0.0
Screenshots
No response
Anything else?
Suggested fix: in reconcileExistingPod, treat a pod in phase Succeeded or Failed
as needing recreation. Delete it as the options-changed path does (waitPodDeleted) and
fall through to creating a new one.
Any way PID 1 can end (OOM kill, crash, a node drain with restartPolicy: Never) leaves the workspace unrecoverable through Devsy until the pod is deleted.
This also blocks one possible fix for issue #1698 (inactivity timeout), which would otherwise end the pod's main process.
Workaround:
kubectl -n <namespace> delete pod <pod>
devsy workspace up ...
Activity
skevetter commentedon Oct 31, 2026
Triage result: fixed in v1.25.0-beta.6 and later prereleases
Reviewed on 30 October 2026 against main at
9bc12ee041ca854c6f81de95fd8c6e49d7cc028c.The original terminal-pod restart defect is implemented and merged. PR #1701 merged as
b86442407a9ad9854213f08931236d470d783b77. v1.25.0-beta.6 contains that commit; the commit comparison confirms ancestry. The fix is also in the currently published v1.25.0-beta.8. The latest stable release is still v1.24.0, the version in this report, so updating only within the stable channel does not deliver this fix.What changed
The original diagnosis was correct for v1.24.0. The terminal-state decision now runs before the unchanged-provider-options reuse shortcut:
classifyPodandreconcileObservedPodclassifySucceededandFailedpods as terminal. A terminateddevsycontainer is also recognized even if another container keeps the pod phaseRunning.workspace upis the resume operation;--recreateand manualkubectl delete podare not needed to solve this report.Evidence and closure check
The existing pod-lifecycle unit tests cover both terminal phases, unchanged-option replacement, running-pod reuse, UID conflicts, ownership rejection, forbidden lookups/deletes, cancellation and PVC preservation. The Kubernetes inactivity/resume E2E suite covers two stop/resume cycles with a preserved file and a separate explicit terminal-pod restart. The Kubernetes inactivity CI job on the final #1701 head completed successfully. PR #1737 subsequently fixed synchronization in a unit-test readiness flag; it does not add another production restart fix and is newer than beta.8.
For customer confirmation, repeat the original reproduction with beta.6 or later and record:
SucceededorFailed.devsy workspace up <workspace>succeeding with a different Pod UID, the same PVC/workspace UID, and the sentinel still present.uppreserving that healthy Pod UID.There is no outstanding implementation change for the behavior reported here. This issue can be resolved as fixed with the prerelease version identified, or kept open for reporter confirmation under the project's release policy. Do not create another terminal-pod recreation ticket. Files outside mounted volumes are not part of the PVC-preservation contract.
This review inspected source, release ancestry and recorded CI results; it did not run a new Kubernetes reproduction.