[Bug][Windows] Failed ocx update leaves a file-less package skeleton; service wrapper restart-loops 1,009 times with no integrity check or backoff

[stephen-drew]

Summary

On Windows 11, an ocx update from 2.13.0 left the global package directory containing 107 directories and zero files.
Every launcher (ocx, ocx.cmd, ocx.ps1, opencodex, opencodex.cmd, opencodex.ps1) was removed from
%APPDATA%\npm, so no documented recovery command was reachable from a normal shell.

The Task Scheduler service wrapper then restarted the missing entry point every 5 seconds for 89 minutes —
1,009 consecutive child exited with code 3 events — until the package was manually rolled back.

This matches the partial-install behaviour described in #281. That issue was closed with the permanent-crash symptom
attributed to an external monitoring script terminating the restarted proxy. That attribution does not fit this case:
no external process was involved. The wrapper was relaunching a package that had no bin/ocx.mjs, no
src/cli/index.ts, and no bundled bun.exe to execute.

Environment

Item	Value
OS	Windows 11 Home, 10.0.26100 build 26100, AMD64
Node	v24.15.0
npm	11.19.0
opencodex (before/after rollback)	2.13.0
Bundled Bun	1.3.14
Service backend	Task Scheduler (opencodex-proxy), wrapper opencodex-service.cmd
Install type	npm global, %APPDATA%\npm\node_modules\@bitkyc08\opencodex

Evidence

1. The abandoned install contained directories only

The failed package was preserved before rollback. Recursive enumeration:

TotalDirs   : 107
TotalFiles  : 0

95 dependency folders were created under node_modules (@bufbuild, @hono,
@modelcontextprotocol, @napi-rs, @oven, bun, ...). The node_modules/bun folder
exists and is completely empty. Missing at the package root: package.json,
bin/ocx.mjs, src/cli/index.ts, node_modules/bun/bin/bun.exe.

So npm created the directory tree, then unpacked no file content and did not roll back.

2. Launchers were removed, recovery commands unreachable

%APPDATA%\npm\ocx.cmd and all sibling launchers were gone. What remained were npm's
rename-aside shims, which still pointed at the now-nonexistent target:

.ocx-cl2A1bun           .opencodex-lwm6eEVg
.ocx.cmd-RmK066DM       .opencodex.cmd-XPiSZ201
.ocx.ps1-AKozk9bb       .opencodex.ps1-p35htcAv

A stale scope directory was also left behind (same pattern as #281).

With no ocx on PATH, the documented remedies (ocx service repair, ocx doctor,
ocx update) cannot be run without manually reconstructing the Bun + CLI invocation.

3. Unbounded restart loop with no integrity check

service.log, filtered to this incident:

[16/08/2026 12:33:51.44] child exited with code 3; restarting in 5s
... 1,009 occurrences, one every ~5 seconds ...
[16/08/2026 14:02:50.11] child exited with code 3; restarting in 5s

First failure was 45 seconds after Windows boot (boot 12:33:06), so the broken package was
already on disk before the restart, and the loop began immediately at logon and never self-limited.

For contrast, code 3 appeared only 25 times in the entire preceding log history
(31/07 x1, 02/08 x13, 08/08 x11). This was not pre-existing noise.

The wrapper loop in opencodex-service.cmd is:

:loop
"%OCX_BUN%" "%OCX_CLI%" start --port 10100 >>"%OCX_SERVICE_LOG%" 2>&1
if %ERRORLEVEL% NEQ 0 (
  ping -n 6 127.0.0.1 >nul
  goto loop
)

There is no check that %OCX_BUN% or %OCX_CLI% exist, no failure counter, and no backoff.
A permanently unrecoverable condition is retried identically forever.

Reproduction

1. Windows, opencodex installed globally via npm and running under the Task Scheduler backend.
2. Run an update that triggers a global npm install while the service is live and Bun binaries are in use.
3. npm hits a locked-file error, leaves the package as a directory skeleton, and does not roll back.
4. Launchers disappear from %APPDATA%\npm; ocx is no longer on PATH.
5. The service wrapper loops indefinitely on the missing entry point.

Impact

All model routing through the proxy is down. Because the CLI is also gone, the failure is not
self-recoverable through the documented commands. Recovery here required restoring the previous
package directory and launcher shims manually, out of band.

Suggested fixes

Updater (relates to #281)

• Stage the new version and swap atomically, so a failed unpack cannot replace a working install.
• On failure, roll back to the previous package and restore launchers rather than leaving a skeleton.
• Stop the service and release Bun file handles before the install step, not after.
• Verify package.json, bin/ocx.mjs, src/cli/index.ts and the bundled runtime exist before declaring success.

Service wrapper (separate, arguably higher value)

• Preflight the launcher and runtime paths before the first spawn; if either is missing, log one clear
  "installation is incomplete, run X" line and stop rather than looping.
• Add exponential backoff and a consecutive-failure ceiling.
• Distinguish "crashed at runtime" from "cannot start at all" in the log.

A single existence check would have converted an 89-minute silent outage into one actionable log line.

Notes

• Rolled back to 2.13.0; the proxy is healthy again and has since survived a restart
  (status=protected, protection=service, rebootSafe=true).
• The failed install directory has been preserved and can be inspected further if useful.
• Happy to test a patched updater on this machine.

[coderabbitai]

🔗 Related PRs

#306 - feat(windows): add tray controls and restart-safety diagnostics [merged]
#533 - fix(update): preserve proxy on npm cache failures [open]
#557 - fix(update): harden npm cache recovery preflight logs [open]
#803 - Codex/pr533 update recovery hardening [open]
#838 - fix(update): recover Windows dashboard proxy after self-update [merged]
#1647 - fix(tray): let the service wrapper exit 0 on an already-live proxy [open]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[github-actions]

Maintainer decision respected

A maintainer has reopened this issue. The automated closure has been deactivated.

[Ingwannu]

Confirmed in current dev, with one important version distinction.

The current updater already stops the managed proxy before package replacement and performs registry-integrity preflight; those protections were added after the reported 2.13.0 build. However, two gaps remain:

• npm can still leave an unusable global package tree on installer failure, and there is no complete transactional restore of both the package directory and npm launchers;
• the current Task Scheduler wrapper still enters the same unconditional five-second loop and does not check whether its baked Bun or CLI entry exists.

I am taking the focused, independently safe wrapper slice first: verify both baked paths before the initial spawn and before every retry, log one actionable installation incomplete failure, and exit instead of retrying a condition that cannot recover in-process. Normal runtime crashes will retain the existing restart behavior. I will add source-level regressions and document why this is intentionally narrower than a transactional npm rollback.

I am not going to improvise a package-directory copy-back in this slice: restoring npm's package tree without the matching global shims and ownership/lock guarantees can make recovery less reliable. The remaining updater rollback boundary should be handled separately with a staged/verified install design.

[Ingwannu]

The focused wrapper fix is now in #1877.

It prevents the exact 1,009-restart failure mode by checking the baked Bun and CLI paths before the first spawn and every retry. A missing path produces one actionable incomplete-install log entry and exits; a proxy that actually starts and later crashes keeps the existing restart behavior.

This is deliberately a partial fix: current dev already stops the proxy before replacement and performs integrity preflight, but transactional restoration of both an npm package tree and its global launchers still needs a separate staged-update design. #1849 should remain open until that updater boundary is resolved.

[lidge-jun]

Split, and staying open. This report contains two defects with different fixes and
different closure conditions, so closing it on the shipped half would have hidden the one
that actually destroyed the install.

Half	Defect	Status
A	Service wrapper restart-loops against a missing install — the 1,009 × child exited with code 3	Fixed by #1877, released in v2.24.0 (verified by git tag --contains 01b212579)
B	ocx update deletes the working install and leaves an unbootable skeleton with no rollback	Open — now tracked in #1942

#1877 stopped the wrapper from hammering a package that is not there. It does nothing about
how the package came to be missing: the update path delegates to npm install -g, which
removes the existing install before the replacement is verified, so a mid-flight failure
leaves 107 directories, zero files, and no launcher to run a recovery command with.

#1942 carries the transactional shape (stage → verify → snapshot → atomic switch → health
probe → restore on failure), the ordering constraint that makes it work, and the
kill-at-every-boundary matrix it needs.

Keeping this issue open as the umbrella so the original evidence — the 107/0 enumeration,
the missing launchers, the 89-minute restart window — stays attached to the story rather
than being split away from it.

On the #281 comparison in the report: agreed that the earlier attribution does not fit here.
There was no external process, and a wrapper relaunching a package with no bin/ocx.mjs and
no bundled bun.exe is a different failure from the one #281 closed on.

[lidge-jun]

One correction to my comment above: I wrote the npm install -g sentence in a way that reads
as present-tense causation for the 107-directories/zero-files enumeration. That evidence is
from v2.13.0, and protections have been added since — a prior comment on this issue already
qualified that.

The conclusion is unchanged, because the primitive is: npm install -g is still
destructive-in-place, and src/update/ has no staging, snapshot, or rollback anywhere on the
path. Verification does exist (checkUpdatePackageIntegrity before the proxy stops,
isRealBunBinary, and an identity-checked /healthz probe after restart) — but it is
detect-only and runs after the old install is already gone, so a failed probe has nothing
left to restore. That is the shape #1942 is about.