Skip to content

fix(selfhost): block deploy-selfhost-prebuilt on container health, sharing wait_for_healthy#8444

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
kai392:fix/critical-issue-prebuilt-health-wait
Jul 24, 2026
Merged

fix(selfhost): block deploy-selfhost-prebuilt on container health, sharing wait_for_healthy#8444
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
kai392:fix/critical-issue-prebuilt-health-wait

Conversation

@kai392

@kai392 kai392 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

deploy-selfhost-prebuilt.sh printed selfhost deploy: complete immediately after docker compose up -d, which only confirms the container was created and started — a freshly built image that crash-loops or never passes its health check still reported success. Its sibling deploy-selfhost-image.sh has always blocked on real health via wait_for_healthy. Both are documented standalone entry points, so an operator running the prebuilt script directly (its own documented usage) got a false success signal.

  • wait_for_healthy moves from deploy-selfhost-image.sh into scripts/lib/selfhost-deploy-common.sh, joining the helpers both deploy scripts already share (require_cmd, env_get/env_put, maybe_infisical_run, compose_file_args).
  • Parameterized rather than reading caller globalswait_for_healthy <service> <timeout_seconds> <log_prefix> <compose_args...>. This is required, not stylistic: compose_args is a function-local array in the prebuilt script (it includes the generated override file), so the old global-reading form could not be shared.
  • deploy-selfhost-image.sh calls the shared version with its existing values, and log_prefix keeps its success line byte-identical (selfhost image deploy: <service> is healthy) — no behavior change there.
  • deploy-selfhost-prebuilt.sh calls it at the end of run_compose_deploy (where compose_args is in scope) and gains HEALTH_TIMEOUT_SECONDS="${SELFHOST_HEALTH_TIMEOUT_SECONDS:-180}", the same override and default the image script already honours. On timeout it exits non-zero with the identical docker compose ps / logs --tail=80 diagnostics; the script's existing EXIT trap still removes the temp override file on that path.
  • docker-compose.yml's healthcheck, selfhost-post-update-check.sh, and selfhost-update.sh are untouched, per the issue's scope.

Closes #8395

Test plan

No Docker daemon in CI and scripts/** is outside Codecov's coverage.include, so this was verified directly:

  • bash -n clean on all three changed scripts
  • Functionally exercised both branches of the shared function by sourcing the lib and stubbing docker: the healthy path prints <prefix>: <service> is healthy and returns 0 (does not exit, so the image script's subsequent env_put/completion lines still run); the timeout path exits 1 and writes error: <service> did not become healthy within <N>s to stderr
  • Confirmed the log_prefix parameter reproduces deploy-selfhost-image.sh's original message exactly, so its output is unchanged
  • CI validate (actionlint/shellcheck)

@kai392
kai392 requested a review from JSONbored as a code owner July 24, 2026 13:19
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.43%. Comparing base (add1f78) to head (e83db04).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8444   +/-   ##
=======================================
  Coverage   92.42%   92.43%           
=======================================
  Files         791      791           
  Lines       79294    79318   +24     
  Branches    23952    23954    +2     
=======================================
+ Hits        73291    73315   +24     
  Misses       4866     4866           
  Partials     1137     1137           
Flag Coverage Δ
shard-1 57.76% <ø> (+<0.01%) ⬆️
shard-2 47.66% <ø> (-0.02%) ⬇️
shard-3 56.59% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 24, 2026
@loopover-orb

loopover-orb Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-24 13:59:32 UTC

3 files · 1 AI reviewer · no blockers · readiness 93/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR moves wait_for_healthy from deploy-selfhost-image.sh into the shared lib, parameterizing it (service, timeout, log_prefix, compose_args) so deploy-selfhost-prebuilt.sh can reuse it instead of reporting success right after `docker compose up -d`. The refactor is clean: the image script's call site preserves its exact prior behavior (same log_prefix, same globals passed explicitly), and the prebuilt script wires HEALTH_TIMEOUT_SECONDS with the same env var/default convention already used by the image script. I traced both call sites and the function body against the full file contents; the parameter shapes match and the exit/trap/cleanup semantics are preserved on the new failure path.

Nits — 4 non-blocking
  • scripts/deploy-selfhost-prebuilt.sh's HEALTH_TIMEOUT_SECONDS is not passed through validate_inputs-style regex validation the way deploy-selfhost-image.sh validates it (`^[0-9]+$`) before use — a malformed SELFHOST_HEALTH_TIMEOUT_SECONDS would only fail inside the arithmetic in wait_for_healthy rather than with a clear error message.
  • scripts/lib/selfhost-deploy-common.sh's wait_for_healthy comment block is fairly long (7 lines) relative to the rest of the file's comment style, though consistent with the file's existing verbose-comment convention.
  • Consider adding the same `[[ "$HEALTH_TIMEOUT_SECONDS" =~ ^[0-9]+$ ]]` guard used in deploy-selfhost-image.sh's validate_inputs to deploy-selfhost-prebuilt.sh for a clearer error message on bad input, per scripts/deploy-selfhost-image.sh's existing pattern.
  • Diff looks like trivial or whitespace-only churn — Reduce whitespace-only or formatting-only churn and keep the diff focused on substantive changes.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8395
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 117 registered-repo PR(s), 62 merged, 4 issue(s).
Contributor context ✅ Confirmed Gittensor contributor kai392; Gittensor profile; 117 PR(s), 4 issue(s).
Improvement ℹ️ Insufficient signal risk: low · value: insufficient-signal · LLM: moderate
Linked issue satisfaction

Addressed
The diff moves wait_for_healthy into the shared common lib parameterized by service/timeout/log_prefix/compose_args, updates deploy-selfhost-image.sh to call the shared version with identical behavior, and adds a call to wait_for_healthy at the end of run_compose_deploy in deploy-selfhost-prebuilt.sh with the same SELFHOST_HEALTH_TIMEOUT_SECONDS override/default, matching the issue's requirements

Review context
  • Author: kai392
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 117 PR(s), 4 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Triage stale or unlinked PRs.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit e3a091e into JSONbored:main Jul 24, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

deploy-selfhost-prebuilt.sh reports complete without waiting for container health, unlike its sibling deploy-selfhost-image.sh

2 participants