fix: run hyperhealth migrations once, and never stamp on failure

[welshDog]

Follow-up to the finding flagged in #313. Two problems on one boot path.

1. Migrations ran three times per boot

The Dockerfile CMD runs alembic upgrade head, then execs uvicorn with --workers 2. Each worker runs the FastAPI lifespan, and lifespan also shelled out to alembic upgrade head. So the CMD migrated once and the two workers migrated again — concurrently, racing each other.

Visible in the logs on every boot:

{"event": "hyperhealth.startup", ...}          x2
{"event": "hyperhealth.migrations_completed"}  x2

Migrations belong in the CMD, before uvicorn forks. Removed the subprocess.run from lifespan (and the now-unused subprocess import).

2. || alembic stamp 001 could mark an unmigrated database as current

001 is head. alembic stamp 001 writes the version row and creates nothing else. Proved with alembic's offline SQL mode, zero risk:

alembic upgrade head --sql   ->  CREATE TABLE alert_policies
                                 CREATE TABLE self_heal_policies
                                 CREATE TABLE check_definitions
                                 CREATE TABLE check_results
                                 INSERT INTO alembic_version_hyperhealth VALUES ('001')

alembic stamp 001 --sql      ->  INSERT INTO alembic_version_hyperhealth VALUES ('001')

If upgrade head failed for any reason that left the DB reachable — a lock timeout, a conflicting object, a failing migration step — the fallback declared the schema current without building it, and every subsequent upgrade head became a no-op.

This isn't hypothetical: on 2026-06-24 the lifespan copy timed out after 30s on every boot and was swallowed at level: warning.

The CMD is now alembic upgrade head && exec uvicorn .... On failure the container exits. Compose has restart: unless-stopped and depends_on: postgres condition: service_healthy, so it retries against a live database rather than lying about the schema.

Verification (on the running stack, after rebuild)

	before	after
migrations_* app events per boot	2	0
alembic invocations per boot	3	1
hyperhealth.startup events	2	2 (unchanged — that's --workers 2)

• alembic_version_hyperhealth still 001; alembic heads → 001 (head)
• alert_policies, self_heal_policies, check_definitions, check_results all present
• hyperhealth-api healthy, hyperhealth-worker healthy, 0 restarts
• GET /health → {"status":"ok","service":"hyperhealth","version":"1.1.0"}

The version table is alembic_version_hyperhealth, dedicated in migrations/env.py, so none of this touches core's shared alembic_version.

Not verified

check_definitions and check_results are both 0 rows after the rebuild. I captured the version and table list beforehand but not the row counts, so I cannot prove they were 0 before. The change cannot delete rows — it removes a no-op upgrade head (the DB was already at head) and a fallback that only ever writes a version row. If those tables are meant to be seeded, Makefile:95 runs seed_checks.py.

--workers 2 is left as-is. It is why lifespan runs twice, which is expected and harmless now that lifespan no longer migrates.

How this was found

Chasing an AST-sweep hit at main.py:111 (subprocess.run inside async def lifespan). That turned out not to be a loop-blocking bug — uvicorn runs lifespan before accepting connections, so blocking there costs nothing. But reading the logs to confirm that surfaced this instead.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

@welshDog, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 8 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2c8b9b4d-ec4c-4173-a9de-8f77830d29a3

📥 Commits

Reviewing files that changed from the base of the PR and between 798a565 and b2dce0e.

📒 Files selected for processing (2)

• agents/hyperhealth/Dockerfile
• agents/hyperhealth/main.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/hyperhealth-migrations-once

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}