Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .workhorse/specs/seedling/healthchecks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
id: SDH
---

# Seedling healthchecks

On a host that runs the Seedling application orchestrator, `bestool tamanu doctor` gathers Seedling-specific health checks from the local Seedling daemon and folds them into the sweep alongside the host and Tamanu checks.
The Seedling checks share the registry, outcomes, grouping, rendering, and status wire format of every other check (see [DOC](../tamanu/doctor.md)); this spec covers only what is specific to Seedling: when they apply, where their data comes from, and what each reports.

## Seedling host context

The Seedling checks apply when the host runs Seedling: the daemon's data directory is configured in the environment and the daemon's control interface is present there.
When no Seedling is configured, every Seedling check skips, so the same doctor invocation runs safely on a host that carries none.

## Obtaining checks from the daemon

The doctor reads Seedling health from the daemon's local control interface, authenticating as a client the daemon trusts.
The daemon is the single source of truth for its own health: the doctor reports what the daemon returns rather than inspecting Seedling's containers or database itself.
When the daemon is reachable but cannot answer a given check, that check is reported as broken rather than failing, so a daemon-side fault is not mistaken for an unhealthy system.

## Checks

The doctor derives one check per Seedling subsystem the daemon reports on.

A reverse-proxy check reports whether the proxy that fronts application traffic is running, and fails when it is not.
A resolver check reports whether the Seedling DNS resolver is running, and fails when it is not.
An applications check reports the health of the Seedling-managed applications: it passes when every app is running (or none are deployed), and warns when some are not running.

Each check carries a one-line summary, and a reason whenever it does not pass, in line with [DOC](../tamanu/doctor.md).
78 changes: 77 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions crates/alertd/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ bestool-tamanu = { version = "0.16.1", path = "../tamanu", features = ["canopy-r
bytes = "1.9.0"
dirs = "6.0.0"
duct = "1.1.0"
# pkcs8 to parse the Seedling daemon's key read-only (see doctor/checks/seedling.rs)
ed25519-dalek = { version = "2.2.0", features = ["pkcs8"] }
futures = { workspace = true }
hickory-resolver = "0.26.1"
jiff = { version = "0.2.28", features = ["serde"] }
Expand All @@ -28,6 +30,7 @@ miette = { workspace = true }
prometheus = "0.14.0"
reqwest = { workspace = true }
rustls = "0.23.40"
seedling-protocol = { git = "https://github.com/beyondessential/seedling.git", tag = "v0.3.0" }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.145"
sysinfo.workspace = true
Expand Down
23 changes: 23 additions & 0 deletions crates/alertd/src/doctor/checks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ pub mod migrations;
pub mod patient_communication_errors;
pub mod pg_tuning;
pub mod report_errors;
pub mod seedling;
pub mod sync_facility_stale;
pub mod sync_lookup;
pub mod sync_restart_loop;
Expand All @@ -62,6 +63,9 @@ pub mod version_drift;
pub struct SweepContext {
pub tamanu: Option<CheckContext>,
pub http_client: reqwest::Client,
/// The local Seedling daemon's status, resolved once per sweep and shared
/// across the Seedling checks. `None` on hosts with no Seedling.
pub seedling: seedling::Probe,
}

/// Shared context handed to every Tamanu-dependent check.
Expand Down Expand Up @@ -276,6 +280,23 @@ pub fn all() -> Vec<CheckEntry> {
entry!("caddy_version", caddy_version, host),
entry!("caddy_certs", caddy_certs, host),
entry!("http_errors", http_errors, host),
// Seedling host/service probes: resolved once per sweep from the local
// Seedling daemon's OI (see SDH), and skip when the host runs no Seedling.
CheckEntry {
name: "seedling_proxy",
on_wire: true,
run: |ctx| Box::pin(seedling::proxy(ctx)),
},
CheckEntry {
name: "seedling_resolver",
on_wire: true,
run: |ctx| Box::pin(seedling::resolver(ctx)),
},
CheckEntry {
name: "seedling_apps",
on_wire: true,
run: |ctx| Box::pin(seedling::apps(ctx)),
},
entry!("tailscale", tailscale, host, off_wire),
entry!("tailscale_config", tailscale_config, host),
// Reports the host's LAN and best-guess WAN addresses as status facts
Expand Down Expand Up @@ -399,6 +420,7 @@ mod tests {
SweepContext {
tamanu: None,
http_client: reqwest::Client::new(),
seedling: None,
}
}

Expand Down Expand Up @@ -427,6 +449,7 @@ mod tests {
has_install: false,
}),
http_client: reqwest::Client::new(),
seedling: None,
}
}

Expand Down
Loading
Loading