Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Ground rules

- One binary, one database file, zero cgo. Dependencies need justification.
- Honest history is the invariant: missing telemetry is `null`, never
- Exact history is the invariant: missing telemetry is `null`, never
interpolated. Any change that fabricates uptime will be rejected.
- Headless-first: every mutation must work via the REST API. The embedded
UI consumes the API; it never bypasses it.
Expand Down
283 changes: 242 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,22 @@ file, no accounts, no SaaS.

- **Probe anything HTTP** — per-service interval, timeout, expected statuses,
headers, body matching, self-signed TLS opt-in.
- **Honest history** — every probe stored; days without data report `null`,
- **Exact history** — every probe stored; days without data report `null`,
never fake green; flap-tolerant day rollups.
- **Incident log included** — open, narrate and resolve incidents via API.
- **Status UI in the box** — the binary serves its own status page with
90-day bars at `/`; no second deployment.
- **Deploy-ready** — bearer auth, rate limits, Prometheus metrics, readiness
probe, versioned API with an embedded OpenAPI contract.

## Quick start

Generate a config, check it, run it:

```sh
cp config.example.yaml config.yaml # point it at your endpoints
go run ./cmd/epmon -config config.yaml
epmon init # interactive walkthrough → epmon.yaml
epmon validate --config epmon.yaml # parses + validates, no side effects
epmon run --config epmon.yaml # probe + serve
```

```sh
Expand All @@ -42,6 +47,9 @@ $ curl -s localhost:8080/api/v1/status | jq .
}
```

Then open `http://localhost:8080/` — the status page with 90-day bars,
served by the same binary. Details in [Status UI](#status-ui).

With Docker:

```sh
Expand All @@ -59,36 +67,58 @@ DATE=$(date -u +%FT%TZ) docker compose up --build -d
docker exec epmon /epmon version # epmon <version> (commit <sha>, built <date>, ...)
```

Interactive API reference lives at `/docs` once it's running; the raw
contract at `/api/v1/openapi.yaml` (or `.json`).
## Installation

## A full round trip

Probes run on their own — here's the human side:
Pick one:

```sh
# something breaks: open an incident (needs an API key, see below)
curl -s -X POST localhost:8080/api/v1/incidents \
-H "Authorization: Bearer $EPMON_API_KEY" \
-d '{"service_id":"api","title":"Elevated latency","severity":"minor"}' | jq .
# {"id":1,...,"state":"investigating","updates":[]}
# from source (Go 1.25+)
go install github.com/epmon-dev/epmon/cmd/epmon@v0.1.0

# narrate as you work
curl -s -X POST localhost:8080/api/v1/incidents/1/updates \
-H "Authorization: Bearer $EPMON_API_KEY" \
-d '{"text":"Slow query found, index added."}' -o /dev/null -w "%{http_code}\n"
# 201
# release binaries (linux/amd64+arm64, darwin/arm64, windows/amd64)
# → https://github.com/epmon-dev/epmon/releases (stamped, no build needed)

# fixed: resolve it
curl -s -X PATCH localhost:8080/api/v1/incidents/1 \
-H "Authorization: Bearer $EPMON_API_KEY" \
-d '{"state":"resolved"}' | jq .state
# "resolved"
# docker
docker compose up --build -d # see VERSION/COMMIT/DATE above for stamping
```

Or `make build` in a checkout — same flags the Dockerfile uses
(`make help` lists every target: `test`, `cross`, `lint`, `check-docs`…).

## CLI reference

# history for the status page: buckets plus uptime over the window
curl -s "localhost:8080/api/v1/services/api/history?days=90" | jq '{uptime_pct, days}'
# {"uptime_pct": 99.97, "days": 90}
```
epmon run [--config PATH] probe + serve (default command)
epmon validate [--config PATH] parse + validate only; exit 0/1, no side effects
epmon healthcheck --endpoint URL GET <url>/healthz; exit 0 on 200, 4 otherwise
epmon version print version/commit/date; exit 0
epmon init [flags] scaffold a config (see below)
```

Exit codes: `0` ok, `1` config error, `2` storage error, `3` listen/bind
error, `4` failed `healthcheck`, `64` usage error.

### `epmon init`

Two modes, one code path — the output always passes `validate`:

```sh
# interactive: prompts with [defaults], empty takes them, Ctrl-C aborts
epmon init --output epmon.yaml

# scripted: same walk driven by flags (repeat --service)
epmon init --non-interactive --output epmon.yaml \
--server-listen 127.0.0.1:8080 \
--service 'api=https://api.example.com/healthz;interval=30s;expect=200,301' \
--service 'web=https://example.com'
```

`--service` takes `id=url[;key=value...]` with keys `name`, `interval`,
`timeout`, `expect`, `body_contains`. Expected statuses accept every form
the loader does (`200`, `2xx`, `200,301`). Header values are
interactive-only, and `$NAME` becomes `${NAME}` so secrets stay references,
never literal tokens. Existing files are never overwritten without
`--force`; nothing is written unless the document validates.

## Configuration

Expand All @@ -100,25 +130,34 @@ always use braces.

```yaml
server:
addr: ":8080"
listen: ":8080" # bind address; addr is the legacy alias
api_keys: ["${EPMON_API_KEY}"] # writes need Bearer; empty = off (dev only)
rate_limit_rpm: 120 # per client IP; 429 + Retry-After past it
rate_limit_rpm: 120 # per client IP; 429 + Retry-After past it
rate_limit_burst: 120
trust_proxy: false # true only behind a sanitizing proxy
cors_allowed_origins: [] # e.g. ["https://status.example.com"]
trust_proxy: false # true only behind a sanitizing proxy
cors_allowed_origins: [] # e.g. ["https://status.example.com"]
max_body_bytes: 1048576
status_page: {enabled: true} # the UI at /; false restores JSON 404s
# tls_cert: "/data/tls/cert.pem" # or terminate TLS at your proxy
# tls_key: "/data/tls/key.pem"

database:
driver: sqlite # adapter name; postgres later without touching callers
dsn: "epmon.db" # adapter connection string (":memory:" = ephemeral)
dsn: "epmon.db" # adapter connection string (":memory:" = ephemeral)
retention_days: 90

probes: # defaults; every field overridable per service
default_interval: 60s
default_timeout: 10s
failure_threshold: 1
concurrency: 64
max_body_bytes: 1048576

history:
timezone: UTC # IANA name; day buckets cut at its midnights

api:
max_page_size: 100 # incident-list page cap (10..1000)

services:
- id: website # required, unique; name defaults to id
Expand All @@ -133,17 +172,116 @@ services:
headers:
Authorization: "Bearer ${TOKEN}"
# insecure_skip_verify: true # only for boxes you own
# enabled: false # skip probing, keep serving history
```

A probe is **up** when it answers within `timeout` with a status in
`expect_status` (after redirects) and — if set — the body contains
`body_contains`. Anything else stores `up: false` with a short error
(`transport: …`, `status: got 500`, `body: …`).
Key rules, enforced at load (unknown keys are errors, with file:line):

- **Service identity:** `id` matches `^[a-z0-9][a-z0-9_-]{0,63}$`, unique;
at least one service is required. Renaming an id archives the old row
and starts fresh history — history moves only via explicit migration.
- **Budgets:** service `interval` 5s–24h, `timeout` ≥1s and strictly below
its interval; unset fields inherit the probe defaults; unset
`expect_status` means `[200]`.
- **Up means:** answered within `timeout`, status in `expect_status`
(after redirects), and — if set — body contains `body_contains`.
Anything else stores `up: false` with a short error (`transport: …`,
`status: got 500`, `body: …`).
- **`enabled: false`** stops probing that service; its history, incidents
and catalog entries keep serving (see [Run modes](#run-modes)).

Config location, in order: `--config <path>`, `EPMON_CONFIG`,
`./epmon.yaml`, `/etc/epmon/epmon.yaml`, then `config.yaml`.
Exit codes: `0` ok, `1` config error, `2` storage error, `3` listen/bind
error, `4` failed `healthcheck`, `64` usage error.

## Run modes

**Full (default).** `epmon run --config epmon.yaml` — probes, API, UI.
This is the mode everything else degrades from.

**Headless API.** The UI is the only optional surface:

```yaml
server:
listen: ":8080"
status_page: {enabled: false}
database:
driver: sqlite
dsn: "epmon.db"
retention_days: 90
services:
- id: website
url: https://example.com
```

Probes, API, metrics and `/healthz` behave exactly as before; `/`
and friends return the JSON 404 again. Use it behind a reverse proxy
that serves its own frontend, or when the binary is a pure data plane.

**API + history, no active probing.** Mark services `enabled: false`
(either in the file or by flipping them later): the scheduler skips
them, nothing new is recorded, and the API keeps serving the catalog,
history, incidents and last-known states. Useful for a retired
environment you still want readable, or a warm standby that must not
generate traffic until you flip it back.

**Ephemeral.** `dsn: ":memory:"` — full behavior, zero disk. Probes run,
API answers, restart wipes everything. Made for CI smoke tests and
`init` output trials:

```sh
epmon init --non-interactive --output /tmp/try.yaml --db-dsn ":memory:" \
--service 'demo=https://example.com;interval=15s'
epmon validate --config /tmp/try.yaml && epmon run --config /tmp/try.yaml
```

**Validate-only.** `epmon validate` in CI or pre-deploy hooks: parses,
substitutes env, enforces every rule above, touches no network or disk
beyond reading the file. Fails closed on unknown keys and unset
`${VAR}` (all missing vars reported at once, with file:line).

## Status UI

The binary serves its own status page — no second deployment:

| Path | What |
|---|---|
| `/` | redirects to the local core's services |
| `/local` | your services: state, latency, 90-day bars, incident feed |
| `/p/:slug`, `/s/:domain` | multi-tenant demo routes (preview/testing) |
| `/embed` | badge snippet docs; `/embed.js` is the 2KB badge script |

Bars are exact history rendered honestly: green days held up, red days
didn't, hatched days had no telemetry — never interpolated. Hover any
bar for the date, outcome and check count. Disable the whole surface
with `status_page.enabled: false` (see [Run modes](#run-modes)).

## A full round trip

Probes run on their own — here's the human side:

```sh
# something breaks: open an incident (needs an API key, see below)
curl -s -X POST localhost:8080/api/v1/incidents \
-H "Authorization: Bearer $EPMON_API_KEY" \
-d '{"service_id":"api","title":"Elevated latency","severity":"minor"}' | jq .
# {"id":1,...,"state":"investigating","updates":[]}

# narrate as you work
curl -s -X POST localhost:8080/api/v1/incidents/1/updates \
-H "Authorization: Bearer $EPMON_API_KEY" \
-d '{"text":"Slow query found, index added."}' -o /dev/null -w "%{http_code}\n"
# 201

# fixed: resolve it
curl -s -X PATCH localhost:8080/api/v1/incidents/1 \
-H "Authorization: Bearer $EPMON_API_KEY" \
-d '{"state":"resolved"}' | jq .state
# "resolved"

# history for the status page: buckets plus uptime over the window
curl -s "localhost:8080/api/v1/services/api/history?days=90" | jq '{uptime_pct, days}'
# {"uptime_pct": 99.97, "days": 90}
```

## API (`/api/v1`)

Expand All @@ -155,10 +293,10 @@ Full contract with schemas: `/docs`, or raw at `/api/v1/openapi.yaml`.
| GET | `/api/v1/status` | `overall` (`operational`\|`partial_outage`\|`unknown`) + last check per service |
| GET | `/api/v1/services` | same per-service states as a list |
| GET | `/api/v1/services/{id}/history?days=90&limit=100` | daily buckets (1–365), `uptime_pct`, newest raw checks (1–500) |
| GET | `/api/v1/incidents[?state=][?service=]` | newest first, update threads embedded |
| GET | `/api/v1/incidents?state=&service=&limit=&offset=` | newest first, update threads embedded; `state` rejects typos with 400 |
| POST | `/api/v1/incidents` | `{service_id?, title, severity?}` → 201 + `Location` |
| GET | `/api/v1/incidents/{id}` | single incident with updates |
| PATCH | `/api/v1/incidents/{id}` | `{title?, severity?, state?}` |
| PATCH | `/api/v1/incidents/{id}` | `{title?, severity?, state?}` (forward-only states) |
| POST | `/api/v1/incidents/{id}/updates` | `{text}` → 201 + `Location` |

Conventions: single resources are bare objects, collections are
Expand Down Expand Up @@ -198,6 +336,12 @@ service at one probe/minute over 90 days.
`sqlite3 /data/epmon.db ".backup '/backup/epmon-$(date -u +%FT%TZ).db'"`
(then `PRAGMA integrity_check` on restores), or mount the volume into
your backup job and back up from there.
- **Upgrades:** stop the binary, replace it, start it. Schema migrations
are forward-only and run at boot; downgrading across a migration
boundary refuses to start — read the release notes first.
- **Config changes:** edit the file, then `epmon validate` before
reloading the process. Unknown keys fail closed so typos can't
silently disable a probe.

## Metrics (`/metrics`, Prometheus text format)

Expand All @@ -217,6 +361,58 @@ won't be renamed.
| `epmon_config_reload_total{result="ok"\|"error"}` | counter | Reserved: config reload outcomes (no reload path exists yet). |
| `epmon_uptime_seconds`, `epmon_build_info` | gauge | Process age and build identity (`version`/`commit` from release ldflags). |

## Troubleshooting

**`bind: address already in use` (exit 3).** Something owns the port.
Point `server.listen` at a free one (`127.0.0.1:18082` for local trials)
— the binary exits instead of hanging, by design.

**`validate` fails on `${VAR}`.** The variable isn't set where you run
it. Export it or use `${VAR:-fallback}`. All missing vars are reported
at once, with file:line — fix them all in one pass.

**A service never leaves `unknown`.** `up: null` means zero checks have
landed: either the scheduler hasn't ticked yet (first probe fires at
boot, so wait one interval), the service is `enabled: false`, or every
probe is erroring before evaluation (check the process log for the
`transport:` reason).

**History shows `up: null` for past days.** That's the design, not data
loss: days before the first recorded check report null. Backfilling
would be fabrication.

**429s from your own monitoring.** You're polling faster than
`rate_limit_rpm` allows. Raise the budget or slow the poller; reads and
the UI share the same bucket.

**Timezone day boundaries look off.** Buckets cut at `history.timezone`
midnights (default UTC), not the viewer's. Set the IANA name you
operate in.

## FAQ

**Can it serve the API without probing?** Yes — set `enabled: false`
on every service ([Run modes](#run-modes)): no traffic generated,
catalog/history/incidents keep serving. There is no separate
API-only binary, by design: one binary, fewer moving parts.

**Why SQLite?** One file, zero cgo, online backup, and far more
write headroom than 1,000 services at 5s intervals need. A second
adapter plugs in behind `store.Store` without touching callers.

**Why is a zero-service config invalid?** A monitor watching nothing is
a scripting bug or a typo away from silence. `epmon init` won't produce
one either.

**How do I rename a service without losing history?** You don't get it
for free: the old id archives, the new one starts fresh. That's the
exact-history invariant — renames are visible by design. Declare the
rename with `aliases` so the intent is recorded alongside the service.

**Where do secrets go?** In the environment, referenced as `${VAR}`.
`epmon init` writes `$NAME` as `${NAME}` automatically and never
persists a literal token.

## Development

```
Expand All @@ -226,12 +422,17 @@ internal/store/ ports: domain types + Check/Incident/Service interfaces
internal/store/sqlite/ SQLite adapter — the only package owning SQL (+tests)
internal/prober/ one HTTP check (+tests)
internal/scheduler/ per-service tick loops, graceful shutdown
internal/api/ handlers, middleware, embedded OpenAPI (+tests)
internal/api/ handlers, middleware, embedded OpenAPI + status UI (+tests)
internal/api/webui/ vendored status SPA (refresh with `make web`)
internal/metrics/ Prometheus exposition (+tests)
scripts/ check-docs.py (README validation), refresh-webui.sh
```

```sh
go build ./... && go vet ./... && go test ./...
make check # fmt + vet + race tests (what CI runs)
make check-docs # every README yaml block must validate
make build # stamped local binary
make help # all 19 targets
```

All green with no external services (tests use `httptest` and temp-file
Expand Down

Large diffs are not rendered by default.

Loading
Loading