feat: daily external port scan of spore

.github/workflows/port-scan.yml

@@ -0,0 +1,76 @@
+name: Port scan
+
+# Scans every TCP port on spore's public address from a GitHub runner,
+# outside every network spore trusts, and alerts if one is open that
+# spore's firewall on main doesn't allow: an undeployed or reverted change,
+# a rule outside networking.firewall, or something at CloudCone.
+on:
+  schedule:
+  - cron: "17 13 * * *"
+  workflow_dispatch:
+  pull_request:
+    paths:
+    - .github/workflows/port-scan.yml
+
+permissions:
+  contents: read
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+    - uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
+      with:
+        github_access_token: ${{ secrets.GITHUB_TOKEN }}
+    - name: Read spore's address and allowed ports
+      id: expected
+      run: |
+        address=$(nix eval --raw .#nixosConfigurations.spore.config.networking.interfaces.eth0.ipv4.addresses \
+          --apply 'as: (builtins.head as).address')
+        ports=$(nix eval --raw .#nixosConfigurations.spore.config.networking.firewall --apply 'fw:
+          toString (fw.allowedTCPPorts
+            ++ builtins.concatMap (r: builtins.genList (i: r.from + i) (r.to - r.from + 1)) fw.allowedTCPPortRanges)')
+        echo "address=$address" >> "$GITHUB_OUTPUT"
+        echo "ports=$ports" >> "$GITHUB_OUTPUT"
+    - name: Scan
+      id: scan
+      env:
+        ADDRESS: ${{ steps.expected.outputs.address }}
+        EXPECTED: ${{ steps.expected.outputs.ports }}
+      run: |
+        nix profile install --inputs-from . nixpkgs#nmap
+        # Connect scan: no root needed. Ports spore drops time out, so
+        # --min-rate keeps the full range to a few minutes.
+        nmap -sT -Pn -p- -T4 --min-rate 1000 --open -oG scan.txt "$ADDRESS" >/dev/null
+        open=$(grep -oE '[0-9]+/open/tcp' scan.txt | cut -d/ -f1 | sort -n | xargs)
+        unexpected=$(comm -23 <(tr ' ' '\n' <<<"$open" | sort) <(tr ' ' '\n' <<<"$EXPECTED" | sort) | sort -n | xargs)
+        echo "Open: ${open:-none}"
+        echo "Allowed: $EXPECTED"
+
+        if [ -z "$open" ]; then
+          # spore's down or the scan is being blocked; either way it saw nothing
+          echo "title=Port scan of spore found no open ports" >> "$GITHUB_OUTPUT"
+          echo "::error::No open ports on $ADDRESS, not even 443"
+          exit 1
+        fi
+        if [ -n "$unexpected" ]; then
+          echo "title=Port scan found ports open on spore that its firewall doesn't allow" >> "$GITHUB_OUTPUT"
+          {
+            echo "fields<<EOF"
+            echo "Unexpected: $unexpected"
+            echo "Allowed: $EXPECTED"
+            echo "EOF"
+          } >> "$GITHUB_OUTPUT"
+          echo "::error::Open on $ADDRESS but not allowed by spore's firewall: $unexpected"
+          exit 1
+        fi
+    - uses: stackptr/actions/slack-notify@8f86bf2d6b69d2225bee5726b080fc6ce1066bd9 # v1.0.0
+      if: failure()
+      with:
+        token: ${{ secrets.SLACK_BOT_TOKEN }}
+        channel: "#alerts"
+        status: failure
+        title: ${{ steps.scan.outputs.title || 'Port scan of spore failed' }}
+        fields: ${{ steps.scan.outputs.fields }}
+        dry-run: ${{ github.event_name == 'pull_request' }}

CLAUDE.md

@@ -90,6 +90,7 @@ Grafana, Loki and Prometheus on glyph. Use them before `journalctl` or SSH when
 - **Loki** (glyph:3100): journald from glyph, spore and zeta via Alloy, oMLX's log from Stroma, plus Claude Code events over OTLP; 30 days.
 - **Prometheus** (glyph:9099): scrapes glyph, spore, zeta and Stroma (node_exporter, mactop, json_exporter for oMLX, Alloy; `hosts/Stroma/monitoring.nix`), plus OTLP pushes; 90 days.
 - **CI failures**: the `notify` job in `.github/workflows/ci.yml` posts to #alerts as Hypha through `stackptr/actions/slack-notify` (pinned by SHA; needs the `SLACK_BOT_TOKEN` repo secret and `permissions: actions: read`). Test a change to it with `dry-run: true`.
+- **Port scan**: `.github/workflows/port-scan.yml` scans every TCP port on spore's public address daily from a GitHub runner and posts to #alerts if one is open that spore's firewall on main doesn't allow, or if nothing is open at all. The expected list comes from evaluating spore's config, so there's nothing to update by hand. Home has no public services to scan beyond glyph's Transmission forward.
 - **Alert rules**: `hosts/glyph/services/grafana-alerts.nix` (folder "Alerts", routed to Slack). Add rules there, not in the UI.
 - **Dashboards**: JSON in `hosts/glyph/services/dashboards/`: Node, ZFS, Log Explorer, PostgreSQL, Disk Health (SMART), Systemd Units, nginx, Apple Silicon (Stroma).
 - **Gatus** (zeta:8080, `status.zx.dev`): out-of-band watchdog in `hosts/zeta/services/gatus.nix` that posts to Slack itself, so it alerts when glyph or Grafana is down. Every minute it checks glyph (reachability, Postgres, Prometheus freshness, Loki ingestion, Grafana through spore), spore, and the public sites through their `*.zx.dev` URLs; the `zx.dev` cert hourly. Hourly, `tailscale-key-expiry.service` asks the Tailscale API (OAuth client in `gatus-env.age`) for node keys expiring within 14 days or servers with expiry enabled, and pushes the result to Gatus's `tailscale/key-expiry` endpoint, which also fires if no result arrives for 3h.