Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion .clinerules.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,20 @@ Test station for reusable VEDA GitHub Actions + Projects-v2 seeds. See `CLAUDE.m
prefilled new-file PR creating `dse-hub/data/changes/<ts>.json`, merged over the canonical arrays at load
(upsert by id, newest `ts` wins). **Don't reintroduce one-file-per-record.** `dse-hub/scripts/compact.mjs`
+ its workflow fold changes in. Details in `docs/DSE_HUB.md`.
- **`algorithm-catalog/` (6th site) has NO generator** — hand-curated `data/{algorithms,events,hazards}.json`,
submissions stage in a cart → one prefilled new-file PR under `data/requests/`, folded into
`data/requests.json` by `scripts/compact.mjs`. Same shape as dse-hub. Details in `docs/ALGORITHM_CATALOG.md`.
- **The catalog's standard lives in ONE place and is mirrored, not re-derived.** `algorithm-catalog/src/rules.ts`
is the source; `scripts/validate_data.py` mirrors it 1:1; `scripts/rules_parity_test.py` fails if they
drift (it also fails on a **new exported const** the Python doesn't know). Change a rule → change both,
then run both scripts. CI runs them on every PR touching `algorithm-catalog/**`.
- **STAC event name = `YYYYMM_Hazard_Location`, EXACTLY 2 underscores** — `^[0-9]{4}(0[1-9]|1[0-2])_[^_]+_[^_]+$`.
This is **deliberately stricter than upstream** `dps/_validate.sh` (whose trailing `.+` lets the location
slot swallow extras). Multi-word hazards AND locations are CamelCase (`TropicalCyclone`, `GulfOfMexico`).
Serialize hazard **ids** (= the CamelCase tokens), never display labels.
- **Products carry two hazard lists.** `hazards` = broad, drives discovery/filtering; `primaryHazards` =
sparse subset, drives the Submit form's auto-selection. Keep `primaryHazards ⊆ hazards` (validator
enforces it) and keep it *sparse* — widening it dumps dozens of chips on the user.
- **Theming = CSS vars + `data-theme` on `<html>`** (`:root` light, `:root[data-theme="dark"]` override);
default **dark**, persisted per-app in `localStorage` (`fte-/leave-/pr-theme`). Only chrome adapts —
**never invert data colors** (person/status palettes, USWDS PR tags). For pr-dashboard, keep the dark
Expand All @@ -36,4 +50,16 @@ Test station for reusable VEDA GitHub Actions + Projects-v2 seeds. See `CLAUDE.m
- **Shared checkout:** `git commit` lands on whatever branch HEAD is on — verify the branch before
committing when multiple sessions share the working tree.
- **Root `.gitignore` ignores `*.json`** → `git add <dir>` silently skips `package.json` / `tsconfig*.json`
/ data JSON; **`git add -f`** them (dse-hub + leave-dashboard both rely on this).
/ data JSON; **`git add -f`** them (dse-hub + leave-dashboard both rely on this). **Exception:**
`algorithm-catalog/` has explicit `!` negation lines, so a plain `git add` works there — prefer that
route for new apps. Verify with `git check-ignore --no-index -v <path>` (plain `git check-ignore` lies
about already-tracked files).
- **Netlify "Deploy Preview canceled" on a PR is usually CORRECT.** Sites with a base directory get an
implicit skip when the PR touched nothing under that dir — so a README/docs-only PR cancels the preview
on all six sites. This happens with or without an `ignore` filter in `netlify.toml`, so omitting `ignore`
does **not** mean "build on every push". Check `git diff --name-only main... -- <app>/` before suspecting
Netlify settings.
- **A blank Netlify base directory fails SILENTLY, not with "base directory not found".** Netlify looks for
`netlify.toml` at the repo root, finds none (by design), logs `Detected 0 framework(s)` /
`Starting to deploy site from '/'`, uploads the raw repo, and reports a **green** deploy that 404s. The
tell is `from '/'` in the deploy log. Set base dir in the UI and leave build command + publish dir EMPTY.
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,20 @@ This repo is a test station for VEDA GitHub Actions + Netlify dashboards. Full c
- **Python 3.12, standard library only** for all generators (no `pip install`, no `requests`, no
`boto3`). AWS pricing is pulled from AWS's **public** Price List Bulk API — **no credentials**.
- **Node 18+** for the dashboards (`cost-dashboard/`, `fte-dashboard/`, …): `cd <dir> && npm ci`.
CI and Netlify both run **Node 24** (Netlify auto-installs it; no `NODE_VERSION` var needed).
- Quick checks: `python3 aws-pricing/test_generate.py` (offline golden test),
`cd cost-dashboard && npm run typecheck && npm run build`.
- Algorithm Catalog checks (stdlib-only, no install): `python3 algorithm-catalog/scripts/validate_data.py`
and `python3 algorithm-catalog/scripts/rules_parity_test.py`, then
`cd algorithm-catalog && npm run typecheck && npm run build`.

## Conventions
- Match surrounding style; keep generators stdlib-only and deterministic (they take `--now`).
- Netlify "Pattern B": each dashboard owns its `<dir>/netlify.toml`; there is **no** root netlify.toml.
The **base directory** is set in the Netlify UI, and build command + publish directory are left EMPTY.
A blank base dir fails *silently* — green deploy, 404 site, `Starting to deploy site from '/'` in the log.
- **`algorithm-catalog/src/rules.ts` and `scripts/validate_data.py` are mirrors.** Edit one → edit the
other → run `scripts/rules_parity_test.py`. It fails on drift, including a new exported constant.

---

Expand Down
8 changes: 8 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,14 @@ sets the attribute before first paint + `<meta name="color-scheme" content="dark
negation lines** in the root `.gitignore`, so a plain `git add algorithm-catalog/` does pick its JSON
up — prefer that route for new apps. Check with `git check-ignore --no-index -v <path>` (plain
`git check-ignore` lies about already-tracked files).
- **Netlify base dir: blank fails GREEN, and "Deploy Preview canceled" is usually correct.** With no
base directory, Netlify reads no `netlify.toml` (there is none at the repo root, by design), logs
`Detected 0 framework(s)` / `Starting to deploy site from '/'`, uploads the raw repo, and reports a
**successful** deploy that 404s — there is no "base directory not found" error. Set base dir in the UI;
leave build command and publish dir EMPTY (`publish` in the toml is relative to the base dir, the UI
field to the repo root — filling both gives `<app>/<app>/dist`). Separately, a base dir earns an
implicit build skip when a commit/PR touched nothing under it, so a **docs-only PR cancels the preview
on all six sites** — that is expected, and it happens with or without an `ignore` filter.
- **STAC event names need EXACTLY 2 underscores, and the catalog is deliberately stricter than
upstream here.** Upstream `dps/_validate.sh:29-36` ends its regex with `.+`, so the LOCATION slot
swallows extra underscores: `202501_Tropical_Cyclone_CA` passes there but silently parses as
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ diff. Intentionally open — it will grow to host other VEDA Actions experiments
### 🗓️ **Leave Dashboard:** https://veda-leave-dashboard.netlify.app
### 🧭 **DSE Hub:** https://veda-dse-hub.netlify.app
### 💵 **AWS Cost Calculator (Disasters Hub):** https://veda-aws-dashboard.netlify.app
### 🛰️ **Algorithm Catalog:** _(Netlify site pending — base dir `algorithm-catalog`)_
### 🛰️ **Algorithm Catalog:** https://veda-algorithm-catalog.netlify.app

## Use this action in another repo
The report is a **reusable composite action** (`action.yml`). Any repo can import it — just point
Expand Down
12 changes: 9 additions & 3 deletions algorithm-catalog/netlify.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,15 @@
[build]
command = "npm run build"
publish = "dist"
# No `ignore` filter: always build & publish on a push so merged changes reliably go live.
# (A folder-scoped `git diff` ignore previously skipped PRODUCTION builds on the older
# dashboards whenever Netlify passed equal CACHED_COMMIT_REF/COMMIT_REF.)
# No explicit `ignore` filter. The older dashboards set one (a folder-scoped `git diff`) and it
# bit them: it skipped PRODUCTION builds whenever Netlify passed equal CACHED_COMMIT_REF and
# COMMIT_REF, silently dropping merged code. The three newest sites dropped it.
#
# Omitting it does NOT mean "build on every push". Because this site has a BASE DIRECTORY set,
# Netlify already applies its own implicit check and skips the build when a commit or PR touched
# nothing under algorithm-catalog/. That is why a README-only PR shows "Deploy Preview canceled"
# on all six sites — expected, not a misconfiguration. To get a deploy preview for this app, the
# PR has to change a file inside algorithm-catalog/.

# SPA: serve index.html for any path (tabs are client-side via the URL hash).
[[redirects]]
Expand Down
25 changes: 22 additions & 3 deletions docs/ALGORITHM_CATALOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,28 @@ Hazard colors and the `auto`/`manual` product badges carry meaning and are fixed
- **Thumbnails are committed binaries** in `public/thumbs/` (600×400 PNG, ≤150 KB, sources and NASA
credit lines in `public/thumbs/CREDITS.md`). `validate_data.py` fails if an algorithm's `thumb` has
no file on disk, so a new algorithm needs its PNG in the same PR.
- Netlify: set **base directory = `algorithm-catalog`**; the first deploy fails if run before
`algorithm-catalog/` exists on the built branch ("base directory not found") — **deploy after the
merge lands on `main`**.
- **Netlify: base directory = `algorithm-catalog` is the ONLY field to set in the UI** — leave build
command and publish directory EMPTY so `algorithm-catalog/netlify.toml` is the single source
(`command = "npm run build"`, `publish = "dist"`, both resolved relative to the base dir).
Two ways this goes wrong, both ending in a 404 on `/`:
- **Base directory left blank** — this fails *silently*, not with "base directory not found".
Netlify looks for `netlify.toml` at the REPO ROOT, where there deliberately isn't one (a root
config's `base` would hijack all six sites), so it logs `Detected 0 framework(s)` /
`No build steps found` / `Starting to deploy site from '/'`, uploads ~192 raw repo files, and
reports a green deploy that serves no `index.html`. **Read the log for `from '/'`** — a correct
deploy says `from 'algorithm-catalog/dist'` and uploads ~13 files.
- **Publish directory also filled in** — `publish` in `netlify.toml` is relative to the base dir
(`dist`), but the UI field is relative to the repo root. Setting both yields
`algorithm-catalog/algorithm-catalog/dist`.
Deploy after the merge lands on `main`, so the base dir exists on the built branch. Netlify
auto-installs **Node 24** here, matching CI — no `NODE_VERSION` var needed.
- **"Deploy Preview canceled" on a PR is usually correct, not broken.** Because the site has a base
directory, Netlify applies an implicit skip when the commit/PR touched nothing under
`algorithm-catalog/` — so a README- or docs-only PR cancels the preview on all six sites. This
happens whether or not `netlify.toml` sets an `ignore` filter, so omitting `ignore` does **not**
mean "build on every push". **A deploy preview for this app requires the PR to change a file
inside `algorithm-catalog/`.** Before debugging Netlify settings, check whether the PR actually
touched the base dir (`git diff --name-only main... -- algorithm-catalog/`).

## Run / test
```bash
Expand Down
48 changes: 48 additions & 0 deletions docs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,51 @@ Netlify reads the `netlify.toml` inside that base, so sites are fully isolated (
site with routing). `pr-dashboard/` is static (no build) — it fetches the report from the
`pr-finder/report` branch at runtime (mirroring how `fte-dashboard` fetches CSVs from `fte-report/all-pis`),
with a bundled snapshot fallback.

**The base directory is the load-bearing setting, and getting it wrong fails green.** With it blank,
Netlify looks for `netlify.toml` at the repo root — where we deliberately have none — so it runs no
build and publishes the repo root, producing a **successful** deploy that serves a 404. There is no
"base directory not found" error to catch it. Set base dir in the UI and leave build command and
publish directory EMPTY, so the in-dir `netlify.toml` is the single source (its `publish` is relative
to the base dir; the UI field is relative to the repo root — filling both yields `<app>/<app>/dist`).

**Setting a base directory also gives you free per-app build skipping**, which is why the three newest
sites carry no `ignore` filter. Netlify skips a build when the commit/PR touched nothing under the base
dir, so a docs-only PR legitimately shows "Deploy Preview canceled" on every site. The older dashboards'
explicit folder-scoped `git diff` ignore was worse than nothing: it also skipped **production** builds
whenever Netlify passed equal `CACHED_COMMIT_REF`/`COMMIT_REF`, silently dropping merged code.

## Algorithm Catalog — enforce the standard the upstream repo never could

`disasters-product-algorithms` has no hazard field at all: hazard is a free-text token in slot 2 of an
`activation_event` string, validated only by a shape regex. Nothing ever checked the *vocabulary*, which
is why `Fire`/`Wildfire`, `Quake`/`Earthquake` and `Storm`/`TropicalStorm`/`Hurricane` all coexist in
production data today. The catalog is the first place that vocabulary is written down, so it also has to
be the place that enforces it.

**One rule set, three enforcement points, mechanically kept in sync.** `src/rules.ts` is the source of
truth; `scripts/validate_data.py` mirrors it line-for-line (stdlib-only, so CI needs no install step);
`scripts/rules_parity_test.py` compares the two as *source text* and fails on any drift — including a
newly exported constant the Python doesn't know about. Enforced live in the form (submit disabled while
any error stands), over the committed data, and in CI on every PR. A shared rule set that is merely
*documented* as duplicated will drift; this one cannot.

**Deliberately stricter than upstream: exactly two underscores.** Upstream's regex ends `_.+$`, so the
location slot swallows extras — `202501_Tropical_Cyclone_CA` passes there while silently parsing as
hazard `Tropical`, which then becomes the GeoTIFF `HAZARD` tag. Ours ends `_[^_]+$`. This knowingly
rejects `202501_Flood_CA_extra`, an explicit upstream pass case
(`tests/integration/test_dps_validate.sh:41-48`). Diverging in this direction is safe: **every name we
accept, DPS accepts too** — we only refuse ones that mis-slot the hazard. The cost is that multi-word
*locations* also need CamelCase (`202512_Hurricane_GulfOfMexico`), so the error message quotes back what
the name would otherwise have been read as.

**Two hazard lists per product, because discovery and defaults want opposite things.** `hazards` is
broad and drives the Algorithms-tab filter, where a false negative (missing a product that would have
helped) is worse than a false positive. `primaryHazards` is a sparse subset and drives the Submit form's
auto-selection, where the reverse is true — auto-selecting from the broad list proposed 28 products for
a single hazard, which is noise, not a starting point. The validator enforces `primaryHazards ⊆ hazards`.

**`.gitignore` negations instead of `git add -f`.** dse-hub relies on contributors remembering to
force-add its JSON past the global `*.json` ignore. This app adds explicit `!` lines instead, so a plain
`git add algorithm-catalog/` is correct and no future contributor (or agent) can silently drop the data
from a commit. Prefer this route for new apps.
Loading