diff --git a/.clinerules.md b/.clinerules.md index fbd8a07..4066861 100644 --- a/.clinerules.md +++ b/.clinerules.md @@ -20,6 +20,20 @@ Test station for reusable VEDA GitHub Actions + Projects-v2 seeds. See `CLAUDE.m prefilled new-file PR creating `dse-hub/data/changes/.json`, merged over the canonical arrays at load (upsert by id, newest `ts` wins). **Don't reintroduce one-file-per-record.** `dse-hub/scripts/compact.mjs` + its workflow fold changes in. Details in `docs/DSE_HUB.md`. +- **`algorithm-catalog/` (6th site) has NO generator** — hand-curated `data/{algorithms,events,hazards}.json`, + submissions stage in a cart → one prefilled new-file PR under `data/requests/`, folded into + `data/requests.json` by `scripts/compact.mjs`. Same shape as dse-hub. Details in `docs/ALGORITHM_CATALOG.md`. +- **The catalog's standard lives in ONE place and is mirrored, not re-derived.** `algorithm-catalog/src/rules.ts` + is the source; `scripts/validate_data.py` mirrors it 1:1; `scripts/rules_parity_test.py` fails if they + drift (it also fails on a **new exported const** the Python doesn't know). Change a rule → change both, + then run both scripts. CI runs them on every PR touching `algorithm-catalog/**`. +- **STAC event name = `YYYYMM_Hazard_Location`, EXACTLY 2 underscores** — `^[0-9]{4}(0[1-9]|1[0-2])_[^_]+_[^_]+$`. + This is **deliberately stricter than upstream** `dps/_validate.sh` (whose trailing `.+` lets the location + slot swallow extras). Multi-word hazards AND locations are CamelCase (`TropicalCyclone`, `GulfOfMexico`). + Serialize hazard **ids** (= the CamelCase tokens), never display labels. +- **Products carry two hazard lists.** `hazards` = broad, drives discovery/filtering; `primaryHazards` = + sparse subset, drives the Submit form's auto-selection. Keep `primaryHazards ⊆ hazards` (validator + enforces it) and keep it *sparse* — widening it dumps dozens of chips on the user. - **Theming = CSS vars + `data-theme` on ``** (`:root` light, `:root[data-theme="dark"]` override); default **dark**, persisted per-app in `localStorage` (`fte-/leave-/pr-theme`). Only chrome adapts — **never invert data colors** (person/status palettes, USWDS PR tags). For pr-dashboard, keep the dark @@ -36,4 +50,16 @@ Test station for reusable VEDA GitHub Actions + Projects-v2 seeds. See `CLAUDE.m - **Shared checkout:** `git commit` lands on whatever branch HEAD is on — verify the branch before committing when multiple sessions share the working tree. - **Root `.gitignore` ignores `*.json`** → `git add ` silently skips `package.json` / `tsconfig*.json` - / data JSON; **`git add -f`** them (dse-hub + leave-dashboard both rely on this). + / data JSON; **`git add -f`** them (dse-hub + leave-dashboard both rely on this). **Exception:** + `algorithm-catalog/` has explicit `!` negation lines, so a plain `git add` works there — prefer that + route for new apps. Verify with `git check-ignore --no-index -v ` (plain `git check-ignore` lies + about already-tracked files). +- **Netlify "Deploy Preview canceled" on a PR is usually CORRECT.** Sites with a base directory get an + implicit skip when the PR touched nothing under that dir — so a README/docs-only PR cancels the preview + on all six sites. This happens with or without an `ignore` filter in `netlify.toml`, so omitting `ignore` + does **not** mean "build on every push". Check `git diff --name-only main... -- /` before suspecting + Netlify settings. +- **A blank Netlify base directory fails SILENTLY, not with "base directory not found".** Netlify looks for + `netlify.toml` at the repo root, finds none (by design), logs `Detected 0 framework(s)` / + `Starting to deploy site from '/'`, uploads the raw repo, and reports a **green** deploy that 404s. The + tell is `from '/'` in the deploy log. Set base dir in the UI and leave build command + publish dir EMPTY. diff --git a/AGENTS.md b/AGENTS.md index 53e8eff..22d3e57 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,12 +7,20 @@ This repo is a test station for VEDA GitHub Actions + Netlify dashboards. Full c - **Python 3.12, standard library only** for all generators (no `pip install`, no `requests`, no `boto3`). AWS pricing is pulled from AWS's **public** Price List Bulk API — **no credentials**. - **Node 18+** for the dashboards (`cost-dashboard/`, `fte-dashboard/`, …): `cd && npm ci`. + CI and Netlify both run **Node 24** (Netlify auto-installs it; no `NODE_VERSION` var needed). - Quick checks: `python3 aws-pricing/test_generate.py` (offline golden test), `cd cost-dashboard && npm run typecheck && npm run build`. +- Algorithm Catalog checks (stdlib-only, no install): `python3 algorithm-catalog/scripts/validate_data.py` + and `python3 algorithm-catalog/scripts/rules_parity_test.py`, then + `cd algorithm-catalog && npm run typecheck && npm run build`. ## Conventions - Match surrounding style; keep generators stdlib-only and deterministic (they take `--now`). - Netlify "Pattern B": each dashboard owns its `/netlify.toml`; there is **no** root netlify.toml. + The **base directory** is set in the Netlify UI, and build command + publish directory are left EMPTY. + A blank base dir fails *silently* — green deploy, 404 site, `Starting to deploy site from '/'` in the log. +- **`algorithm-catalog/src/rules.ts` and `scripts/validate_data.py` are mirrors.** Edit one → edit the + other → run `scripts/rules_parity_test.py`. It fails on drift, including a new exported constant. --- diff --git a/CLAUDE.md b/CLAUDE.md index 23932f0..cbf1b98 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -141,6 +141,14 @@ sets the attribute before first paint + `//dist`). Separately, a base dir earns an + implicit build skip when a commit/PR touched nothing under it, so a **docs-only PR cancels the preview + on all six sites** — that is expected, and it happens with or without an `ignore` filter. - **STAC event names need EXACTLY 2 underscores, and the catalog is deliberately stricter than upstream here.** Upstream `dps/_validate.sh:29-36` ends its regex with `.+`, so the LOCATION slot swallows extra underscores: `202501_Tropical_Cyclone_CA` passes there but silently parses as diff --git a/README.md b/README.md index 26e4ba9..31fdcde 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ diff. Intentionally open — it will grow to host other VEDA Actions experiments ### 🗓️ **Leave Dashboard:** https://veda-leave-dashboard.netlify.app ### 🧭 **DSE Hub:** https://veda-dse-hub.netlify.app ### 💵 **AWS Cost Calculator (Disasters Hub):** https://veda-aws-dashboard.netlify.app -### 🛰️ **Algorithm Catalog:** _(Netlify site pending — base dir `algorithm-catalog`)_ +### 🛰️ **Algorithm Catalog:** https://veda-algorithm-catalog.netlify.app ## Use this action in another repo The report is a **reusable composite action** (`action.yml`). Any repo can import it — just point diff --git a/algorithm-catalog/netlify.toml b/algorithm-catalog/netlify.toml index 5601ace..0c7fdd9 100644 --- a/algorithm-catalog/netlify.toml +++ b/algorithm-catalog/netlify.toml @@ -9,9 +9,15 @@ [build] command = "npm run build" publish = "dist" - # No `ignore` filter: always build & publish on a push so merged changes reliably go live. - # (A folder-scoped `git diff` ignore previously skipped PRODUCTION builds on the older - # dashboards whenever Netlify passed equal CACHED_COMMIT_REF/COMMIT_REF.) + # No explicit `ignore` filter. The older dashboards set one (a folder-scoped `git diff`) and it + # bit them: it skipped PRODUCTION builds whenever Netlify passed equal CACHED_COMMIT_REF and + # COMMIT_REF, silently dropping merged code. The three newest sites dropped it. + # + # Omitting it does NOT mean "build on every push". Because this site has a BASE DIRECTORY set, + # Netlify already applies its own implicit check and skips the build when a commit or PR touched + # nothing under algorithm-catalog/. That is why a README-only PR shows "Deploy Preview canceled" + # on all six sites — expected, not a misconfiguration. To get a deploy preview for this app, the + # PR has to change a file inside algorithm-catalog/. # SPA: serve index.html for any path (tabs are client-side via the URL hash). [[redirects]] diff --git a/docs/ALGORITHM_CATALOG.md b/docs/ALGORITHM_CATALOG.md index a14e322..127853a 100644 --- a/docs/ALGORITHM_CATALOG.md +++ b/docs/ALGORITHM_CATALOG.md @@ -79,9 +79,28 @@ Hazard colors and the `auto`/`manual` product badges carry meaning and are fixed - **Thumbnails are committed binaries** in `public/thumbs/` (600×400 PNG, ≤150 KB, sources and NASA credit lines in `public/thumbs/CREDITS.md`). `validate_data.py` fails if an algorithm's `thumb` has no file on disk, so a new algorithm needs its PNG in the same PR. -- Netlify: set **base directory = `algorithm-catalog`**; the first deploy fails if run before - `algorithm-catalog/` exists on the built branch ("base directory not found") — **deploy after the - merge lands on `main`**. +- **Netlify: base directory = `algorithm-catalog` is the ONLY field to set in the UI** — leave build + command and publish directory EMPTY so `algorithm-catalog/netlify.toml` is the single source + (`command = "npm run build"`, `publish = "dist"`, both resolved relative to the base dir). + Two ways this goes wrong, both ending in a 404 on `/`: + - **Base directory left blank** — this fails *silently*, not with "base directory not found". + Netlify looks for `netlify.toml` at the REPO ROOT, where there deliberately isn't one (a root + config's `base` would hijack all six sites), so it logs `Detected 0 framework(s)` / + `No build steps found` / `Starting to deploy site from '/'`, uploads ~192 raw repo files, and + reports a green deploy that serves no `index.html`. **Read the log for `from '/'`** — a correct + deploy says `from 'algorithm-catalog/dist'` and uploads ~13 files. + - **Publish directory also filled in** — `publish` in `netlify.toml` is relative to the base dir + (`dist`), but the UI field is relative to the repo root. Setting both yields + `algorithm-catalog/algorithm-catalog/dist`. + Deploy after the merge lands on `main`, so the base dir exists on the built branch. Netlify + auto-installs **Node 24** here, matching CI — no `NODE_VERSION` var needed. +- **"Deploy Preview canceled" on a PR is usually correct, not broken.** Because the site has a base + directory, Netlify applies an implicit skip when the commit/PR touched nothing under + `algorithm-catalog/` — so a README- or docs-only PR cancels the preview on all six sites. This + happens whether or not `netlify.toml` sets an `ignore` filter, so omitting `ignore` does **not** + mean "build on every push". **A deploy preview for this app requires the PR to change a file + inside `algorithm-catalog/`.** Before debugging Netlify settings, check whether the PR actually + touched the base dir (`git diff --name-only main... -- algorithm-catalog/`). ## Run / test ```bash diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index c2fb41b..cb04864 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -99,3 +99,51 @@ Netlify reads the `netlify.toml` inside that base, so sites are fully isolated ( site with routing). `pr-dashboard/` is static (no build) — it fetches the report from the `pr-finder/report` branch at runtime (mirroring how `fte-dashboard` fetches CSVs from `fte-report/all-pis`), with a bundled snapshot fallback. + +**The base directory is the load-bearing setting, and getting it wrong fails green.** With it blank, +Netlify looks for `netlify.toml` at the repo root — where we deliberately have none — so it runs no +build and publishes the repo root, producing a **successful** deploy that serves a 404. There is no +"base directory not found" error to catch it. Set base dir in the UI and leave build command and +publish directory EMPTY, so the in-dir `netlify.toml` is the single source (its `publish` is relative +to the base dir; the UI field is relative to the repo root — filling both yields `//dist`). + +**Setting a base directory also gives you free per-app build skipping**, which is why the three newest +sites carry no `ignore` filter. Netlify skips a build when the commit/PR touched nothing under the base +dir, so a docs-only PR legitimately shows "Deploy Preview canceled" on every site. The older dashboards' +explicit folder-scoped `git diff` ignore was worse than nothing: it also skipped **production** builds +whenever Netlify passed equal `CACHED_COMMIT_REF`/`COMMIT_REF`, silently dropping merged code. + +## Algorithm Catalog — enforce the standard the upstream repo never could + +`disasters-product-algorithms` has no hazard field at all: hazard is a free-text token in slot 2 of an +`activation_event` string, validated only by a shape regex. Nothing ever checked the *vocabulary*, which +is why `Fire`/`Wildfire`, `Quake`/`Earthquake` and `Storm`/`TropicalStorm`/`Hurricane` all coexist in +production data today. The catalog is the first place that vocabulary is written down, so it also has to +be the place that enforces it. + +**One rule set, three enforcement points, mechanically kept in sync.** `src/rules.ts` is the source of +truth; `scripts/validate_data.py` mirrors it line-for-line (stdlib-only, so CI needs no install step); +`scripts/rules_parity_test.py` compares the two as *source text* and fails on any drift — including a +newly exported constant the Python doesn't know about. Enforced live in the form (submit disabled while +any error stands), over the committed data, and in CI on every PR. A shared rule set that is merely +*documented* as duplicated will drift; this one cannot. + +**Deliberately stricter than upstream: exactly two underscores.** Upstream's regex ends `_.+$`, so the +location slot swallows extras — `202501_Tropical_Cyclone_CA` passes there while silently parsing as +hazard `Tropical`, which then becomes the GeoTIFF `HAZARD` tag. Ours ends `_[^_]+$`. This knowingly +rejects `202501_Flood_CA_extra`, an explicit upstream pass case +(`tests/integration/test_dps_validate.sh:41-48`). Diverging in this direction is safe: **every name we +accept, DPS accepts too** — we only refuse ones that mis-slot the hazard. The cost is that multi-word +*locations* also need CamelCase (`202512_Hurricane_GulfOfMexico`), so the error message quotes back what +the name would otherwise have been read as. + +**Two hazard lists per product, because discovery and defaults want opposite things.** `hazards` is +broad and drives the Algorithms-tab filter, where a false negative (missing a product that would have +helped) is worse than a false positive. `primaryHazards` is a sparse subset and drives the Submit form's +auto-selection, where the reverse is true — auto-selecting from the broad list proposed 28 products for +a single hazard, which is noise, not a starting point. The validator enforces `primaryHazards ⊆ hazards`. + +**`.gitignore` negations instead of `git add -f`.** dse-hub relies on contributors remembering to +force-add its JSON past the global `*.json` ignore. This app adds explicit `!` lines instead, so a plain +`git add algorithm-catalog/` is correct and no future contributor (or agent) can silently drop the data +from a commit. Prefer this route for new apps.