From fe185597105021122fc35f648b2bee522bd962bc Mon Sep 17 00:00:00 2001 From: Kyle Lesinger Date: Fri, 14 Aug 2026 06:40:39 -0500 Subject: [PATCH 1/3] Add Algorithm Catalog Netlify URL to README; correct the base-dir gotcha MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The site is live at veda-algorithm-catalog.netlify.app, so replace the "Netlify site pending" placeholder with the real link, matching the other five dashboards. Also correct the Netlify note in docs/ALGORITHM_CATALOG.md. It predicted the first deploy would fail with "base directory not found" if run too early. That is not what happens when the base directory is left blank — the deploy goes GREEN and serves a 404. Netlify looks for netlify.toml at the repo root, finds none (deliberately — a root config's `base` would hijack all six sites), logs "Detected 0 framework(s) / No build steps found / Starting to deploy site from '/'", and uploads ~192 raw repo files with no index.html among them. Documents the tell to look for in the deploy log (`from '/'` vs `from 'algorithm-catalog/dist'`, ~192 files vs ~13), and the second way to get the same 404: filling in the publish directory as well, since `publish` in netlify.toml is relative to the base dir while the UI field is relative to the repo root, so setting both resolves algorithm-catalog/algorithm-catalog/dist. Notes that Netlify auto-installs Node 24 for this site, matching CI, so no NODE_VERSION variable is needed. --- README.md | 2 +- docs/ALGORITHM_CATALOG.md | 18 +++++++++++++++--- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 26e4ba9..31fdcde 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ diff. Intentionally open — it will grow to host other VEDA Actions experiments ### 🗓️ **Leave Dashboard:** https://veda-leave-dashboard.netlify.app ### 🧭 **DSE Hub:** https://veda-dse-hub.netlify.app ### 💵 **AWS Cost Calculator (Disasters Hub):** https://veda-aws-dashboard.netlify.app -### 🛰️ **Algorithm Catalog:** _(Netlify site pending — base dir `algorithm-catalog`)_ +### 🛰️ **Algorithm Catalog:** https://veda-algorithm-catalog.netlify.app ## Use this action in another repo The report is a **reusable composite action** (`action.yml`). Any repo can import it — just point diff --git a/docs/ALGORITHM_CATALOG.md b/docs/ALGORITHM_CATALOG.md index a14e322..411d500 100644 --- a/docs/ALGORITHM_CATALOG.md +++ b/docs/ALGORITHM_CATALOG.md @@ -79,9 +79,21 @@ Hazard colors and the `auto`/`manual` product badges carry meaning and are fixed - **Thumbnails are committed binaries** in `public/thumbs/` (600×400 PNG, ≤150 KB, sources and NASA credit lines in `public/thumbs/CREDITS.md`). `validate_data.py` fails if an algorithm's `thumb` has no file on disk, so a new algorithm needs its PNG in the same PR. -- Netlify: set **base directory = `algorithm-catalog`**; the first deploy fails if run before - `algorithm-catalog/` exists on the built branch ("base directory not found") — **deploy after the - merge lands on `main`**. +- **Netlify: base directory = `algorithm-catalog` is the ONLY field to set in the UI** — leave build + command and publish directory EMPTY so `algorithm-catalog/netlify.toml` is the single source + (`command = "npm run build"`, `publish = "dist"`, both resolved relative to the base dir). + Two ways this goes wrong, both ending in a 404 on `/`: + - **Base directory left blank** — this fails *silently*, not with "base directory not found". + Netlify looks for `netlify.toml` at the REPO ROOT, where there deliberately isn't one (a root + config's `base` would hijack all six sites), so it logs `Detected 0 framework(s)` / + `No build steps found` / `Starting to deploy site from '/'`, uploads ~192 raw repo files, and + reports a green deploy that serves no `index.html`. **Read the log for `from '/'`** — a correct + deploy says `from 'algorithm-catalog/dist'` and uploads ~13 files. + - **Publish directory also filled in** — `publish` in `netlify.toml` is relative to the base dir + (`dist`), but the UI field is relative to the repo root. Setting both yields + `algorithm-catalog/algorithm-catalog/dist`. + Deploy after the merge lands on `main`, so the base dir exists on the built branch. Netlify + auto-installs **Node 24** here, matching CI — no `NODE_VERSION` var needed. ## Run / test ```bash From 9f435800e9ac3ec62cf2d1f666d916d76398eed8 Mon Sep 17 00:00:00 2001 From: Kyle Lesinger Date: Fri, 14 Aug 2026 06:45:43 -0500 Subject: [PATCH 2/3] Correct the netlify.toml ignore-filter comment: base dir already skips builds MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The comment claimed that omitting an `ignore` filter means the site always builds on every push. That is wrong. Because the site has a BASE DIRECTORY set, Netlify applies its own implicit check and skips the build when nothing under algorithm-catalog/ changed — with or without an `ignore` filter. Confirmed against two PRs: #220 touched aws-pricing/ + cost-dashboard/ and only veda-aws-dashboard produced a preview (the other four canceled); #227 touched only README.md + docs/ and all six canceled. So "Deploy Preview canceled" on a docs-only PR is correct behavior, not a misconfiguration — a preview for this app requires the PR to change a file inside algorithm-catalog/. Documented in docs/ALGORITHM_CATALOG.md so the next person doesn't go hunting through Netlify settings, as we just did. The rationale for not setting an `ignore` filter is unchanged and still stands: the older dashboards' folder-scoped `git diff` ignore skipped PRODUCTION builds when Netlify passed equal CACHED_COMMIT_REF/COMMIT_REF, silently dropping merged code. --- algorithm-catalog/netlify.toml | 12 +++++++++--- docs/ALGORITHM_CATALOG.md | 7 +++++++ 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/algorithm-catalog/netlify.toml b/algorithm-catalog/netlify.toml index 5601ace..0c7fdd9 100644 --- a/algorithm-catalog/netlify.toml +++ b/algorithm-catalog/netlify.toml @@ -9,9 +9,15 @@ [build] command = "npm run build" publish = "dist" - # No `ignore` filter: always build & publish on a push so merged changes reliably go live. - # (A folder-scoped `git diff` ignore previously skipped PRODUCTION builds on the older - # dashboards whenever Netlify passed equal CACHED_COMMIT_REF/COMMIT_REF.) + # No explicit `ignore` filter. The older dashboards set one (a folder-scoped `git diff`) and it + # bit them: it skipped PRODUCTION builds whenever Netlify passed equal CACHED_COMMIT_REF and + # COMMIT_REF, silently dropping merged code. The three newest sites dropped it. + # + # Omitting it does NOT mean "build on every push". Because this site has a BASE DIRECTORY set, + # Netlify already applies its own implicit check and skips the build when a commit or PR touched + # nothing under algorithm-catalog/. That is why a README-only PR shows "Deploy Preview canceled" + # on all six sites — expected, not a misconfiguration. To get a deploy preview for this app, the + # PR has to change a file inside algorithm-catalog/. # SPA: serve index.html for any path (tabs are client-side via the URL hash). [[redirects]] diff --git a/docs/ALGORITHM_CATALOG.md b/docs/ALGORITHM_CATALOG.md index 411d500..127853a 100644 --- a/docs/ALGORITHM_CATALOG.md +++ b/docs/ALGORITHM_CATALOG.md @@ -94,6 +94,13 @@ Hazard colors and the `auto`/`manual` product badges carry meaning and are fixed `algorithm-catalog/algorithm-catalog/dist`. Deploy after the merge lands on `main`, so the base dir exists on the built branch. Netlify auto-installs **Node 24** here, matching CI — no `NODE_VERSION` var needed. +- **"Deploy Preview canceled" on a PR is usually correct, not broken.** Because the site has a base + directory, Netlify applies an implicit skip when the commit/PR touched nothing under + `algorithm-catalog/` — so a README- or docs-only PR cancels the preview on all six sites. This + happens whether or not `netlify.toml` sets an `ignore` filter, so omitting `ignore` does **not** + mean "build on every push". **A deploy preview for this app requires the PR to change a file + inside `algorithm-catalog/`.** Before debugging Netlify settings, check whether the PR actually + touched the base dir (`git diff --name-only main... -- algorithm-catalog/`). ## Run / test ```bash From 61ee323d4f9c7e892170ed04b0f748948ef85dab Mon Sep 17 00:00:00 2001 From: Kyle Lesinger Date: Fri, 14 Aug 2026 06:50:31 -0500 Subject: [PATCH 3/3] Sync docs: Algorithm Catalog rules, decisions, and the Netlify base-dir traps Records what this session established, split by audience so nothing is duplicated three times. .clinerules.md (must-follow rules + gotchas) - algorithm-catalog is the 6th site, no generator, cart -> prefilled PR - the standard lives in src/rules.ts and is MIRRORED in validate_data.py; rules_parity_test.py fails on drift, including a new exported constant - STAC event name: exactly 2 underscores, CamelCase hazards AND locations, serialize hazard ids not labels - products carry hazards (broad, discovery) + primaryHazards (sparse, auto-select); keep the subset relation and keep it sparse - .gitignore negation exception, and how to actually test it (git check-ignore --no-index -v; the plain form lies about tracked files) - the two Netlify traps below docs/DECISIONS.md (rationale) - why the catalog enforces a vocabulary upstream never did, and why the rule set is mirrored mechanically rather than merely documented as duplicated - why exactly-2-underscores is deliberately stricter than upstream, and why that direction is the safe one (everything we accept, DPS accepts) - why two hazard lists exist: discovery and defaults want opposite error tradeoffs, and auto-selecting from the broad list proposed 28 products - why negations beat `git add -f` for new apps - Netlify: base dir is load-bearing, and it earns free per-app build skipping AGENTS.md - Node 24 on CI and Netlify (auto-installed, no NODE_VERSION var) - the catalog's stdlib-only quick checks - rules.ts/validate_data.py mirror rule CLAUDE.md - the two Netlify traps, in the Gotchas section Both Netlify traps cost real time this session and neither is discoverable from the code: a blank base directory produces a GREEN deploy that 404s (no "base directory not found" error), and "Deploy Preview canceled" on a docs-only PR is correct behavior, not a misconfiguration. --- .clinerules.md | 28 ++++++++++++++++++++++++++- AGENTS.md | 8 ++++++++ CLAUDE.md | 8 ++++++++ docs/DECISIONS.md | 48 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 91 insertions(+), 1 deletion(-) diff --git a/.clinerules.md b/.clinerules.md index fbd8a07..4066861 100644 --- a/.clinerules.md +++ b/.clinerules.md @@ -20,6 +20,20 @@ Test station for reusable VEDA GitHub Actions + Projects-v2 seeds. See `CLAUDE.m prefilled new-file PR creating `dse-hub/data/changes/.json`, merged over the canonical arrays at load (upsert by id, newest `ts` wins). **Don't reintroduce one-file-per-record.** `dse-hub/scripts/compact.mjs` + its workflow fold changes in. Details in `docs/DSE_HUB.md`. +- **`algorithm-catalog/` (6th site) has NO generator** — hand-curated `data/{algorithms,events,hazards}.json`, + submissions stage in a cart → one prefilled new-file PR under `data/requests/`, folded into + `data/requests.json` by `scripts/compact.mjs`. Same shape as dse-hub. Details in `docs/ALGORITHM_CATALOG.md`. +- **The catalog's standard lives in ONE place and is mirrored, not re-derived.** `algorithm-catalog/src/rules.ts` + is the source; `scripts/validate_data.py` mirrors it 1:1; `scripts/rules_parity_test.py` fails if they + drift (it also fails on a **new exported const** the Python doesn't know). Change a rule → change both, + then run both scripts. CI runs them on every PR touching `algorithm-catalog/**`. +- **STAC event name = `YYYYMM_Hazard_Location`, EXACTLY 2 underscores** — `^[0-9]{4}(0[1-9]|1[0-2])_[^_]+_[^_]+$`. + This is **deliberately stricter than upstream** `dps/_validate.sh` (whose trailing `.+` lets the location + slot swallow extras). Multi-word hazards AND locations are CamelCase (`TropicalCyclone`, `GulfOfMexico`). + Serialize hazard **ids** (= the CamelCase tokens), never display labels. +- **Products carry two hazard lists.** `hazards` = broad, drives discovery/filtering; `primaryHazards` = + sparse subset, drives the Submit form's auto-selection. Keep `primaryHazards ⊆ hazards` (validator + enforces it) and keep it *sparse* — widening it dumps dozens of chips on the user. - **Theming = CSS vars + `data-theme` on ``** (`:root` light, `:root[data-theme="dark"]` override); default **dark**, persisted per-app in `localStorage` (`fte-/leave-/pr-theme`). Only chrome adapts — **never invert data colors** (person/status palettes, USWDS PR tags). For pr-dashboard, keep the dark @@ -36,4 +50,16 @@ Test station for reusable VEDA GitHub Actions + Projects-v2 seeds. See `CLAUDE.m - **Shared checkout:** `git commit` lands on whatever branch HEAD is on — verify the branch before committing when multiple sessions share the working tree. - **Root `.gitignore` ignores `*.json`** → `git add ` silently skips `package.json` / `tsconfig*.json` - / data JSON; **`git add -f`** them (dse-hub + leave-dashboard both rely on this). + / data JSON; **`git add -f`** them (dse-hub + leave-dashboard both rely on this). **Exception:** + `algorithm-catalog/` has explicit `!` negation lines, so a plain `git add` works there — prefer that + route for new apps. Verify with `git check-ignore --no-index -v ` (plain `git check-ignore` lies + about already-tracked files). +- **Netlify "Deploy Preview canceled" on a PR is usually CORRECT.** Sites with a base directory get an + implicit skip when the PR touched nothing under that dir — so a README/docs-only PR cancels the preview + on all six sites. This happens with or without an `ignore` filter in `netlify.toml`, so omitting `ignore` + does **not** mean "build on every push". Check `git diff --name-only main... -- /` before suspecting + Netlify settings. +- **A blank Netlify base directory fails SILENTLY, not with "base directory not found".** Netlify looks for + `netlify.toml` at the repo root, finds none (by design), logs `Detected 0 framework(s)` / + `Starting to deploy site from '/'`, uploads the raw repo, and reports a **green** deploy that 404s. The + tell is `from '/'` in the deploy log. Set base dir in the UI and leave build command + publish dir EMPTY. diff --git a/AGENTS.md b/AGENTS.md index 53e8eff..22d3e57 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,12 +7,20 @@ This repo is a test station for VEDA GitHub Actions + Netlify dashboards. Full c - **Python 3.12, standard library only** for all generators (no `pip install`, no `requests`, no `boto3`). AWS pricing is pulled from AWS's **public** Price List Bulk API — **no credentials**. - **Node 18+** for the dashboards (`cost-dashboard/`, `fte-dashboard/`, …): `cd && npm ci`. + CI and Netlify both run **Node 24** (Netlify auto-installs it; no `NODE_VERSION` var needed). - Quick checks: `python3 aws-pricing/test_generate.py` (offline golden test), `cd cost-dashboard && npm run typecheck && npm run build`. +- Algorithm Catalog checks (stdlib-only, no install): `python3 algorithm-catalog/scripts/validate_data.py` + and `python3 algorithm-catalog/scripts/rules_parity_test.py`, then + `cd algorithm-catalog && npm run typecheck && npm run build`. ## Conventions - Match surrounding style; keep generators stdlib-only and deterministic (they take `--now`). - Netlify "Pattern B": each dashboard owns its `/netlify.toml`; there is **no** root netlify.toml. + The **base directory** is set in the Netlify UI, and build command + publish directory are left EMPTY. + A blank base dir fails *silently* — green deploy, 404 site, `Starting to deploy site from '/'` in the log. +- **`algorithm-catalog/src/rules.ts` and `scripts/validate_data.py` are mirrors.** Edit one → edit the + other → run `scripts/rules_parity_test.py`. It fails on drift, including a new exported constant. --- diff --git a/CLAUDE.md b/CLAUDE.md index 23932f0..cbf1b98 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -141,6 +141,14 @@ sets the attribute before first paint + `//dist`). Separately, a base dir earns an + implicit build skip when a commit/PR touched nothing under it, so a **docs-only PR cancels the preview + on all six sites** — that is expected, and it happens with or without an `ignore` filter. - **STAC event names need EXACTLY 2 underscores, and the catalog is deliberately stricter than upstream here.** Upstream `dps/_validate.sh:29-36` ends its regex with `.+`, so the LOCATION slot swallows extra underscores: `202501_Tropical_Cyclone_CA` passes there but silently parses as diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index c2fb41b..cb04864 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -99,3 +99,51 @@ Netlify reads the `netlify.toml` inside that base, so sites are fully isolated ( site with routing). `pr-dashboard/` is static (no build) — it fetches the report from the `pr-finder/report` branch at runtime (mirroring how `fte-dashboard` fetches CSVs from `fte-report/all-pis`), with a bundled snapshot fallback. + +**The base directory is the load-bearing setting, and getting it wrong fails green.** With it blank, +Netlify looks for `netlify.toml` at the repo root — where we deliberately have none — so it runs no +build and publishes the repo root, producing a **successful** deploy that serves a 404. There is no +"base directory not found" error to catch it. Set base dir in the UI and leave build command and +publish directory EMPTY, so the in-dir `netlify.toml` is the single source (its `publish` is relative +to the base dir; the UI field is relative to the repo root — filling both yields `//dist`). + +**Setting a base directory also gives you free per-app build skipping**, which is why the three newest +sites carry no `ignore` filter. Netlify skips a build when the commit/PR touched nothing under the base +dir, so a docs-only PR legitimately shows "Deploy Preview canceled" on every site. The older dashboards' +explicit folder-scoped `git diff` ignore was worse than nothing: it also skipped **production** builds +whenever Netlify passed equal `CACHED_COMMIT_REF`/`COMMIT_REF`, silently dropping merged code. + +## Algorithm Catalog — enforce the standard the upstream repo never could + +`disasters-product-algorithms` has no hazard field at all: hazard is a free-text token in slot 2 of an +`activation_event` string, validated only by a shape regex. Nothing ever checked the *vocabulary*, which +is why `Fire`/`Wildfire`, `Quake`/`Earthquake` and `Storm`/`TropicalStorm`/`Hurricane` all coexist in +production data today. The catalog is the first place that vocabulary is written down, so it also has to +be the place that enforces it. + +**One rule set, three enforcement points, mechanically kept in sync.** `src/rules.ts` is the source of +truth; `scripts/validate_data.py` mirrors it line-for-line (stdlib-only, so CI needs no install step); +`scripts/rules_parity_test.py` compares the two as *source text* and fails on any drift — including a +newly exported constant the Python doesn't know about. Enforced live in the form (submit disabled while +any error stands), over the committed data, and in CI on every PR. A shared rule set that is merely +*documented* as duplicated will drift; this one cannot. + +**Deliberately stricter than upstream: exactly two underscores.** Upstream's regex ends `_.+$`, so the +location slot swallows extras — `202501_Tropical_Cyclone_CA` passes there while silently parsing as +hazard `Tropical`, which then becomes the GeoTIFF `HAZARD` tag. Ours ends `_[^_]+$`. This knowingly +rejects `202501_Flood_CA_extra`, an explicit upstream pass case +(`tests/integration/test_dps_validate.sh:41-48`). Diverging in this direction is safe: **every name we +accept, DPS accepts too** — we only refuse ones that mis-slot the hazard. The cost is that multi-word +*locations* also need CamelCase (`202512_Hurricane_GulfOfMexico`), so the error message quotes back what +the name would otherwise have been read as. + +**Two hazard lists per product, because discovery and defaults want opposite things.** `hazards` is +broad and drives the Algorithms-tab filter, where a false negative (missing a product that would have +helped) is worse than a false positive. `primaryHazards` is a sparse subset and drives the Submit form's +auto-selection, where the reverse is true — auto-selecting from the broad list proposed 28 products for +a single hazard, which is noise, not a starting point. The validator enforces `primaryHazards ⊆ hazards`. + +**`.gitignore` negations instead of `git add -f`.** dse-hub relies on contributors remembering to +force-add its JSON past the global `*.json` ignore. This app adds explicit `!` lines instead, so a plain +`git add algorithm-catalog/` is correct and no future contributor (or agent) can silently drop the data +from a commit. Prefer this route for new apps.