Repository navigation
Environment: restore OISST and disclose SST fallback sources - #158
Conversation
|
Review finding — [P2] OISST refresh can fail around New Year within the allowed publication lag. Calendar-year chunks make December 31 a fixed non-final endpoint even when final OISST is only published through mid/late December. The new fatal historical-error handling then aborts the weekly refresh. ERDDAP requires coordinate values to be inside the published axis range (documentation). Fix prepared in Validation: lint passed; Svelte checks returned zero errors/warnings; 12 fetch tests and 107 application tests passed (two opt-in real-data tests skipped). Verified all 4,840 snapshot dates and all non-OISST columns are unchanged, with 4,828 restored OISST values. No other actionable findings from the diff review. No live full NOAA refresh or real-data recomputation was performed during this review. |
- /ocean: the coverage line moves from the page header into the SST chart's note, with formatted numbers and singular/plural wording. - AGENTS.md and docs/dev/OISST-154.md: upwell redirects ncdcOisst21Agg to coastwatch.pfeg.noaa.gov (only oisst_rclone is served by upwell itself); note that ERDDAP failures now stop the ocean job for Coral Reef Watch too. - erddap.test.mjs: drop a redundant re-import. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Reviewed against NOAA's live data and the real survey data (aggregates only). It's right, and I've pushed small fixes in aa459c5. What I checked
Why the stricter fetch matters. The first weekly refresh on 2 Nov failed:
With this PR, either ERDDAP failure stops the ocean job before it writes. The cost is that one outage leaves the whole ocean snapshot (waves too) unchanged that week and fails the workflow, as chlorophyll already does. I filed #159 so one failed job stops blocking the others and the weather job gets through a 429. Fixes pushed (aa459c5)
|
The site labelled its SST as OISST, but the committed OISST column was entirely empty and analyses used Coral Reef Watch. This restores 4,828 daily OISST values through 2026-10-19, fixes source credits, and shows live OISST/fallback/missing coverage on /ocean. All other snapshot columns and dates are unchanged.
The fetch now uses NOAA upwell ncdcOisst21Agg with 0–360 longitude and the surface selector. That aggregation silently omits 19 historical dates; the fetch recovers them from NOAA's oisst_rclone representation of the same final product. It validates daily continuity and bounded publication lag before writing. Historical request failures, final-chunk outages and all-null variables are fatal. Only an absent final chunk may be tolerated, subject to the whole-series checks. Requests have a timeout. The --only=oisst repair mode preserves independent ocean variables.
Source copy is shared and explicitly describes OISST plus CRW fallback. The current snapshot has ten CRW fallback days and two days without either source. No import logic changed; the external fingerprint already rebuilds caches.
Real-data recomputation preserves the calendar interpretation of the raw SST association: fully adjusted RR per °C is 0.974 (95% CI 0.835–1.136), versus 0.939 before. Nursery-minus-satellite mean changes from +0.43 to +0.42 °C. Seasonal sampling-night means change by −0.078 to +0.247 °C. Aggregate validation and source details are in docs/dev/OISST-154.md; private records and caches are excluded.
Validation: nine fetch regression tests, 107 unit tests, lint, svelte-check (zero errors/warnings), read-only real-data import and before/after analysis comparison, production build, Playwright/axe (37 tests). Fetch tests are included in pnpm test.
Closes #154.