Skip to content

fix(cdec): bound only sub-daily snow values; retry cut-off JSON once - #95

Merged
egagli merged 2 commits into
mainfrom
claude/busy-brown-ralg4x
Nov 6, 2026
Merged

egagli merged 2 commits into
mainfrom
claude/busy-brown-ralg4x

Conversation

@egagli

@egagli egagli commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Fixes two CDEC problems that showed up in global_snow_networks' nightly run of 2026-11-06, the first one using 0.3.7.

1. The 0–15 m bound now applies only to hourly and event series

0.3.7 (#94) set SWE and snow depth outside 0–15 m to None at every duration. But CDEC's daily series has many small negative readings, which are sensor noise around zero. For example, ADM has 723 snow-depth values down to −7.6 cm, and FLV has 231 SWE values down to −18.7 cm. So a refresh of those stations came back with fewer values than the archives built from the same series, and the GSN shrink guard rejected the update for about 17 stations (ADM, FLL, CRL, SPT, PDS, TUN, FDC, HOR, FLV, INN, LLP, BSK, …). Those stations then got no new data.

The bound now applies only to the hourly (H) and event (E) series, the sensor-network data it was written for (−799,899 cm at G01, 21,049 cm at DPK). Daily and monthly series come back as CDEC publishes them, as they did in 0.3.6.

Live check (2026-11-06): ADM's full daily record returns 7,122 snow-depth values, 723 of them negative. The archive has 7,121.

2. A cut-off JSON response is retried once, then raises CDECError

A full-record JSONDataServlet response arrived cut off (62 MB, ending mid-object). The resulting requests.exceptions.JSONDecodeError wasn't a CDECError, so it got past the caller's handler and stopped the GSN CDEC job at batch 3 of 8. _get_json now retries once and then raises CDECError.

Tests

pytest -m "not live" tests/stations: 497 pass. New tests: a daily series keeps its negatives; a cut-off body is retried and then raises CDECError. The existing event-series bound test is unchanged.

Release as 0.3.8; global_snow_networks will require it.


Generated by Claude Code

0.3.7 nulled SWE and snow depth outside 0-15 m at every duration. CDEC's
daily series carries small negative readings (sensor noise around zero:
723 at ADM down to -7.6 cm, 231 SWE values at FLV down to -18.7 cm), so
every refresh of those stations returned fewer values than archives built
from the same series, and global_snow_networks' shrink guard refused the
update for ~17 stations on 2026-11-06. The bound now applies to the
hourly and event series it was meant for; daily and monthly series are
returned as CDEC publishes them, as in 0.3.6.

A full-record JSONDataServlet response also arrived cut off (62 MB,
ending mid-object) and escaped as requests' JSONDecodeError. _get_json
now retries once, then raises CDECError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015LSmHDJN224tLPVAUrvQTE
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015LSmHDJN224tLPVAUrvQTE
@egagli
egagli merged commit 2d8a8fd into main Nov 6, 2026
11 checks passed
egagli pushed a commit to egagli/global_snow_networks that referenced this pull request Nov 7, 2026
0.3.8 (egagli/easysnowdata#95) applies the 0-15 m bound to CDEC's hourly
and event series only, so refreshes of ADM, HOR, FLV and ~14 other daily
stations no longer lose their small negative readings and trip the
shrink guard; it also retries a cut-off CDEC response once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015LSmHDJN224tLPVAUrvQTE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants