Repository navigation
fix(cdec): bound only sub-daily snow values; retry cut-off JSON once - #95
Merged
Merged
Conversation
0.3.7 nulled SWE and snow depth outside 0-15 m at every duration. CDEC's daily series carries small negative readings (sensor noise around zero: 723 at ADM down to -7.6 cm, 231 SWE values at FLV down to -18.7 cm), so every refresh of those stations returned fewer values than archives built from the same series, and global_snow_networks' shrink guard refused the update for ~17 stations on 2026-11-06. The bound now applies to the hourly and event series it was meant for; daily and monthly series are returned as CDEC publishes them, as in 0.3.6. A full-record JSONDataServlet response also arrived cut off (62 MB, ending mid-object) and escaped as requests' JSONDecodeError. _get_json now retries once, then raises CDECError. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LSmHDJN224tLPVAUrvQTE
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LSmHDJN224tLPVAUrvQTE
egagli
pushed a commit
to egagli/global_snow_networks
that referenced
this pull request
Nov 7, 2026
0.3.8 (egagli/easysnowdata#95) applies the 0-15 m bound to CDEC's hourly and event series only, so refreshes of ADM, HOR, FLV and ~14 other daily stations no longer lose their small negative readings and trip the shrink guard; it also retries a cut-off CDEC response once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LSmHDJN224tLPVAUrvQTE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes two CDEC problems that showed up in global_snow_networks' nightly run of 2026-11-06, the first one using 0.3.7.
1. The 0–15 m bound now applies only to hourly and event series
0.3.7 (#94) set SWE and snow depth outside 0–15 m to
Noneat every duration. But CDEC's daily series has many small negative readings, which are sensor noise around zero. For example, ADM has 723 snow-depth values down to −7.6 cm, and FLV has 231 SWE values down to −18.7 cm. So a refresh of those stations came back with fewer values than the archives built from the same series, and the GSN shrink guard rejected the update for about 17 stations (ADM, FLL, CRL, SPT, PDS, TUN, FDC, HOR, FLV, INN, LLP, BSK, …). Those stations then got no new data.The bound now applies only to the hourly (
H) and event (E) series, the sensor-network data it was written for (−799,899 cm at G01, 21,049 cm at DPK). Daily and monthly series come back as CDEC publishes them, as they did in 0.3.6.Live check (2026-11-06): ADM's full daily record returns 7,122 snow-depth values, 723 of them negative. The archive has 7,121.
2. A cut-off JSON response is retried once, then raises
CDECErrorA full-record JSONDataServlet response arrived cut off (62 MB, ending mid-object). The resulting
requests.exceptions.JSONDecodeErrorwasn't aCDECError, so it got past the caller's handler and stopped the GSN CDEC job at batch 3 of 8._get_jsonnow retries once and then raisesCDECError.Tests
pytest -m "not live" tests/stations: 497 pass. New tests: a daily series keeps its negatives; a cut-off body is retried and then raisesCDECError. The existing event-series bound test is unchanged.Release as 0.3.8; global_snow_networks will require it.
Generated by Claude Code