release: 1.10.1 — CWOP adapter patch (pre-release review fixes)#84
Merged
Conversation
Adds the APRS-IS Citizen Weather Observer Program adapter: scan/nearby/stream/snapshot/latest/history/persist_observations, 6-layer QC, monthly parquet persistence. Parity firewall intact — no research()/merge/live wiring added.
|
Parity ticket gate: PASSED See |
…loor P1-A/P1-B/P2-E/P3 (CHANGELOG, [1.10.0]): - scan(): documented as scan(stations) batch fetch; real signature is scan(lat, lon, radius_km=25.0, *, listen_seconds=60, min_reports=1) — a radius discovery-listen on APRS-IS. Bullet rewritten to match. - nearby(): documented a nonexistent min_count param and icao arg; real signature is nearby(station, *, radius_km=25.0, listen_seconds=60). - snapshot(): bullet omitted the real duration_seconds + qc params. - stream(): yields QC-scored CWOPObservations, not raw "APRS-IS packets". P1-C (packages/weather/pyproject.toml): mostlyright.weather.cwop imports NoCWOPDataError from core, which first ships in core 1.10.0. Bump the core dependency floor >=1.0.0 → >=1.10.0 so a partial upgrade (weather 1.10.0 + core 1.9.0) cannot ImportError on `import mostlyright.weather.cwop`. The core [research] extra floor (mostlyrightmd-weather>=1.7.0) is independent and unaffected. uv.lock re-synced the editable workspace versions to 1.10.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_store_field` guarded non-temperature integer fields with `raw.isdigit()`
then called `int(raw)` unconditionally. `str.isdigit()` is True for Unicode
digit chars (²³¹ and others) that `int()` rejects with ValueError, so a
crafted/corrupt packet carrying such a char in any fixed-width weather field
raised an uncaught ValueError out of the PUBLIC parse_weather_fields /
parse_aprs_line surface. Pre-check `raw.isascii() and raw.isdigit()` so the
field is omitted (never recorded) and the parse never raises — mirroring the
temp path's existing try/except.
Note: the leading ddd/sss wind-extension path is NOT affected — its `\d{3}`
regex and int() agree on Unicode decimal digits, so no change there.
TDD: RED reproduced the exact ValueError from a superscript humidity field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The DHM/HMS/positionless timestamp parsers each build candidates across neighbouring periods (month/date/year) and funneled through `_closest_to`, which picked the candidate with the smallest absolute delta to `ref` (the socket receive_time = knowledge_time) with NO past-only constraint. So a clock-fast HMS, or a day-28 DHM received on the 21st, could resolve observed_at 7 days INTO THE FUTURE of knowledge_time. The cwop schema defines knowledge_time = event_time + lag with lag >= 0, so a future observed_at corrupts dedup and event-time backtest windows in a leakage-firewall SDK. `_closest_to` now restricts to candidates that are non-future within a 10-minute honest-clock-drift tolerance and picks the nearest of those, only falling back to the nearest overall when EVERY candidate is far ahead (so a degenerate input still resolves rather than returning None). Existing month-/year-rollover tests already asserted the past-preference outcome and remain green; this hardens it into an invariant across all three formats. TDD: RED showed a day-ahead DHM resolving 7 days into the future. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…bs (P2-C) `qc_reliability_score` (Layer 6) could return 0.0 — actual_reports_30d=0 + qc_pass_rate_30d=0.0 + cadence None gives components [0.0, 0.0] → mean 0.0. In `qc_cwop` any layer score <= 0.0 short-circuits the weighted geometric mean to 0.0 → "dropped", so a physically-valid current reading was discarded on historical metadata alone. Reliability is a reputation/metadata signal; like the buddy (0.05) and solar (0.2) layers it must be able to pull the combined score down but never single-handedly veto it. Floor the layer at RELIABILITY_FLOOR=0.05 (still < 0.5 so it FLAGS the problem) and clamp qc_pass_rate_30d into [0, 1] so a bogus out-of-range value can't push the layer score above 1.0 or below 0.0. TDD: RED showed a worst-case reliability context dropping an in-bounds reading to score 0.0, and an unclamped qc_pass_rate_30d=1.5 yielding 1.25. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ped (P2-D) `write_cwop_cache` built the parquet table via `pa.Table.from_pylist(merged)`, which infers the schema from the FIRST row only and silently drops any key absent from it. On heterogeneous input (e.g. a packet with no temp reading followed by one carrying temp_f) the public write path lost columns. Build an explicit pyarrow schema from the schema.cwop.v1 column set (_build_cwop_pa_schema), normalize every row to the full ordered key set, and pass `schema=` to from_pylist so all declared columns are preserved regardless of per-row key presence. Keys outside schema.cwop.v1 are intentionally excluded from the persisted partition. Existing dtype roundtrip / dedup / Int64 behavior is unchanged. TDD: RED showed temp_f/humidity from a later row dropped because the first row omitted them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…er (PKG-03) The PKG-03 weather pin assertion only accepted the exact string >=1.0.0,<2.0. P1-C bumped weather's core floor to >=1.10.0,<2.0 (so a partial upgrade can't import NoCWOPDataError, which only exists in core 1.10.0). That is still an active-major-bounded pin — same major, capped <2.0, legacy 0.0.x forbidden — so generalize the check to a regex accepting any >=1.x.y,<2.0 (or the pre-1.0 >=0.1.x,<0.2) floor. The markets test is unchanged (markets still pins >=1.0.0,<2.0). PKG-03 intent preserved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_run_live_scan` early-exited the listen loop on `len(station_index) >= min_reports`, counting EVERY parsed packet — including positionless reports (lat/lon None) that the post-scan filter then drops as distance inf. With the default min_reports=1 a positionless first packet broke the loop before any coordinate-bearing station was seen, so scan()/nearby() could return/raise empty even though a later positioned in-radius packet in the same window would have qualified. Add `_placeable_within_radius_count` (mirrors the post-scan filter: position present AND within radius) and gate the early-exit on it, so the loop only stops once enough stations that will actually be RETURNED have arrived. Also read the final station index after the loop so the returned set is the freshest snapshot regardless of how the loop terminated. The positionless-still-filtered-out and outside-radius-raises behaviors are unchanged. TDD: RED showed a positionless first packet pre-empting a later positioned in-radius station, returning empty. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…amp (P2-G) The MMDDHHNN timestamp on a positionless `_` weather report is OPTIONAL, but the parser consumed the first 8 chars after `_` as a timestamp attempt whenever the remainder was >= 8 chars — even when the parse FAILED. So `_c220s004g005t077...` lost the swallowed leading wind dir/speed fields, and `_t085h60b10168` was dropped entirely. (The short <8-char no-timestamp case was already handled.) Only strip the first 8 chars when `_parse_timestamp_positionless` actually returns a value; otherwise treat the whole remainder (from after `_`) as weather data. No realistic collision: valid weather data starts with a field-prefix char or the `ddd/sss` wind form, never 8 leading bare digits. This is a separate path from the `_closest_to` non-future resolution fix, which is untouched. TDD: RED showed `_c220s004...` losing wind dir/speed and `_t085h60b10168` being dropped to None. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
✅ Docs-required check: PASS API-surface change includes docs updates — no reminder needed. API-surface files changed: Docs files changed: |
Supersedes 1.10.0, which was published before the CWOP review loop completed. Bumps all three PyPI distributions and four npm packages 1.10.0 -> 1.10.1, refreshes uv.lock, and documents the patch in CHANGELOG. No public API change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
helloiamvu
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mainvia feat(weather): CWOP adapter — APRS-IS parser, transport, 6-layer QC, public API #82What's in 1.10.0
mostlyright.weather.cwoppublic surface:scan,nearby,stream,snapshot,latest,history,persist_observations~/.mostlyright/cache/cwop/NoCWOPDataErrorsubclass ofNoLiveDataErrorschema.cwop.v1(parity firewall intact — noresearch()/merge/live wiring)Test plan
uv run pytest -m "not live" -q) passes onmain(all CWOP code landed with tests in feat(weather): CWOP adapter — APRS-IS parser, transport, 6-layer QC, public API #82)v1.10.0to trigger CI trusted publishing to PyPI + npm🤖 Generated with Claude Code