Skip to content

Environment: import in-situ temperature loggers - #94

Merged
Luen merged 3 commits into
mainfrom
in-situ-temperature
Nov 3, 2026
Merged

Luen merged 3 commits into
mainfrom
in-situ-temperature

Conversation

@Luen

@Luen Luen commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

What

Imports the raw temperature logger exports (#10) and uses them across the site. Drop the folder anywhere under DATA_DIR (e.g. data/Temperature loggers/); like the workbooks, the CSVs are only read, never edited or committed.

Importer (import/loggers.ts, import/insitu.ts)

  • HOBO exports in English and French, comma- or tab-separated, UTF-8 or Mac Roman. Their clocks are Moorea time even where the header says GMT+10:00 (the light channel peaks at midday), so the label is ignored and a clock that looks off is flagged.
  • EnvLogger exports with their own serial, name, time zone (UTC−10 early, UTC later) and the phone's GPS at download. App logs, empty downloads and unknown CSVs are skipped and listed on Data quality.
  • Each file is placed at a nursery by download GPS (≤ 400 m), logger name, file name, the EnvLogger register workbook or the season workbooks' HOBO log. Where a logger's name and its download GPS disagree, a warning goes to Data quality for the lab to check.
  • Cleaning: trims deployment and retrieval hours, drops readings outside 15–40 °C, merges repeated downloads of the same logger. Builds logger_files, logger_daily (days with ≥ 18 site-hours) and logger_diel, and adds sets.insitu_temp (logger temperature during the set).
  • The EnvLogger register is recognised by its header before the /gps|location/ file-name rule, so it no longer produces spurious "unknown site" warnings.
  • CSVs are in the import fingerprint; IMPORTER_VERSION → 4.

Pages

  • Water quality: new in-situ section (stat tiles, nursery water vs offshore SST, daily cycle in warm and cool months, daily maximum by nursery, logger coverage). Nurseries are only ranked if logged for at least a year in both seasons; a summer-only logger would otherwise look hottest.
  • Nursery profiles: temperature over time (daily high, mean, low) and the daily cycle at that nursery.
  • Drivers: new per-set variable "Logger water temperature during the set".
  • Findings: nursery water vs offshore SST, computed live.

Notes for the lab (on the Data quality page after deploy): a few HOBO files whose light channel suggests the clock was off, one EnvLogger named for one nursery but downloaded at another, a lab test logger, and empty downloads.

Checks

pnpm lint, pnpm check, pnpm test (56 passed, new parser tests plus a fixture import test), real-data import (opt-in), pnpm build, pnpm test:e2e (29 passed, axe light and dark). Checked both pages by hand against the real data, including at phone width. Fixtures are synthetic (invented serials and readings).

Closes #10

🤖 Generated with Claude Code

Luen and others added 2 commits November 3, 2026 10:18
- loggers.ts reads raw HOBO (English/French, comma/tab, UTF-8/Mac Roman)
  and EnvLogger CSVs anywhere under DATA_DIR, converting every reading to
  Moorea local time; app logs, empty downloads and unknown CSVs are skipped.
- insitu.ts assigns each file to a nursery (download GPS, logger name,
  register workbook, file name or the workbooks' HOBO log), trims
  deployment edges, drops implausible readings, merges repeated downloads
  and builds site-hourly, daily (>= 18 h) and diel tables.
- The EnvLogger register is recognised by its header before the
  GPS/location file-name rule, so it no longer floods Data quality.
- sets.insitu_temp: logger temperature during each set (Drivers variable).
- Water quality: logger section (stats, nursery vs offshore SST, daily
  cycle, daily maximum by nursery, coverage); nursery profiles get their
  own temperature and daily-cycle charts; new live finding.
- CSVs are part of the import fingerprint; IMPORTER_VERSION 4.
- Synthetic HOBO + EnvLogger fixtures, parser unit tests, docs.

Closes #10

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The light-peak clock check raised false alarms: HOBO 7 and 11 peak in
light at 17:00 and 07:00 because of shading, yet their temperature cycles
line up hour for hour with their partner loggers. Replace it with a check
of each download's hourly series against the loggers at the other
nurseries (compareSeries): r < 0.5 leaves the download out
(logger_files.excluded) and a best alignment shifted by >= 3 h warns of a
clock problem. On the real data this drops two downloads that do not
follow the lagoon (HOBO 17b Oct18-Feb19, HOBO 20 Nov16-Apr17).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Luen
Luen merged commit 2b813ed into main Nov 3, 2026
5 checks passed
@Luen
Luen deleted the in-situ-temperature branch November 3, 2026 00:47
@Luen Luen mentioned this pull request Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HOBO loggers: ingest raw in-situ temperature / DO exports

1 participant