Release v1.1.0: spectrogram/PSD, EDF, user options, per-file other:: config

[AlexisJanin]

Merges the accumulated dev work into main as v1.1.0 — 64 commits since v1.0.0, touching 175 files (+16,957 / −7,775).

What's in it

Two new plot types. Spectrogram and PSD, for high-rate signals (EEG, pressure waveforms). Both are re-renderings of the same samples, which is the side of the ADR-0006 line this library stays on.

A .edf datasource — European Data Format, the usual EEG/polysomnography container.

A third configuration tier, user_options. Per-person app behaviour and display fallbacks, edited in the Settings modal. It only ever fills gaps database_options left — it never overrides (ADR-0005).

Per-file configuration for other/. Each file gets its own other:: scope with its own time_shift, timezone, grouping and trace style (ADR-0009).

Breaking: philips_waves, philips_numerics and syringe are removed. They did no format-specific parsing, so once other/ gained per-file scopes they no longer earned a module (ADR-0008). The changelog carries the migration, including three traps: the syringe timezone default moves Paris → UTC, marker traces need restoring via trace_options, and signals take the qualified other::<stem>::<column> form.

Substantially faster loading. Columns are selected and pruned before reading, with row pushdown as a strictly optional accelerator (ADR-0007).

Full detail in CHANGELOG.md.

⚠️ Upgrading: the parquet cache must be rewritten once

This release changes what clinical_scope_output/ holds, and nothing detects a cache written by an older version. After updating, un-tick "Re-use data if already loaded once" (quick_load) for one run per patient. A stale cache gives EIT recordings without their %Local columns, and timestamps carrying whatever timezone the old run happened to stamp in.

Pre-merge hardening on this PR

CI had been red since 2026-07-29 — 13 consecutive failures. Two independent causes, both in ruff format --check:

1. Three Python files were committed unformatted (one also breaking E501).
2. dev = ["ruff"] is unpinned, so CI had moved to ruff 0.16.2 while local venvs sat on 0.15.x. 0.16 formats Python code blocks inside Markdown (a 4th file) and, under select = ["ALL"], added CPY001 — 66 hits across every module.

Because the format step fails first and fail-fast: true cancels the sibling job, lint and tests had never once run on this branch. They do now: 848 tests pass on Python 3.11 and 3.13, and both matrix jobs are green on the merge head.

Also landed here:

• Two silent defects fixed, both interactions between user_options and code predating it — see the review comment below.
• Three ADRs back-filled, plus the stale PSD db_range in the tutorial corrected against measured demo data.
• Version bumped in both pyproject.toml and CITATION.cff; the two had drifted (1.0.1 vs 1.0.0) and docs/RELEASING.md tracks both.

After the review: loader purity (ADR-0010)

An audit run after the three-axis review found loaders resolving configuration — a timezone stamped in by five of them, and eit narrowing its parser on field_display. Because _load's output is the parquet cache, any such value outlives the setting that produced it and is unreachable from a later run's config. ADR-0010 states the rule: _load transcribes, _format interprets.

• Five loaders stop stamping a timezone; _format localizes to the same instants as before. test_load_config_independence.py calls each caching source's _load twice under differing configs and asserts the frames match.
• Behaviour change: a mindray_scope folder mixing naive .csv and offset-bearing .xml now raises with both file names, instead of being silently rescued by the default timezone.
• EIT %Local N ratios move into _load — they resolve no option, so they are transcription. That puts them in the cache where field_display can select them: a configured EIT read goes from all 71 columns to 9 of 75.
• Column pruning is now gated on provenance rather than index dtype, so a cache with a non-temporal index no longer falls back to reading every column.

This is the work that makes the cache-rewrite note above necessary.

Review

A three-axis review (Standards / Spec / Maintainability) was run over the full main...dev diff; findings and their triage are in the review comment on this PR. The deferred findings are tracked in #79, #80 and #81.

Issues shipped here

#53, #57, #63, #66, #68, #69, #71, #72, #73, #74, #75 — all already closed except #71 (spectrogram), which is labelled status: done but still open. No closing keywords are used in this body, so merging changes no issue state.

[AlexisJanin]

Three-axis review — main...dev

Three parallel agents reviewed the full 163-file diff along independent axes, then every load-bearing claim was verified against the code before being reported here. The axes are kept separate on purpose: code can pass one and fail another.

Findings marked ✅ fixed landed in this PR. Findings marked ↩︎ accepted were triaged as not worth acting on. Findings marked 📋 follow-up are proposed as issues.

---

Standards

Does the code follow this repo's documented standards?

The branch is disciplined overall: no ADR is violated, D213 is universal, the abbreviation sweep is thorough, and the clinician-facing docs correctly avoid linking to docs/adr/, CONTEXT.md or CLAUDE.md. Findings cluster in one pattern — the "shared literals belong in constants.py" rule applied to some new code and not the rest, often inside the same function.

Hard violations

1. Constants left inline. Worst cases, all verified:

   • data_callbacks.py:407,522 — Input({"name": "user_options.display_timezone"}) hand-spelled, while constants.py:330 defines NAME = "display_timezone". This is the sharpest one: Dash pattern-matching IDs that stop matching don't error, the callback just never fires. A rename breaks it silently. 📋
   • database_options_xlsx.py:285-322 writes "label"/"unit"/"range"/"color" as bare strings while lines 244-247 directly above use cst.* — a half-finished migration inside one loop. 📋
   • inspection.py:75 declares a status enum in a comment (# "ok" | "load_error" | …), retyped as literals at wrapper.py:626, base.py:704,715,779,786. registry.py:288 then adds a second undeclared enum. 📋
   • file_utils.py:475,493,506 — the pushdown kind discriminator; file_utils.py:263 — tz_localize("UTC") where cst.LIBRARY_TZ exists. 📋
   • signal_container.py:235 — plot_priority = 10000 bare, and the diff deleted the comment explaining it while promoting the sibling TraceOptions defaults into cst.TraceDefaults. 📋

2. Tests restate the constant they exercise. test_main_visualization.py:145 is verbatim CLAUDE.md's counter-example (== cst.DEFAULT_SUBPLOT_HEIGHT * …) — a test that can never fail when the constant changes. Also test_timezone.py:96,99,102, test_datetime_pushdown.py:176-246, test_signal_container.py:596-597. Adjacent lines get it right (== 220, == "closest"). 📋

3. CHANGELOG.md documents only the datasource removal — ✅ fixed: the other ~40 commits are now written up, and the version bumped to 1.1.0 in both pyproject.toml and CITATION.cff.

Top judgement calls

• timezone.py:255-285 — _resolve_effective_tz was extracted but the identical block left in place in apply_timezone_to_dataframe. 📋
• base.py:170-172 — comment says "the library default" but assigns cst.DISPLAY_TIMEZONE (Europe/Paris), not LIBRARY_TZ; base.py:443 repeats the wrong claim by reference. 📋
• constants.py:592 adds a shared AdditionalInformations.TIMEZONE, but all nine source modules keep their own copy and the new edf/options.py:37 adds a tenth. 📋
• scripts/bench_pushdown.py — a 438-line dev harness sitting in the user-facing scripts folder, whose docstring prescribes a git stash A/B workflow. 📋

---

Spec

Does the code do what the originating issues asked?

#63, #66, #68, #69, #75 are fully and correctly implemented — every acceptance criterion checked against code.

Missing or partial — all triaged ↩︎ accepted:

• Remove useless datasources #74 the retired-folder warning fires from wrapper.py:340 (main) and :568 (inspect) but not from the extract entry points.
• Remove useless datasources #74 no test coverage for the migration path (registry.py:170-230).
• Remove useless datasources #74 the syringe-timezone migration case isn't covered by test_other.py::TestTimezone.
• Add datasource: eeg (.edf files) #72 licence-audit pass — verified separately, nothing to change (basic licence).
• Add datasource: eeg (.edf files) #72 /anonymize-timeseries EDF header fields — the EDF file is open-source, no de-identification needed.
• Add new plot type: spectrogram #71 demo EDF sits 20 years off the other sources (recording_start defaults to ""); the correct value is documented in the README.

Implemented but wrong

• Add datasource: eeg (.edf files) #72 pyproject.toml adds pyedflib as a hard runtime dependency, contradicting the issue text and PR Spectrogram & PSD plot types, EEG datasource, other-source cleanup #76's body. ↩︎ This was a deliberate supersede — preferred over the alternatives.
• Remove useless datasources #74 the changelog listed 4 trace_options keys where constants.py:579 has 6, omitting exactly the two (marker_symbol, marker_size) the adjacent "Marker traces" migration note depends on. ✅ fixed
• spectral tutorial.md:788 still showed PSD "db_range": [40, 90], stale since the density re-scale. Measured on the demo EEG the data spans 6.5–41 dB, so the documented band sat entirely above it and would render empty. ✅ fixed to [0, 40], matching the two spectrogram examples the same commit did update.

---

Maintainability

What will the next change to this code cost?

Fix now — two were live bugs, both verified by execution, both now ✅ fixed:

1. DisplayFallbacks.hovermode was overwritten by a hardcoded literal. annotation_callbacks.py:861 set hovermode to "closest" if point_mode_active else "x unified", while signal_container.py:1100 sets it from the carrier. render_annotations fires on any annotation create/edit/delete or mode toggle, so "Closest point only" chosen in Settings held until the user touched an annotation, then reverted for the session. The comment claimed the two stayed in step — the capability set was shared, the value was not.

2. Spectrogram dB bounds had no ordering check. Verified: {db_min: 100, db_max: 40} → spectrogram_db_range == (100.0, 40.0), reaching go.Heatmap as zmin > zmax. Reachable by typing two in-range numbers into Settings. The independent clamp also turns an out-of-range pair into (100.0, -100.0). Correction to the agent's report: it claimed _coerce's missing FLOAT branch let unclamped values reach the render layer — it doesn't, bounded_number catches them. The real defect was the missing min ≤ max check. Both are fixed; _coerce now handles FLOAT for consistency with its own docstring.

3. trace_options accept/interpret drift. 📋 Validated in any section (constants.py:498, database_options_parser.py:266-288) and written for any datasource by the XLSX writer — but read at exactly one site, other/find_load_format.py:90. Ten scopes accept it, one interprets it. Put trace_options in a servo_u section and it validates cleanly and is silently ignored. Deferred rather than fixed here because making it apply everywhere changes rendering for configs that currently validate-but-ignore — that belongs in its own PR, not a merge-readiness pass.

Fix later 📋

• A fourth plot type costs 8 src files, ~20 sites. Measured: PSD touched 7 src files / +680 lines with the abstraction already built, and still missed a site — the code says so at other/find_load_format.py:130-132 ("forgetting to is what made 'psd' validate cleanly yet never render"). Two duplicate (SPECTROGRAM, PSD) loops; the to_plotly_trace hovertemplate if/elif is the one behaviour never converted to a capability set.
• The :: convention has 16 sites (7 construct, 9 parse). "Strip the other:: prefix" is hand-written three times, in three spellings. The separator is centralised; the operations are not.
• Duplicate validation — every other::<stem> issue logs twice on Process. One line.
• CLAUDE.md's "so no signature grows" is false — 7 signatures grew, and registry.py:41 weakened MainModule to Callable[..., ...] to absorb it. The salvageable claim (the next fallback costs a field plus a read site) holds for the render path.

Positive verdicts — leave alone

• The datasource removal is clean. No dead scaffolding for the three removed sources; RETIRED_DATASOURCE_FOLDERS is live migration support with 5 readers; docs, tests, goldens and example data all updated.
• The PlotType capability sets, the import-time plot-type/section-key consistency check, schema-driven user options (a new one costs zero UI sites), _DERIVED_PLOTS, _make_bounds_computer. Also: do not unify the xlsx sheet blocks or the dual-shape config readers — those differences are real.

---

Bottom line

Axis	Findings	Worst in that axis
Standards	9	Hand-spelled Dash callback IDs that break silently on rename
Spec	9 (7 accepted)	PSD db_range documented above the data — now fixed
Maintainability	8	Hovermode silently discarding a user setting — now fixed

The three I'd fix first, from anywhere: the trace_options accept/interpret gap (ten scopes accept, one reads); the hand-spelled user_options.display_timezone callback IDs; and the tests that restate their own constants, which are the ones least likely to be caught by anything else.

Merge readiness: CI green on both matrix Pythons, 848 tests passing, no ADR violated, breaking change documented with a migration path. Nothing outstanding blocks the merge.

[AlexisJanin]

Follow-ups from the review above are now tracked:

• Decide a ruff policy: select = ["ALL"] with an unpinned linter re-breaks CI #79 — ruff policy: select = ["ALL"] with an unpinned linter re-breaks CI. Includes the fail-fast masking, which is worth fixing whichever pinning option wins.
• trace_options is accepted in ten config scopes but read in only one #80 — trace_options accepted in ten scopes, read in one. Needs a decision on what the feature is before it can be fixed.
• Consistency cleanups from the v1.1.0 review (tracking) #81 — tracking checklist for the smaller consistency findings (constants left inline, tests restating their own constants, small duplication) plus the measured cost of adding a fourth plot type.

The 📋 markers in the review map onto these three.