v0.6.0 — web parity + analysis depth (Features A–I)

CHANGELOG.md

@@ -5,6 +5,61 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.6.0] — 2026-06-03
+
+Web parity + analysis depth. Closes the web memory/parity gap from 0.5.0 and
+adds six standards-grade analysis features, all parity-tested Python↔JS.
+
+### Added — web streaming + parity (Features A, B)
+- **Streaming columnar web parse** — `web/column_store.js` + `parseTrendColumnar`
+  / `parseTrendColumnarStream` decode `trend.bin` straight into packed
+  `Float32Array` columns, streamed from the dropped `File` in 8 MB
+  record-aligned `Blob.slice` chunks and **Transferred** back from the worker.
+  The 7-day file (589,877 recs / 438 MB) now parses + fully analyses at
+  **~278 MB peak RSS** vs the old ~1.6 GB. Analysis, charts, range select, and
+  tariff all read the resident store; exports materialise records transiently.
+  Legacy `parseTrendBin` retained for small-file / CSV paths.
+- **`web/analysis.js`** — JS port of `analysis.py` (Welford moments, percentile
+  sketch, `wholeSessionStats`, `classifyItic`/`eventItic`, `timeOfDayProfile`,
+  `correlateMarkers`). A Statistics panel + time-of-day chart in the web UI; the
+  stats table is embedded in the HTML export.
+- Fixed a latent `Math.max(...arr)` stack overflow in the insights engine that
+  would have crashed the web app on a ~590 K-element session.
+
+### Added — CT-reversal detection (Feature C)
+- **`--auto-reverse-cts`** — detects a reversed-CT install (real power negative
+  for ≥ 50 % of non-outage time) and applies `--reverse-cts` automatically with
+  a loud notice. A matching web banner + one-click apply. Flags the real ES.004
+  (52 % negative P, mean −37 kW). Python + JS.
+
+### Added — multi-session stitching (Feature D)
+- **`fluke-analyze stitch S1 S2 … -o OUT`** — concatenates consecutive sessions
+  into one continuous, gap-aware timeline with per-source provenance, then runs
+  the normal analysis over the stitched series (beats the meter's 7-day cap).
+  `web/multi_session.js` gains `stitchStores` + `buildStitched`. Validated on
+  ES.001 + ES.002 → 79,897 records with a detected 802 s gap.
+
+### Added — executive summary (Feature E)
+- **Auto-narrative** — deterministic, rule-based plain-English summary
+  (`narrative.md` + top of `summary.txt`, HTML, XLSX). Python + JS parity.
+
+### Added — power-quality standards (Feature F)
+- **IEEE 519** voltage-THD compliance per phase (p95 vs 8 %/5 %) and **IEEE 1159
+  / SARFI-90/80/70/50/10** indices, in stats + reports. `docs/PQ_STANDARDS.md`.
+
+### Added — demand + timezone + per-asset rules (Features G, H, I)
+- **`--demand-window`** rolling peak-demand (default 15 min) with peak window +
+  series, in stats/XLSX. Python + JS.
+- **`--tz ZONE`** renders report timestamps in local + UTC (default UTC
+  unchanged); anchors still accept ISO offsets. Python + JS.
+- **`--rules-file FILE`** (JSON/TOML) overrides `EventRules` thresholds keyed by
+  asset name (defaults + per-asset). `docs/RULES_FILE.md`. Python + JS.
+
+### Tests
+- Python 176 → 223; web 82 → 114. New Python↔JS golden-parity harness covers
+  stats, ITIC, CT reversal, narrative, IEEE 519/SARFI, demand, and timezone
+  formatting.
+
 ## [0.5.0] — 2026-06-02
 
 Large-session hardening — the tool now survives week-long captures

README.md

@@ -96,6 +96,10 @@ fluke-analyze path/to/ES.NNN -o output/ \
 | `--marks FILE.csv` | | Load markers from a CSV (`time,label`). |
 | `--tod-profile [HH:MM-HH:MM]` | off | Time-of-day (diurnal) profile — avg/min/max envelope per bin across all days. Bare flag = 24 h. Writes `time_of_day_profile.csv` + an XLSX sheet. |
 | `--tod-bin MINS` | `1` | Time-of-day bin width in minutes. |
+| `--demand-window SECS` | `900` | Rolling peak-demand window. Reports peak demand + the window it occurred in (`demand.json` + XLSX). See [`docs/DEMAND.md`](docs/DEMAND.md). |
+| `--tz ZONE` | UTC | Render report timestamps in local + UTC for an IANA zone (e.g. `America/Chicago`). Default UTC only. |
+| `--auto-reverse-cts` | off | Auto-detect a reversed-CT install (sustained negative real power) and apply `--reverse-cts` automatically, with a loud notice. See `docs/CT_REVERSAL.md`. |
+| `--rules-file FILE` | off | JSON/TOML EventRules overrides keyed by asset name. See [`docs/RULES_FILE.md`](docs/RULES_FILE.md). |
 | `--no-stats` | | Skip whole-session statistics (`stats.json`/`stats.csv` + XLSX sheet). |
 | `--format` | `png` | `png` or `svg` |
 | `--no-xlsx` | | Skip the XLSX workbook |
@@ -134,6 +138,25 @@ Overlays selected quantities across sessions on the same axes (aligned by
 relative-time-from-session-start) and writes a side-by-side
 `compare_summary.csv`. See [`docs/COMPARE.md`](docs/COMPARE.md).
 
+### Multi-session stitching
+
+```bash
+fluke-analyze stitch ES.001 ES.002 [...] -o OUT
+```
+
+Concatenates consecutive captures of the same asset into one continuous,
+gap-aware timeline (beating the meter's 7-day cap), with per-source provenance
+in `stitch.json`, then runs the normal analysis over the stitched series. See
+[`docs/STITCHING.md`](docs/STITCHING.md).
+
+## Power-quality standards & per-asset rules
+
+- **IEEE 519 / IEEE 1159 / SARFI** — voltage-THD compliance and SARFI dip
+  indices in every run (`pq_standards.json`). See
+  [`docs/PQ_STANDARDS.md`](docs/PQ_STANDARDS.md).
+- **Per-asset thresholds** — override `EventRules` from JSON/TOML keyed by asset
+  with `--rules-file`. See [`docs/RULES_FILE.md`](docs/RULES_FILE.md).
+
 ## Event detection
 
 Auto-detected event kinds (full rules + rationale in [`docs/EVENT_RULES.md`](docs/EVENT_RULES.md)):

ROADMAP.md

@@ -4,6 +4,34 @@ Shipped releases live in [CHANGELOG.md](CHANGELOG.md). This file is the
 backlog of ideas we've discussed but not yet committed to a specific
 release.
 
+## Shipped in v0.6 — web parity + analysis depth
+
+Closed the v0.5 web memory/parity gap and added standards-grade analysis, all
+parity-tested Python↔JS:
+
+- **Web streaming columnar parse** — the two deferred v0.5 web items below
+  ("typed-array record storage" + "chunked/streaming parse with progress") are
+  now shipped together: `web/column_store.js` + a chunked `Blob.slice` parser in
+  the worker that Transfers packed `Float32Array` columns back. ES.004 (438 MB,
+  589,877 recs) now parses + fully analyses at ~278 MB peak RSS vs ~1.6 GB.
+- **`web/analysis.js`** — full JS stats port (Welford, percentile sketch,
+  whole-session stats, ITIC, time-of-day, marker correlation) with a Statistics
+  panel + ToD chart; closes the "web has no stats" gap.
+- **CT-reversal auto-detection** (`--auto-reverse-cts` + web banner).
+- **Multi-session stitching** (`fluke-analyze stitch`).
+- **Auto-narrative / executive summary** — the rule-based, no-LLM version of
+  Theme D below (an LLM mostly restates the structured findings, as predicted).
+- **IEEE 519 + IEEE 1159 / SARFI** power-quality indices.
+- **Rolling peak demand** (`--demand-window`) — the kWh-side of Theme B's
+  "demand charges".
+- **Per-asset threshold config** (`--rules-file`) — the file-driven form of
+  Theme E's "custom event-rule editor" (a web slider editor is still open).
+- **Timezone-aware reports** (`--tz`).
+
+Still open from the backlog: fleet/monitoring (Theme A), full TOU/financial
+rigor (Theme B), comparison polish (Theme C), the rest of power-user/ecosystem
+(Theme E), and polish (Theme F).
+
 ## Shipped in v0.5 — large-session hardening
 
 The v0.5 release was driven by a real ~6.8-day P115RE-MAC03 capture
@@ -18,19 +46,17 @@ remain the forward backlog.
 ### Deferred from the v0.5 web pass
 
 The Python core is fully week-hardened. The browser app got chart
-decimation (the biggest uPlot win) this pass; the remaining 7-day
-robustness work is parked here:
-
-- **Typed-array record storage** — `parser.js` currently allocates one
-  object + one `Float32Array(180)` per record (~425 MB+ for a week).
-  Replace with a single flat `Float32Array` (or per-column arrays mirroring
-  the Python `ColumnStore`) indexed by record, eliminating per-record
-  object overhead.
-- **Chunked / streaming parse with progress** — parse in slices off a
-  `File`/`Blob` stream so a 438 MB session never has to be held as one
-  `ArrayBuffer` plus a parallel object array.
-- **IndexedDB large-session verification** — confirm the cache layer
-  holds a 438 MB session and evicts sanely under quota pressure.
+decimation in v0.5; the streaming/typed-array work below **shipped in v0.6**:
+
+- ~~**Typed-array record storage**~~ — **DONE (v0.6)**: `web/column_store.js`
+  keeps the analysis channels as packed `Float32Array` columns instead of
+  per-record objects.
+- ~~**Chunked / streaming parse with progress**~~ — **DONE (v0.6)**:
+  `parseTrendColumnarStream` reads the `File` in 8 MB record-aligned
+  `Blob.slice` chunks; the full `ArrayBuffer` is never resident.
+- **IndexedDB large-session verification** — still open: confirm the cache
+  layer holds a 438 MB session and evicts sanely under quota pressure. (The
+  streaming path no longer caches the raw buffer, reducing pressure.)
 
 ## v0.5 candidates
 

docs/DEMAND.md

@@ -0,0 +1,40 @@
+# Demand analysis (`--demand-window`)
+
+Utilities bill on **demand** — a sliding/block-window average of real power, not
+the instantaneous peak. The 15-minute interval is the most common. This
+analyzer computes a trailing rolling mean of `P_total_avg_W` and reports the
+peak demand and when it occurred.
+
+```bash
+fluke-analyze ES.004 --demand-window 900   # 900 s = 15 min (default)
+```
+
+## What it computes
+
+For a window of `W` seconds (1 record = 1 s), at each index `i ≥ W−1` the
+trailing demand is `mean(P_total over the last W records)`. The analyzer reports:
+
+| Field | Meaning |
+|---|---|
+| `peak_demand_w` / `peak_demand_kw` | the highest rolling-window demand |
+| `peak_window_start` / `peak_window_end` | the window that produced the peak |
+| `mean_demand_w` | mean of all full-window demands |
+| `n_windows` | number of full windows evaluated |
+| `series` | an optional decimated demand series for charting |
+
+Non-finite `P` samples are treated as 0 in the running sum. If the session is
+shorter than one window, no peak is reported (`n_windows = 0`).
+
+## Outputs
+
+- CLI: `demand.json` + a one-line `[demand]` summary + Peak-demand rows in the
+  XLSX **Summary** sheet.
+- Web: surfaced in the Statistics panel and the exported HTML report
+  (`web/analysis.js → demandAnalysis`).
+
+## Notes
+
+- The window is **trailing** (right-aligned), matching how interval meters
+  accumulate demand within each interval.
+- For true utility block-demand (fixed 15-min boundaries) rather than a sliding
+  window, split first with `--split-by 15m` and read each bucket's mean kW.

docs/PQ_STANDARDS.md

@@ -0,0 +1,71 @@
+# Power-quality standards: IEEE 519 & IEEE 1159 / SARFI
+
+This analyzer reports two standards-based power-quality summaries alongside the
+event log and whole-session statistics. Both are computed identically in the
+Python CLI and the web app (parity-tested).
+
+## IEEE 519-2014 — harmonic (THD) limits
+
+IEEE 519 sets voltage-distortion limits at the point of common coupling. For
+systems at or below 1 kV (the case for the 3540 FC's typical 277/480 V service):
+
+| Quantity | Limit |
+|---|---|
+| Total voltage THD | **8.0 %** |
+| Planning level / single-harmonic guidance | **5.0 %** |
+
+**How it is assessed.** IEEE 519 evaluates compliance against the 95th
+percentile of the measured distortion, not the instantaneous peak. The analyzer
+therefore reports the **p95 of `V_THD_pct_<phase>_avg`** per phase and marks a
+phase:
+
+- `compliant` when p95 ≤ 8.0 %,
+- `exceeds_planning` when p95 > 5.0 % (a yellow flag even if still compliant).
+
+`all_voltage_compliant` is true only when all three phases pass.
+
+**Current THD.** IEEE 519 current limits are expressed as TDD (total demand
+distortion) and depend on the short-circuit ratio Isc/IL, which the meter does
+not record. The analyzer therefore reports **p95 of `I_THD_pct_<phase>_avg`**
+per phase as informational context, without a hard pass/fail.
+
+Output: `pq_standards.json → ieee519`.
+
+## IEEE 1159 / IEEE 1564 — SARFI indices
+
+The **System Average RMS (variation) Frequency Index, SARFI-X**, counts the
+number of voltage variation events whose **residual voltage dipped below X % of
+nominal**. For a single monitoring point (one meter, one asset) the index is the
+event count itself.
+
+The analyzer reports the standard magnitude thresholds:
+
+| Index | Counts events with residual voltage below |
+|---|---|
+| SARFI-90 | 90 % (i.e. any dip ≥ 10 %) |
+| SARFI-80 | 80 % |
+| SARFI-70 | 70 % |
+| SARFI-50 | 50 % |
+| SARFI-10 | 10 % (near-interruption / outage) |
+
+Residual voltage is taken from each detected `dip` (severity = residual
+fraction) and `outage` (deepest L-N voltage ÷ nominal). Swells and non-voltage
+events are excluded. Because the bins are cumulative, SARFI-90 ≥ SARFI-80 ≥ … ≥
+SARFI-10 by construction.
+
+Output: `pq_standards.json → sarfi`.
+
+## Where it shows up
+
+- **CLI:** a one-line `[pq]` summary during analysis; full detail in
+  `pq_standards.json`.
+- **Web:** computed inline from the resident ColumnStore via
+  `web/analysis.js` (`ieee519Compliance`, `sarfiIndices`).
+- **Reports:** surfaced in the HTML/XLSX statistics area.
+
+## Caveats
+
+- THD field semantics are confidence `M` in `spec/field_map.json`; treat the THD
+  numbers as strong-inference until cross-checked against a reference meter.
+- SARFI here is a single-site event count, not the multi-site customer-weighted
+  utility metric. It is directly comparable across captures of the same asset.

docs/RULES_FILE.md

@@ -0,0 +1,76 @@
+# Per-asset threshold config (`--rules-file`)
+
+Event detection uses a single set of thresholds (`EventRules`) tuned to IEEE
+1159 / NEMA defaults. When you know an asset's real behaviour — its actual trip
+voltage, an expected swell ceiling, a noisier-than-usual feeder — you can
+override those thresholds per asset with `--rules-file FILE`.
+
+```
+fluke-analyze ES.004 --rules-file fleet_rules.json
+```
+
+The asset is matched on the session's `asset_name` (from the `*-config.json`).
+
+## File format (JSON or TOML)
+
+Two optional sections:
+
+- `defaults` — applied to every asset.
+- `assets` — a map of `asset_name → overrides`. A special `"default"` asset key
+  is used when the session's asset has no explicit entry.
+
+Per-asset values win over `defaults`, which win over the built-in defaults.
+
+### JSON
+
+```json
+{
+  "defaults": {
+    "dip_pct_of_nominal": 0.92
+  },
+  "assets": {
+    "P115RE-MAC03": {
+      "outage_v_threshold": 60.0,
+      "swell_pct_of_nominal": 1.08
+    },
+    "default": {
+      "freq_excursion_hz": 0.4
+    }
+  }
+}
+```
+
+### TOML
+
+```toml
+[defaults]
+dip_pct_of_nominal = 0.92
+
+[assets."P115RE-MAC03"]
+outage_v_threshold = 60.0
+swell_pct_of_nominal = 1.08
+```
+
+A **flat file** with only threshold keys (no `defaults`/`assets`) is treated as
+defaults for every asset.
+
+## Overridable keys
+
+| Key | Default | Meaning |
+|---|---|---|
+| `outage_v_threshold` | 50.0 | any phase L-N below this V is an outage |
+| `dip_pct_of_nominal` | 0.90 | < this fraction of nominal = dip |
+| `swell_pct_of_nominal` | 1.10 | > this fraction of nominal = swell |
+| `high_current_sigma` | 2.0 | mean + Nσ on any phase = high current |
+| `freq_excursion_hz` | 0.5 | \|f − nominal\| over this = excursion |
+| `imbalance_pct_threshold` | 2.5 | NEMA % imbalance threshold |
+| `power_step_pct_of_mean` | 0.50 | ΔP in 1 s over this × mean \|P\| = step |
+| `min_duration_secs` | 1 | events shorter than this are ignored (int) |
+| `gap_tolerance_secs` | 1 | merge runs split by ≤ this many samples (int) |
+| `nominal_freq_hz` | 60.0 | line frequency baseline |
+
+Unknown keys are rejected with an error listing the valid ones. `min_duration_secs`
+and `gap_tolerance_secs` are coerced to integers; all others to floats.
+
+On load the CLI prints a one-line `[rules]` note showing exactly which
+thresholds changed (`key: old -> new`) for the matched asset.