diff --git a/CHANGELOG.md b/CHANGELOG.md index dd27fd1..617ca57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,64 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.8.0] — 2026-06-03 + +Active/standby load-state split for bimodal loads — classify each record by +current, report the two states separately, correct the session energy three +ways, headline the active-state power factor, and harden the auto reverse-CTs +heuristic to decide on the dominant high-current state. Parity-tested Python↔JS. +Built from the real P115RE coating-rectifier session. + +### Added — load-state split (`load_states`) +- **Current-gated classifier.** Each record is **active** when its mean + per-phase average current `(I_a_avg+I_b_avg+I_c_avg)/3` is **≥ the threshold** + (default **50 A**), else **standby**. Current — not power — because the power + *sign* at low current is exactly the thing in question. Configurable with + **`--standby-threshold-a N`**. +- **`load_states.csv` + `load_states.json`** — one row per state (active, + standby): records, hours, duty %, kWh, P avg/min/max (kW), I avg (A), S avg + (kVA), PF avg, V_LN avg, V_THD p95. JSON also carries the threshold, the three + energy figures, and the standby-sign caveat. Emitted automatically (cheap); + `--load-states` is an explicit opt-in. A compact table is embedded in + `summary.txt` and the HTML report. +- **Three explicit energy figures** (never silently changing the historic + number): `energy_as_measured_kWh` (signed sum — current behavior), + `energy_active_kWh` (active records only), and + `energy_net_clip_standby_kWh` (standby real power clipped to ≥0). The standby + real-power sign is unreliable at low current, so the active/clip figures are + the defensible consumption. +- **Headline PF is the active-state PF.** The narrative, summary, and HTML + report now headline the active-state power factor (the blended whole-session + PF is meaningless for a bimodal load); the raw whole-session PF is kept but + de-emphasized. + +### Changed — magnitude-weighted reverse-CTs auto-detect +- `detect_ct_reversal` / `--auto-reverse-cts` now decide on the **dominant + high-current (active) state** — *is real power negative when current is + high?* — instead of the fragile whole-session negative-P count, which a + bimodal load defeats. The whole-session count fields are still reported for + context (`basis: "active"` vs `"whole_session"`, with `active_records`, + `active_frac_negative`, `active_mean_p_w`); the operator notice keys off the + active state. **Manual `--reverse-cts` behavior is unchanged** — only the AUTO + heuristic + its printed notice were improved. + +### Changed — shift integration +- `shift_comparison.csv/json` rows gain **`active_records`**, + **`active_duty_pct`**, **`active_kWh`**, and **`active_PF_avg`** so each shift + shows its active load too. `summary.txt`'s shift table shows the new columns. + +### Parity / tests / docs +- JS port in `web/analysis.js` (`classifyLoadStates`, `loadStateRows`, + `sessionEnergy`, `activeStatePf`; active-state `detectCtReversal`; active + columns on `shiftComparisonRows`) with a Python↔JS parity test + (`load_states_parity.test.js`) against a shared bimodal golden fixture. +- New Python tests: classifier (threshold, balanced bimodal fixture), the three + energy figures, the magnitude-weighted reverse-CTs decision (active-positive + and active-negative cases + fallback), shift active columns, narrative + active-PF/energy, and CLI `load_states` outputs. +- `docs/LOAD_STATES.md` (concept, the energy caveat, the standby-sign + explanation); README options; version bump to 0.8.0. + ## [0.7.0] — 2026-06-03 Generalized, named, configurable shift/period splitting — so usage can be diff --git a/README.md b/README.md index 02139ca..69c6a8b 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,7 @@ This project gives you: - **A Python CLI** for scripting, batch jobs, and publication-quality gnuplot output. `pip install -e .` and you're going. - **Auto event detection** — outages, voltage dips, swells, high-current peaks, frequency excursions, NEMA imbalance spikes, sudden load steps. - **CT reversal correction** — `--reverse-cts` flag handles iFlex probes installed backwards (extremely common mistake). See [`docs/CT_REVERSAL.md`](docs/CT_REVERSAL.md). +- **Active/standby load-state split** — for bimodal loads (e.g. a rectifier that toggles between a heavy active draw and a light standby), classify each record by current, report the two states separately, headline the active-state power factor, and correct the session energy three ways. See [`docs/LOAD_STATES.md`](docs/LOAD_STATES.md). ## Web app — no install @@ -100,7 +101,9 @@ fluke-analyze path/to/ES.NNN -o output/ \ | `--tod-bin MINS` | `1` | Time-of-day bin width in minutes. | | `--demand-window SECS` | `900` | Rolling peak-demand window. Reports peak demand + the window it occurred in (`demand.json` + XLSX). See [`docs/DEMAND.md`](docs/DEMAND.md). | | `--tz ZONE` | UTC | Render report timestamps in local + UTC for an IANA zone (e.g. `America/Chicago`). Default UTC only. | -| `--auto-reverse-cts` | off | Auto-detect a reversed-CT install (sustained negative real power) and apply `--reverse-cts` automatically, with a loud notice. See `docs/CT_REVERSAL.md`. | +| `--auto-reverse-cts` | off | Auto-detect a reversed-CT install and apply `--reverse-cts` automatically, with a loud notice. Decides on the dominant **high-current (active)** state — robust for bimodal loads. See `docs/CT_REVERSAL.md`. | +| `--standby-threshold-a A` | `50` | Per-phase mean current (A) at/above which a record is **active** load (else standby). Drives the active/standby split (`load_states.csv`/`.json`), the energy correction (three figures), and the magnitude-weighted reverse-CTs decision. See [`docs/LOAD_STATES.md`](docs/LOAD_STATES.md). | +| `--load-states` | auto | Force the active/standby load-state report (emitted by default; flag is an explicit opt-in). | | `--rules-file FILE` | off | JSON/TOML EventRules overrides keyed by asset name. See [`docs/RULES_FILE.md`](docs/RULES_FILE.md). | | `--no-stats` | | Skip whole-session statistics (`stats.json`/`stats.csv` + XLSX sheet). | | `--format` | `png` | `png` or `svg` | diff --git a/ROADMAP.md b/ROADMAP.md index 0d127c8..106d68b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,6 +4,28 @@ Shipped releases live in [CHANGELOG.md](CHANGELOG.md). This file is the backlog of ideas we've discussed but not yet committed to a specific release. +## Shipped in v0.8 — active/standby load-state split + +Bimodal-load handling, built from the real P115RE coating-rectifier session, +parity-tested Python↔JS: + +- **Current-gated classifier** (`--standby-threshold-a`, default 50 A) splitting + each record into active vs standby, reported separately in + `load_states.csv`/`.json`. +- **Three explicit energy figures** (as-measured / active-only / net-clip + standby) — the standby real-power sign is unreliable at low current, so the + historic signed sum understates consumption; the report never silently + changes it. +- **Headline PF = active-state PF** in the narrative/summary/HTML (the blended + whole-session PF is meaningless for a bimodal load). +- **Magnitude-weighted reverse-CTs auto-detect** — decides on the dominant + high-current (active) state instead of the fragile whole-session negative-P + count. Manual `--reverse-cts` unchanged. +- Shift rows gain `active_duty_pct` / `active_kWh` / `active_PF_avg`. +- New `analysis.py` funcs (`classify_load_states`, `load_state_rows`, + `session_energy`, `active_state_pf`) + JS port + parity test + + `docs/LOAD_STATES.md`. + ## Shipped in v0.7 — generalized shift splitting Operator-defined, named, midnight-wrapping shift windows so usage can be diff --git a/docs/LOAD_STATES.md b/docs/LOAD_STATES.md new file mode 100644 index 0000000..867f9de --- /dev/null +++ b/docs/LOAD_STATES.md @@ -0,0 +1,129 @@ +# Load states: active vs standby (`load_states`) + +Many factory loads are **bimodal** — they alternate between a heavy *active* +draw and a light *standby* state. Blending the two into one session mean buries +the real consumption and produces a meaningless average power factor. The +load-state report classifies every record as **active** or **standby**, reports +the two states separately, surfaces the **active-state PF**, and corrects the +session energy. + +This was built from the real P115RE coating-rectifier session (ES.004, +`--reverse-cts`, `America/Chicago`): + +| state | duty | I/phase | P_total | P1 (fundamental) | PF | +|---------|------|---------|---------|------------------|-------| +| active | ~49% | ~239 A | +97 kW | — | +0.47 | +| standby | ~47% | ~16 A | −7.6 kW | −7.7 kW | −0.64 | + +## Why classify by *current*, not power + +The whole question is whether the **power sign** is trustworthy. With the global +`--reverse-cts`, the active state reads +97 kW (correct — that is the coating +draw), but the standby state reads −7.6 kW. A rectifier in standby should draw +small *positive* core/copper losses, not export. The standby reading is balanced +across all three phases and the **fundamental** P1 is also negative (−7.7 kW), so +it is not a harmonic artifact — it is simply that **no single CT polarity makes +both states physical**, and the low-current sign is unreliable. + +So we gate on **current**, which is unambiguous: + +> A record is **active** when its mean per-phase average current +> `(I_a_avg + I_b_avg + I_c_avg) / 3` is **≥ the threshold** (default **50 A**), +> otherwise **standby**. + +A single dropped phase is ignored (the mean is taken over the finite phases), so +one NaN does not drag a record into standby. The threshold is configurable with +`--standby-threshold-a N`. A single threshold is used (no transition band); set +it between the two clusters — for the P115RE the active state is ~239 A and the +standby state is ~16 A, so the 50 A default sits comfortably between them. + +## The three energy figures + +Because the standby sign is unreliable, the as-measured signed energy is +**understated** (the bogus −7.6 kW standby subtracts). The report surfaces all +three figures explicitly — it never silently changes the historic number: + +- **`energy_as_measured_kWh`** — the signed sum of `P_total_avg_W` over all + records. This is the existing/historic behavior. *(P115RE: ~6,054 kWh.)* +- **`energy_active_kWh`** — energy from the **active** records only. + *(P115RE: ~6,638 kWh.)* +- **`energy_net_clip_standby_kWh`** — active records pass through unchanged; + standby real power is **clipped to ≥ 0** (a rectifier in standby never + exports). *(P115RE: ~6,684 kWh.)* + +All three use the tool's standard convention: per record (1 s) energy = +`P_total_avg_W / 1000 / 3600`, summed; non-finite samples are skipped. + +> **Caveat (carried in every output):** standby real-power sign is unreliable at +> low current, so the **active** and **clip** figures are the defensible +> consumption — not the as-measured signed sum. + +## Headline power factor + +For a bimodal load the blended whole-session PF is meaningless (the P115RE +blended PF is −0.09). The **meaningful** figure is the **active-state PF** +(~0.47), so that is what the narrative, summary, and HTML report headline. The +raw whole-session PF is still reported, but de-emphasized and labeled. + +## Outputs + +`load_states.csv` — one row per state (active, then standby): + +| column | meaning | +|-----------------|------------------------------------------| +| `state` | `active` or `standby` | +| `records` | record count in the state | +| `hours` | hours (records / 3600) | +| `duty_pct` | percent of all records in the state | +| `kWh` | mean P × hours for the state | +| `P_avg_kW` | mean real power (kW) | +| `P_min_kW` | min real power (kW) | +| `P_max_kW` | max real power (kW) | +| `I_avg_A` | mean per-phase current (A) | +| `S_avg_kVA` | mean apparent power (kVA) | +| `PF_avg` | mean power factor | +| `V_LN_avg_V` | mean L-N voltage (outage zeros excluded) | +| `V_THD_p95_pct` | 95th-percentile V_THD | + +`load_states.json` carries the same `states` rows plus `standby_threshold_a`, +the three `energy` figures, and the caveat `note`. A compact load-state table +(and the three energy figures) is embedded in `summary.txt` and the HTML report. + +The report is emitted automatically (it is cheap — a couple of streaming +passes); `--load-states` is accepted as an explicit opt-in but is not required. + +## Integration with shifts + +When `--split-by shifts` is used, each shift row in `shift_comparison.csv/json` +also carries its **active** load, using the same current cut: + +- `active_records` — active record count in the shift +- `active_duty_pct` — percent of the shift's records that are active +- `active_kWh` — active-only energy for the shift +- `active_PF_avg` — active-state PF for the shift + +## The magnitude-weighted reverse-CTs decision + +The same active/standby insight hardens the **auto** reverse-CTs heuristic +(`--auto-reverse-cts`). A naive count-based test ("is P negative for ≥ 50 % of +records?") is fragile for bimodal loads: the P115RE reads negative more than half +the time (standby) while the real consumption — the active state — is clearly +positive. So the auto-detect now decides on the **dominant high-current (active) +state**: *is real power negative when current is high?* The whole-session count +fields are still reported for context, but `reversed` and the operator notice key +off the active state (`basis: "active"`). If there is no active population at all +(everything below the threshold), it falls back to the whole-session count +(`basis: "whole_session"`). The manual `--reverse-cts` behavior is unchanged — +only the AUTO heuristic was improved. + +## Tuning the threshold + +```bash +# Default 50 A active/standby cut +fluke-analyze ES.004 --reverse-cts --tz America/Chicago + +# Lower the cut so a small steady draw counts as active +fluke-analyze ES.004 --reverse-cts --standby-threshold-a 25 --tz America/Chicago +``` + +See also [CT_REVERSAL.md](CT_REVERSAL.md) and [SHIFTS.md](SHIFTS.md). diff --git a/python/pyproject.toml b/python/pyproject.toml index 50bb0bb..07ef864 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "fluke-3540-analyzer" -version = "0.7.0" +version = "0.8.0" description = "Parser, event detector, and chart generator for Fluke 3540 FC three-phase power-quality sessions. See https://github.com/GrumpyTanker/fluke-3540-analyzer." requires-python = ">=3.10" license = {text = "MIT"} diff --git a/python/src/fluke_3540/__init__.py b/python/src/fluke_3540/__init__.py index 7350e37..42ee55c 100644 --- a/python/src/fluke_3540/__init__.py +++ b/python/src/fluke_3540/__init__.py @@ -4,4 +4,4 @@ The canonical field map and binary layout live in spec/field_map.json at the repo root and are shared with the JavaScript port. See README.md. """ -__version__ = "0.7.0" +__version__ = "0.8.0" diff --git a/python/src/fluke_3540/analysis.py b/python/src/fluke_3540/analysis.py index 8b01ee4..24629e3 100644 --- a/python/src/fluke_3540/analysis.py +++ b/python/src/fluke_3540/analysis.py @@ -192,71 +192,162 @@ def whole_session_stats( # load reads as a persistent *generator* (P_total < 0). We flag a session when # real power is negative for a high fraction of NON-OUTAGE time — outage samples # (all phases collapsed) are excluded because P there is ~0/noise. +# +# Bimodal loads (e.g. a rectifier that toggles between a heavy ACTIVE draw and a +# light STANDBY state) defeat a naive count-based test: at low current the real- +# power sign is unreliable, so a session can read negative-P more than half the +# time while the *real* consumption — the high-current active state — is clearly +# positive (or clearly negative if the CTs really are backwards). The auto-detect +# therefore decides on the DOMINANT HIGH-CURRENT (active) state: it asks whether +# real power is negative when current is high. The whole-session count-based +# fields are still reported (for context / back-compat), but ``reversed`` and the +# operator notice key off the active state. + +# Default per-phase current (A) above which a record counts as "active" load. +# Shared with the load-state classifier so the two views agree. +STANDBY_CURRENT_THRESHOLD_A = 50.0 + + +def _mean_phase_current(ia, ib, ic, i: int) -> float: + """Mean of the three per-phase avg currents at record ``i`` (NaN->skipped). + + Returns the average over the finite phases (0.0 if none are finite), so a + single dropped phase doesn't drag the classifier toward standby. + """ + s = 0.0 + k = 0 + for c in (ia[i], ib[i], ic[i]): + if c == c and c not in (math.inf, -math.inf): # finite + s += c + k += 1 + return (s / k) if k else 0.0 + def detect_ct_reversal( store: ColumnStore, neg_fraction_threshold: float = 0.50, outage_v_threshold: float = 50.0, + active_threshold_a: float = STANDBY_CURRENT_THRESHOLD_A, ) -> dict: - """Detect a likely reversed-CT install from sustained negative real power. + """Detect a likely reversed-CT install, deciding on the active (high-I) state. Returns a dict: { - "reversed": bool, # True if neg fraction >= threshold - "frac_negative": float, # fraction of non-outage records with P<0 + "reversed": bool, # decision (see basis below) + "frac_negative": float, # whole-session: fraction non-outage P<0 "non_outage_records": int, "negative_records": int, "mean_p_w": float, # mean P over finite non-outage records - "threshold": float, + "threshold": float, # neg_fraction_threshold + # --- magnitude-weighted (active-state) decision --- + "basis": "active" | "whole_session", + "active_threshold_a": float, + "active_records": int, # non-outage records with mean I >= thresh + "active_negative_records": int, + "active_frac_negative": float, + "active_mean_p_w": float, # mean P over the active state } - A correctly-wired load draws positive real power essentially all the time, - so even a modestly-sustained negative-P fraction is a strong reversal signal; - the default 0.50 threshold (negative more often than positive) catches it - while staying clear of brief regen/export blips. Non-finite P samples are - skipped (the real meter occasionally emits NaN). ``reversed`` True means the - data looks like a load wired with backwards CTs — re-run with - ``--reverse-cts`` (or ``--auto-reverse-cts``) to correct it. + DECISION BASIS. When the session has a meaningful active (high-current) + population, ``reversed`` is True iff real power is negative for at least + ``neg_fraction_threshold`` of the ACTIVE records — i.e. the real consumption + looks like export. This is robust for bimodal loads where the low-current + standby sign is noise. If there is no active population (everything is below + ``active_threshold_a``), the test falls back to the whole-session count-based + fraction (``basis`` = "whole_session"). Non-finite P samples are skipped (the + real meter occasionally emits NaN). ``reversed`` True means the data looks + like a load wired with backwards CTs — re-run with ``--reverse-cts`` (or + ``--auto-reverse-cts``) to correct it. """ p = store.col("P_total_avg_W") va = store.col("V_LN_a_avg_V") vb = store.col("V_LN_b_avg_V") vc = store.col("V_LN_c_avg_V") + ia = store.col("I_a_avg_A") + ib = store.col("I_b_avg_A") + ic = store.col("I_c_avg_A") n = store.n non_outage = 0 negative = 0 p_sum = 0.0 p_count = 0 + active = 0 + active_negative = 0 + active_p_sum = 0.0 + active_p_count = 0 for i in range(n): if va[i] > outage_v_threshold and vb[i] > outage_v_threshold and vc[i] > outage_v_threshold: non_outage += 1 pv = p[i] - if pv == pv and pv not in (math.inf, -math.inf): # finite + finite = pv == pv and pv not in (math.inf, -math.inf) + if finite: p_sum += pv p_count += 1 if pv < 0: # NaN < 0 is False, so non-finite never counts as negative negative += 1 + if _mean_phase_current(ia, ib, ic, i) >= active_threshold_a: + active += 1 + if finite: + active_p_sum += pv + active_p_count += 1 + if pv < 0: + active_negative += 1 frac = (negative / non_outage) if non_outage else 0.0 mean_p = (p_sum / p_count) if p_count else 0.0 + active_frac = (active_negative / active) if active else 0.0 + active_mean_p = (active_p_sum / active_p_count) if active_p_count else 0.0 + + if active > 0: + basis = "active" + reversed_ = active_frac >= neg_fraction_threshold + else: + basis = "whole_session" + reversed_ = frac >= neg_fraction_threshold + return { - "reversed": frac >= neg_fraction_threshold, + "reversed": reversed_, "frac_negative": frac, "non_outage_records": non_outage, "negative_records": negative, "mean_p_w": mean_p, "threshold": neg_fraction_threshold, + "basis": basis, + "active_threshold_a": active_threshold_a, + "active_records": active, + "active_negative_records": active_negative, + "active_frac_negative": active_frac, + "active_mean_p_w": active_mean_p, } def ct_reversal_notice(result: dict) -> str: - """A loud, explicit operator-facing notice for a flagged CT reversal.""" - pct = result["frac_negative"] * 100.0 + """A loud, explicit operator-facing notice for a flagged CT reversal. + + Keys the headline numbers off the ACTIVE (high-current) state when that is + the decision basis, since the low-current standby sign is unreliable. + """ + if result.get("basis") == "active": + pct = result["active_frac_negative"] * 100.0 + mean_kw = result["active_mean_p_w"] / 1000.0 + basis_line = ( + f" Real power is NEGATIVE for {pct:.1f}% of ACTIVE (high-current, " + f"I >= {result['active_threshold_a']:.0f} A/phase) time " + f"(active mean P = {mean_kw:.1f} kW). A load should draw positive " + "real power when it is actually running." + ) + else: + pct = result["frac_negative"] * 100.0 + mean_kw = result["mean_p_w"] / 1000.0 + basis_line = ( + f" Real power (P_total) is NEGATIVE for {pct:.1f}% of non-outage " + f"time (mean P = {mean_kw:.1f} kW). A load should draw positive " + "real power." + ) return ( " !! CT REVERSAL DETECTED !!\n" - f" Real power (P_total) is NEGATIVE for {pct:.1f}% of non-outage time " - f"(mean P = {result['mean_p_w'] / 1000:.1f} kW). A load should draw " - "positive real power — this signature means one or more iFlex CT probes " - "are clipped on backwards.\n" + f"{basis_line}\n" + " This signature means one or more iFlex CT probes are clipped on " + "backwards.\n" " Re-run with --reverse-cts to negate P/Q/PF/energy, or " "--auto-reverse-cts to apply the correction automatically." ) @@ -839,7 +930,9 @@ def start_date(i: int) -> str: def shift_comparison_rows(store: ColumnStore, ss: ShiftSet, events: Sequence, tz=None, nominal_ln_v: float = 277.0, - demand_window: int = 15) -> list[dict]: + demand_window: int = 15, + standby_threshold_a: float = STANDBY_CURRENT_THRESHOLD_A, + ) -> list[dict]: """The headline per-shift-name aggregate comparison. One row per named shift (plus ``unassigned`` if any records land there), @@ -847,7 +940,10 @@ def shift_comparison_rows(store: ColumnStore, ss: ShiftSet, events: Sequence, are filed to a shift by the shift their ``t_start`` falls in. ``demand_window`` is in MINUTES (peak rolling demand within the shift's - records). Per-row energy/avg/percentiles are memory-bounded. + records). Per-row energy/avg/percentiles are memory-bounded. Each row also + carries its active-state load (``active_duty_pct``, ``active_kWh``, + ``active_PF_avg``) using ``standby_threshold_a`` as the active/standby + current cut. """ by_name = aggregate_shifts(store, ss, tz=tz) win_by_name = {sh.name: sh for sh in ss.shifts} @@ -872,17 +968,20 @@ def shift_comparison_rows(store: ColumnStore, ss: ShiftSet, events: Sequence, window = sh.window_str if sh is not None else "—" bucket_events = ev_by_name.get(name, []) rows.append(_shift_row(name, window, sub, bucket_events, - nominal_ln_v, demand_window)) + nominal_ln_v, demand_window, + standby_threshold_a)) return rows def _shift_row(name: str, window: str, sub: ColumnStore, bucket_events, - nominal_ln_v: float, demand_window: int) -> dict: + nominal_ln_v: float, demand_window: int, + standby_threshold_a: float = STANDBY_CURRENT_THRESHOLD_A) -> dict: """Aggregate one shift's records into a comparison row.""" nrec = sub.n p = sub.col("P_total_avg_W") pf = sub.col("PF_total_avg") va = sub.col("V_LN_a_avg_V"); vb = sub.col("V_LN_b_avg_V"); vc = sub.col("V_LN_c_avg_V") + ia = sub.col("I_a_avg_A"); ib = sub.col("I_b_avg_A"); ic = sub.col("I_c_avg_A") vth_a = sub.col("V_THD_pct_a_avg") vth_b = sub.col("V_THD_pct_b_avg") vth_c = sub.col("V_THD_pct_c_avg") @@ -893,14 +992,25 @@ def _shift_row(name: str, window: str, sub: ColumnStore, bucket_events, v_mom = _RunningMoments() v_sketch = _PercentileSketch(0.0, 400.0) vthd_sketch = _PercentileSketch(0.0, 50.0) + # Active-state (high-current) sub-aggregates for the shift's own load split. + act_p_mom = _RunningMoments() + act_pf_mom = _RunningMoments() + act_records = 0 for i in range(nrec): pv = p[i] + is_active = _mean_phase_current(ia, ib, ic, i) >= standby_threshold_a + if is_active: + act_records += 1 if math.isfinite(pv): p_mom.add(pv) p_min = min(p_min, pv); p_max = max(p_max, pv) + if is_active: + act_p_mom.add(pv) pfi = pf[i] if math.isfinite(pfi): pf_mom.add(pfi) + if is_active: + act_pf_mom.add(pfi) for vv in (va[i], vb[i], vc[i]): if math.isfinite(vv) and vv > 50.0: # ignore outage zeros v_mom.add(vv); v_sketch.add(vv) @@ -912,6 +1022,11 @@ def _shift_row(name: str, window: str, sub: ColumnStore, bucket_events, hours = p_mom.n / 3600.0 # 1 record == 1 s kwh = p_mean / 1000.0 * hours + act_p_mean = act_p_mom.mean if act_p_mom.n else 0.0 + act_hours = act_p_mom.n / 3600.0 + act_kwh = act_p_mean / 1000.0 * act_hours + act_duty_pct = (act_records / nrec * 100.0) if nrec else 0.0 + # Peak rolling demand within the shift's gathered records. demand = demand_analysis(sub, window_secs=max(1, demand_window * 60)) @@ -946,9 +1061,208 @@ def q(sketch, p_): "n_dips": n_dip, "n_swells": n_swell, "outage_minutes": outage_minutes, + # Active-state load (current-gated) for this shift. + "active_records": act_records, + "active_duty_pct": act_duty_pct, + "active_kWh": act_kwh, + "active_PF_avg": (act_pf_mom.mean if act_pf_mom.n else 0.0), + } + + +# --- Load-state split (active vs standby, current-gated) -------------------- +# +# Real bimodal loads (e.g. a coating rectifier) alternate between a heavy ACTIVE +# draw and a light STANDBY state. The two states have very different — and not +# uniformly trustworthy — power signatures, so blending them into one session +# mean buries the real consumption and produces a meaningless mean PF. +# +# We classify each record by mean per-phase CURRENT (not power, because the +# power SIGN at low current is exactly the thing in question): a record is +# ``active`` when (I_a_avg + I_b_avg + I_c_avg)/3 >= the threshold (default +# 50 A), else ``standby``. The two states are then aggregated and reported +# separately, and three energy figures are surfaced (see ``session_energy``). + +LOAD_STATES = ("active", "standby") + + +def classify_load_states( + store: ColumnStore, + threshold_a: float = STANDBY_CURRENT_THRESHOLD_A, +) -> dict[str, list[int]]: + """Partition record indices into ``active`` / ``standby`` by mean current. + + Returns ``{"active": [ascending idx], "standby": [ascending idx]}``. A + record is active when its mean per-phase avg current is >= ``threshold_a``. + Records with no finite phase current read 0 A and fall to standby. + """ + ia = store.col("I_a_avg_A") + ib = store.col("I_b_avg_A") + ic = store.col("I_c_avg_A") + active: list[int] = [] + standby: list[int] = [] + for i in range(store.n): + if _mean_phase_current(ia, ib, ic, i) >= threshold_a: + active.append(i) + else: + standby.append(i) + return {"active": active, "standby": standby} + + +def _load_state_row(name: str, sub: ColumnStore, total_records: int) -> dict: + """Aggregate one load state's gathered records into a comparison row.""" + nrec = sub.n + p = sub.col("P_total_avg_W") + pf = sub.col("PF_total_avg") + s = sub.col("S_total_avg_VA") + va = sub.col("V_LN_a_avg_V"); vb = sub.col("V_LN_b_avg_V"); vc = sub.col("V_LN_c_avg_V") + ia = sub.col("I_a_avg_A"); ib = sub.col("I_b_avg_A"); ic = sub.col("I_c_avg_A") + vth_a = sub.col("V_THD_pct_a_avg") + vth_b = sub.col("V_THD_pct_b_avg") + vth_c = sub.col("V_THD_pct_c_avg") + + p_mom = _RunningMoments() + p_min = math.inf; p_max = -math.inf + pf_mom = _RunningMoments() + s_mom = _RunningMoments() + i_mom = _RunningMoments() + v_mom = _RunningMoments() + vthd_sketch = _PercentileSketch(0.0, 50.0) + for i in range(nrec): + pv = p[i] + if math.isfinite(pv): + p_mom.add(pv) + p_min = min(p_min, pv); p_max = max(p_max, pv) + pfi = pf[i] + if math.isfinite(pfi): + pf_mom.add(pfi) + sv = s[i] + if math.isfinite(sv): + s_mom.add(sv) + i_mom.add(_mean_phase_current(ia, ib, ic, i)) + for vv in (va[i], vb[i], vc[i]): + if math.isfinite(vv) and vv > 50.0: # ignore outage zeros + v_mom.add(vv) + for tv in (vth_a[i], vth_b[i], vth_c[i]): + if math.isfinite(tv): + vthd_sketch.add(tv) + + p_mean = p_mom.mean if p_mom.n else 0.0 + hours = p_mom.n / 3600.0 # 1 record == 1 s + kwh = p_mean / 1000.0 * hours + + def q(sketch, p_): + v = sketch.quantile(p_) + return 0.0 if (v != v) else v + + return { + "state": name, + "records": nrec, + "hours": hours, + "duty_pct": (nrec / total_records * 100.0) if total_records else 0.0, + "kWh": kwh, + "P_avg_kW": p_mean / 1000.0, + "P_min_kW": (p_min / 1000.0 if p_min != math.inf else 0.0), + "P_max_kW": (p_max / 1000.0 if p_max != -math.inf else 0.0), + "I_avg_A": (i_mom.mean if i_mom.n else 0.0), + "S_avg_kVA": (s_mom.mean / 1000.0 if s_mom.n else 0.0), + "PF_avg": (pf_mom.mean if pf_mom.n else 0.0), + "V_LN_avg_V": (v_mom.mean if v_mom.n else 0.0), + "V_THD_p95_pct": q(vthd_sketch, 0.95), } +def load_state_rows( + store: ColumnStore, + threshold_a: float = STANDBY_CURRENT_THRESHOLD_A, +) -> list[dict]: + """Per-load-state comparison rows (one each for ``active`` then ``standby``). + + Each row carries records/hours/duty_pct/kWh, P avg/min/max (kW), I_avg (A), + S_avg (kVA), PF_avg, V_LN_avg (V), and V_THD_p95 (%). Rows are always in the + fixed order (active, standby) so downstream tables are stable. + """ + groups = classify_load_states(store, threshold_a) + total = store.n + rows: list[dict] = [] + for name in LOAD_STATES: + sub = gather_store(store, groups[name]) + rows.append(_load_state_row(name, sub, total)) + return rows + + +def session_energy( + store: ColumnStore, + threshold_a: float = STANDBY_CURRENT_THRESHOLD_A, +) -> dict: + """Three explicitly-labeled session energy figures (kWh) + the caveat note. + + Returns:: + + { + "energy_as_measured_kWh": float, # signed sum — current behavior + "energy_active_kWh": float, # active (high-I) records only + "energy_net_clip_standby_kWh": float, # standby real power clipped >=0 + "standby_threshold_a": float, + "note": str, + } + + All three use the same kWh convention as the rest of the tool: per record + (1 s) energy = P_total_avg_W / 1000 / 3600, summed. Non-finite P samples are + skipped. ``energy_as_measured_kWh`` is unchanged from the historic signed + sum; the active / clip figures correct for the unreliable low-current + standby sign and are the defensible consumption. + """ + p = store.col("P_total_avg_W") + ia = store.col("I_a_avg_A") + ib = store.col("I_b_avg_A") + ic = store.col("I_c_avg_A") + per_kwh = 1.0 / 1000.0 / 3600.0 # W * 1 s -> kWh + + as_measured = 0.0 + active = 0.0 + net_clip = 0.0 + for i in range(store.n): + pv = p[i] + if not (pv == pv and pv not in (math.inf, -math.inf)): # skip non-finite + continue + e = pv * per_kwh + as_measured += e + if _mean_phase_current(ia, ib, ic, i) >= threshold_a: + active += e + net_clip += e # active records pass through unchanged + else: + # standby: clip real power to >= 0 (a rectifier in standby draws + # small positive losses, never exports). + if pv > 0: + net_clip += e + return { + "energy_as_measured_kWh": as_measured, + "energy_active_kWh": active, + "energy_net_clip_standby_kWh": net_clip, + "standby_threshold_a": threshold_a, + "note": ( + "Standby real-power SIGN is unreliable at low current, so the " + "as-measured signed sum can understate consumption. energy_active " + "(active records only) and energy_net_clip_standby (standby real " + "power clipped to >=0) are the defensible consumption figures." + ), + } + + +def active_state_pf( + rows: Sequence[dict], +) -> float | None: + """Pull the active-state mean PF out of :func:`load_state_rows` output. + + Returns the active row's ``PF_avg`` (the meaningful headline PF for a + bimodal load), or ``None`` if there is no active row. + """ + for r in rows: + if r.get("state") == "active": + return r.get("PF_avg") + return None + + # --- Event markers / correlation (--mark / --marks) ------------------------- @dataclass(frozen=True) diff --git a/python/src/fluke_3540/cli.py b/python/src/fluke_3540/cli.py index 33d20fd..33d5723 100644 --- a/python/src/fluke_3540/cli.py +++ b/python/src/fluke_3540/cli.py @@ -131,10 +131,23 @@ def build_argparser() -> argparse.ArgumentParser: "Bare flag = all phases; pass a comma list like 'a,c' to " "only flip those phases (plus totals).") ap.add_argument("--auto-reverse-cts", action="store_true", - help="Auto-detect a reversed-CT install (sustained negative " - "real power on a load) and apply --reverse-cts " - "automatically, with a loud notice. No-op if the data " - "already reads as a normal load.") + help="Auto-detect a reversed-CT install and apply " + "--reverse-cts automatically, with a loud notice. The " + "heuristic decides on the dominant high-current " + "(active) state: is real power negative when current " + "is high? No-op if the active state already reads as a " + "normal load.") + ap.add_argument("--standby-threshold-a", dest="standby_threshold_a", + type=float, default=None, metavar="A", + help="Per-phase mean current (A) at/above which a record " + "counts as ACTIVE load (else standby). Default 50. " + "Drives the active/standby load-state split, the " + "energy correction, and the magnitude-weighted " + "reverse-CTs decision. See docs/LOAD_STATES.md.") + ap.add_argument("--load-states", dest="load_states", action="store_true", + help="Force the active/standby load-state report " + "(load_states.csv/json). Emitted by default in --auto; " + "this flag is only needed to opt in elsewhere.") ap.add_argument("--every", type=int, default=1, metavar="K", help="Emit every K-th record into the CSV (default 1, all)") ap.add_argument("--max-csv-rows", type=int, default=None, metavar="N", @@ -224,6 +237,13 @@ def build_argparser() -> argparse.ArgumentParser: return ap +def _standby_threshold(args: argparse.Namespace) -> float: + """Resolve the active/standby current cut (A), defaulting to the module value.""" + from .analysis import STANDBY_CURRENT_THRESHOLD_A + v = getattr(args, "standby_threshold_a", None) + return float(v) if v is not None else STANDBY_CURRENT_THRESHOLD_A + + def _is_csv_input(path: Path) -> bool: return path.is_file() and path.suffix.lower() == ".csv" @@ -313,7 +333,8 @@ def _parse_session(args: argparse.Namespace, outdir: Path, # the single-pass parse with the correction applied. store = res["store"] from .analysis import ct_reversal_notice, detect_ct_reversal - ct = detect_ct_reversal(store) + thr = _standby_threshold(args) + ct = detect_ct_reversal(store, active_threshold_a=thr) if ct["reversed"]: print(ct_reversal_notice(ct)) already_reversed = bool(reverse_cts) @@ -327,10 +348,15 @@ def _parse_session(args: argparse.Namespace, outdir: Path, log=print, progress_every=100_000, ) store = res["store"] - ct_after = detect_ct_reversal(store) - print(f" after auto-reverse: P now negative for " - f"{ct_after['frac_negative'] * 100:.1f}% of non-outage time " - f"(mean P = {ct_after['mean_p_w'] / 1000:.1f} kW)") + ct_after = detect_ct_reversal(store, active_threshold_a=thr) + if ct_after.get("basis") == "active": + print(f" after auto-reverse: active-state P now negative " + f"for {ct_after['active_frac_negative'] * 100:.1f}% " + f"(active mean P = {ct_after['active_mean_p_w'] / 1000:.1f} kW)") + else: + print(f" after auto-reverse: P now negative for " + f"{ct_after['frac_negative'] * 100:.1f}% of non-outage time " + f"(mean P = {ct_after['mean_p_w'] / 1000:.1f} kW)") return full_csv, min_csv, res["config"], store @@ -514,6 +540,59 @@ def _write_stats(outdir: Path, store: ColumnStore) -> dict: return stats +def _write_load_states(outdir: Path, store: ColumnStore, + threshold_a: float) -> dict: + """Active/standby load-state split + the three energy figures. + + Writes load_states.csv (one row per state) + load_states.json (rows + + threshold + the three energy figures + the standby-sign caveat). Returns the + payload so the narrative/summary can surface the active PF + energy. + """ + from .analysis import load_state_rows, session_energy + rows = load_state_rows(store, threshold_a=threshold_a) + energy = session_energy(store, threshold_a=threshold_a) + cols = ["state", "records", "hours", "duty_pct", "kWh", "P_avg_kW", + "P_min_kW", "P_max_kW", "I_avg_A", "S_avg_kVA", "PF_avg", + "V_LN_avg_V", "V_THD_p95_pct"] + with (outdir / "load_states.csv").open("w", newline="", encoding="utf-8") as fh: + w = _csv.writer(fh) + w.writerow(cols) + for r in rows: + w.writerow([_fmt_load_state_cell(c, r[c]) for c in cols]) + payload = { + "standby_threshold_a": threshold_a, + "states": rows, + "energy": energy, + "note": energy["note"], + } + (outdir / "load_states.json").write_text( + json.dumps(payload, indent=2), encoding="utf-8") + by = {r["state"]: r for r in rows} + a = by.get("active", {}) + s = by.get("standby", {}) + print(f"[load] active {a.get('duty_pct', 0):.0f}% duty: " + f"I={a.get('I_avg_A', 0):.0f} A P={a.get('P_avg_kW', 0):+.1f} kW " + f"PF={a.get('PF_avg', 0):+.2f} | standby " + f"I={s.get('I_avg_A', 0):.0f} A P={s.get('P_avg_kW', 0):+.1f} kW") + print(f"[load] energy kWh — as-measured {energy['energy_as_measured_kWh']:.0f} | " + f"active {energy['energy_active_kWh']:.0f} | " + f"net(clip standby≥0) {energy['energy_net_clip_standby_kWh']:.0f}") + return payload + + +def _fmt_load_state_cell(col: str, val): + """Format one load_states.csv cell (round floats; pass ints/strings).""" + if col == "state" or isinstance(val, int): + return val + if col in ("kWh", "P_avg_kW", "P_min_kW", "P_max_kW", "S_avg_kVA"): + return f"{val:.2f}" + if col == "PF_avg": + return f"{val:.3f}" + if col in ("hours", "duty_pct", "I_avg_A", "V_LN_avg_V", "V_THD_p95_pct"): + return f"{val:.2f}" + return f"{val:.2f}" + + def _write_markers(outdir: Path, markers, events: Sequence[Event]) -> None: from .analysis import correlate_markers corr = correlate_markers(markers, events) @@ -644,12 +723,14 @@ def _run_shifts(args: argparse.Namespace, outdir: Path, store: ColumnStore, # (1) Headline aggregate comparison ------------------------------------ rows = shift_comparison_rows(store, ss, events, tz=tz, nominal_ln_v=nominal_ln_v, - demand_window=demand_min) + demand_window=demand_min, + standby_threshold_a=_standby_threshold(args)) cols = ["shift", "window", "records", "hours", "kWh", "P_total_avg_W", "P_total_min_W", "P_total_max_W", "peak_demand_kW", "peak_demand_window_secs", "PF_avg", "V_LN_avg_V", "V_LN_p5_V", "V_LN_p95_V", "V_THD_p95_pct", "n_outages", "n_dips", "n_swells", - "outage_minutes"] + "outage_minutes", "active_records", "active_duty_pct", + "active_kWh", "active_PF_avg"] with (outdir / "shift_comparison.csv").open("w", newline="", encoding="utf-8") as fh: w = _csv.writer(fh) w.writerow(cols) @@ -705,11 +786,11 @@ def _fmt_shift_cell(col: str, val): return val if col == "kWh": return f"{val:.2f}" - if col in ("hours", "outage_minutes"): + if col in ("hours", "outage_minutes", "active_kWh"): return f"{val:.2f}" - if col == "PF_avg": + if col in ("PF_avg", "active_PF_avg"): return f"{val:.3f}" - if col == "peak_demand_kW": + if col in ("peak_demand_kW",): return f"{val:.2f}" return f"{val:.1f}" @@ -738,12 +819,14 @@ def _run_extra_analyses(args: argparse.Namespace, outdir: Path, renderer can surface a Statistics sheet. """ nominal_ln_v = _infer_nominal_ln_v(store, args.nominal_ln_v) + threshold_a = _standby_threshold(args) # ITIC always augments events.json (cheap, high-value for the deliverable). _augment_events_itic(outdir, events, nominal_ln_v) - # CT-reversal status snapshot for reports/web (cheap; one pass). + # CT-reversal status snapshot for reports/web (cheap; one pass). The decision + # is made on the dominant active (high-current) state. from .analysis import detect_ct_reversal, ieee519_compliance, sarfi_indices - ct = detect_ct_reversal(store) + ct = detect_ct_reversal(store, active_threshold_a=threshold_a) (outdir / "ct_reversal.json").write_text(json.dumps(ct, indent=2), encoding="utf-8") # IEEE 519 (THD) + IEEE 1159 / SARFI power-quality (Feature F). @@ -773,6 +856,12 @@ def _run_extra_analyses(args: argparse.Namespace, outdir: Path, if not getattr(args, "no_stats", False): stats = _write_stats(outdir, store) + # Active/standby load-state split + energy correction. Cheap (a couple of + # streaming passes), so emit it alongside the other always-on artifacts + # (narrative/pq/demand). --load-states is accepted as an explicit opt-in but + # the report is produced unconditionally here. + load_states = _write_load_states(outdir, store, threshold_a) + time_shift = store.time_shift markers = _parse_markers(args, time_shift) if markers: @@ -790,14 +879,17 @@ def _run_extra_analyses(args: argparse.Namespace, outdir: Path, else: _run_split_by(args, outdir, store, events, config, nominal_ln_v) - # Auto-narrative / executive summary (Feature E) — needs stats + ct. - narrative = _write_narrative(outdir, store, events, findings, stats, ct, config) + # Auto-narrative / executive summary (Feature E) — needs stats + ct + the + # load-state split (for the active-state PF + the corrected energy). + narrative = _write_narrative(outdir, store, events, findings, stats, ct, + config, load_states) - return stats, tod_rows, narrative, demand, shift_rows + return stats, tod_rows, narrative, demand, shift_rows, load_states def _write_narrative(outdir: Path, store: ColumnStore, events, findings, - stats: dict, ct: dict, config: dict) -> str: + stats: dict, ct: dict, config: dict, + load_states: dict | None = None) -> str: """Build the executive summary, write narrative.md, return the prose.""" from .narrative import build_narrative, narrative_markdown duration = None @@ -806,6 +898,7 @@ def _write_narrative(outdir: Path, store: ColumnStore, events, findings, narrative = build_narrative( events, findings, stats or None, ct, config=config, total_records=store.n, duration_secs=duration, + load_states=load_states, ) (outdir / "narrative.md").write_text( narrative_markdown(narrative, config), encoding="utf-8") @@ -820,7 +913,8 @@ def _write_summary_txt(outdir: Path, events: Sequence[Event], narrative: str | None = None, tz=None, tz_name: str | None = None, store: "ColumnStore | None" = None, - shift_rows: "list[dict] | None" = None) -> None: + shift_rows: "list[dict] | None" = None, + load_states: "dict | None" = None) -> None: lines: list[str] = ["Fluke 3540 FC Session Summary", "=" * 32, ""] if narrative: lines.append("Executive Summary") @@ -848,16 +942,43 @@ def _write_summary_txt(outdir: Path, events: Sequence[Event], for f in findings: lines.append(f" [{f.severity:5s}] {f.kind:25s} {f.headline}") lines.append("") + if load_states and load_states.get("states"): + thr = load_states.get("standby_threshold_a", 50.0) + lines.append(f"Load states (active vs standby, cut at {thr:.0f} A/phase)") + lines.append("-" * 16) + lines.append(f" {'state':<8} {'duty%':>6} {'records':>8} " + f"{'I_avg_A':>8} {'P_avg_kW':>9} {'S_avg_kVA':>10} " + f"{'PF':>6} {'kWh':>9}") + for r in load_states["states"]: + lines.append( + f" {r['state']:<8} {r['duty_pct']:>6.1f} {r['records']:>8d} " + f"{r['I_avg_A']:>8.1f} {r['P_avg_kW']:>9.2f} " + f"{r['S_avg_kVA']:>10.2f} {r['PF_avg']:>6.3f} {r['kWh']:>9.2f}") + en = load_states.get("energy", {}) + if en: + lines.append("") + lines.append(f" Energy as-measured (signed): " + f"{en['energy_as_measured_kWh']:>10.1f} kWh") + lines.append(f" Energy active-only: " + f"{en['energy_active_kWh']:>10.1f} kWh") + lines.append(f" Energy net (standby clip >=0): " + f"{en['energy_net_clip_standby_kWh']:>10.1f} kWh") + lines.append(" Note: standby real-power sign is unreliable at low " + "current; active/clip are the defensible consumption.") + lines.append("") if shift_rows: lines.append("Shift comparison") lines.append("-" * 16) lines.append(f" {'shift':<10} {'window':>13} {'records':>8} " - f"{'kWh':>9} {'Pavg_kW':>8} {'peak_kW':>8} {'PF':>5}") + f"{'kWh':>9} {'Pavg_kW':>8} {'peak_kW':>8} {'PF':>5} " + f"{'actDuty%':>8} {'actkWh':>9} {'actPF':>6}") for r in shift_rows: lines.append( f" {r['shift']:<10} {r['window']:>13} {r['records']:>8d} " f"{r['kWh']:>9.2f} {r['P_total_avg_W']/1000:>8.2f} " - f"{r['peak_demand_kW']:>8.2f} {r['PF_avg']:>5.3f}") + f"{r['peak_demand_kW']:>8.2f} {r['PF_avg']:>5.3f} " + f"{r.get('active_duty_pct', 0):>8.1f} " + f"{r.get('active_kWh', 0):>9.2f} {r.get('active_PF_avg', 0):>6.3f}") lines.append("") lines.append(f"Events detected: {len(events)}") for ev in events: @@ -960,6 +1081,12 @@ def _render_phase(args: argparse.Namespace, outdir: Path, full_csv: Path, related_event_ids=tuple(d.get("related_event_ids", [])), recommended_actions=tuple(d.get("recommended_actions", [])), )) + # Reload the load-state split from disk (also covers --plot-only). + load_states_payload = None + ls_path = outdir / "load_states.json" + if ls_path.exists(): + import json as _json + load_states_payload = _json.loads(ls_path.read_text(encoding="utf-8")) write_html_report( html_path, charts_dir=charts_dir, config=config, @@ -967,6 +1094,7 @@ def _render_phase(args: argparse.Namespace, outdir: Path, full_csv: Path, events=events, snapshots=snaps, findings=loaded_findings, narrative=narrative, + load_states=load_states_payload, ) if args.pdf: @@ -1117,13 +1245,14 @@ def print(*a, **kw): # noqa: A001 — intentional shadow # Post-detection analysis features (markers, stats, tod, split) run on # the in-memory store + on-disk CSVs. - stats, tod_rows, narrative, demand, shift_rows = _run_extra_analyses( + (stats, tod_rows, narrative, demand, shift_rows, + load_states) = _run_extra_analyses( args, outdir, store, events, findings, config, full_csv, min_csv) # Re-write summary.txt with the executive narrative + tz-aware time range. _write_summary_txt(outdir, events, snaps, findings, config, narrative=narrative, tz=getattr(args, "_tz", None), tz_name=getattr(args, "tz", None), store=store, - shift_rows=shift_rows) + shift_rows=shift_rows, load_states=load_states) if getattr(args, "json_mode", False): _emit_json(events, snaps, findings, config) diff --git a/python/src/fluke_3540/narrative.py b/python/src/fluke_3540/narrative.py index 7e61c5e..cf2edfb 100644 --- a/python/src/fluke_3540/narrative.py +++ b/python/src/fluke_3540/narrative.py @@ -45,12 +45,16 @@ def build_narrative( config: dict | None = None, total_records: int | None = None, duration_secs: float | None = None, + load_states: dict | None = None, ) -> str: """Return a deterministic plain-English executive summary string. ``events`` are Event objects (id/kind/t_start/t_end/severity/affected_phases), ``findings`` are Finding objects (kind/severity/headline), ``stats`` is the whole_session_stats dict, ``ct_reversal`` is detect_ct_reversal output. + ``load_states`` is the _write_load_states payload (states + energy); when + present, the headline PF is the ACTIVE-state PF and the corrected energy + figures are surfaced. """ sentences: list[str] = [] config = config or {} @@ -105,8 +109,48 @@ def build_narrative( else: sentences.append("No outages, dips, or swells were detected.") - # 3) Power factor / imbalance from stats + findings - if stats and "PF_total_avg" in stats: + # 2b) Active/standby load split + corrected energy (bimodal loads). + if load_states and load_states.get("states"): + by = {r.get("state"): r for r in load_states["states"]} + a = by.get("active") + s = by.get("standby") + if a and a.get("records"): + bits = ( + f"The load is bimodal: an ACTIVE state ({a['duty_pct']:.0f}% " + f"duty, {a['I_avg_A']:.0f} A/phase, {a['P_avg_kW']:+.0f} kW, " + f"PF {a['PF_avg']:+.2f})") + if s and s.get("records"): + bits += ( + f" and a STANDBY state ({s['duty_pct']:.0f}% duty, " + f"{s['I_avg_A']:.0f} A/phase, {s['P_avg_kW']:+.0f} kW)") + sentences.append(bits + ".") + energy = load_states.get("energy") + if energy: + sentences.append( + f"Energy: {energy['energy_as_measured_kWh']:.0f} kWh " + f"as-measured (signed), {energy['energy_active_kWh']:.0f} kWh " + f"active-only, {energy['energy_net_clip_standby_kWh']:.0f} kWh " + "net (standby clipped >=0). Standby real-power sign is " + "unreliable at low current, so the active/clip figures are the " + "defensible consumption.") + + # 3) Power factor / imbalance from stats + findings. For a bimodal load the + # headline PF is the ACTIVE-state PF (the blended whole-session PF is + # meaningless); the raw whole-session PF is kept but de-emphasized. + active_pf = None + if load_states and load_states.get("states"): + for r in load_states["states"]: + if r.get("state") == "active" and r.get("records"): + active_pf = r.get("PF_avg") + break + if active_pf is not None: + whole = "" + if stats and "PF_total_avg" in stats: + whole = (f" (whole-session blended PF {stats['PF_total_avg']['mean']:.2f}, " + "not meaningful for a bimodal load)") + sentences.append( + f"Active-state power factor averaged {active_pf:.2f}{whole}.") + elif stats and "PF_total_avg" in stats: pf = stats["PF_total_avg"] sentences.append( f"Power factor (total) averaged {pf['mean']:.2f} " diff --git a/python/src/fluke_3540/plots/html_report.py b/python/src/fluke_3540/plots/html_report.py index 703ccca..7b87a11 100644 --- a/python/src/fluke_3540/plots/html_report.py +++ b/python/src/fluke_3540/plots/html_report.py @@ -199,6 +199,53 @@ def _insights_html(findings: Sequence[Finding]) -> str: return "\n".join(out) +def _load_states_html(load_states: Mapping | None) -> str: + """Compact active-vs-standby load-state table + the three energy figures.""" + if not load_states or not load_states.get("states"): + return "" + thr = load_states.get("standby_threshold_a", 50.0) + rows = load_states["states"] + parts = [ + f"

Load states " + f"(active vs standby, cut at {thr:.0f} A/phase)

", + "" + "" + "" + "", + ] + for r in rows: + parts.append( + "" + f"" + f"" + f"" + f"" + f"" + f"" + f"" + f"" + "" + ) + parts.append("
stateduty %recordsI avg (A)P avg (kW)S avg (kVA)PFkWh
{html.escape(str(r['state']))}{r['duty_pct']:.1f}{r['records']}{r['I_avg_A']:.1f}{r['P_avg_kW']:+.2f}{r['S_avg_kVA']:.2f}{r['PF_avg']:+.3f}{r['kWh']:.2f}
") + en = load_states.get("energy") + if en: + parts.append( + "" + "" + f"" + f"" + f"" + "
energy figurekWh
as-measured (signed)" + f"{en['energy_as_measured_kWh']:.1f}
active-only{en['energy_active_kWh']:.1f}
net (standby clipped ≥0)" + f"{en['energy_net_clip_standby_kWh']:.1f}
" + "

Standby real-power sign is unreliable at low " + "current, so the active / clip figures are the defensible " + "consumption.

" + ) + parts.append("
") + return "".join(parts) + + def render_report_html( *, title: str, @@ -210,6 +257,7 @@ def render_report_html( findings: Sequence[Finding] = (), generated_at: dt.datetime | None = None, narrative: str | None = None, + load_states: Mapping | None = None, ) -> str: generated_at = generated_at or dt.datetime.now(dt.timezone.utc) body = [] @@ -222,6 +270,9 @@ def render_report_html( ) body.append("

Summary

") body.append(_summary_dl_html(summary_stats, config)) + ls_html = _load_states_html(load_states) + if ls_html: + body.append(ls_html) if findings: body.append(_insights_html(findings)) body.append("

Events

") @@ -254,6 +305,7 @@ def write_html_report( findings: Sequence[Finding] = (), title: str | None = None, narrative: str | None = None, + load_states: Mapping | None = None, ) -> Path: """High-level wrapper: read PNGs from charts_dir, write a self-contained HTML report. @@ -272,7 +324,7 @@ def write_html_report( render_report_html( title=title, config=config, summary_stats=summary_stats, events=events, snapshots=snapshots, charts=charts, - findings=findings, narrative=narrative, + findings=findings, narrative=narrative, load_states=load_states, ), encoding="utf-8", ) diff --git a/python/tests/fixtures/analysis_golden.json b/python/tests/fixtures/analysis_golden.json index a1aed2e..28ec558 100644 --- a/python/tests/fixtures/analysis_golden.json +++ b/python/tests/fixtures/analysis_golden.json @@ -295,6 +295,60 @@ "i_max_A": 110.0 } ], + "load_states": { + "base_epoch_ms": 1779840000000, + "n_active": 50, + "n_standby": 50, + "threshold_a": 50.0, + "rows": [ + { + "state": "active", + "records": 50, + "hours": 0.013888888888888888, + "duty_pct": 50.0, + "kWh": 1.347222222222222, + "P_avg_kW": 97.0, + "P_min_kW": 97.0, + "P_max_kW": 97.0, + "I_avg_A": 239.0, + "S_avg_kVA": 206.0, + "PF_avg": 0.4699999988079071, + "V_LN_avg_V": 277.0, + "V_THD_p95_pct": 4.0062500000000005 + }, + { + "state": "standby", + "records": 50, + "hours": 0.013888888888888888, + "duty_pct": 50.0, + "kWh": -0.10555555555555554, + "P_avg_kW": -7.6, + "P_min_kW": -7.6, + "P_max_kW": -7.6, + "I_avg_A": 16.0, + "S_avg_kVA": 11.8, + "PF_avg": -0.6399999856948853, + "V_LN_avg_V": 277.0, + "V_THD_p95_pct": 4.0062500000000005 + } + ], + "energy": { + "energy_as_measured_kWh": 1.2416666666666634, + "energy_active_kWh": 1.347222222222222, + "energy_net_clip_standby_kWh": 1.347222222222222, + "standby_threshold_a": 50.0 + }, + "ct": { + "reversed": false, + "basis": "active", + "active_threshold_a": 50.0, + "active_records": 50, + "active_negative_records": 0, + "active_frac_negative": 0.0, + "active_mean_p_w": 97000.0, + "frac_negative": 0.5 + } + }, "itic_points": [ [ 70.0, @@ -347,7 +401,13 @@ "non_outage_records": 100, "negative_records": 70, "mean_p_w": -12000.0, - "threshold": 0.5 + "threshold": 0.5, + "basis": "active", + "active_threshold_a": 50.0, + "active_records": 100, + "active_negative_records": 70, + "active_frac_negative": 0.7, + "active_mean_p_w": -12000.0 }, "narrative": "Asset MAC03 captured over 168.0 h (590,000 one-second records). The most significant event was a 18.5 min outage at 2024-01-13 22:01 UTC, preceded by a phase-c dip to 72%. Power factor (total) averaged 0.81 (p5 0.70, p95 0.95). Power factor below 0.85 for 99.8% of non-outage time. WARNING: real power is negative for 52% of non-outage time \u2014 the iFlex CTs are likely reversed; re-run with --reverse-cts. Bottom line: 1 alert-level finding(s) warrant follow-up.", "ieee519": { @@ -452,7 +512,11 @@ "n_outages": 0, "n_dips": 0, "n_swells": 0, - "outage_minutes": 0.0 + "outage_minutes": 0.0, + "active_records": 2160, + "active_duty_pct": 100.0, + "active_kWh": 22.46690277777777, + "active_PF_avg": 0.9399999976158142 }, { "shift": "night", @@ -473,7 +537,11 @@ "n_outages": 0, "n_dips": 0, "n_swells": 0, - "outage_minutes": 0.0 + "outage_minutes": 0.0, + "active_records": 2160, + "active_duty_pct": 100.0, + "active_kWh": 19.396847222222263, + "active_PF_avg": 0.9399999976158142 } ], "occurrences_utc": [ @@ -540,7 +608,11 @@ "n_outages": 0, "n_dips": 0, "n_swells": 0, - "outage_minutes": 0.0 + "outage_minutes": 0.0, + "active_records": 1440, + "active_duty_pct": 100.0, + "active_kWh": 14.70750000000002, + "active_PF_avg": 0.939999997615815 }, { "shift": "B", @@ -561,7 +633,11 @@ "n_outages": 0, "n_dips": 0, "n_swells": 0, - "outage_minutes": 0.0 + "outage_minutes": 0.0, + "active_records": 1440, + "active_duty_pct": 100.0, + "active_kWh": 12.94979166666667, + "active_PF_avg": 0.939999997615815 }, { "shift": "C", @@ -582,7 +658,11 @@ "n_outages": 0, "n_dips": 0, "n_swells": 0, - "outage_minutes": 0.0 + "outage_minutes": 0.0, + "active_records": 1440, + "active_duty_pct": 100.0, + "active_kWh": 14.206458333333323, + "active_PF_avg": 0.939999997615815 } ] }, diff --git a/python/tests/test_analysis_parity_golden.py b/python/tests/test_analysis_parity_golden.py index 0eca3b4..e752c81 100644 --- a/python/tests/test_analysis_parity_golden.py +++ b/python/tests/test_analysis_parity_golden.py @@ -17,8 +17,9 @@ from fluke_3540.analysis import ( ShiftSet, classify_itic, demand_analysis, detect_ct_reversal, event_itic, - ieee519_compliance, sarfi_indices, shift_comparison_rows, shift_occurrences, - time_of_day_profile, whole_session_stats, + ieee519_compliance, load_state_rows, sarfi_indices, session_energy, + shift_comparison_rows, shift_occurrences, time_of_day_profile, + whole_session_stats, ) from fluke_3540.events import Event from fluke_3540.insights import Finding @@ -165,9 +166,46 @@ def test_emit_analysis_golden(): shift_store, shift_abc, events=[], tz=chi, nominal_ln_v=277.0, demand_window=15) + # --- Load-state split golden (active/standby + 3 energy figures) ------ + # A deterministic bimodal session the JS test recreates exactly: 50 active + # (high current, +P) then 50 standby (low current, -P). The JS parity test + # builds the same records and compares load_state_rows / session_energy / + # the active-state detect_ct_reversal decision. + ls_base = dt.datetime(2026, 5, 27, 0, 0, 0, tzinfo=dt.timezone.utc) + ls_n_active, ls_n_standby = 50, 50 + ls_overrides: dict = {} + plant_window(ls_overrides, 0, ls_n_active - 1, { + "I_a_avg_A": 239.0, "I_b_avg_A": 239.0, "I_c_avg_A": 239.0, + "P_total_avg_W": 97_000.0, "S_total_avg_VA": 206_000.0, + "PF_total_avg": 0.47, "V_THD_pct_a_avg": 3.0, "V_THD_pct_b_avg": 2.5, + "V_THD_pct_c_avg": 4.0}) + plant_window(ls_overrides, ls_n_active, ls_n_active + ls_n_standby - 1, { + "I_a_avg_A": 16.0, "I_b_avg_A": 16.0, "I_c_avg_A": 16.0, + "P_total_avg_W": -7_600.0, "S_total_avg_VA": 11_800.0, + "PF_total_avg": -0.64, "V_THD_pct_a_avg": 3.0, "V_THD_pct_b_avg": 2.5, + "V_THD_pct_c_avg": 4.0}) + ls_store = ColumnStore.from_records( + make_records(ls_n_active + ls_n_standby, base=ls_base, + overrides=ls_overrides)) + ls_rows = load_state_rows(ls_store, threshold_a=50.0) + ls_energy = session_energy(ls_store, threshold_a=50.0) + ls_ct = detect_ct_reversal(ls_store, active_threshold_a=50.0) + golden = { "stats": stats, "tod_rows": tod, + "load_states": { + "base_epoch_ms": int(ls_base.timestamp() * 1000), + "n_active": ls_n_active, + "n_standby": ls_n_standby, + "threshold_a": 50.0, + "rows": ls_rows, + "energy": {k: v for k, v in ls_energy.items() if k != "note"}, + "ct": {k: ls_ct[k] for k in ( + "reversed", "basis", "active_threshold_a", "active_records", + "active_negative_records", "active_frac_negative", + "active_mean_p_w", "frac_negative")}, + }, "itic_points": itic_points, "itic": itic, "event_itic": event_itic_out, @@ -216,3 +254,11 @@ def test_emit_analysis_golden(): # 3 days × 2 shifts, but the first record (00:00) is night and the run # continues; occurrences = day/night alternation. Expect 6-7 occurrences. assert len(shift_occ_utc) >= 6 + # Load states: active reads positive at high current -> NOT reversed despite + # standby dragging the whole-session negative fraction past 50%. + ls_by = {r["state"]: r for r in ls_rows} + assert ls_by["active"]["records"] == 50 + assert ls_by["standby"]["records"] == 50 + assert ls_ct["basis"] == "active" + assert ls_ct["reversed"] is False + assert ls_energy["energy_as_measured_kWh"] < ls_energy["energy_active_kWh"] diff --git a/python/tests/test_cli_features.py b/python/tests/test_cli_features.py index 9f4b186..dbace67 100644 --- a/python/tests/test_cli_features.py +++ b/python/tests/test_cli_features.py @@ -219,3 +219,99 @@ def test_split_by_union_of_events_equals_whole(tmp_path: Path): b_end = start + dt.timedelta(seconds=period.seconds) union += [e for e in whole if start <= e.t_start < b_end] assert {(e.kind, e.t_start) for e in union} == {(e.kind, e.t_start) for e in whole} + + +# --- Active/standby load-state split (CLI) ----------------------------------- + +def _bimodal_trend(path: Path, active_n: int, standby_n: int, + base: dt.datetime) -> None: + """Write a trend.bin: active_n high-current/+P records then standby_n + low-current/-P records (the P115RE-like bimodal shape).""" + import struct + from conftest import FIELD_INDEX, dt_to_filetime + from fluke_3540.parser import (DATA_FLOATS, HEADER_BYTES, RECORD_MAGIC) + + def floats_for(active: bool) -> list[float]: + f = [0.0] * DATA_FLOATS + for ph in ("a", "b", "c"): + for st in ("min", "max", "avg"): + f[FIELD_INDEX[f"V_LN_{ph}_{st}_V"]] = 277.0 + i = 239.0 if active else 16.0 + for ph in ("a", "b", "c"): + f[FIELD_INDEX[f"I_{ph}_avg_A"]] = i + f[FIELD_INDEX["freq_avg_Hz"]] = 60.0 + f[FIELD_INDEX["P_total_avg_W"]] = 97_000.0 if active else -7_600.0 + f[FIELD_INDEX["S_total_avg_VA"]] = 206_000.0 if active else 11_800.0 + f[FIELD_INDEX["PF_total_avg"]] = 0.47 if active else -0.64 + return f + + with path.open("wb") as fh: + for n in range(active_n + standby_n): + start_ft = dt_to_filetime(base + dt.timedelta(seconds=n)) + end_ft = dt_to_filetime(base + dt.timedelta(seconds=n + 1)) + header = ( + RECORD_MAGIC + + struct.pack("> 32 & 0xFFFFFFFF, start_ft & 0xFFFFFFFF) + + struct.pack("> 32 & 0xFFFFFFFF, end_ft & 0xFFFFFFFF) + + struct.pack(" all 100 active. + base = dt.datetime(2024, 1, 13, 22, 0, 0, tzinfo=dt.timezone.utc) + d = tmp_path / "ES.LS2"; d.mkdir() + _bimodal_trend(d / "trend.bin", active_n=50, standby_n=50, base=base) + out = tmp_path / "out" + rc = main([str(d), "-o", str(out), "--parse-only", "--no-stats", + "--standby-threshold-a", "10"]) + assert rc == 0 + payload = json.loads((out / "load_states.json").read_text()) + assert payload["standby_threshold_a"] == 10.0 + by = {r["state"]: r for r in payload["states"]} + assert by["active"]["records"] == 100 + assert by["standby"]["records"] == 0 + + +def test_shift_comparison_has_active_columns_cli(tmp_path: Path): + base = dt.datetime(2024, 1, 13, 22, 0, 0, tzinfo=dt.timezone.utc) + d = tmp_path / "ES.LS3"; d.mkdir() + _bimodal_trend(d / "trend.bin", active_n=120, standby_n=120, base=base) + out = tmp_path / "out" + rc = main([str(d), "-o", str(out), "--parse-only", "--split-by", "shifts", + "--no-xlsx"]) + assert rc == 0 + rows = list(csv.DictReader((out / "shift_comparison.csv").open())) + header = rows[0].keys() + for col in ("active_records", "active_duty_pct", "active_kWh", "active_PF_avg"): + assert col in header, f"missing shift column {col}" diff --git a/python/tests/test_ct_reversal.py b/python/tests/test_ct_reversal.py index dbd7f27..bf63551 100644 --- a/python/tests/test_ct_reversal.py +++ b/python/tests/test_ct_reversal.py @@ -1,6 +1,8 @@ """Tests for CT-reversal auto-detection (Feature C).""" from __future__ import annotations +import pytest + from fluke_3540.analysis import ct_reversal_notice, detect_ct_reversal from fluke_3540.store import ColumnStore @@ -69,3 +71,76 @@ def test_notice_is_loud_and_mentions_flags(): assert "CT REVERSAL DETECTED" in notice assert "--reverse-cts" in notice assert "--auto-reverse-cts" in notice + + +# --- magnitude-weighted (active-state) decision ------------------------------ + +def test_bimodal_decides_on_active_state_positive(): + # The real P115RE shape WITH --reverse-cts already correct: active draws + # high current + POSITIVE power (correct), standby collapses to low current + # + bogus NEGATIVE power. Whole-session count is 47% negative (near the + # 50% line), but the ACTIVE state is clearly positive -> NOT reversed. + overrides: dict = {} + plant_window(overrides, 0, 48, { # 49 active records, +97 kW, 239 A + "I_a_avg_A": 239.0, "I_b_avg_A": 239.0, "I_c_avg_A": 239.0, + "P_total_avg_W": 97_000.0}) + plant_window(overrides, 49, 99, { # 51 standby records, -7.6 kW, 16 A + "I_a_avg_A": 16.0, "I_b_avg_A": 16.0, "I_c_avg_A": 16.0, + "P_total_avg_W": -7_600.0}) + store = ColumnStore.from_records(make_records(100, overrides=overrides)) + res = detect_ct_reversal(store) + # whole-session count-based fraction is past 50% (the fragile signal)… + assert res["frac_negative"] >= 0.50 + # …but the decision is made on the active state, which is positive. + assert res["basis"] == "active" + assert res["active_records"] == 49 + assert res["active_negative_records"] == 0 + assert res["active_frac_negative"] == 0.0 + assert res["active_mean_p_w"] == pytest.approx(97_000.0) + assert res["reversed"] is False + + +def test_bimodal_decides_on_active_state_reversed(): + # Same bimodal shape but the ACTIVE state reads NEGATIVE (true backwards + # CTs): active is the high-current state and it exports -> reversed=True, + # even though standby happens to read small positive here. + overrides: dict = {} + plant_window(overrides, 0, 48, { # active, -97 kW (backwards), 239 A + "I_a_avg_A": 239.0, "I_b_avg_A": 239.0, "I_c_avg_A": 239.0, + "P_total_avg_W": -97_000.0}) + plant_window(overrides, 49, 99, { # standby, small +loss, 16 A + "I_a_avg_A": 16.0, "I_b_avg_A": 16.0, "I_c_avg_A": 16.0, + "P_total_avg_W": 500.0}) + store = ColumnStore.from_records(make_records(100, overrides=overrides)) + res = detect_ct_reversal(store) + # whole-session count-based fraction is BELOW 50% (only active is negative)… + assert res["frac_negative"] < 0.50 + # …but the active high-current state is fully negative -> reversed. + assert res["basis"] == "active" + assert res["active_frac_negative"] == pytest.approx(1.0) + assert res["reversed"] is True + + +def test_no_active_population_falls_back_to_whole_session(): + # Everything below the active threshold -> no active state -> the legacy + # whole-session count-based decision is used. + store = ColumnStore.from_records(make_records(100, defaults={ + "I_a_avg_A": 10.0, "I_b_avg_A": 10.0, "I_c_avg_A": 10.0, + "P_total_avg_W": -5_000.0})) + res = detect_ct_reversal(store) + assert res["basis"] == "whole_session" + assert res["active_records"] == 0 + assert res["reversed"] is True # 100% of non-outage is negative + + +def test_active_threshold_configurable(): + # 30 A current with the default 50 A threshold -> standby (fallback); + # lower the threshold to 20 A -> those become active and drive the decision. + store = ColumnStore.from_records(make_records(100, defaults={ + "I_a_avg_A": 30.0, "I_b_avg_A": 30.0, "I_c_avg_A": 30.0, + "P_total_avg_W": -5_000.0})) + res_default = detect_ct_reversal(store) + assert res_default["basis"] == "whole_session" + res_low = detect_ct_reversal(store, active_threshold_a=20.0) + assert res_low["basis"] == "active" + assert res_low["active_records"] == 100 diff --git a/python/tests/test_load_states.py b/python/tests/test_load_states.py new file mode 100644 index 0000000..57803e3 --- /dev/null +++ b/python/tests/test_load_states.py @@ -0,0 +1,176 @@ +"""Tests for the active/standby load-state split (current-gated). + +Real bimodal loads (a coating rectifier) alternate between a heavy ACTIVE draw +and a light STANDBY state. We classify each record by mean per-phase CURRENT, +report the two states separately, surface the active-state PF, and correct the +session energy three ways. +""" +from __future__ import annotations + +import datetime as dt + +import pytest + +from fluke_3540.analysis import ( + LOAD_STATES, + STANDBY_CURRENT_THRESHOLD_A, + active_state_pf, + classify_load_states, + load_state_rows, + session_energy, +) +from fluke_3540.parser import Record +from fluke_3540.store import ColumnStore + +from conftest import make_records, plant_window + + +def _bimodal_store(active_n=50, standby_n=50, + active_i=239.0, standby_i=16.0, + active_p=97_000.0, standby_p=-7_600.0, + active_pf=0.47, standby_pf=-0.64): + """A balanced bimodal P115RE-like session: an active +cluster and a standby + -cluster, classified by current. Active records draw high current + positive + power; standby collapses to low current + (bogus) negative power.""" + overrides: dict = {} + plant_window(overrides, 0, active_n - 1, { + "I_a_avg_A": active_i, "I_b_avg_A": active_i, "I_c_avg_A": active_i, + "P_total_avg_W": active_p, "S_total_avg_VA": active_p / active_pf, + "PF_total_avg": active_pf, + }) + plant_window(overrides, active_n, active_n + standby_n - 1, { + "I_a_avg_A": standby_i, "I_b_avg_A": standby_i, "I_c_avg_A": standby_i, + "P_total_avg_W": standby_p, "S_total_avg_VA": abs(standby_p / standby_pf), + "PF_total_avg": standby_pf, + }) + recs = make_records(active_n + standby_n, overrides=overrides) + return ColumnStore.from_records(recs) + + +# --- classifier -------------------------------------------------------------- + +def test_classify_threshold_default(): + store = _bimodal_store(active_n=50, standby_n=50) + groups = classify_load_states(store) # default 50 A + assert groups["active"] == list(range(0, 50)) + assert groups["standby"] == list(range(50, 100)) + + +def test_classify_threshold_configurable(): + # With a 20 A threshold the 16 A standby is still standby; with a 10 A + # threshold it becomes active. + store = _bimodal_store(standby_i=16.0) + g20 = classify_load_states(store, threshold_a=20.0) + assert len(g20["active"]) == 50 and len(g20["standby"]) == 50 + g10 = classify_load_states(store, threshold_a=10.0) + assert len(g10["active"]) == 100 and g10["standby"] == [] + + +def test_classify_uses_mean_of_three_phases(): + # One phase high, two phases zero -> mean 80 A -> active at default 50 A. + overrides = {0: {"I_a_avg_A": 240.0, "I_b_avg_A": 0.0, "I_c_avg_A": 0.0}} + store = ColumnStore.from_records(make_records(1, overrides=overrides)) + g = classify_load_states(store) + assert g["active"] == [0] + # Drop it below: 120 A on one phase -> mean 40 A -> standby. + overrides = {0: {"I_a_avg_A": 120.0, "I_b_avg_A": 0.0, "I_c_avg_A": 0.0}} + store = ColumnStore.from_records(make_records(1, overrides=overrides)) + g = classify_load_states(store) + assert g["standby"] == [0] + + +def test_classify_boundary_inclusive(): + # Exactly at threshold counts as active (>=). + overrides = {0: {"I_a_avg_A": 50.0, "I_b_avg_A": 50.0, "I_c_avg_A": 50.0}} + store = ColumnStore.from_records(make_records(1, overrides=overrides)) + assert classify_load_states(store, threshold_a=50.0)["active"] == [0] + + +# --- load_state_rows --------------------------------------------------------- + +def test_load_state_rows_schema_and_order(): + store = _bimodal_store() + rows = load_state_rows(store) + assert [r["state"] for r in rows] == list(LOAD_STATES) == ["active", "standby"] + for r in rows: + for k in ("state", "records", "hours", "duty_pct", "kWh", "P_avg_kW", + "P_min_kW", "P_max_kW", "I_avg_A", "S_avg_kVA", "PF_avg", + "V_LN_avg_V", "V_THD_p95_pct"): + assert k in r, f"missing schema key {k}" + + +def test_load_state_rows_values(): + store = _bimodal_store(active_n=50, standby_n=50, + active_i=239.0, standby_i=16.0, + active_p=97_000.0, standby_p=-7_600.0, + active_pf=0.47, standby_pf=-0.64) + rows = load_state_rows(store) + by = {r["state"]: r for r in rows} + a, s = by["active"], by["standby"] + assert a["records"] == 50 and s["records"] == 50 + assert a["duty_pct"] == pytest.approx(50.0) + assert s["duty_pct"] == pytest.approx(50.0) + assert a["I_avg_A"] == pytest.approx(239.0) + assert s["I_avg_A"] == pytest.approx(16.0) + assert a["P_avg_kW"] == pytest.approx(97.0) + assert s["P_avg_kW"] == pytest.approx(-7.6) + assert a["PF_avg"] == pytest.approx(0.47) + assert s["PF_avg"] == pytest.approx(-0.64) + # active-state energy: 97 kW * 50 s = 50/3600 h + assert a["kWh"] == pytest.approx(97.0 * (50 / 3600.0)) + + +def test_active_state_pf_helper(): + store = _bimodal_store(active_pf=0.47) + rows = load_state_rows(store) + assert active_state_pf(rows) == pytest.approx(0.47) + # No active records -> None. + standby_only = ColumnStore.from_records( + make_records(10, defaults={"I_a_avg_A": 5.0, "I_b_avg_A": 5.0, + "I_c_avg_A": 5.0})) + rows2 = load_state_rows(standby_only) + assert active_state_pf(rows2) == 0.0 # active row present but empty -> 0.0 + + +# --- three energy figures ---------------------------------------------------- + +def test_three_energy_figures_distinct(): + # 50 active @ +97 kW, 50 standby @ -7.6 kW, 1 s each. + store = _bimodal_store(active_n=50, standby_n=50, + active_p=97_000.0, standby_p=-7_600.0) + e = session_energy(store) + per_s = 1.0 / 1000.0 / 3600.0 + as_measured = (50 * 97_000.0 + 50 * -7_600.0) * per_s + active = 50 * 97_000.0 * per_s + clip = 50 * 97_000.0 * per_s # standby negative clipped to 0 + assert e["energy_as_measured_kWh"] == pytest.approx(as_measured) + assert e["energy_active_kWh"] == pytest.approx(active) + assert e["energy_net_clip_standby_kWh"] == pytest.approx(clip) + # The understated as-measured < the corrected figures. + assert e["energy_as_measured_kWh"] < e["energy_active_kWh"] + assert e["energy_as_measured_kWh"] < e["energy_net_clip_standby_kWh"] + assert e["standby_threshold_a"] == STANDBY_CURRENT_THRESHOLD_A + assert "unreliable" in e["note"].lower() + + +def test_clip_keeps_positive_standby(): + # If standby draws small POSITIVE losses, clip == as_measured == active+standby. + store = _bimodal_store(active_p=97_000.0, standby_p=2_000.0, + standby_pf=0.3) + e = session_energy(store) + assert e["energy_net_clip_standby_kWh"] == pytest.approx( + e["energy_as_measured_kWh"]) + assert e["energy_active_kWh"] < e["energy_net_clip_standby_kWh"] + + +def test_energy_skips_nonfinite_p(): + overrides = { + 0: {"I_a_avg_A": 200.0, "I_b_avg_A": 200.0, "I_c_avg_A": 200.0, + "P_total_avg_W": 100_000.0}, + 1: {"I_a_avg_A": 200.0, "I_b_avg_A": 200.0, "I_c_avg_A": 200.0, + "P_total_avg_W": float("nan")}, + } + store = ColumnStore.from_records(make_records(2, overrides=overrides)) + e = session_energy(store) + per_s = 1.0 / 1000.0 / 3600.0 + assert e["energy_active_kWh"] == pytest.approx(100_000.0 * per_s) diff --git a/python/tests/test_narrative.py b/python/tests/test_narrative.py index a92ee2e..c1ca0ed 100644 --- a/python/tests/test_narrative.py +++ b/python/tests/test_narrative.py @@ -52,6 +52,37 @@ def test_narrative_includes_pf_and_ct(): assert "Bottom line: 1 alert-level finding" in n +def test_narrative_prefers_active_state_pf_and_energy(): + stats = { + "PF_total_avg": {"mean": -0.09, "p5": -0.70, "p95": 0.50}, + "_thresholds": {"total_records": 1000}, + } + load_states = { + "standby_threshold_a": 50.0, + "states": [ + {"state": "active", "records": 490, "duty_pct": 49.0, + "I_avg_A": 239.0, "P_avg_kW": 97.0, "PF_avg": 0.47, "kWh": 6638.0}, + {"state": "standby", "records": 470, "duty_pct": 47.0, + "I_avg_A": 16.0, "P_avg_kW": -7.6, "PF_avg": -0.64, "kWh": -100.0}, + ], + "energy": { + "energy_as_measured_kWh": 6054.0, + "energy_active_kWh": 6638.0, + "energy_net_clip_standby_kWh": 6684.0, + }, + } + n = build_narrative([], [], stats, None, config={"asset_name": "P115RE"}, + load_states=load_states) + # headline PF is the ACTIVE-state PF, not the meaningless blended -0.09 + assert "Active-state power factor averaged 0.47" in n + assert "not meaningful for a bimodal load" in n + # bimodal description + three energy figures + assert "bimodal" in n.lower() + assert "6054 kWh as-measured" in n + assert "6638 kWh active-only" in n + assert "6684 kWh net" in n + + def test_narrative_dip_only_no_outage(): dip = _ev(0, "dip", 50, 53, 0.65, ("a",)) n = build_narrative([dip], [], None, None) diff --git a/python/tests/test_shifts.py b/python/tests/test_shifts.py index 85b6e89..4975cec 100644 --- a/python/tests/test_shifts.py +++ b/python/tests/test_shifts.py @@ -318,6 +318,46 @@ def test_shift_comparison_rows_schema_and_values(): assert k in d, f"missing schema key {k}" +def test_shift_comparison_has_active_state_columns(): + # Each shift row exposes its active-state load (current-gated). Build a day + # shift that is half active (high current) and half standby (low current), + # and a night shift that is all standby. + overrides = {} + # 17:59:00 day window: 30 active (200 A, 80 kW) + 30 standby (10 A, -5 kW) + for i in range(30): + overrides[i] = {"I_a_avg_A": 200.0, "I_b_avg_A": 200.0, + "I_c_avg_A": 200.0, "P_total_avg_W": 80_000.0, + "PF_total_avg": 0.5} + for i in range(30, 60): + overrides[i] = {"I_a_avg_A": 10.0, "I_b_avg_A": 10.0, + "I_c_avg_A": 10.0, "P_total_avg_W": -5_000.0, + "PF_total_avg": -0.6} + # 18:00:00 night window: 60 standby + for i in range(60, 120): + overrides[i] = {"I_a_avg_A": 10.0, "I_b_avg_A": 10.0, + "I_c_avg_A": 10.0, "P_total_avg_W": -5_000.0, + "PF_total_avg": -0.6} + base = dt.datetime(2026, 5, 29, 17, 59, 0, tzinfo=dt.timezone.utc) + recs = make_records(120, base=base, overrides=overrides) + store = ColumnStore.from_records(recs) + ss = ShiftSet.parse("day=06:00-18:00,night=18:00-06:00") + rows = shift_comparison_rows(store, ss, events=[], tz=None, + nominal_ln_v=277.0, demand_window=15, + standby_threshold_a=50.0) + by = {r["shift"]: r for r in rows} + for k in ("active_records", "active_duty_pct", "active_kWh", "active_PF_avg"): + assert k in by["day"], f"missing active column {k}" + # day: 30 of 60 are active. + assert by["day"]["active_records"] == 30 + assert by["day"]["active_duty_pct"] == pytest.approx(50.0) + assert by["day"]["active_PF_avg"] == pytest.approx(0.5) + assert by["day"]["active_kWh"] == pytest.approx(80.0 * (30 / 3600.0)) + # night: no active records. + assert by["night"]["active_records"] == 0 + assert by["night"]["active_duty_pct"] == pytest.approx(0.0) + assert by["night"]["active_kWh"] == pytest.approx(0.0) + + def test_shift_comparison_multi_day_aggregates_across_occurrences(): # Two day-shift windows on two different dates aggregate into ONE day row. # Day A: 2026-05-29 08:00 ×30 @ 5kW ; Day B: 2026-05-30 08:00 ×30 @ 15kW. diff --git a/web/analysis.js b/web/analysis.js index 2d43506..cdef4ad 100644 --- a/web/analysis.js +++ b/web/analysis.js @@ -158,42 +158,100 @@ export function wholeSessionStats(source, spec, opts = {}) { // --- CT-reversal auto-detection (Feature C) -------------------------------- // // Mirrors python analysis.detect_ct_reversal: a load wired with backwards iFlex -// CTs reads as a persistent generator (P_total < 0). Flag when real power is -// negative for a high fraction of NON-OUTAGE time. +// CTs reads as a persistent generator (P_total < 0). For bimodal loads the +// low-current standby sign is noise, so the decision is made on the dominant +// high-current (ACTIVE) state — is real power negative when current is high? +// The whole-session count fields are still reported for context / back-compat. + +// Default per-phase current (A) above which a record counts as "active" load. +export const STANDBY_CURRENT_THRESHOLD_A = 50.0; + +// Mean of the three per-phase avg currents at record i (NaN-skipped). +function meanPhaseCurrent(ia, ib, ic, i) { + let s = 0.0; + let k = 0; + for (const c of [ia[i], ib[i], ic[i]]) { + if (Number.isFinite(c)) { s += c; k += 1; } + } + return k ? s / k : 0.0; +} export function detectCtReversal(source, spec, opts = {}) { const negFractionThreshold = opts.negFractionThreshold ?? 0.50; const outageVThreshold = opts.outageVThreshold ?? 50.0; + const activeThresholdA = opts.activeThresholdA ?? STANDBY_CURRENT_THRESHOLD_A; const src = asColumnSource(source, spec); const p = src.column('P_total_avg_W'); const va = src.column('V_LN_a_avg_V'); const vb = src.column('V_LN_b_avg_V'); const vc = src.column('V_LN_c_avg_V'); + const ia = src.column('I_a_avg_A'); + const ib = src.column('I_b_avg_A'); + const ic = src.column('I_c_avg_A'); let nonOutage = 0; let negative = 0; let pSum = 0.0; let pCount = 0; + let active = 0; + let activeNegative = 0; + let activePSum = 0.0; + let activePCount = 0; for (let i = 0; i < src.length; i++) { if (va[i] > outageVThreshold && vb[i] > outageVThreshold && vc[i] > outageVThreshold) { nonOutage += 1; const pv = p[i]; - if (Number.isFinite(pv)) { pSum += pv; pCount += 1; } + const finite = Number.isFinite(pv); + if (finite) { pSum += pv; pCount += 1; } if (pv < 0) negative += 1; // NaN < 0 is false, so non-finite never counts + if (meanPhaseCurrent(ia, ib, ic, i) >= activeThresholdA) { + active += 1; + if (finite) { activePSum += pv; activePCount += 1; } + if (pv < 0) activeNegative += 1; + } } } const frac = nonOutage ? negative / nonOutage : 0.0; const meanP = pCount ? pSum / pCount : 0.0; + const activeFrac = active ? activeNegative / active : 0.0; + const activeMeanP = activePCount ? activePSum / activePCount : 0.0; + let basis; + let reversed; + if (active > 0) { + basis = 'active'; + reversed = activeFrac >= negFractionThreshold; + } else { + basis = 'whole_session'; + reversed = frac >= negFractionThreshold; + } return { - reversed: frac >= negFractionThreshold, + reversed, frac_negative: frac, non_outage_records: nonOutage, negative_records: negative, mean_p_w: meanP, threshold: negFractionThreshold, + basis, + active_threshold_a: activeThresholdA, + active_records: active, + active_negative_records: activeNegative, + active_frac_negative: activeFrac, + active_mean_p_w: activeMeanP, }; } export function ctReversalNotice(result) { + if (result.basis === 'active') { + const pct = result.active_frac_negative * 100.0; + const meanKw = result.active_mean_p_w / 1000.0; + return ( + 'CT REVERSAL DETECTED — ' + + `real power is negative for ${pct.toFixed(1)}% of ACTIVE (high-current, ` + + `I >= ${result.active_threshold_a.toFixed(0)} A/phase) time ` + + `(active mean P = ${meanKw.toFixed(1)} kW). A load should draw positive ` + + 'real power when running: one or more iFlex CT probes are likely clipped ' + + 'on backwards. Toggle "Reverse CTs" (all phases) to correct P/Q/PF/energy.' + ); + } const pct = result.frac_negative * 100.0; return ( 'CT REVERSAL DETECTED — ' + @@ -204,6 +262,141 @@ export function ctReversalNotice(result) { ); } +// --- Load-state split (active vs standby, current-gated) ------------------- +// JS port of python analysis.classify_load_states / load_state_rows / +// session_energy / active_state_pf. Classify by mean per-phase CURRENT, report +// the two states separately, and surface three explicitly-labeled energy +// figures (the standby real-power SIGN is unreliable at low current). + +export const LOAD_STATES = ['active', 'standby']; + +export function classifyLoadStates(source, spec, opts = {}) { + const thresholdA = opts.thresholdA ?? STANDBY_CURRENT_THRESHOLD_A; + const src = asColumnSource(source, spec); + const ia = src.column('I_a_avg_A'); + const ib = src.column('I_b_avg_A'); + const ic = src.column('I_c_avg_A'); + const active = []; + const standby = []; + for (let i = 0; i < src.length; i++) { + if (meanPhaseCurrent(ia, ib, ic, i) >= thresholdA) active.push(i); + else standby.push(i); + } + return { active, standby }; +} + +function loadStateRow(name, src, idxs, totalRecords) { + const p = src.column('P_total_avg_W'); + const pf = src.column('PF_total_avg'); + const s = src.column('S_total_avg_VA'); + const va = src.column('V_LN_a_avg_V'); + const vb = src.column('V_LN_b_avg_V'); + const vc = src.column('V_LN_c_avg_V'); + const ia = src.column('I_a_avg_A'); + const ib = src.column('I_b_avg_A'); + const ic = src.column('I_c_avg_A'); + const vthA = src.column('V_THD_pct_a_avg'); + const vthB = src.column('V_THD_pct_b_avg'); + const vthC = src.column('V_THD_pct_c_avg'); + + const pMom = new RunningMoments(); + let pMin = Infinity; let pMax = -Infinity; + const pfMom = new RunningMoments(); + const sMom = new RunningMoments(); + const iMom = new RunningMoments(); + const vMom = new RunningMoments(); + const vthdSketch = new PercentileSketch(0.0, 50.0); + for (const i of idxs) { + const pv = p[i]; + if (Number.isFinite(pv)) { + pMom.add(pv); + pMin = Math.min(pMin, pv); pMax = Math.max(pMax, pv); + } + const pfi = pf[i]; + if (Number.isFinite(pfi)) pfMom.add(pfi); + const sv = s[i]; + if (Number.isFinite(sv)) sMom.add(sv); + iMom.add(meanPhaseCurrent(ia, ib, ic, i)); + for (const vv of [va[i], vb[i], vc[i]]) { + if (Number.isFinite(vv) && vv > 50.0) vMom.add(vv); + } + for (const tv of [vthA[i], vthB[i], vthC[i]]) { + if (Number.isFinite(tv)) vthdSketch.add(tv); + } + } + const pMean = pMom.n ? pMom.mean : 0.0; + const hours = pMom.n / 3600.0; + const kwh = (pMean / 1000.0) * hours; + const q = (sk, qq) => { const v = sk.quantile(qq); return Number.isNaN(v) ? 0.0 : v; }; + return { + state: name, + records: idxs.length, + hours, + duty_pct: totalRecords ? (idxs.length / totalRecords) * 100.0 : 0.0, + kWh: kwh, + P_avg_kW: pMean / 1000.0, + P_min_kW: pMin === Infinity ? 0.0 : pMin / 1000.0, + P_max_kW: pMax === -Infinity ? 0.0 : pMax / 1000.0, + I_avg_A: iMom.n ? iMom.mean : 0.0, + S_avg_kVA: sMom.n ? sMom.mean / 1000.0 : 0.0, + PF_avg: pfMom.n ? pfMom.mean : 0.0, + V_LN_avg_V: vMom.n ? vMom.mean : 0.0, + V_THD_p95_pct: q(vthdSketch, 0.95), + }; +} + +export function loadStateRows(source, spec, opts = {}) { + const thresholdA = opts.thresholdA ?? STANDBY_CURRENT_THRESHOLD_A; + const src = asColumnSource(source, spec); + const groups = classifyLoadStates(source, spec, { thresholdA }); + const total = src.length; + return LOAD_STATES.map((name) => loadStateRow(name, src, groups[name], total)); +} + +export function sessionEnergy(source, spec, opts = {}) { + const thresholdA = opts.thresholdA ?? STANDBY_CURRENT_THRESHOLD_A; + const src = asColumnSource(source, spec); + const p = src.column('P_total_avg_W'); + const ia = src.column('I_a_avg_A'); + const ib = src.column('I_b_avg_A'); + const ic = src.column('I_c_avg_A'); + const perKwh = 1.0 / 1000.0 / 3600.0; + let asMeasured = 0.0; + let active = 0.0; + let netClip = 0.0; + for (let i = 0; i < src.length; i++) { + const pv = p[i]; + if (!Number.isFinite(pv)) continue; + const e = pv * perKwh; + asMeasured += e; + if (meanPhaseCurrent(ia, ib, ic, i) >= thresholdA) { + active += e; + netClip += e; + } else if (pv > 0) { + netClip += e; + } + } + return { + energy_as_measured_kWh: asMeasured, + energy_active_kWh: active, + energy_net_clip_standby_kWh: netClip, + standby_threshold_a: thresholdA, + note: ( + 'Standby real-power SIGN is unreliable at low current, so the ' + + 'as-measured signed sum can understate consumption. energy_active ' + + '(active records only) and energy_net_clip_standby (standby real ' + + 'power clipped to >=0) are the defensible consumption figures.' + ), + }; +} + +export function activeStatePf(rows) { + for (const r of rows) { + if (r.state === 'active') return r.PF_avg; + } + return null; +} + // --- ITIC / CBEMA classification ------------------------------------------- const ITIC_LOWER = [ @@ -697,6 +890,7 @@ export function shiftComparisonRows(source, spec, ss, opts = {}) { const tzName = opts.tz ?? null; const events = opts.events ?? []; const demandMin = Math.max(1, Math.floor(opts.demandWindow ?? 15)); + const standbyThresholdA = opts.standbyThresholdA ?? STANDBY_CURRENT_THRESHOLD_A; const src = asColumnSource(source, spec); const byName = aggregateShifts(source, spec, ss, { tz: tzName }); const winByName = new Map(ss.shifts.map((sh) => [sh.name, sh])); @@ -717,6 +911,9 @@ export function shiftComparisonRows(source, spec, ss, opts = {}) { const va = src.column('V_LN_a_avg_V'); const vb = src.column('V_LN_b_avg_V'); const vc = src.column('V_LN_c_avg_V'); + const ia = src.column('I_a_avg_A'); + const ib = src.column('I_b_avg_A'); + const ic = src.column('I_c_avg_A'); const vthA = src.column('V_THD_pct_a_avg'); const vthB = src.column('V_THD_pct_b_avg'); const vthC = src.column('V_THD_pct_c_avg'); @@ -733,14 +930,26 @@ export function shiftComparisonRows(source, spec, ss, opts = {}) { const vMom = new RunningMoments(); const vSketch = new PercentileSketch(0.0, 400.0); const vthdSketch = new PercentileSketch(0.0, 50.0); + // Active-state (high-current) sub-aggregates for the shift's own load split. + const actPMom = new RunningMoments(); + const actPfMom = new RunningMoments(); + let actRecords = 0; const gatheredP = new Float64Array(idxs.length); for (let k = 0; k < idxs.length; k++) { const i = idxs[k]; const pv = p[i]; gatheredP[k] = Number.isFinite(pv) ? pv : 0.0; - if (Number.isFinite(pv)) { pMom.add(pv); pMin = Math.min(pMin, pv); pMax = Math.max(pMax, pv); } + const isActive = meanPhaseCurrent(ia, ib, ic, i) >= standbyThresholdA; + if (isActive) actRecords += 1; + if (Number.isFinite(pv)) { + pMom.add(pv); pMin = Math.min(pMin, pv); pMax = Math.max(pMax, pv); + if (isActive) actPMom.add(pv); + } const pfi = pf[i]; - if (Number.isFinite(pfi)) pfMom.add(pfi); + if (Number.isFinite(pfi)) { + pfMom.add(pfi); + if (isActive) actPfMom.add(pfi); + } for (const vv of [va[i], vb[i], vc[i]]) { if (Number.isFinite(vv) && vv > 50.0) { vMom.add(vv); vSketch.add(vv); } } @@ -754,6 +963,11 @@ export function shiftComparisonRows(source, spec, ss, opts = {}) { const kwh = (pMean / 1000.0) * hours; const peakKw = peakRollingDemandKw(gatheredP, demandMin * 60); + const actPMean = actPMom.n ? actPMom.mean : 0.0; + const actHours = actPMom.n / 3600.0; + const actKwh = (actPMean / 1000.0) * actHours; + const actDutyPct = idxs.length ? (actRecords / idxs.length) * 100.0 : 0.0; + let nOut = 0; let nDip = 0; let nSwell = 0; let outageSecs = 0.0; for (const e of evs) { if (e.kind === 'outage') { nOut += 1; outageSecs += (e.tEndMs - e.tStartMs) / 1000; } @@ -782,6 +996,10 @@ export function shiftComparisonRows(source, spec, ss, opts = {}) { n_dips: nDip, n_swells: nSwell, outage_minutes: outageSecs / 60.0, + active_records: actRecords, + active_duty_pct: actDutyPct, + active_kWh: actKwh, + active_PF_avg: actPfMom.n ? actPfMom.mean : 0.0, }); } return rows; diff --git a/web/tests/load_states_parity.test.js b/web/tests/load_states_parity.test.js new file mode 100644 index 0000000..77ab00c --- /dev/null +++ b/web/tests/load_states_parity.test.js @@ -0,0 +1,152 @@ +// Load-state parity — web/analysis.js active/standby split, the three energy +// figures, and the magnitude-weighted reverse-CTs decision must match +// python/.../analysis.py. We recreate the exact deterministic bimodal session +// the Python golden generator builds (test_analysis_parity_golden.py +// `load_states` block) and compare against fixtures/analysis_golden.json. +import { strict as assert } from 'node:assert'; +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import { dirname, resolve } from 'node:path'; +import { test } from 'node:test'; + +import { + classifyLoadStates, loadStateRows, sessionEnergy, activeStatePf, + detectCtReversal, +} from '../analysis.js'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const repoRoot = resolve(__dirname, '..', '..'); +const spec = JSON.parse(readFileSync(resolve(repoRoot, 'spec', 'field_map.json'), 'utf8')); +const golden = JSON.parse(readFileSync( + resolve(repoRoot, 'python', 'tests', 'fixtures', 'analysis_golden.json'), 'utf8')); + +const FI = new Map(spec.fields.map((f) => [f.name, f.index])); +const G = golden.load_states; + +// Mirror the Python `make_records` defaults + the load_states bimodal overrides. +function buildBimodalSession() { + const base = {}; + for (const f of spec.fields) base[f.name] = 0.0; + for (const ph of ['a', 'b', 'c']) { + for (const st of ['min', 'max', 'avg']) { + base[`V_LN_${ph}_${st}_V`] = 277.0; + base[`I_${ph}_${st}_A`] = 100.0; + } + } + for (const pair of ['ab', 'bc', 'ca']) { + for (const st of ['min', 'max', 'avg']) base[`V_LL_${pair}_${st}_V`] = 480.0; + } + base.freq_min_Hz = 60.0; base.freq_max_Hz = 60.0; base.freq_avg_Hz = 60.0; + base.P_total_avg_W = 50_000.0; + base.P_total_min_W = 50_000.0; base.P_total_max_W = 50_000.0; + + const nA = G.n_active; + const nS = G.n_standby; + const active = { + I_a_avg_A: 239.0, I_b_avg_A: 239.0, I_c_avg_A: 239.0, + P_total_avg_W: 97_000.0, S_total_avg_VA: 206_000.0, + PF_total_avg: 0.47, V_THD_pct_a_avg: 3.0, V_THD_pct_b_avg: 2.5, + V_THD_pct_c_avg: 4.0, + }; + const standby = { + I_a_avg_A: 16.0, I_b_avg_A: 16.0, I_c_avg_A: 16.0, + P_total_avg_W: -7_600.0, S_total_avg_VA: 11_800.0, + PF_total_avg: -0.64, V_THD_pct_a_avg: 3.0, V_THD_pct_b_avg: 2.5, + V_THD_pct_c_avg: 4.0, + }; + const records = []; + for (let n = 0; n < nA + nS; n++) { + const merged = { ...base, ...(n < nA ? active : standby) }; + const floats = new Float32Array(spec.data_floats); + for (const [name, val] of Object.entries(merged)) { + if (FI.has(name)) floats[FI.get(name)] = val; + } + records.push({ + index: n, + startMs: G.base_epoch_ms + n * 1000, + endMs: G.base_epoch_ms + (n + 1) * 1000, + floats, + }); + } + return records; +} + +const REL = 1e-4; +const PCT_ABS = 1.0; // percentile sketch is bin-width bounded + +function approx(actual, expected, absTol, msg) { + const tol = Math.max(absTol, Math.abs(expected) * REL); + assert.ok(Math.abs(actual - expected) <= tol, + `${msg}: got ${actual}, expected ${expected} (tol ${tol})`); +} + +test('classifyLoadStates: splits by mean per-phase current at the threshold', () => { + const recs = buildBimodalSession(); + const g = classifyLoadStates(recs, spec, { thresholdA: G.threshold_a }); + assert.equal(g.active.length, G.n_active); + assert.equal(g.standby.length, G.n_standby); + assert.deepEqual(g.active.slice(0, 3), [0, 1, 2]); +}); + +test('loadStateRows: matches Python golden row-for-row', () => { + const recs = buildBimodalSession(); + const rows = loadStateRows(recs, spec, { thresholdA: G.threshold_a }); + const by = Object.fromEntries(rows.map((r) => [r.state, r])); + const gby = Object.fromEntries(G.rows.map((r) => [r.state, r])); + assert.deepEqual(Object.keys(by).sort(), Object.keys(gby).sort()); + for (const name of Object.keys(gby)) { + const j = by[name]; + const g = gby[name]; + assert.equal(j.records, g.records, `${name}.records`); + approx(j.hours, g.hours, 1e-6, `${name}.hours`); + approx(j.duty_pct, g.duty_pct, 1e-6, `${name}.duty_pct`); + approx(j.kWh, g.kWh, 1e-4, `${name}.kWh`); + approx(j.P_avg_kW, g.P_avg_kW, 1e-3, `${name}.P_avg_kW`); + approx(j.P_min_kW, g.P_min_kW, 1e-3, `${name}.P_min_kW`); + approx(j.P_max_kW, g.P_max_kW, 1e-3, `${name}.P_max_kW`); + approx(j.I_avg_A, g.I_avg_A, 1e-3, `${name}.I_avg_A`); + approx(j.S_avg_kVA, g.S_avg_kVA, 1e-2, `${name}.S_avg_kVA`); + approx(j.PF_avg, g.PF_avg, 1e-4, `${name}.PF_avg`); + approx(j.V_LN_avg_V, g.V_LN_avg_V, 1e-3, `${name}.V_LN_avg_V`); + approx(j.V_THD_p95_pct, g.V_THD_p95_pct, PCT_ABS, `${name}.V_THD_p95_pct`); + } +}); + +test('sessionEnergy: three figures match Python golden', () => { + const recs = buildBimodalSession(); + const e = sessionEnergy(recs, spec, { thresholdA: G.threshold_a }); + approx(e.energy_as_measured_kWh, G.energy.energy_as_measured_kWh, 1e-6, + 'energy_as_measured_kWh'); + approx(e.energy_active_kWh, G.energy.energy_active_kWh, 1e-6, + 'energy_active_kWh'); + approx(e.energy_net_clip_standby_kWh, G.energy.energy_net_clip_standby_kWh, + 1e-6, 'energy_net_clip_standby_kWh'); + assert.equal(e.standby_threshold_a, G.energy.standby_threshold_a); + // The understated as-measured < the corrected figures. + assert.ok(e.energy_as_measured_kWh < e.energy_active_kWh); + assert.ok(e.energy_as_measured_kWh < e.energy_net_clip_standby_kWh); +}); + +test('activeStatePf: returns the active row PF', () => { + const recs = buildBimodalSession(); + const rows = loadStateRows(recs, spec, { thresholdA: G.threshold_a }); + approx(activeStatePf(rows), 0.47, 1e-4, 'active PF'); +}); + +test('detectCtReversal: magnitude-weighted decision matches Python golden', () => { + const recs = buildBimodalSession(); + const ct = detectCtReversal(recs, spec, { activeThresholdA: G.threshold_a }); + assert.equal(ct.basis, G.ct.basis, 'basis'); + assert.equal(ct.reversed, G.ct.reversed, 'reversed'); + assert.equal(ct.active_records, G.ct.active_records, 'active_records'); + assert.equal(ct.active_negative_records, G.ct.active_negative_records, + 'active_negative_records'); + approx(ct.active_frac_negative, G.ct.active_frac_negative, 1e-9, + 'active_frac_negative'); + approx(ct.active_mean_p_w, G.ct.active_mean_p_w, 1e-3, 'active_mean_p_w'); + // The whole-session count fraction is past 50% (the fragile signal) but the + // decision is made on the positive active state, so reversed stays false. + approx(ct.frac_negative, G.ct.frac_negative, 1e-9, 'frac_negative'); + assert.ok(ct.frac_negative >= 0.50); + assert.equal(ct.reversed, false); +});