Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,48 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.7.0] — 2026-06-03

Generalized, named, configurable shift/period splitting — so usage can be
compared across operator-defined shifts (day vs night, A/B/C), with windows
that may wrap past midnight. Parity-tested Python↔JS.

### Added — shift splitting (`--split-by shifts`)
- **`--split-by shifts`** activates named shift mode, with **`--shifts
"name=HH:MM-HH:MM,..."`** (comma-separated; a window where `end<=start` wraps
past midnight, e.g. `night=18:00-06:00`) or the JSON **`--shifts-file`**
(`{"shifts":[{"name","start","end"}]}`, mirroring `--rules-file`). With neither
flag the default is `day=06:00-18:00,night=18:00-06:00`.
- **Timezone-correct windows:** shift windows are evaluated in the report
timezone (`--tz`), localizing each UTC record before applying the `HH:MM`
rule (UTC if `--tz` unset, and the output says so). Validated on a real
Central-time session: day/night split lands exactly on 06:00 / 18:00 Central.
- **Two outputs:**
- `shift_comparison.csv` + `shift_comparison.json` — the headline per-shift
aggregate (records, hours, kWh, P avg/min/max, peak rolling demand, PF avg,
V_LN avg/p5/p95, V_THD p95, event counts, outage minutes), gathering each
shift's non-contiguous records across the whole session.
- Per-occurrence contiguous buckets under `<out>/shifts/<name>_<date>/`
(session.csv, events.json+ITIC, summary.txt); a midnight-spanning night is
one occurrence labeled by its start date.
- Gap/overlap validation (warns when windows don't tile 24 h), first-matching
window wins on overlap, and an `unassigned` shift for records matching none.
- Shift-comparison table embedded in `summary.txt`.

### Added — model + parity
- `analysis.py`: `Shift` / `ShiftSet` (parse / from_spec / default /
coverage_issues), `gather_store` (non-contiguous index slicing),
`aggregate_shifts`, `shift_occurrences`, `shift_comparison_rows`; new
`shifts_file.py` loader.
- `web/analysis.js`: full JS port (`ShiftSet`, `localMinuteOfDay`,
`aggregateShifts`, `shiftOccurrences`, `shiftComparisonRows`) with a
Python↔JS parity test (`web/tests/shifts_parity.test.js`) on a shared
synthetic multi-day fixture.
- Docs: new [`docs/SHIFTS.md`](docs/SHIFTS.md); README options table updated.

### Tests
- Python 251 passing (was 223); web 120 passing (was 114).

## [0.6.0] — 2026-06-03

Web parity + analysis depth. Closes the web memory/parity gap from 0.5.0 and
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ fluke-analyze path/to/ES.NNN -o output/ \
| `--max-csv-rows N` | off | Cap the full CSV at ~N rows for week-long sessions; stride is auto-raised and the downsampling is logged. 1-min CSV + analysis keep full resolution. |
| `--anchor-start ISO_TIME` | off | Pin the real wall-clock **start** to correct a wrong meter RTC (shifts every timestamp). Mutually exclusive with `--anchor-end`. |
| `--anchor-end ISO_TIME` | off | Pin the real wall-clock **end** (e.g. a known shutdown time). |
| `--split-by PERIOD` | off | Partition into time buckets (`hour`\|`day`\|`week` or a duration like `30m`/`6h`/`2d`). Emits a full per-bucket report (`<label>/`) + `buckets_summary.csv` roll-up. |
| `--split-by PERIOD` | off | Partition into time buckets (`hour`\|`day`\|`week`, a duration like `30m`/`6h`/`2d`, or `shifts` for named shift windows). Emits a full per-bucket report (`<label>/`) + `buckets_summary.csv` roll-up. |
| `--shifts SPEC` | `day=06:00-18:00,night=18:00-06:00` | With `--split-by shifts`: named windows `name=HH:MM-HH:MM,...`. `end<=start` wraps past midnight. Evaluated in `--tz`. Emits `shift_comparison.csv`/`.json` + per-occurrence reports. See [`docs/SHIFTS.md`](docs/SHIFTS.md). |
| `--shifts-file FILE` | off | JSON shift schedule (`{"shifts":[{"name","start","end"}]}`); alternative to `--shifts`. |
| `--mark "ISO=LABEL"` | | Add an event marker (repeatable); cross-referenced to nearest events in `markers.json`. |
| `--marks FILE.csv` | | Load markers from a CSV (`time,label`). |
| `--tod-profile [HH:MM-HH:MM]` | off | Time-of-day (diurnal) profile — avg/min/max envelope per bin across all days. Bare flag = 24 h. Writes `time_of_day_profile.csv` + an XLSX sheet. |
Expand Down
21 changes: 21 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,27 @@ Shipped releases live in [CHANGELOG.md](CHANGELOG.md). This file is the
backlog of ideas we've discussed but not yet committed to a specific
release.

## Shipped in v0.7 — generalized shift splitting

Operator-defined, named, midnight-wrapping shift windows so usage can be
compared across shifts (day vs night, A/B/C), parity-tested Python↔JS:

- **`--split-by shifts`** + `--shifts "name=HH:MM-HH:MM,..."` / `--shifts-file`
(JSON). Windows that wrap past midnight; default day/night.
- Windows evaluated in the **report timezone** (`--tz`), not raw UTC — verified
on a real Central-time session.
- Headline `shift_comparison.csv`/`.json` aggregate per shift name (energy,
power, peak demand, PF, V_LN/THD percentiles, event counts, outage minutes)
plus per-occurrence contiguous buckets. This delivers the shift slice of the
open **comparison polish (Theme C)** backlog.
- New model in `analysis.py` (`Shift`/`ShiftSet`, `gather_store`,
`aggregate_shifts`, `shift_occurrences`, `shift_comparison_rows`) + JS port +
parity test + `docs/SHIFTS.md`.

Still open from comparison polish: cross-session / period-over-period diffing
and a web UI surface for shifts (the core logic + parity are shipped; web
rendering is best-effort).

## Shipped in v0.6 — web parity + analysis depth

Closed the v0.5 web memory/parity gap and added standards-grade analysis, all
Expand Down
112 changes: 112 additions & 0 deletions docs/SHIFTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Generalized shift splitting (`--split-by shifts`)

Define **named time periods within a day** — which may wrap past midnight — and
compare power usage across them (day vs night, or A/B/C shifts). This is the
generalized successor to the clock-aligned `--split-by hour|day|week`: instead of
fixed grid buckets, you define the windows that match how the plant actually
runs.

```bash
# Default day/night
fluke-analyze ES.004 --split-by shifts --tz America/Chicago

# Explicit two-shift
fluke-analyze ES.004 --split-by shifts \
--shifts "day=06:00-18:00,night=18:00-06:00" --tz America/Chicago

# Three 8-hour shifts
fluke-analyze ES.004 --split-by shifts \
--shifts "A=06:00-14:00,B=14:00-22:00,C=22:00-06:00" --tz America/Chicago

# From a committed schedule file
fluke-analyze ES.004 --split-by shifts --shifts-file shifts.json --tz America/Chicago
```

## Defining shifts

A shift is `name=HH:MM-HH:MM`. Multiple shifts are comma-separated. **A window
where `end <= start` wraps past midnight**: `night=18:00-06:00` covers
`[18:00, 24:00)` plus `[00:00, 06:00)`. Windows are half-open: start inclusive,
end exclusive.

- **Two shifts:** `day=06:00-18:00,night=18:00-06:00` (the default when
`--split-by shifts` is given with no `--shifts`/`--shifts-file`).
- **Three shifts:** `A=06:00-14:00,B=14:00-22:00,C=22:00-06:00`.

### File form (`--shifts-file`)

```json
{
"shifts": [
{"name": "day", "start": "06:00", "end": "18:00"},
{"name": "night", "start": "18:00", "end": "06:00"}
]
}
```

A bare top-level list of `{name, start, end}` objects is also accepted.

## Timezone contract (important)

Captured timestamps are stored in **UTC**. Shift windows are interpreted in the
**report timezone** set by `--tz` (e.g. `America/Chicago`). Each record's
timestamp is localized to that zone *before* the `HH:MM` rule is applied. If you
omit `--tz`, windows are evaluated in **UTC** and the output says so. Always pass
`--tz` so a `06:00-18:00` shift means 6 AM local, not 6 AM UTC.

Validated on a real Central-time session: `day=06:00-18:00,night=18:00-06:00`
splits exactly on 06:00 / 18:00 **Central**, 43 200 one-second records per
half-day.

## Assignment rules

- Each record is assigned to the **first** window it matches (so overlapping
windows resolve deterministically).
- Records that match **no** window go to an `unassigned` shift, with a printed
notice.
- If the windows don't tile 24 h, a gap/overlap **warning** is printed to
stderr (and recorded in `shift_comparison.json → coverage_issues`). Shifts
ideally tile the day with no gaps or overlaps.

## Outputs

### 1. `shift_comparison.csv` / `shift_comparison.json` — the headline

One row **per shift name**, aggregating **all** records of that shift across the
whole session (a "night" recurs daily; those non-contiguous records are gathered
into one row). This is what lets you compare day vs night usage.

| Column | Meaning |
|---|---|
| `shift` | shift name (or `unassigned`) |
| `window` | the `HH:MM-HH:MM` window |
| `records` | number of records in the shift |
| `hours` | hours covered (1 record = 1 s) |
| `kWh` | energy = mean power × hours |
| `P_total_avg_W` / `P_total_min_W` / `P_total_max_W` | real-power avg/min/max |
| `peak_demand_kW` | peak rolling demand within the shift (15 min default; from `--demand-window`) |
| `peak_demand_window_secs` | the demand window used |
| `PF_avg` | average power factor |
| `V_LN_avg_V` / `V_LN_p5_V` / `V_LN_p95_V` | L-N voltage avg + p5/p95 (outage zeros excluded) |
| `V_THD_p95_pct` | 95th-percentile voltage THD |
| `n_outages` / `n_dips` / `n_swells` | event counts (filed by event start time) |
| `outage_minutes` | total outage minutes in the shift |

The JSON also carries `tz`, `spec`, `demand_window_secs`, and `coverage_issues`.

### 2. Per-occurrence buckets — `<out>/shifts/<name>_<date>/`

Each **individual** shift instance becomes its own contiguous, time-ordered
bucket reusing the standard per-bucket report machinery (`session.csv`,
`events.json` with ITIC, `summary.txt`). A night spanning midnight is **one**
occurrence labeled by its **start date** (e.g. `night 2026-05-29`).

A shift-comparison table is also embedded in the top-level `summary.txt`.

## Parity

The shift model (parse, midnight-wrap, tz-localized assignment, aggregate,
occurrences, comparison) is ported to the web side (`web/analysis.js`:
`ShiftSet`, `aggregateShifts`, `shiftOccurrences`, `shiftComparisonRows`) and
covered by a Python↔JS parity test on a shared synthetic multi-day fixture
(`web/tests/shifts_parity.test.js` vs `analysis_golden.json`).
2 changes: 1 addition & 1 deletion python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "fluke-3540-analyzer"
version = "0.6.0"
version = "0.7.0"
description = "Parser, event detector, and chart generator for Fluke 3540 FC three-phase power-quality sessions. See https://github.com/GrumpyTanker/fluke-3540-analyzer."
requires-python = ">=3.10"
license = {text = "MIT"}
Expand Down
2 changes: 1 addition & 1 deletion python/src/fluke_3540/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
The canonical field map and binary layout live in spec/field_map.json at
the repo root and are shared with the JavaScript port. See README.md.
"""
__version__ = "0.6.0"
__version__ = "0.7.0"
Loading