Skip to content

Phase column and phase_statistics on phased X/mR (#144, PR 3) #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cnicholas merged 1 commit into from
Oct 18, 2026
Merged

Phase column and phase_statistics on phased X/mR (#144, PR 3) #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
Minitab's staged Xbar-S chart. Residual values, calibrations and `by=[time]` are refused with
messages that say what to use instead; a phase with no replicated subgroup keeps its points without
limits and is named in `metadata['insufficient_phases']`. (#144)
- **`result.phase_statistics(chart)`.** One row per phase of a phased Xbar/S chart: `phase`,
`subgroups`, `N`, `center`, `s_bar`, `sigma_hat`, `lpl`, `upl`. (#144)
- **`result.phase_statistics(chart, stratum=None)`.** One row per phase of a phased chart. Xbar/S:
`phase`, `subgroups`, `N`, `center`, `s_bar`, `sigma_hat`, `lpl`, `upl`. X/mR: `phase`, `points`,
`center`, `mR_bar`, `lpl`, `upl`, per stratum on a stratified chart (`stratum=`, or through
`focus()`). (#144)

### Changed
- **`formulate(response=..., factors=[])` with no time is the same study.** It used to treat the
Expand All @@ -36,12 +38,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Xbar and S chart data carry `n`.** `get_chart('Xbar')` / `get_chart('S')`, and the Excel chart
sheets, gain an `n` column after the subgroup label. Unstratified Xbar/S metadata names the label
column as `group_col` (#143).
- **Phased Xbar/S chart data and `chart_table` carry a `phase` column** after the subgroup label, and
the metadata carries `phase_by`, `lane_boundaries`, `single_point_phases` and `insufficient_phases`.
`n_avg` is not reported on a phased chart, whose `n_mode='average'` is applied per phase; the
per-phase mean size is `phase_statistics(chart)['N']`. (#144)
- **Lane dividers on an Xbar or S chart are labelled with the phase.** These charts have no time
column and so no cell band under the axis, which is where X charts name their lanes. (#144)
- **Phased chart data and `chart_table` carry a `phase` column** (the lane label) after the subgroup
label, on X/mR as well as Xbar/S. Phased Xbar/S metadata carries `phase_by`, `lane_boundaries`,
`single_point_phases` and `insufficient_phases`; `n_avg` is not reported on a phased chart, whose
`n_mode='average'` is applied per phase, so the per-phase mean size is `phase_statistics(chart)['N']`.
(#144)

### Fixed
- **Xbar/S `chart_table` keeps the subgroup label and `n` on every `by=`.** With a `by=` other than
Expand All @@ -54,6 +57,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
centres and statistics are unchanged (#143).
- **`execute()` no longer documents `phased=True` as requiring `by=[]`.** Stratified phasing of X/mR
(`by=['factor 1'], phased=True`) has been accepted since 75bcdd1; the docstring never caught up. (#144)
- **The companion mR of a phased X chart reports `single_point_phases`.** Only the solo mR did. (#144)

## [0.4.0] - 2026-10-14

Expand Down
24 changes: 13 additions & 11 deletions docs/reference/api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,17 +330,19 @@ directly, and is not a top-level export.)
### Phases

```python
result.phase_statistics(chart) # -> DataFrame, one row per phase (phased Xbar/S)
```

A chart executed with `phased=True` gives each phase — one combination of the grouping
factors, a contiguous run of subgroups in time order — its own center line and limits.
On Xbar/S the chart data gains a `phase` column (the lane label) next to the subgroup
label, and `chart_table` shows it. `phase_statistics(chart)` returns the numbers behind
the stepped lines: `phase`, `subgroups`, `N` (the subgroup size the phase's limits assume;
`NaN` when sizes vary within it), `center`, `s_bar`, `sigma_hat`, `lpl`, `upl` (`NaN` when
the limits step within the phase). It raises `ValidationError` on a chart that is not
phased, and on phased X/mR, whose per-phase values are on each row of `get_chart()`.
result.phase_statistics(chart, stratum=None) # -> DataFrame, one row per phase
```

A chart executed with `phased=True` gives each phase — one factor combination, a
contiguous run of points in time order — its own center line and limits. The chart data
gains a `phase` column (the lane label) next to the subgroup label, and `chart_table`
shows it. `phase_statistics(chart)` returns the numbers behind the stepped lines. On
Xbar/S: `phase`, `subgroups`, `N` (the subgroup size the phase's limits assume; `NaN`
when sizes vary within it), `center`, `s_bar`, `sigma_hat`, `lpl`, `upl` (`NaN` when the
limits step within the phase). On X/mR: `phase`, `points`, `center`, `mR_bar`, `lpl`,
`upl`. A stratified phased X/mR chart (`by=['factor 1'], phased=True`) has phases per
stratum: pass `stratum=`, or call it on `focus(stratum)`. It raises `ValidationError` on a
chart that is not phased.

Phased metadata carries `phased=True`, `run_rules_applicable=False`, `phase_by` (the
factors a phase is one combination of), `lane_boundaries` (the divider positions),
Expand Down
69 changes: 60 additions & 9 deletions processbehavior/analysis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,25 @@ def _raise_no_replication(chart: str, sds, insufficient: list, total: int) -> No
)


def _mr_phase_table(phase_stats: pd.DataFrame, out: pd.DataFrame, keys: list[str], round_to: int) -> pd.DataFrame:
"""One row per phase of a phased X or mR chart: label, points, centre, mR-bar and limits.

``phase_stats`` is the per-phase aggregation keyed by ``keys`` (``['_phase_id']``, or
``[stratify_col, '_phase_id']``); ``out`` carries the ``phase`` label on every row.
"""
labels = out.groupby(keys, sort=False)['phase'].first().reset_index()
table = phase_stats.merge(labels, on=keys, how='left').rename(columns={'_n': 'points', 'mR': 'mR_bar'})
return table[keys + ['phase', 'points', 'center', 'mR_bar', 'lpl', 'upl']].round(round_to)


def _split_phase_table(table: pd.DataFrame, stratify_col: str) -> dict:
"""The per-stratum phase tables of a stratified phased chart, keyed by stratum."""
return {
stratum: group.drop(columns=[stratify_col, '_phase_id']).reset_index(drop=True)
for stratum, group in table.groupby(stratify_col, sort=False)
}


def _build_stratified_mr_payload(
*,
mr_spec: _MRChartSpec,
Expand All @@ -345,6 +364,7 @@ def _build_stratified_mr_payload(
phased: bool = False,
insufficient_strata: list | None = None,
single_point_phases: int | None = None,
phases: dict | None = None,
) -> dict:
"""Build the {chart_type: {...}} payload for a stratified MR-family chart.

Expand Down Expand Up @@ -379,6 +399,7 @@ def _build_stratified_mr_payload(
'statistics': chart_statistics,
'metadata': metadata,
'strata': published_strata,
**({'phases': phases} if phased and phases is not None else {}),
},
}

Expand All @@ -392,6 +413,7 @@ def _build_ungrouped_mr_payload(
lane_boundaries,
phased: bool = False,
single_point_phases: int | None = None,
phases: pd.DataFrame | None = None,
) -> dict:
"""Build the {chart_type: {...}} payload for an ungrouped MR-family chart."""
metadata: dict = {
Expand All @@ -410,6 +432,7 @@ def _build_ungrouped_mr_payload(
'data': chart_out,
'statistics': chart_statistics,
'metadata': metadata,
**({'phases': phases} if phased and phases is not None else {}),
},
}

Expand Down Expand Up @@ -2239,10 +2262,13 @@ def _calculate_mr_chart_from_precomputed(
phase_stats = precomputed['phase_stats']

# Recompute R-specific limits per phase within each stratum
r_phase_stats = phase_stats[[stratify_col, '_phase_id', 'mR']].copy()
r_phase_stats = phase_stats[[stratify_col, '_phase_id', '_n', 'mR']].copy()
r_lims = _mr_family_limits(mr_spec, mean=0, mR=r_phase_stats['mR'])
r_phase_stats = _join_limits(r_phase_stats, r_lims)
r_phase_stats['center'] = r_phase_stats['mR']
r_phases = _split_phase_table(
_mr_phase_table(r_phase_stats, out, [stratify_col, '_phase_id'], spec.round_to), stratify_col
)

# Replace X limits with mR limits
out = out.drop(columns=['center', 'lpl', 'upl', 'beyond_limits'], errors='ignore')
Expand All @@ -2261,7 +2287,7 @@ def _calculate_mr_chart_from_precomputed(
# Format output
chart_out = self._build_output_columns(
df=out,
value_cols=extra_cols + [plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
value_cols=extra_cols + ['phase', plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
)

chart_statistics = {
Expand All @@ -2286,6 +2312,8 @@ def _calculate_mr_chart_from_precomputed(
strata=strata,
lane_boundaries=xmr_lane_boundaries,
phased=True,
single_point_phases=precomputed.get('single_point_phases'),
phases=r_phases,
)

# --- Non-phased precomputed path ---
Expand Down Expand Up @@ -2396,12 +2424,14 @@ def _calculate_mr_chart_stratified( # noqa: C901
else:
out['_phase_key'] = list(map(tuple, out[collapsed_factors].to_numpy()))

# 2. Assign within-stratum phase IDs via transitions of _phase_key
# 2. Assign within-stratum phase IDs via transitions of _phase_key, and the
# phase's display label (the lane label) on every row
out['_phase_id'] = (
out.groupby(stratify_col, sort=False)['_phase_key']
.transform(lambda s: (s != s.shift()).cumsum())
.astype('int64')
)
out['phase'] = _lane_labels(out, collapsed_factors)

# 3. Per-phase mR (reset at phase boundaries within each stratum)
out['mr'] = out.groupby([stratify_col, '_phase_id'], sort=False)[mr_source_col].diff().abs()
Expand Down Expand Up @@ -2455,12 +2485,15 @@ def _calculate_mr_chart_stratified( # noqa: C901

# 11. Track single-point phases
n_single_point = int((phase_stats['_n'] < 2).sum())
phases = _split_phase_table(
_mr_phase_table(phase_stats, out, [stratify_col, '_phase_id'], spec.round_to), stratify_col
)

# Format output
extra_cols = [c for c in stratify_by if c != spec.rsg_var_name and c != spec.time_var]
chart_out = self._build_output_columns(
df=out,
value_cols=extra_cols + [plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
value_cols=extra_cols + ['phase', plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
)

result.update(_build_stratified_mr_payload(
Expand All @@ -2474,12 +2507,14 @@ def _calculate_mr_chart_stratified( # noqa: C901
lane_boundaries=lane_boundaries,
phased=True,
single_point_phases=n_single_point,
phases=phases,
))

if _return_intermediates:
result['_intermediates'] = {
'out': out,
'phase_stats': phase_stats,
'single_point_phases': n_single_point,
'stratify_col': stratify_col,
'stratify_by': stratify_by,
'strata': strata,
Expand Down Expand Up @@ -2614,7 +2649,7 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
xmr_lane_boundaries = _precomputed['lane_boundaries']

# Recompute R-specific limits per phase
r_phase_stats = phase_stats[['_phase_id', 'mR']].copy()
r_phase_stats = phase_stats[['_phase_id', '_n', 'mR']].copy()
r_lims = r_phase_stats.apply(
lambda row: calculate_limits(
mean=0,
Expand All @@ -2628,6 +2663,11 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
)
r_phase_stats = _join_limits(r_phase_stats, r_lims)
r_phase_stats['center'] = r_phase_stats['mR']
r_phases = (
_mr_phase_table(r_phase_stats, out, ['_phase_id'], spec.round_to)
.drop(columns=['_phase_id'])
.reset_index(drop=True)
)

# Replace X limits with mR limits
out = out.drop(columns=['center', 'lpl', 'upl'], errors='ignore')
Expand All @@ -2646,7 +2686,7 @@ def _calculate_mr_chart_ungrouped( # noqa: C901

chart_out = self._build_output_columns(
df=out,
value_cols=[plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
value_cols=['phase', plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
)

chart_statistics = {
Expand All @@ -2665,6 +2705,8 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
plot_col=plot_col,
lane_boundaries=xmr_lane_boundaries,
phased=True,
single_point_phases=_precomputed.get('single_point_phases'),
phases=r_phases,
))
return result

Expand Down Expand Up @@ -2738,8 +2780,10 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
# === Phased limits path ===
if phased and collapsed_factors:
self._reject_calibration('phased X/mR')
# 1. Assign phase_id: contiguous runs of the same rsg_key
# 1. Assign phase_id: contiguous runs of the same rsg_key, and the phase's display
# label (the lane label) on every row
out['_phase_id'] = (out['rsg_key'] != out['rsg_key'].shift()).cumsum().astype('int64')
out['phase'] = _lane_labels(out, collapsed_factors)

# 2. Per-phase moving range (reset at boundaries)
out['mr'] = out.groupby('_phase_id', sort=False)[mr_source_col].diff().abs()
Expand Down Expand Up @@ -2806,7 +2850,7 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
# 9. Format output
chart_out = self._build_output_columns(
df=out,
value_cols=[plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
value_cols=['phase', plot_col, 'center', 'lpl', 'upl', 'beyond_limits'],
)

# 10. Statistics: limits_vary=True signals stepped rendering
Expand All @@ -2818,8 +2862,13 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
'limits_vary': True,
}

# 11. Track single-point phases for visibility
# 11. Track single-point phases for visibility, and the per-phase table
n_single_point = int((phase_stats['_n'] < 2).sum())
phases = (
_mr_phase_table(phase_stats, out, ['_phase_id'], spec.round_to)
.drop(columns=['_phase_id'])
.reset_index(drop=True)
)

result.update(_build_ungrouped_mr_payload(
mr_spec=mr_spec,
Expand All @@ -2829,6 +2878,7 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
lane_boundaries=lane_boundaries,
phased=True,
single_point_phases=n_single_point,
phases=phases,
))

if _return_intermediates:
Expand All @@ -2839,6 +2889,7 @@ def _calculate_mr_chart_ungrouped( # noqa: C901
'is_stratified': False,
'phased': True,
'phase_stats': phase_stats,
'single_point_phases': n_single_point,
}

return result
Expand Down
55 changes: 41 additions & 14 deletions processbehavior/analysis_result.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -656,21 +656,34 @@ def statistics(self, chart: str, stratum: str | None = None) -> dict:
)
return dict(stats[stratum])

def phase_statistics(self, chart: str) -> pd.DataFrame:
def phase_statistics(self, chart: str, stratum: str | None = None) -> pd.DataFrame:
"""One row per phase of a phased chart: the numbers behind its stepped lines.

Columns: ``phase`` (the lane label, as in ``chart_table``), ``subgroups`` (points
charted in it), ``N`` (the subgroup size its limits assume: the common size, the mean
size under ``n_mode='average'``, or NaN when sizes vary within the phase), ``center``,
``s_bar``, ``sigma_hat``, ``lpl`` and ``upl`` (NaN when the limits step within the
phase; ``get_chart`` carries them per row). A phase with no replicated subgroup keeps
its ``subgroups`` and is NaN elsewhere; ``metadata['insufficient_phases']`` names it.
Xbar/S columns: ``phase`` (the lane label, as in ``chart_table``), ``subgroups``
(points charted in it), ``N`` (the subgroup size its limits assume: the common size,
the mean size under ``n_mode='average'``, or NaN when sizes vary within the phase),
``center``, ``s_bar``, ``sigma_hat``, ``lpl`` and ``upl`` (NaN when the limits step
within the phase; ``get_chart`` carries them per row). A phase with no replicated
subgroup keeps its ``subgroups`` and is NaN elsewhere; ``metadata['insufficient_phases']``
names it.

X/mR columns: ``phase``, ``points``, ``center``, ``mR_bar``, ``lpl``, ``upl``. On the
mR chart ``center`` is ``mR_bar``.

Parameters
----------
chart : str
Chart name, e.g. ``'Xbar'`` or ``'X'``.
stratum : str, optional
Required on a stratified phased chart (X/mR with ``by=[factor]``), where each
stratum has its own phases; rejected otherwise. ``focus(stratum)`` gives the same
table without the argument.

Raises
------
ValidationError
On a chart that is not phased, and on phased X/mR, whose per-phase values are on
each row of ``get_chart`` (a later release adds this table there; issue #144).
On a chart that is not phased, or when a stratum is needed and not given, given
and not needed, or unknown.
ChartNotAvailableError
If the chart is not in this result.
"""
Expand All @@ -680,16 +693,30 @@ def phase_statistics(self, chart: str) -> pd.DataFrame:
f"Chart '{name}' not found. Available charts: {self.all_charts}", chart=name, available=self.all_charts
)
payload = self.charts[name]
if not payload.get('metadata', {}).get('phased'):
if not payload.get('metadata', {}).get('phased') or payload.get('phases') is None:
raise ValidationError(
f'Chart {chart!r} is not phased, so it has no per-phase statistics. '
'execute(..., phased=True) gives each phase its own centre line and limits.'
)
phases = payload.get('phases')
if phases is None:
phases = payload['phases']
if isinstance(phases, dict):
available = list(phases)
if stratum is None:
raise ValidationError(
f'Chart {chart!r} is stratified, so a stratum is required to return one set of '
f'phases.\nAvailable strata: {available}\n'
f'Use result.phase_statistics({chart!r}, stratum={available[0]!r}) '
f'or result.focus({available[0]!r}).phase_statistics({chart!r}).'
)
if stratum not in phases:
raise ValidationError(
f'Stratum {stratum!r} not found in chart {chart!r}.\nAvailable strata: {available}'
)
return phases[stratum].copy()
if stratum is not None:
raise ValidationError(
'phase_statistics is available on phased Xbar and S charts; a phased X or mR chart '
f"carries each phase's centre and limits on its rows (get_chart({chart!r}))."
f'This result is not stratified, so stratum={stratum!r} does not apply.\n'
f'Use result.phase_statistics({chart!r}) with no stratum.'
)
return phases.copy()

Expand Down
Loading
Loading