Add distribution drift detection

stats had A/B experiment tests but no Population Stability Index and no
Kolmogorov-Smirnov two-sample test for the reference-vs-current "is
today's data shaped like the baseline" check. Add psi, ks_two_sample,
categorical_drift, and a detect_drift wrapper that pair with data_profile.
Wired through facade, executor (AC_detect_drift / AC_categorical_drift),
MCP, and the Script Builder with a headless test batch and EN/Zh docs.

Author: JE-Chen

README.md

@@ -13,6 +13,7 @@
 
 ## Table of Contents
 
+- [What's new (2026-06-21) — Distribution Drift Detection](#whats-new-2026-06-21--distribution-drift-detection)
 - [What's new (2026-06-21) — Layered Configuration Resolver](#whats-new-2026-06-21--layered-configuration-resolver)
 - [What's new (2026-06-21) — Server-Sent Events (SSE) Client Parser](#whats-new-2026-06-21--server-sent-events-sse-client-parser)
 - [What's new (2026-06-21) — Dotenv (.env) Parsing](#whats-new-2026-06-21--dotenv-env-parsing)
@@ -134,6 +135,12 @@
 
 ---
 
+## What's new (2026-06-21) — Distribution Drift Detection
+
+Check whether today's data is shaped like the baseline. Full reference: [`docs/source/Eng/doc/new_features/v82_features_doc.rst`](docs/source/Eng/doc/new_features/v82_features_doc.rst).
+
+- **`psi` / `ks_two_sample` / `categorical_drift` / `detect_drift`** (`AC_detect_drift`, `AC_categorical_drift`): `stats` had A/B experiment tests but no Population Stability Index and no KS two-sample test for reference-vs-current distributions. This adds PSI (quantile-binned log-ratio), the KS statistic with a Kolmogorov p-value, and a categorical chi-square + total-variation summary — pairing with `data_profile`. `detect_drift` gives a one-call `{psi, drifted, ks}` verdict. Pure-stdlib, deterministic.
+
 ## What's new (2026-06-21) — Layered Configuration Resolver
 
 Compose config with `defaults < file < env < CLI` precedence. Full reference: [`docs/source/Eng/doc/new_features/v81_features_doc.rst`](docs/source/Eng/doc/new_features/v81_features_doc.rst).

README/README_zh-CN.md

@@ -12,6 +12,7 @@
 
 ## 目录
 
+- [本次更新 (2026-06-21) — 分布漂移检测](#本次更新-2026-06-21--分布漂移检测)
 - [本次更新 (2026-06-21) — 分层配置解析器](#本次更新-2026-06-21--分层配置解析器)
 - [本次更新 (2026-06-21) — Server-Sent Events (SSE) 客户端解析器](#本次更新-2026-06-21--server-sent-events-sse-客户端解析器)
 - [本次更新 (2026-06-21) — Dotenv (.env) 解析](#本次更新-2026-06-21--dotenv-env-解析)
@@ -133,6 +134,12 @@
 
 ---
 
+## 本次更新 (2026-06-21) — 分布漂移检测
+
+检查今天的数据形状是否与基准一致。完整参考:[`docs/source/Zh/doc/new_features/v82_features_doc.rst`](../docs/source/Zh/doc/new_features/v82_features_doc.rst)。
+
+- **`psi` / `ks_two_sample` / `categorical_drift` / `detect_drift`**(`AC_detect_drift`、`AC_categorical_drift`):`stats` 有 A/B 实验检定,但没有 Population Stability Index,也没有针对 reference-vs-current 分布的 KS 双样本检定。本功能加入 PSI(分位分箱的 log-ratio)、KS 统计量与 Kolmogorov p 值,以及类别卡方 + total-variation 摘要 —— 与 `data_profile` 搭配。`detect_drift` 给出一次性的 `{psi, drifted, ks}` 判定。纯标准库、确定。
+
 ## 本次更新 (2026-06-21) — 分层配置解析器
 
 以 `defaults < file < env < CLI` 优先级组合配置。完整参考:[`docs/source/Zh/doc/new_features/v81_features_doc.rst`](../docs/source/Zh/doc/new_features/v81_features_doc.rst)。

README/README_zh-TW.md

@@ -12,6 +12,7 @@
 
 ## 目錄
 
+- [本次更新 (2026-06-21) — 分布漂移偵測](#本次更新-2026-06-21--分布漂移偵測)
 - [本次更新 (2026-06-21) — 分層設定解析器](#本次更新-2026-06-21--分層設定解析器)
 - [本次更新 (2026-06-21) — Server-Sent Events (SSE) 用戶端解析器](#本次更新-2026-06-21--server-sent-events-sse-用戶端解析器)
 - [本次更新 (2026-06-21) — Dotenv (.env) 解析](#本次更新-2026-06-21--dotenv-env-解析)
@@ -133,6 +134,12 @@
 
 ---
 
+## 本次更新 (2026-06-21) — 分布漂移偵測
+
+檢查今天的資料形狀是否與基準一致。完整參考:[`docs/source/Zh/doc/new_features/v82_features_doc.rst`](../docs/source/Zh/doc/new_features/v82_features_doc.rst)。
+
+- **`psi` / `ks_two_sample` / `categorical_drift` / `detect_drift`**(`AC_detect_drift`、`AC_categorical_drift`):`stats` 有 A/B 實驗檢定,但沒有 Population Stability Index,也沒有針對 reference-vs-current 分布的 KS 雙樣本檢定。本功能加入 PSI(分位分箱的 log-ratio)、KS 統計量與 Kolmogorov p 值,以及類別卡方 + total-variation 摘要 —— 與 `data_profile` 搭配。`detect_drift` 給出一次性的 `{psi, drifted, ks}` 判定。純標準函式庫、具決定性。
+
 ## 本次更新 (2026-06-21) — 分層設定解析器
 
 以 `defaults < file < env < CLI` 優先序組合設定。完整參考:[`docs/source/Zh/doc/new_features/v81_features_doc.rst`](../docs/source/Zh/doc/new_features/v81_features_doc.rst)。

docs/source/Eng/doc/new_features/v82_features_doc.rst

@@ -0,0 +1,42 @@
+Distribution Drift Detection
+============================
+
+``stats`` has two-sample tests for A/B *experiment* outcomes (proportions and
+means), but no Population Stability Index and no Kolmogorov-Smirnov two-sample
+test for the canonical "is today's data shaped like the baseline" check. This
+adds PSI, KS, and a categorical-drift summary that pair with ``data_profile``.
+
+Pure standard library (``math`` / ``bisect`` / ``collections`` + reuse of
+``stats.percentile``); imports no ``PySide6``. Every function is pure
+(sequences in, dict/float out), so it is fully deterministic in CI.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import psi, ks_two_sample, categorical_drift, detect_drift
+
+    score = psi(reference, current)               # Population Stability Index
+    ks = ks_two_sample(reference, current)        # {statistic, p_value}
+    report = detect_drift(reference, current)     # {psi, drifted, ks}
+
+    cat = categorical_drift(ref_labels, cur_labels)
+    # {chi_square, total_variation, categories}
+
+``psi`` bins ``current`` against ``reference`` quantile edges and sums the
+log-ratio contribution per bin (0 for identical distributions, growing as they
+diverge). ``ks_two_sample`` returns the maximum empirical-CDF gap and a p-value
+from the Kolmogorov distribution. ``categorical_drift`` compares label
+frequencies via a chi-square statistic and the total-variation distance.
+``detect_drift`` wraps the numeric path into one report with a ``drifted``
+verdict at ``threshold`` (default ``0.25``).
+
+Executor commands
+-----------------
+
+``AC_detect_drift`` takes ``reference`` / ``current`` numeric lists (and
+optional ``threshold`` / ``bins``) and returns ``{psi, drifted, ks}``.
+``AC_categorical_drift`` returns the categorical summary. Both are exposed as
+MCP tools (``ac_detect_drift`` / ``ac_categorical_drift``) and as Script Builder
+commands under **Data**.

docs/source/Eng/eng_index.rst

@@ -104,6 +104,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v79_features_doc
    doc/new_features/v80_features_doc
    doc/new_features/v81_features_doc
+   doc/new_features/v82_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

docs/source/Zh/doc/new_features/v82_features_doc.rst

@@ -0,0 +1,35 @@
+分布漂移偵測
+==========
+
+``stats`` 具備針對 A/B *實驗*結果(比例與平均)的雙樣本檢定,但沒有 Population Stability Index,也沒有
+Kolmogorov-Smirnov 雙樣本檢定來做經典的「今天的資料形狀是否與基準一致」檢查。本功能加入 PSI、KS,以及
+與 ``data_profile`` 搭配的類別漂移摘要。
+
+純標準函式庫(``math`` / ``bisect`` / ``collections`` + 重用 ``stats.percentile``);不匯入 ``PySide6``。
+每個函式皆為純函式(輸入序列、輸出 dict/float),因此在 CI 中完全具決定性。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import psi, ks_two_sample, categorical_drift, detect_drift
+
+    score = psi(reference, current)               # Population Stability Index
+    ks = ks_two_sample(reference, current)        # {statistic, p_value}
+    report = detect_drift(reference, current)     # {psi, drifted, ks}
+
+    cat = categorical_drift(ref_labels, cur_labels)
+    # {chi_square, total_variation, categories}
+
+``psi`` 以 ``reference`` 的分位邊界將 ``current`` 分箱,並加總每箱的 log-ratio 貢獻(分布相同為 0,
+分歧越大值越大)。``ks_two_sample`` 回傳最大經驗 CDF 差距與 Kolmogorov 分布的 p 值。``categorical_drift``
+以卡方統計量與 total-variation 距離比較類別頻率。``detect_drift`` 把數值路徑包成一份報告,並以 ``threshold``
+(預設 ``0.25``)給出 ``drifted`` 判定。
+
+執行器命令
+----------
+
+``AC_detect_drift`` 接受 ``reference`` / ``current`` 數值清單(以及選用的 ``threshold`` / ``bins``)
+回傳 ``{psi, drifted, ks}``。``AC_categorical_drift`` 回傳類別摘要。兩者皆以 MCP 工具
+(``ac_detect_drift`` / ``ac_categorical_drift``)以及 Script Builder 中 **Data** 分類下的命令提供。

docs/source/Zh/zh_index.rst

@@ -104,6 +104,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v79_features_doc
    doc/new_features/v80_features_doc
    doc/new_features/v81_features_doc
+   doc/new_features/v82_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

je_auto_control/__init__.py

@@ -159,6 +159,10 @@
 )
 # Data profiling: per-column stats + schema inference from observed rows
 from je_auto_control.utils.data_profile import infer_schema, profile_rows
+# Distribution drift: PSI, KS two-sample, categorical drift
+from je_auto_control.utils.data_drift import (
+    categorical_drift, detect_drift, ks_two_sample, psi,
+)
 # i18n / l10n testing: pseudo-localize, overflow + catalog checks
 from je_auto_control.utils.i18n_test import (
     check_catalog, check_overflow, pseudo_localize, pseudo_localize_catalog,
@@ -823,6 +827,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "merge_results", "shard_flows",
     "extract_fields", "mask_rows", "validate_rows",
     "infer_schema", "profile_rows",
+    "categorical_drift", "detect_drift", "ks_two_sample", "psi",
     "check_catalog", "check_overflow", "pseudo_localize",
     "pseudo_localize_catalog",
     "Checkpoint", "CheckpointStore", "run_resumable",

je_auto_control/gui/script_builder/command_schema.py

@@ -1697,6 +1697,29 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Show the value and winning layer for a dotted config key.",
     ))
+    specs.append(CommandSpec(
+        "AC_detect_drift", "Data", "Data Drift: Detect (PSI + KS)",
+        fields=(
+            FieldSpec("reference", FieldType.STRING,
+                      placeholder="[1.0, 2.0, 3.0, ...]"),
+            FieldSpec("current", FieldType.STRING,
+                      placeholder="[1.1, 2.2, 9.9, ...]"),
+            FieldSpec("threshold", FieldType.FLOAT, optional=True,
+                      default=0.25),
+            FieldSpec("bins", FieldType.INT, optional=True, default=10),
+        ),
+        description="Numeric drift: Population Stability Index + KS two-sample.",
+    ))
+    specs.append(CommandSpec(
+        "AC_categorical_drift", "Data", "Data Drift: Categorical",
+        fields=(
+            FieldSpec("reference", FieldType.STRING,
+                      placeholder='["a", "b", "a", ...]'),
+            FieldSpec("current", FieldType.STRING,
+                      placeholder='["a", "c", "c", ...]'),
+        ),
+        description="Categorical drift: chi-square + total-variation distance.",
+    ))
     specs.append(CommandSpec(
         "AC_rate_limit", "Flow", "Rate Limit (Token Bucket)",
         fields=(

je_auto_control/utils/data_drift/__init__.py

@@ -0,0 +1,6 @@
+"""Distribution drift detection for AutoControl data checks."""
+from je_auto_control.utils.data_drift.data_drift import (
+    categorical_drift, detect_drift, ks_two_sample, psi,
+)
+
+__all__ = ["categorical_drift", "detect_drift", "ks_two_sample", "psi"]