reorg B10: market-review 基准面板 L5 并入 provider seam

.github/copilot-instructions.md

@@ -59,7 +59,7 @@ gunicorn app:app -b 0.0.0.0:5000  # production
 
 Before suggesting non-trivial changes, consult these:
 
-- **[docs/plans/business_line_reorg.md](../docs/plans/business_line_reorg.md)** — the 2026-09 business-line reorg (provider seam + canonical schema, ticker-only Parameters bar, readiness prefetch). Batches **B1–B9 landed**; §10 lists the deferred follow-ups. ADRs [0011](../docs/decisions/0011-pluggable-data-provider-seam.md) / [0012](../docs/decisions/0012-parameter-ownership-and-prefetch.md) are **Accepted** — don't re-litigate.
+- **[docs/plans/business_line_reorg.md](../docs/plans/business_line_reorg.md)** — the 2026-09 business-line reorg (provider seam + canonical schema, ticker-only Parameters bar, readiness prefetch). Batches **B1–B10 landed**; §10 lists the two remaining deferred follow-ups (risk-free-rate global setting, ADR 0011 `symbol` column). ADRs [0011](../docs/decisions/0011-pluggable-data-provider-seam.md) / [0012](../docs/decisions/0012-parameter-ownership-and-prefetch.md) are **Accepted** — don't re-litigate.
 - **[docs/constraints.md](../docs/constraints.md)** — external/historical constraints (yfinance limits, SQLite choice, single-machine assumption, intentional "magic numbers"). Read this before flagging anything as tech debt.
 - **[docs/glossary.md](../docs/glossary.md)** — domain terms (IV vs HV, Greeks, regime, anomaly flags). Read this before assuming a term means what you think it means.
 - **[docs/decisions/](../docs/decisions/)** — Architecture Decision Records. Each ADR explains the context, options considered, and accepted trade-offs for a major design choice.

CLAUDE.md

@@ -12,10 +12,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 > **Business-line reorg (2026-09) — [ADR 0011](docs/decisions/0011-pluggable-data-provider-seam.md)
 > (provider seam + canonical schema) + [0012](docs/decisions/0012-parameter-ownership-and-prefetch.md)
 > (ticker-only Parameters bar + readiness prefetch), both Accepted.**
-> Batches B1–B9 **landed**; the architecture below reflects the end state. Only
+> Batches B1–B10 **landed**; the architecture below reflects the end state. Two
 > deferred follow-ups remain — see [`docs/plans/business_line_reorg.md`](docs/plans/business_line_reorg.md)
-> §10 (risk-free-rate global setting, `market_review_prices` L5 → provider seam,
-> ADR 0011 `symbol` column). Do not re-litigate the Accepted ADRs.
+> §10: the risk-free-rate global setting and the ADR 0011 `symbol` column
+> (ticker→symbol rename, waits for a real second provider). Do not re-litigate
+> the Accepted ADRs.
 
 ## Commands
 

CODEBUDDY.md

@@ -12,10 +12,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 > **Business-line reorg (2026-09) — [ADR 0011](docs/decisions/0011-pluggable-data-provider-seam.md)
 > (provider seam + canonical schema) + [0012](docs/decisions/0012-parameter-ownership-and-prefetch.md)
 > (ticker-only Parameters bar + readiness prefetch), both Accepted.**
-> Batches B1–B9 **landed**; the architecture below reflects the end state. Only
+> Batches B1–B10 **landed**; the architecture below reflects the end state. Two
 > deferred follow-ups remain — see [`docs/plans/business_line_reorg.md`](docs/plans/business_line_reorg.md)
-> §10 (risk-free-rate global setting, `market_review_prices` L5 → provider seam,
-> ADR 0011 `symbol` column). Do not re-litigate the Accepted ADRs.
+> §10: the risk-free-rate global setting and the ADR 0011 `symbol` column
+> (ticker→symbol rename, waits for a real second provider). Do not re-litigate
+> the Accepted ADRs.
 
 ## Commands
 

data_pipeline/read/_query.py

@@ -76,6 +76,72 @@ def get_cleaned_daily(ticker: str, start: dt.date | None = None, end: dt.date |
     return df
 
 
+# DOMAIN: default lookback for the market-review close panel when the caller
+# passes no explicit range. Matches the 400-day window the old
+# market_review_prices ladder used (services/market_review/fetch.py).
+_PANEL_LOOKBACK_DAYS = 400
+
+
+def get_close_panel(
+    symbols: list[str],
+    start: dt.date | None = None,
+    end: dt.date | None = None,
+) -> pd.DataFrame:
+    """Wide close-price panel from ``clean_bars`` for *symbols* (ADR 0011, L5).
+
+    Replaces the old ``market_review_prices`` acquisition path: every symbol —
+    the primary ticker and each benchmark — is coverage-healed through the same
+    ``needs_backfill`` / background ``ensure_range`` machinery the rest of the
+    read layer uses, then its ``close`` series is read from ``clean_bars``.
+
+    Missing ranges are kicked once up front and then awaited **together** for a
+    single short grace window, so a cold panel costs ~one grace period, not one
+    per symbol. Whatever coverage exists at the end of that window is returned;
+    the background backfills keep filling and the next call sees the rest.
+
+    Returns a date-indexed frame with one column per symbol that had any data
+    (input order), or an empty frame when nothing is available.
+    """
+    start = start or (dt.date.today() - dt.timedelta(days=_PANEL_LOOKBACK_DAYS))
+    end = end or dt.date.today()
+    init_db()
+
+    kicked: list[str] = []
+    for sym in symbols:
+        try:
+            _u.manual_update(sym, days=7)
+            if _bf.needs_backfill(sym, start, end):
+                _kick_backfill(sym, start, end)
+                kicked.append(sym)
+            else:
+                _bf.ensure_range(sym, start, end)
+        except Exception as exc:  # noqa: BLE001 — one bad symbol must not sink the panel
+            logger.warning("close-panel coverage check failed for %s: %s", sym, exc)
+
+    if kicked:
+        deadline = time.monotonic() + _BACKFILL_WAIT_SECONDS
+        while time.monotonic() < deadline:
+            if not any(_bf.needs_backfill(s, start, end) for s in kicked):
+                break
+            time.sleep(0.25)
+
+    series: dict[str, pd.Series] = {}
+    for sym in symbols:
+        # fetch_df indexes by date when the column is present.
+        df = fetch_df(
+            "SELECT date, close FROM clean_bars WHERE ticker=? AND date>=? AND date<=? ORDER BY date",
+            (sym, start.isoformat(), end.isoformat()),
+        )
+        if df.empty or "close" not in df.columns:
+            continue
+        s = df["close"].dropna()
+        if not s.empty:
+            series[sym] = s
+    if not series:
+        return pd.DataFrame()
+    return pd.DataFrame(series).sort_index()
+
+
 def get_processed(
     ticker: str, frequency: str = "D", start: dt.date | None = None, end: dt.date | None = None
 ) -> pd.DataFrame:

data_pipeline/read/facade.py

@@ -8,8 +8,8 @@
     *make data ready* is delegated to ``orchestrate``.
 Contracts:
   - ``DataService`` staticmethods — initialize, manual_update, seed_history,
-    has_data_for_date, ensure_range, get_cleaned_daily, get_processed,
-    get_processed_data, get_latest_spot.
+    has_data_for_date, ensure_range, get_cleaned_daily, get_close_panel,
+    get_processed, get_processed_data, get_latest_spot.
 Dependencies UPWARD:
   - store (db), orchestrate (backfill / update)
 Dependencies DOWNWARD:
@@ -73,6 +73,11 @@ def ensure_range(ticker: str, start, end) -> bool:
     def get_cleaned_daily(ticker: str, start=None, end=None):
         return _q.get_cleaned_daily(ticker, start, end)
 
+    @staticmethod
+    def get_close_panel(symbols: list[str], start=None, end=None):
+        """Wide close-price panel over ``clean_bars`` (market review, ADR 0011 L5)."""
+        return _q.get_close_panel(symbols, start, end)
+
     @staticmethod
     def get_processed(ticker: str, frequency: str = "D", start=None, end=None):
         return _q.get_processed(ticker, frequency, start, end)

data_pipeline/store/db.py

@@ -210,17 +210,9 @@ def init_db(db_path: str | None = None):
     _create_table(cur, "raw_prices", _BARS_COLUMNS)
     _create_table(cur, "clean_prices", _CLEAN_BARS_COLUMNS)
     _create_table(cur, "processed_prices", _FEATURE_BARS_COLUMNS)
-    # Market review benchmark close prices
-    cur.execute(
-        """
-        CREATE TABLE IF NOT EXISTS market_review_prices (
-            ticker TEXT NOT NULL,
-            date TEXT NOT NULL,
-            close REAL,
-            PRIMARY KEY (ticker, date)
-        )
-        """
-    )
+    # NOTE (batch B10): the market-review benchmark panel dropped its own
+    # ``market_review_prices`` table — benchmark symbols are stored in
+    # ``clean_bars`` like any other ticker (ADR 0011 L5).
     # Market regime daily log (see core.regime)
     cur.execute(
         """
@@ -308,7 +300,6 @@ def get_conn(db_path: str | None = None):
         "raw_prices",
         "clean_prices",
         "processed_prices",
-        "market_review_prices",
         "regime_log",
         "data_quality_log",
         "tracked_strategies",

data_pipeline/store/repos.py

@@ -112,45 +112,15 @@ def update_tracked_strategy_closed(position_id: int, closed_date: str, closed_va
         return cur.rowcount
 
 
-# ── Market review (benchmark / instrument close panel) ────────────
+# ── Schema bootstrap ─────────────────────────────────────────────
 def ensure_schema() -> None:
     """Bootstrap the SQLite schema (idempotent). Safe to call before any read."""
     init_db()
 
 
-def fetch_market_review_latest_dates(tickers: list[str]) -> dict[str, str | None]:
-    """Return ``{ticker: latest date string | None}`` from ``market_review_prices``."""
-    out: dict[str, str | None] = {}
-    with get_conn() as conn:
-        for t in tickers:
-            row = conn.execute("SELECT MAX(date) FROM market_review_prices WHERE ticker = ?", (t,)).fetchone()
-            out[t] = row[0] if row and row[0] else None
-    return out
-
-
-def upsert_market_review_prices(rows: Iterable[tuple[str, str, float]]) -> None:
-    """Insert or replace ``(ticker, date, close)`` rows in ``market_review_prices``."""
-    rows = list(rows)
-    if not rows:
-        return
-    with get_conn() as conn:
-        conn.executemany(
-            "INSERT INTO market_review_prices (ticker, date, close) "
-            "VALUES (?, ?, ?) ON CONFLICT(ticker, date) DO UPDATE SET close=excluded.close",
-            rows,
-        )
-        conn.commit()
-
-
-def fetch_market_review_panel(range_start: str) -> pd.DataFrame:
-    """Return ``market_review_prices`` rows with ``date >= range_start`` as a DataFrame."""
-    with get_conn() as conn:
-        return pd.read_sql_query(
-            "SELECT ticker, date, close FROM market_review_prices WHERE date >= ? ORDER BY date",
-            conn,
-            params=(range_start,),
-            parse_dates=["date"],
-        )
+# NOTE (batch B10): the market-review benchmark panel no longer has its own
+# table / ladder here — benchmark symbols flow through ``clean_bars`` like any
+# other ticker and are read via ``DataService.get_close_panel`` (ADR 0011 L5).
 
 
 # ── Regime log ───────────────────────────────────────────────────

docs/architecture_review.md

@@ -80,7 +80,7 @@ Rescoped in batch B1 of [ADR 0011](decisions/0011-pluggable-data-provider-seam.m
 | ~~`data_pipeline/providers/yf_client.py` (391 lines, fan-in 11)~~ — **resolved 2026-09-10 (B1)** | it sat 9 lines below the 400-line god-file threshold | the option-chain section was extracted pre-emptively, as prescribed, into `providers/yf_snapshot.py`; `yf_client.py` is now a ~35-line shim. The pressure moved to `providers/yf_snapshot.py` (≈340 lines) and `providers/yfinance_provider.py` (≈290 lines) — watch them before adding endpoints |
 | `static/sim/black_scholes.js` + `core/options/greeks` (risk-free rate) | the same constant is hard-coded in the client simulation **and** the server-side Greeks; changing one without the other makes the two pricings diverge silently | batch B8 retired the Config tab, so there is no global-setting surface to put it in; making it a parameter (store entry + form field + backend path) is a small feature — do it before anyone edits either constant |
 | `data_pipeline/providers/yf_client.py` (compat shim, ADR 0011 B1) | re-exports `fetch_spot` / `fetch_option_chain` / `fetch_close_panel` / `fetch_daily_ohlcv` with their old yfinance-shaped contracts; it was a **"one release"** bridge so importers did not have to change in the B1 PR | once `grep -rn "providers.yf_client\|yf_client import" services/ data_pipeline/read/` is empty (importers moved to the canonical `providers.get_provider()` shapes), delete `yf_client.py` and its re-exports from `providers/__init__.py` in one commit |
-| `services/market_review/fetch.py` → `market_review_prices` (L5 in plan §4.1) | a second acquisition path outside the provider seam: its own L1/L2/L3 close-panel ladder writes a `market_review_prices` table that `data_pipeline/orchestrate/readiness.py` does **not** plan, so the benchmark panel still lazy-fetches on the Market Review slice | fold the ladder into `providers` + a canonical `bars` read (ADR 0011's L5 exit); until then, add `market_review` benchmark tickers to `KIND_DATASETS` so the readiness pass warms them |
+| ~~`services/market_review/fetch.py` → `market_review_prices` (L5 in plan §4.1)~~ — **resolved 2026-09-11 (B10)** | a second acquisition path outside the provider seam: its own L1/L2/L3 close-panel ladder wrote a `market_review_prices` table that the readiness pass did **not** plan, so the benchmark panel lazy-fetched on the Market Review slice | folded into the seam: benchmark symbols are ordinary `clean_bars` tickers now, read via `DataService.get_close_panel` (heals through `ensure_range`); `market_review_prices` + its three repo functions deleted; `services/market/readiness.py` expands the plan with the benchmark symbols so `POST /` prefetches them. `services/market_review/fetch.py` keeps only its 5-min L1 memo |
 | ~~`services/market/analysis/assessment.py` option overlay~~ — **retired 2026-09-11 (B9, plan §10 F5-a)** | B7 left the projection-vs-positions overlay + the sizing max-loss unfed (positions moved to the Portfolio tab) | removed: the `option_data` branches in `assessment.py`, `FormService.parse_option_data`, `MarketAnalyzer.analyze_options` / `MarketChartAssembly.analyze_options`, and `core/market/option_pnl.py` + `core/market/charts/option_pnl.py` (whole files). Assessment sizing is now debit-only; position P&L lives in the Portfolio tab |
 
 ## 3. Guardrails (how the score is kept)

docs/decisions/0011-pluggable-data-provider-seam.md

@@ -151,6 +151,11 @@ writes both families) and `scripts/migrate_canonical_tables.py` backfills an exi
   update `docs/constraints.md` §1, `docs/l0_architecture.md`,
   `docs/architecture_review.md` §2, and `scripts/doc_guard.py::_ALLOWED_DEPS`
   in the same batches.
+- Landed 2026-09-11 (batch B10): L5 closed. `services/market_review/fetch.py`'s
+  parallel ladder and the `market_review_prices` table are gone — the benchmark
+  close panel is now `DataService.get_close_panel` over `clean_bars`, so every
+  acquisition path runs through `providers/`. Deferred still: the `symbol` column
+  (ticker→symbol rename), which waits for a real second provider.
 
 ## References