Skip to content

feat(math): PCA warm start in clojure-legacy engine mode (PR-B)#2640

Draft
jucor wants to merge 1 commit into
spr/edge/d61d82adfrom
spr/edge/be96f1da
Draft

feat(math): PCA warm start in clojure-legacy engine mode (PR-B)#2640
jucor wants to merge 1 commit into
spr/edge/d61d82adfrom
spr/edge/be96f1da

Conversation

@jucor

@jucor jucor commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Threads the previous tick's unit components into the power-iteration PCA as
start vectors when POLISMATH_ENGINE_MODE=clojure-legacy, matching Clojure
(conversation.clj:381-387 passes :start-vectors (get-in conv [:pca :comps])
into powerit-pca, pca.clj:86-105; new columns absorbed by 1-padding,
pca.clj:46-49). Default 'improved' mode is unchanged (cold recompute).

  • pca_project_dataframe gains start_vectors (default None) and require_powerit
    (default False). With both absent the code path is BYTE-IDENTICAL to before.
    When a warm start is supplied — or required by legacy mode — and
    POLISMATH_PCA_IMPL=sklearn is set, we warn and fall back to power iteration:
    sklearn's SVD has no start-vector hook, so running it would silently drop the
    warm start (documented at the guard).
  • powerit_pca already supported start_vectors + 1-padding + all-zero->None;
    this just wires production callers to it.
  • Conversation._compute_pca: in legacy mode sets require_powerit=True and, when
    prev_pca has real comps, start_vectors = prev comps (empty/cold -> None, the
    deterministic random cold draw). prev_pca is captured in recompute() (PR-A).

Semantic judgment calls (integrator, please verify vs Clojure):

  • Warm start is POSITIONAL: prev comps align to current columns by index, and
    new (appended) comments are 1-padded — exactly Clojure's behavior. Column
    removal/reorder would misalign, but Clojure is append-only here and
    _power_iteration truncates a too-long start vector defensively (pre-existing,
    pca.py:124-127) where Clojure would error.
  • "legacy implies powerit" is enforced via require_powerit=True even on the
    cold first tick, so an operator setting PCA_IMPL=sklearn + legacy still gets
    powerit (with a warning).

TDD RED evidence (before implementing):
test_start_vectors_reach_powerit:
"TypeError: pca_project_dataframe() got an unexpected keyword argument
'start_vectors'"
test_legacy_tick2_receives_tick1_comps: "assert None is not None"
(tick 2 ran cold; no comps threaded)

Tests: 8 new (start_vectors threading, sklearn-conflict warning, improved
byte-identity, two-tick legacy warm start via spy, angle stability, cold-tick
cross-mode identity). Full suite 422 passed, 17 skipped, 47 xfailed.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

commit-id:be96f1da


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

@jucor
jucor marked this pull request as draft July 18, 2026 13:35
@jucor
jucor changed the base branch from spr/edge/2324207e to spr/edge/d61d82ad July 18, 2026 13:37
@jucor
jucor requested a review from Copilot July 21, 2026 08:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Clojure-parity PCA warm-start behavior to the Python Delphi math pipeline when POLISMATH_ENGINE_MODE=clojure-legacy, by threading the previous tick’s PCA components into the power-iteration solver as start vectors, while keeping the default improved mode cold/recompute behavior unchanged.

Changes:

  • Extend pca_project_dataframe to accept start_vectors and require_powerit, and ensure sklearn PCA is not used when warm-start/powerit is required (warn + fall back).
  • In Conversation._compute_pca, when in legacy mode, require power-iteration and feed previous tick’s pca['comps'] as warm-start vectors when available.
  • Add a dedicated test module covering threading, fallback warning, and two-tick warm-start behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
delphi/polismath/pca_kmeans_rep/pca.py Adds warm-start/powerit forcing controls and sklearn-conflict warning/fallback; threads start_vectors into powerit_pca.
delphi/polismath/conversation/conversation.py Wires legacy engine mode to require power-iteration and optionally seed PCA from previous tick comps.
delphi/tests/test_pca_warm_start.py New tests validating warm-start threading, warning/fallback behavior, and cold-path invariance across modes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread delphi/polismath/conversation/conversation.py Outdated
Comment thread delphi/polismath/pca_kmeans_rep/pca.py Outdated
Comment thread delphi/tests/test_pca_warm_start.py Outdated
@jucor
jucor force-pushed the spr/edge/d61d82ad branch from 5144dc9 to b193ee4 Compare July 21, 2026 11:11
@jucor
jucor force-pushed the spr/edge/be96f1da branch 2 times, most recently from e5d3b9f to d75b16a Compare July 21, 2026 19:49
@jucor
jucor force-pushed the spr/edge/d61d82ad branch from b193ee4 to 29beda7 Compare July 21, 2026 19:49
Threads the previous tick's unit components into the power-iteration PCA as
start vectors when POLISMATH_ENGINE_MODE=clojure-legacy, matching Clojure
(conversation.clj:381-387 passes :start-vectors (get-in conv [:pca :comps])
into powerit-pca, pca.clj:86-105; new columns absorbed by 1-padding,
pca.clj:46-49). Default 'improved' mode is unchanged (cold recompute).

- pca_project_dataframe gains start_vectors (default None) and require_powerit
  (default False). With both absent the code path is BYTE-IDENTICAL to before.
  When a warm start is supplied — or required by legacy mode — and
  POLISMATH_PCA_IMPL=sklearn is set, we warn and fall back to power iteration:
  sklearn's SVD has no start-vector hook, so running it would silently drop the
  warm start (documented at the guard).
- powerit_pca already supported start_vectors + 1-padding + all-zero->None;
  this just wires production callers to it.
- Conversation._compute_pca: in legacy mode sets require_powerit=True and, when
  prev_pca has real comps, start_vectors = prev comps (empty/cold -> None, the
  deterministic random cold draw). prev_pca is captured in recompute() (PR-A).

Semantic judgment calls (integrator, please verify vs Clojure):
- Warm start is POSITIONAL: prev comps align to current columns by index, and
  new (appended) comments are 1-padded — exactly Clojure's behavior. Column
  removal/reorder would misalign, but Clojure is append-only here and
  _power_iteration truncates a too-long start vector defensively (pre-existing,
  pca.py:124-127) where Clojure would error.
- "legacy implies powerit" is enforced via require_powerit=True even on the
  cold first tick, so an operator setting PCA_IMPL=sklearn + legacy still gets
  powerit (with a warning).

TDD RED evidence (before implementing):
  test_start_vectors_reach_powerit:
    "TypeError: pca_project_dataframe() got an unexpected keyword argument
     'start_vectors'"
  test_legacy_tick2_receives_tick1_comps: "assert None is not None"
  (tick 2 ran cold; no comps threaded)

Tests: 8 new (start_vectors threading, sklearn-conflict warning, improved
byte-identity, two-tick legacy warm start via spy, angle stability, cold-tick
cross-mode identity). Full suite 422 passed, 17 skipped, 47 xfailed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

commit-id:be96f1da
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 02:01
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 02:02
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 02:10
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 02:10
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 03:54
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 03:54
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 06:16
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 06:16
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 06:50
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 06:50
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 08:11
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 08:11
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 09:58
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 09:58
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 22, 2026 11:21
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 22, 2026 11:21
@jucor
jucor changed the base branch from spr/edge/d61d82ad to edge July 24, 2026 06:02
@jucor
jucor changed the base branch from edge to spr/edge/d61d82ad July 24, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants