cellpy 2.2 (Stage 5) — tracking issue

[jepegit]

cellpy 2.2 = the deferred "complete cellpy 2" features (architecture plan §3): additive,
version-cleanly. Mirror of 2.1 — where 2.0 finalized what can't be shimmed and 2.1 spent the
shims, 2.2 delivers the parked features. Kept deliberately lean: v2-migration bug reports
ship as v2.1.x patches off master, not folded in here.

Plan: stage5-github-issues.md (revision 2026-09-08 in §11) · coordinating: cellpy2-architecture-plan.md §Stage 5.

Epic L — Live / incremental refresh (cellpy-only; design done)

• L6: golden equality test — incremental update() == full load #778 L6 — golden equality test (author first: incremental update() == full load)
• L1: SupportsIncrementalLoad protocol + LoadMarker/IncrementalChunk (cellpy-hosted) #779 L1 — SupportsIncrementalLoad protocol + marker/chunk types (cellpy-hosted)
• L2: load_since() for cheap-partial loaders (arbin_res/sql, neware_txt, maccor_txt) #780 L2 — load_since() for cheap-partial loaders (arbin_res/sql, neware_txt, maccor_txt)
• Allow for c.update() #164 L3 — CellpyCell.update() (change-detect via check_file_ids/NEWEST + protocol/fallback; persist marker)
• L4: live.py poll loop; retire processor.py; fix batch_core lstrip #781 L4 — live.py poll loop; retire processor.py (batch_core lstrip item moot — module gone since batch v3)
• L5: batch live refresh — b.update(live=True) / b.poll() #782 L5 — batch live refresh (b.update(live=True) / b.poll()) on top of Batch.update()

Epic S — Step/summary science (additive columns, current schema; S1–S3 core-first)

• Estimate internal resistance at start and end of (dis)charge #313 S1 — internal resistance at start/end of (dis)charge
• Split single-step CCCV-steps into CC and CV substeps #312 S2 — split single-step CCCV into CC/CV substeps + CV-share
• handle full cells that starts with a discharge step #359 S3 — full cells that start with a discharge (cycle_mode/counter) — convention delta
• handle Reference Performance Tests #888 S4 — reference-performance-test (RPT) filters on cellpy.filters (cellpy-only; added 2026-09-08)

Epic I — Instruments / IO

• Add support for Biologic mpr data version 3 #270 I1 — Biologic mpr v3
• Better support for csv-type files #338 I2 — better csv support (format inspection; discovery half largely covered by list_instruments() still logs loader-probe warnings on every call #786/Missing external tools fail silently: mdb-export raises bare FileNotFoundError, pyodbc ImportError hides two loaders #938)
• exporter from Arbin db #306 I3 — Arbin-db exporter (one file per test)
• Custom loader silently omits a declared-but-absent column #761 I4 — loader robustness: declared-but-absent column must not be silently dropped (posture precedent: Missing external tools fail silently: mdb-export raises bare FileNotFoundError, pyodbc ImportError hides two loaders #938)
• Support Multiple Cell PEC CSV Files #827 I5 — PEC csv with several cells in one file → tuple of Data (added 2026-09-08)

Epic R — Remote paths & discovery

• Regression: scp:// SSH-config Host aliases no longer resolve after UPath OtherPath switch #687 R1 — regression: scp://<ssh-config-alias>/... resolution (or fail-loud + docs)
• feat: filefinder / batch dump scoped to project subfolder with fuzzy match hints #691 R2 — fuzzy match hints on a missing project folder (project scoping + raise shipped in Wire auto_use_file_list in journal_from_db / find_files #900)

Epic M — External metadata sources / BatBase (read path in 2.2; cross-repo)

• Implement BatBaseClient (OAuth2 client-credentials) cellpy/cellpy-connectors#1 M0 — BatBaseClient (OAuth2 client-credentials, keyring/env, configure batbase) + get passthrough / cellpy connectors batbase get (yolo)
• Pluggable external metadata sources (query an API for cell/test metadata) #784 M1 — MetadataSource Protocol + MetaResolver hook (read-only) — merged via feat: pluggable external metadata sources — Protocol, registry, resolver hook (#784) #1106 (2026-09-26); CellMeta.uuid core-first as CellMeta.uuid: cell-level join key for external metadata sources cellpy/cellpy-core#151
• BatBase MetadataSource adapter (on top of BatBaseClient) cellpy/cellpy-connectors#2 M2 — BatBase MetadataSource adapter (/api/test-cellpy-tag/ → MetaRecord) — merged feat: BatBase MetadataSource adapter for cellpy (#2) cellpy/cellpy-connectors#9 (2026-09-26); API side ife-bat/batbase#473 → PR ife-bat/batbase#475 (open)
• Deferred to 2.3 (stage 6 §6): M3 push. M4 Epic M: consume file pointers from external metadata sources (skip filefinder) #1107 (file pointers from the source, skip filefinder) merged early in feat: file pointers from external metadata sources — cellpy.get(source=) and batch.from_source (#1107) #1119 (2026-10-01); follow-ups Epic M: use FileRef size/mtime from the metadata source to skip stat-ing raw files on update() #1124 (size/mtime short-circuit in update()) and BatBase adapter: map experiment files[] to MetaRecord.files (FileRef) cellpy/cellpy-connectors#11 (adapter maps BatBase files[] → FileRef). Server side is done: ife-bat/batbase#473 (annotations, PR Stage 1.1: cellpy-file format spec in cellpy_file/format.py (#446) #475) and docs: Stage 0 foundations complete (#439) #474 (TestDataFile + files + writable /api/test-data-file/, PR 476 golden benchmark fixes #478) merged 2026-09-26; CellMeta.uuid in core (feat(metadata): add CellMeta.uuid cell-level join key (#151) cellpy/cellpy-core#152, awaiting release + re-pin). User guide: docs/guides/metadata_sources.md (docs: how-to guide for pulling cell metadata from a lab database (BatBase) (#1023) #1108).
• M3 push / journal write-back → 2.3 (Stage 6)

Milestone extras (unstaged)

• Make a function to plot the initial OCV before cycle 1 starts for a batch object #352 — initial OCV (before cycle 1) batch plot

Deferred → 2.3 (v.2.3 milestone)

SPEED-30 value+unit+dtype versioned headers (structural headline) · #73 GITT/PITT analysis · #889 Fredrik ICA.
Design-only, post-2.2: original-vs-processed data curation (#206).

Sequencing

Only S1–S3 are core-first (core PR → PyPI → cellpy re-pin); L / I / R / S4 are cellpy-only.
L6 first, then L1 → L2 → L3 → L4/L5. S1/S2 share one schema addition; S3 is a
user-visible convention change → #-delta register + comparator exception. I1–I5 and R1
parallel; R2 is a small follow-on to #900. Startable now: S-oracle characterization,
#888, #270, #306, #827, #687, #691, #784, #352.

Progress (2026-09-26)

• Epic L shipped (all six issues closed): L6: golden equality test — incremental update() == full load #778 oracle (test: golden equality oracle for incremental updates (L6, #778) #1016), L1: SupportsIncrementalLoad protocol + LoadMarker/IncrementalChunk (cellpy-hosted) #779 protocol (Add an optional incremental-load protocol #1100), L2: load_since() for cheap-partial loaders (arbin_res/sql, neware_txt, maccor_txt) #780 load_since() (Add load_since() to arbin_res, arbin_sql, neware_txt, and maccor_txt #1101), Allow for c.update() #164 CellpyCell.update() (Add CellpyCell.update() to refresh a cell from a grown raw source #1102), L4: live.py poll loop; retire processor.py; fix batch_core lstrip #781 live.poll() + processor.py removed (Add live.poll() and retire utils.processor #1103), L5: batch live refresh — b.update(live=True) / b.poll() #782 Batch.refresh() / update(live=True) / poll() (Add batch live refresh: Batch.refresh, update(live=True) and poll #1104). Design notes: .issueflows/04-designs-and-guides/incremental-load-protocol.md.
• I4 Custom loader silently omits a declared-but-absent column #761 closed 2026-09-17 (fix: raise when custom loader misses a declared column #1049).
• Remaining: Epic S (Estimate internal resistance at start and end of (dis)charge #313 Split single-step CCCV-steps into CC and CV substeps #312 handle full cells that starts with a discharge step #359 core-first; handle Reference Performance Tests #888), Epic I (Add support for Biologic mpr data version 3 #270 Better support for csv-type files #338 exporter from Arbin db #306 Support Multiple Cell PEC CSV Files #827), Epic R (Regression: scp:// SSH-config Host aliases no longer resolve after UPath OtherPath switch #687 feat: filefinder / batch dump scoped to project subfolder with fuzzy match hints #691), Epic M read path (Pluggable external metadata sources (query an API for cell/test metadata) #784 + BatBase MetadataSource adapter (on top of BatBaseClient) cellpy/cellpy-connectors#2; M0 Implement BatBaseClient (OAuth2 client-credentials) cellpy/cellpy-connectors#1 shipped 2026-09-26 via feat: BatBase client with OAuth2 client-credentials and read passthrough (#1) cellpy/cellpy-connectors#8), extra Make a function to plot the initial OCV before cycle 1 starts for a batch object #352.
• Stage 6 (2.3) draft plan: cellpy-design-and-development/roadmap/stages/stage6-github-issues.md.
• Orchestration note: the epic's issues are all yolo: no, so /iflow-drive halted at L1: SupportsIncrementalLoad protocol + LoadMarker/IncrementalChunk (cellpy-hosted) #779; the rest was run by hand. Tracked as iflow-drive stops at the first yolo:no issue and cannot resolve stacked-PR / squash conflicts; should carry through under the overnight confirm issue-flow#386.

Progress (2026-09-08)

• Stage 5 epics: not started (all checklist items still open).
• Reactive stream shipped: v2.1.1.post4 … v2.1.3.post3, and v2.1.4 (2026-09-08, on cellpycore==0.2.5) — ≈85 issues closed out-of-band. Patch line now empty.
• Check capacity calculation behavior #989 (capacity rebase-on-load, 2.1.4) recorded as Δ8 in the delta register; stage5 decision Utils #6 amended (number-changing patch fixes get a Δ-row).
• Milestone also holds Pluggable external metadata sources (query an API for cell/test metadata) #784 (metadata sources) and Make a function to plot the initial OCV before cycle 1 starts for a batch object #352 (initial OCV batch plot).
• Day-to-day status: CURRENT.md.

Progress (2026-07-31)

• Stage 5 epics: not started (all checklist items still open).
• Reactive stream shipped: v2.1.1 … v2.1.1.post3 (app-builder collect/plot work); v.2.1.2 still open (batch - read custom json #345/Lightweight metadata read: read_meta(path) without a full get() #799/Per-instrument metadata schema (which fields an instrument import needs) #800).