Skip to content

Add CellpyCell.update() to refresh a cell from a grown raw source - #1102

Merged
jepegit merged 4 commits into
masterfrom
164-cell-update
Sep 26, 2026
Merged

jepegit merged 4 commits into
masterfrom
164-cell-update

Conversation

@jepegit

@jepegit jepegit commented Sep 25, 2026

Copy link
Copy Markdown
Owner

Closes #164. Epic #783 (Stage 2, L3). Stacked on #1101 (780-load-since); retarget to master once that merges.

What

CellpyCell.update(force=False, **loader_kwargs) -> bool refreshes a cell whose raw source is still being written.

  • Change detection: fresh FileID size/mtime against the stored raw_data_files; database sources always count as changed. Unchanged → False, no work.
  • Incremental path (single file, native schema, harmonized raw, loader matches SupportsIncrementalLoad: arbin_res, arbin_sql, neware_txt, maccor_txt): derive a LoadMarker from the loaded raw (rewound to the last cycle start, both seek fields filled), load_since, stamp test_id, align dtypes, then core.update_core_data + summary extras + scaled columns.
  • Fallback full reload for multi-file cells, non-incremental loaders, or when core rejects the chunk (e.g. single-cycle head). Keeps meta_common (mass, area, nominal capacity), cycle_mode, cell_name.
  • Cellpy-file round trip: loader recreated from data._provenance["source_type"]; **loader_kwargs (e.g. model="UIO") forwarded to set_instrument.
  • FileID stats, last_data_point, and raw_data_files_length refreshed after each update.
  • tests/incremental_support.incremental_update (the L6: golden equality test — incremental update() == full load #778 oracle) now delegates to CellpyCell._update_from_raw_rows.

Tests

tests/test_cell_update.py (9 essential): no-op, growth == full cellpy.get, two growths, FileID refresh, cellpy-file round trip, single-cycle fallback keeps meta, force, no source raises, non-incremental loader routes to full reload.

uv run pytest -m essential: 939 passed, 74 skipped.

Docs

.issueflows/04-designs-and-guides/incremental-load-protocol.md (#164 section), docs/agents/index.md, root AGENTS.md quick facts, HISTORY.md, test registry.

Made with Cursor

jepegit and others added 3 commits September 26, 2026 00:20
Keep the stage-2 halt notes locally. This replaces the leftover #1074 auto status in the same file.

Co-authored-by: Cursor <cursoragent@cursor.com>
arbin_res, arbin_sql, neware_txt, and maccor_txt now match the optional
SupportsIncrementalLoad protocol. Each returns the harmonized rows read since
a LoadMarker and the next marker. The marker rewinds to the first row of the
last cycle read, because harmonize's reset-granularity normalisation rebases
each cycle against its first row; a mid-cycle chunk would be rebased against
the wrong row. Text loaders seek by data row (row_count), arbin by Data_Point
(last_source_datapoint_num) through the existing data_points filter (.res) or
one extra WHERE clause (SQL Server). Other loaders stay full-read.

Tests cover protocol membership, chunk equality with a full harmonize(parse())
read, marker rewind, the empty past-the-end chunk, parse-cache hygiene, and
the #778 equality oracle driven by a real chunk.

Closes #780

Co-authored-by: Cursor <cursoragent@cursor.com>
…#164)

Change detection from file size/mtime, incremental append through
load_since + cellpycore update_core_data for arbin_res, arbin_sql,
neware_txt and maccor_txt, and a full-reload fallback that keeps cell
metadata. Works on cells loaded from a cellpy-file. The #778 test oracle
now delegates to the shipped engine.

Co-authored-by: Cursor <cursoragent@cursor.com>
Base automatically changed from 780-load-since to master September 26, 2026 07:35
@jepegit
jepegit enabled auto-merge (squash) September 26, 2026 07:35
@jepegit
jepegit disabled auto-merge September 26, 2026 07:38
@jepegit
jepegit merged commit e1f3351 into master Sep 26, 2026
6 checks passed
@jepegit
jepegit deleted the 164-cell-update branch September 26, 2026 07:40
@jepegit jepegit mentioned this pull request Sep 26, 2026
10 of 21 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow for c.update()

1 participant