diff --git a/HISTORY.md b/HISTORY.md index 2c24a422..34f3ea2d 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -46,6 +46,12 @@ `b.summaries` and the QC report (`b.last_report`) on ticks that changed. (#782) +* Docs: how-to guide "Pull cell metadata from a lab database" (BatBase via + `cellpy-connectors`: install, `cellpy connectors configure batbase`, + `fetch_meta` key kinds, `refresh_after`, `external_links`, field/unit map, + troubleshooting), wired into the guides index, How-do-I and `llms.txt`. + (#1023) + ## [2.1.5.post6] - 2026-09-25 * `summary_collector(...).plot()` keeps a lone charge or discharge series diff --git a/docs/guides/index.md b/docs/guides/index.md index b9d65aa6..7110aadc 100644 --- a/docs/guides/index.md +++ b/docs/guides/index.md @@ -27,6 +27,9 @@ something specific. If you just want the one-liner, start with batch utility reads, and which columns you actually have to fill in - [Work with remote files](remote_paths.md) — load raw data and cellpy files over SSH / SFTP +- [Pull cell metadata from a lab database](metadata_sources.md) — let + BatBase (or another metadata source plugin) supply mass, area, nominal + capacity and cell type **Extending cellpy** diff --git a/docs/guides/metadata_sources.md b/docs/guides/metadata_sources.md new file mode 100644 index 00000000..a57a7235 --- /dev/null +++ b/docs/guides/metadata_sources.md @@ -0,0 +1,194 @@ +# Pull cell metadata from a lab database (BatBase) + +You have the raw file from the cycler. The **mass, electrode area, nominal +capacity and cell type** live somewhere else — in your lab's database. This +page shows how to let cellpy fetch them instead of typing them in, using +IFE's BatBase as the example. The same steps apply to any *metadata source* +plugin. + +If you only want the one-liner, see [How do I…?](../how_do_i.md#set-the-cell-up-correctly). + +## What you get + +```python +import cellpy + +c = cellpy.get("20250925_siba001_01_hcicc_lif_01.res", instrument="arbin_res") +c.fetch_meta("batbase") # looks the cell up by its name in BatBase +c.refresh_after() # recompute the summary with the new mass etc. + +c.data.meta_common.mass # → 2.35 (mg, from the electrode record) +c.data.meta_common.nom_cap # → 3579.0 (mAh/g) +c.external_links["batbase"] # → where it came from (row id + URL) +``` + +`fetch_meta` is read-only towards the database and applies the record the same +way a batch-journal row would: **above** what the instrument file wrote, +**below** anything you pass explicitly (`mass=…`) afterwards. If the database +is unreachable, the cell still loads — you get a warning and no metadata layer. + +## 1. Install the connector + +Metadata sources are plugins. BatBase lives in the `cellpy-connectors` +package; install it into the same environment as cellpy: + +```bash +pip install cellpy-connectors # or: uv pip install cellpy-connectors +``` + +Check that cellpy sees it: + +```python +from cellpy.readers import metadata_sources +metadata_sources.names() # → ('batbase',) +``` + +If the tuple is empty, the package went into a different environment than the +one running cellpy. + +## 2. Give cellpy your credentials — once + +BatBase issues each user an **API client id and secret** (OAuth2 client +credentials; ask your BatBase admin, or create one on your BatBase profile +page). Store them in your operating system's keyring: + +```bash +cellpy connectors configure batbase +``` + +You are prompted for the id and the secret; nothing is echoed and nothing is +written to a file. On a machine without a keyring (a compute cluster, CI), use +environment variables instead: + +| Variable | Meaning | +| --- | --- | +| `CELLPY_BATBASE_CLIENT_ID` | the client id | +| `CELLPY_BATBASE_CLIENT_SECRET` | the client secret | +| `CELLPY_BATBASE_URL` | the server, e.g. `https://batbase.example.org` (defaults to a local dev server) | + +Then prove the connection works: + +```bash +cellpy connectors batbase check +``` + +It prints a small JSON block with `"authenticated": true` and the scope +(`read`). A wrong secret says so in plain words; fix it with `configure` +again. cellpy never reads these secrets from its YAML config file, so do not +put them there. + +## 3. Fetch + +`fetch_meta(source, key, kind=…)` needs to know **what to look for**. BatBase +accepts four kinds of key: + +| `kind=` | `key` is … | Typical use | +| --- | --- | --- | +| `"cell_name"` (default) | the cellpy **label** BatBase stores for the test, falling back to the test name or the cell name | one cell, named the same in both places | +| `"tag"` | a **cellpy tag** name (add `project=` if the same tag exists in several projects) | all tests of a campaign | +| `"external_id"` | the journal row **id** | you copied the id from the BatBase page | +| `"test_name"` | BatBase's own test name (`20250925_siba001_01_hcicc_lif`) | scripted lookups | + +```python +c.fetch_meta("batbase") # key = c.cell_name +c.fetch_meta("batbase", "siba_01", kind="cell_name") # a different label +c.fetch_meta("batbase", "SAL_010", kind="tag", project="3") +c.fetch_meta("batbase", "42", kind="external_id") +``` + +The call returns the matching records (a tuple). Empty means *nothing found, +nothing changed*: + +```python +records = c.fetch_meta("batbase", "typo_here") +if not records: + print("not in BatBase — set the mass by hand") +``` + +A tag usually matches **several** tests. `fetch_meta` applies the *first* +record and logs a warning; when you want to pick yourself, look before you +apply: + +```python +records = c.fetch_meta("batbase", "SAL_010", kind="tag", apply=False) +for r in records: + print(r.external_id, r.test.get("cell_name"), r.cell.get("mass")) +``` + +## 4. Recompute, then check what changed + +Fetching writes the metadata but does **not** rebuild the summary. Do that +explicitly: + +```python +c.refresh_after() # cheap: only the mass/area/nom_cap-dependent columns +# or c.make_summary() for a full rebuild +``` + +To see exactly which fields BatBase supplied: + +```python +link = c.external_links["batbase"] +link.fields # ('active_electrode_area', 'cell_name', 'cell_type', 'mass', ...) +link.source_uri # 'https://…/api/test-cellpy-journal/42/' +link.fetched_at # when +``` + +The link is saved inside the cellpy file, so a colleague opening +`my_cell.cellpy` later can see where the mass came from. + +## What BatBase fills in + +| BatBase | cellpy (`meta_common`) | Unit after fetch | +| --- | --- | --- | +| electrode mass (`mass`, `total_mass`) | `mass`, `tot_mass` | mg | +| electrode `area`, `loading` | `active_electrode_area`, `active_electrode_loading` | cm², mg/cm² | +| nominal capacity + unit | `nom_cap`, `nom_cap_specifics` | mAh/g (gravimetric), mAh/cm² (areal) or mAh (absolute) | +| cell configuration `hc` / `fc` / `3e` / `sym` | `cell_type` | `half_cell` / `full_cell` / … | +| test mode normal / inverted | `cycle_mode` | `cathode` or `full_cell` / `anode` | +| test label | `cell_name` | — | +| comments, schedule file | `comment`, `schedule_file_name` | — | + +The units follow cellpy's conventions described in +[Units, mass, area and C-rates](units.md); you do not convert anything +yourself. If BatBase has no electrode record for the cell, those fields are +simply not touched. + +## When it does not work + +**`fetch_meta` returns `()` and logs "had nothing for …"** — the key did not +match. Check the label in BatBase, or try `kind="external_id"` with the row id +from the web page. Also make sure you are a *member* of the project in +BatBase: the API only shows rows from projects you belong to. + +**`MetadataSourceAuthError`** — credentials missing or rejected. This is never +swallowed, even without `strict=True`, because silently loading a cell +*without* its mass is worse than stopping. Run `cellpy connectors batbase +check`. + +**Warning "metadata source 'batbase' unavailable … continuing without it"** — +read the bit in parentheses. `UnknownMetadataSource` means the connector +package is not installed in *this* environment (step 1); a connection error +means network or server down. Either way the cell loaded without the database +layer. Re-run `fetch_meta` later, or pass `strict=True` if your script must +not continue without it (then these become exceptions). + +**The summary still shows the old mass** — you forgot step 4 +(`c.refresh_after()`). + +## Batch workflows + +The batch utility resolves metadata through the same layers, so a journal +built from the Excel sheet ([Set up the cellpy database](batch_database.md)) +and a record fetched from BatBase end up in the same place. Pulling a whole +batch from BatBase in one call (`batch.from_source(...)`) and letting BatBase +tell cellpy *where the raw files are* are planned for cellpy 2.3 +([#1107](https://github.com/jepegit/cellpy/issues/1107)). + +## For developers + +The plugin contract (`MetadataSource`, `MetaQuery`, `MetaRecord`, the +`cellpy.metadata_sources` entry-point group) is documented in the +[API reference](../api/readers.md#external-metadata-sources) and in +`cellpy.readers.metadata_sources.testing.check_metadata_source`, a conformance +check you can run against your own source. diff --git a/docs/how_do_i.md b/docs/how_do_i.md index 64e60d86..0ea49a75 100644 --- a/docs/how_do_i.md +++ b/docs/how_do_i.md @@ -164,6 +164,15 @@ c = cellpy.get("my_cell.res", nominal_capacity="3579 mAh/g") → all four: [Units, mass, area and C-rates](guides/units.md) +**…get mass, area and nominal capacity from our lab database instead of typing them?** + +```python +c.fetch_meta("batbase") # needs the cellpy-connectors package + credentials +c.refresh_after() +``` + +→ [Pull cell metadata from a lab database](guides/metadata_sources.md) + --- ## Find the numbers diff --git a/docs/llms.txt b/docs/llms.txt index 05496a92..188ca10b 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -20,6 +20,7 @@ - [How do I…?](https://cellpy.readthedocs.io/en/latest/how_do_i/) - [Compute ICA / DVA](https://cellpy.readthedocs.io/en/latest/guides/ica/) +- [Pull cell metadata from a lab database](https://cellpy.readthedocs.io/en/latest/guides/metadata_sources/) - [Batch processing](https://cellpy.readthedocs.io/en/latest/examples/batch_utility/cellpy_batch_processing/) ## Reference diff --git a/zensical.toml b/zensical.toml index 2b49ca3a..8765fb28 100644 --- a/zensical.toml +++ b/zensical.toml @@ -66,6 +66,7 @@ nav = [ { "Get your data out" = "guides/exporting.md" }, { "Set up the cellpy database" = "guides/batch_database.md" }, { "Work with remote files" = "guides/remote_paths.md" }, + { "Pull cell metadata from a lab database" = "guides/metadata_sources.md" }, { "Write an instrument loader plugin" = "guides/writing_a_loader_plugin.md" }, { "Write a CLI plugin" = "guides/writing_a_cli_plugin.md" }, ] },