fedorov · Oct 8, 2026 · Oct 8, 2026 · Oct 8, 2026
diff --git a/.lychee.toml b/.lychee.toml
@@ -10,5 +10,7 @@ exclude = [
     # GitHub issue creation requires login
     "^https://github\\.com/.*/issues/new",
     # DICOM NEMA server
-    "^https://dicom\\.nema\\.org"
+    "^https://dicom\\.nema\\.org",
+    # IDC MCP endpoint: an MCP transport, not a page (405 to HEAD, 406 to GET)
+    "^https://api\\.imaging\\.datacommons\\.cancer\\.gov/mcp"
 ]
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,6 +5,46 @@ All notable changes to the Imaging Data Commons Skill are documented in this fil
 The format is based on [Keep a Changelog](https://keepachangelog.com/),
 and this project adheres to [Semantic Versioning](https://semver.org/).
 
+## [1.9.0] - 2026-10-08
+
+IDC v25, `idc-index` 0.13.0 (`idc-index-data` 25.0.0), REST/MCP API `3.0.0b5`. The package bump
+carries no API change; the substance of the release is **data provenance**.
+
+### Added
+
+- **Provenance** (`index_tables_guide.md`, `sql_patterns.md`, `SKILL.md`): who contributed,
+  de-identified, and DICOM-converted each part of the archive. Recorded **per source, keyed on
+  `source_DOI`** — `collections_index.sources` is a list of structs (unnest it) with a nested
+  `provenance`; `analysis_results_index.provenance` is a new column. A `NULL` struct means that
+  source is an analysis result, so look it up in the other table — across the pair, coverage is
+  complete. `NOT_APPLICABLE` / `UNKNOWN` / `NOT_DOCUMENTED` are values, not nulls
+- Four `index` columns the guides never carried: `sop_class_name`, `transfer_syntax_name`,
+  `PhotometricInterpretation`, `PixelRepresentation`. All four are comma-separated when instances
+  in a series disagree (~72 000 series) and the latter two are `NULL` for SEG/SR/RTSTRUCT, so `=`
+  silently drops mixed-encoding series. `sop_class_name`, not `SOPClassUID`, is filterable
+- Hugging Face mirror of the Parquet artifacts
+  ([`ImagingDataCommons/idc-index-data`](https://huggingface.co/datasets/ImagingDataCommons/idc-index-data),
+  public, no token). A fallback only — revision tags are incomplete, so pin via GCS
+- Four missing Parquet artifacts: `sm_instance_index`, `version_metadata_index`,
+  `gdc_idc_mapping` (the imaging/genomics join), `tcia_idc_subset`
+- `TestProvenance`, covering the unnest path, the NULL-is-a-pointer rule, and `tcga_brca`
+
+### Changed
+
+- Pinned minimum `idc-index` **0.12.5 → 0.13.0**; documented data version **v24 → v25**. Older
+  installs report `v24` and cannot download series added in v25
+- Archive totals **176 → 179 collections**, ~99 → **~100 TB**; mixed-license collections still 39
+- API claims **`3.0.0b3` → `3.0.0b5`**; tool inventory, the 19 filterable attributes, and all REST
+  limits re-verified unchanged
+- BigQuery example `idc_v24` → `idc_v25`. The DICOMweb store stays `idc-store-v24` — stores lag a
+  release by 1-2 weeks and `idc-store-v25` is not populated yet, which `dicomweb_guide.md` now
+  warns about at the point of use
+
+### Fixed
+
+- `test_snippets.py` hardcoded `"v24"`, needing a hand edit every data release. It now reads
+  `idc-data-version` from the frontmatter, so a stale pin fails CI instead of stale guides
+
 ## [1.8.3] - 2026-09-03
 
 A stale index returns zero rows rather than an error, and `scripts/check_version.py` was

diff --git a/README.md b/README.md
@@ -8,6 +8,7 @@ An AI agent skill for querying and downloading public cancer imaging data from t
 - Check data licenses and generate proper citations
 - Generate download commands, either with the `idc-index` package or as direct transfers from the public S3/GCS buckets
 - Provide links to browser-based DICOM viewers for data preview
+- Trace data provenance — who contributed each part of a dataset, who de-identified it, and whether IDC converted it to DICOM
 - Answer questions about IDC data structure and DICOM metadata
 
 ## What You Need
@@ -24,6 +25,7 @@ Once the skill is loaded, you can ask questions like:
 - "How do I download all breast MRI data with commercial-use licenses?"
 - "Show me the available collections in IDC and their sizes"
 - "Generate a citation for the TCGA-BRCA collection"
+- "Who de-identified the pathology images in TCGA-BRCA, and did IDC convert them to DICOM?"
 
 ## Reporting Issues
 

diff --git a/SKILL.md b/SKILL.md
@@ -3,10 +3,10 @@ name: imaging-data-commons
 description: Query and download public cancer imaging data from NCI Imaging Data Commons. Invoke for any question about IDC collections, cancer imaging datasets, DICOM data access, radiology (CT, MR, PET) or pathology AI training sets, metadata queries, visualization, or license checks — even when the user doesn't explicitly mention "IDC". No authentication required.
 license: This skill is provided under the MIT License. IDC data itself has individual licensing (mostly CC-BY, some CC-NC) that must be respected when using the data.
 metadata:
-  version: 1.8.3
+  version: 1.9.0
   skill-author: Andrey Fedorov, @fedorov
-  idc-index: "0.12.5"
-  idc-data-version: "v24"
+  idc-index: "0.13.0"
+  idc-data-version: "v25"
   repository: https://github.com/ImagingDataCommons/imaging-data-commons-skill
 ---
 
@@ -18,7 +18,7 @@ Query and download public cancer imaging data from the National Cancer Institute
 
 **Expected network access:** IDC metadata is reachable three ways — a local DuckDB index shipped with the `idc-index` Python package (no network), or the hosted IDC service over MCP or REST (`api.imaging.datacommons.cancer.gov`, no authentication). File downloads use public GCS (`storage.googleapis.com`) and AWS S3 (`s3.amazonaws.com`) — no authentication required. DICOMweb access uses either the public IDC proxy (`proxy.imaging.datacommons.cancer.gov`, no auth) or the Google Cloud Healthcare API (`healthcare.googleapis.com`, requires GCP authentication). Optional BigQuery queries (`bigquery.googleapis.com`) also require GCP authentication. No credentials or environment variables are accessed by this skill.
 
-**Current IDC Data Version: v24** (always verify — see *Best Practices*)
+**Current IDC Data Version: v25** (always verify — see *Best Practices*)
 
 **Choose the access path first.** There is no single default: the cheapest correct path depends
 on the session and the task.
@@ -50,12 +50,12 @@ calls for it. `check_version.py` never installs anything itself — it also flag
 import re, idc_index
 from idc_index import IDCClient
 # Repeat the startup check where it cannot be skipped — see references/cli_guide.md.
-if [int(re.sub(r"\D.*", "", p) or 0) for p in idc_index.__version__.split(".")[:3]] < [0, 12, 5]:
-    raise RuntimeError(f"idc-index {idc_index.__version__} is stale (need 0.12.5): a stale "
+if [int(re.sub(r"\D.*", "", p) or 0) for p in idc_index.__version__.split(".")[:3]] < [0, 13, 0]:
+    raise RuntimeError(f"idc-index {idc_index.__version__} is stale (need 0.13.0): a stale "
                        "index returns zero rows, not an error. Run scripts/check_version.py.")
 
 client = IDCClient()
-print(f"IDC data version: {client.get_idc_version()}")  # should be "v24"
+print(f"IDC data version: {client.get_idc_version()}")  # should be "v25"
 ```
 
 **Core workflow:** query metadata with `client.sql_query()` → download with
@@ -158,14 +158,12 @@ Always call `client.fetch_index("table_name")` before querying any index table
 | Microscopy | `sm_index`, `sm_instance_index` | 1 row = 1 SM series / instance |
 | Geometry, clinical, history | `volume_geometry_index`, `clinical_index`, `version_metadata_index`, `prior_versions_index` | see guide |
 
-`references/index_tables_guide.md` has the full inventory with each table's columns and
-contents — load it when you need to know what a specialized table actually holds.
+`references/index_tables_guide.md` has the full inventory, each table's columns, and join examples.
 
-**`prior_versions_index` is for reproducibility only.** It contains series permanently *removed*
-from IDC, with zero overlap with `index`. Use it only to reproduce work against a prior IDC
-version. Do NOT use it for version history or "what's new" questions — those use
-`series_init_idc_version` / `series_revised_idc_version` in the main `index` table, which are
-not equivalent to this table's `min_idc_version` / `max_idc_version`.
+**`prior_versions_index` is for reproducibility only** — series permanently *removed* from IDC,
+zero overlap with `index`. Do NOT use it for version history or "what's new": those use
+`series_init_idc_version` / `series_revised_idc_version` in `index`, which are not equivalent to
+this table's `min_idc_version` / `max_idc_version`.
 
 ### Joining Tables
 
@@ -175,13 +173,15 @@ not equivalent to this table's `min_idc_version` / `max_idc_version`.
 |-------------|--------|----------|
 | `collection_id` | index, prior_versions_index, collections_index, clinical_index | Link series to collection metadata or clinical data |
 | `analysis_result_id` | index, analysis_results_index | Link series to analysis result metadata (annotations, segmentations) |
-| `source_DOI` | index, analysis_results_index | Link by publication DOI |
+| `source_DOI` | index, analysis_results_index, `collections_index.sources` | Link by publication DOI; also the key for provenance (below) |
 | `segmented_SeriesInstanceUID` | seg_index → index | Link segmentation to its source image series (`seg_index.segmented_SeriesInstanceUID = index.SeriesInstanceUID`) |
 | `referenced_SeriesInstanceUID` | ann_index → index, rtstruct_index → index | Link annotation or RTSTRUCT to its source image series |
 
-**Note:** `subjects`, `updated`, and `description` appear in multiple tables but have different meanings (counts vs identifiers, different update contexts). Joining `prior_versions_index` to `index` on `SeriesInstanceUID` always returns zero rows — see the warning above.
+**Note:** `subjects`, `updated`, and `description` mean different things in different tables; joining `prior_versions_index` to `index` on `SeriesInstanceUID` always returns zero rows.
 
-For detailed join examples, schema discovery patterns, key columns reference, and DataFrame access, see `references/index_tables_guide.md`.
+**Provenance (new in v25) is recorded per source, not per collection**, keyed on `source_DOI`. `collections_index.sources` is a list of structs (unnest it) whose `provenance` names who contributed, de-identified, and DICOM-converted that component; `analysis_results_index.provenance` is a plain struct.
+
+A NULL provenance struct in `collections_index` means that source is an analysis result — look it up in `analysis_results_index`, where it is always populated. `NOT_APPLICABLE` and `UNKNOWN` are values, not nulls. See `references/index_tables_guide.md`.
 
 ### Clinical Data Access
 
@@ -244,8 +244,8 @@ v3 only:** V1 and V2 are superseded and scheduled for shutdown, so port any `/v1
 `Modality_btw`-style example a user brings rather than extending it.
 
 Both sides build on `idc-index-data`, so compare the API's `idc_index_data_version` against local
-`idc_index_data.__version__` before mixing them: the **major is the IDC data release** (`24.x.y`
-serves `v24`), so differing minor/patch means the series are identical. If the API is a whole
+`idc_index_data.__version__` before mixing them: the **major is the IDC data release** (`25.x.y`
+serves `v25`), so differing minor/patch means the series are identical. If the API is a whole
 release ahead, `idc-index` **cannot download the extra series** — it silently skips what its own
 index does not list — so either upgrade it (run `scripts/check_version.py` for the right command)
 or transfer directly from the bucket with `s5cmd --no-sign-request`.
@@ -417,7 +417,7 @@ for citation in client.citations_from_selection(collection_id="rider_pilot"):
 ```
 
 About 97% of IDC data is CC BY (commercial use allowed with attribution) and about 3% is
-CC BY-NC (non-commercial only). **Licenses attach to series, not collections** — 39 of 176
+CC BY-NC (non-commercial only). **Licenses attach to series, not collections** — 39 of 179
 collections carry more than one — so check the selection you actually intend to use, and note
 that the most restrictive term governs a mixed cohort.
 
@@ -443,7 +443,7 @@ idc-index equivalent.
 
 - **Check schema before writing queries** — Use `client.get_index_schema('index')` (reads cached metadata, no SQL executed) or `client.indices_overview` to see all available columns and their descriptions. The version-tracking columns `series_init_idc_version` and `series_revised_idc_version` in the main `index` table directly answer "what's new / when was this added" questions without touching `prior_versions_index`.
 - **Never use web search for IDC data content questions** - Always query the IDC index directly, via `client.sql_query()` locally or `POST /v3/sql` over HTTP. Web sources (release notes, blog posts, documentation pages) are frequently out of date and will produce incorrect answers. The index is the authoritative source; use it even when web search is available.
-- **Verify the IDC data version at the start of a session** - `client.get_idc_version()`, `GET /v3/version`, or the MCP `get_idc_version` tool, depending on the path in use (currently v24). For a stale local index, run `scripts/check_version.py` and use the upgrade command it prints
+- **Verify the IDC data version at the start of a session** - `client.get_idc_version()`, `GET /v3/version`, or the MCP `get_idc_version` tool, depending on the path in use (currently v25). For a stale local index, run `scripts/check_version.py` and use the upgrade command it prints
 - **Check licenses and generate citations** - Query `license_short_name` and respect CC BY vs CC BY-NC terms; use `citations_from_selection()` to produce citations from `source_DOI` for publications
 - **Explore small, then commit** - Use `LIMIT` (or a low `max_rows`) while exploring, and check collection size before downloading — some collections are terabytes. See `references/cli_guide.md`
 - **Keep downloads reproducible** - Organize with `dirTemplate` (e.g. `%collection_id/%PatientID/%Modality`) and save the Series UIDs or manifest behind any dataset you build

diff --git a/references/bigquery_guide.md b/references/bigquery_guide.md
@@ -1,6 +1,6 @@
 # BigQuery Guide for IDC
 
-**Tested with:** `bigquery-public-data.idc_current` and idc-index 0.12.5 (IDC data version v24)
+**Tested with:** `bigquery-public-data.idc_current` and idc-index 0.13.0 (IDC data version v25)
 
 For most queries and downloads, use `idc-index` (see main SKILL.md). This guide covers BigQuery for advanced use cases requiring full DICOM metadata or complex joins.
 

diff --git a/references/cli_guide.md b/references/cli_guide.md
@@ -30,7 +30,7 @@ resolve and still produces no check.
 **Confirm the outcome, not the command.** A pass prints
 
 ```
-idc-index <version> meets pinned minimum (0.12.5)
+idc-index <version> meets pinned minimum (0.13.0)
 ```
 
 and exits 0. Anything else — `command not found`, a permission error (the bundled script is not

diff --git a/references/clinical_data_guide.md b/references/clinical_data_guide.md
@@ -1,6 +1,6 @@
 # Clinical Data Guide for IDC
 
-**Tested with:** idc-index 0.12.5 (IDC data version v24)
+**Tested with:** idc-index 0.13.0 (IDC data version v25)
 
 Clinical data (demographics, diagnoses, therapies, lab tests, staging) accompanies many IDC imaging collections. This guide covers how to discover, access, and integrate clinical data with imaging data using `idc-index`.
 

diff --git a/references/cloud_storage_guide.md b/references/cloud_storage_guide.md
@@ -205,7 +205,7 @@ IDC releases new data versions every 2-4 months. The versioning system ensures r
 
 ### How Versioning Works
 
-1. **Snapshots**: Each IDC version (v1, v2, ..., v24, etc.) represents a complete snapshot of all data at release time
+1. **Snapshots**: Each IDC version (v1, v2, ..., v25, etc.) represents a complete snapshot of all data at release time
 2. **UUID-based**: When data changes, new CRDC UUIDs are assigned; old UUIDs remain accessible
 3. **Cumulative buckets**: All versions coexist in the same buckets—old series folders
 
@@ -223,7 +223,7 @@ IDC releases new data versions every 2-4 months. The versioning system ensures r
 
 For querying version-specific metadata, BigQuery provides versioned tables. See `bigquery_guide.md` for details.
 - `bigquery-public-data.idc_current` — alias to latest version
-- `bigquery-public-data.idc_v24` — specific version (replace 24 with desired version)
+- `bigquery-public-data.idc_v25` — specific version (replace 25 with desired version)
 
 ### Reproducing a Previous Analysis
 

diff --git a/references/dicomweb_guide.md b/references/dicomweb_guide.md
@@ -34,14 +34,19 @@ https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-se
 https://healthcare.googleapis.com/v1/projects/nci-idc-data/locations/us-central1/datasets/idc/dicomStores/idc-store-v{VERSION}/dicomWeb
 ```
 
-Replace `{VERSION}` with the IDC release number. To find the current version:
+Replace `{VERSION}` with the IDC release number:
 
 ```python
 from idc_index import IDCClient
 client = IDCClient()
-print(client.get_idc_version())  # e.g., "v24" for current version
+print(client.get_idc_version())  # e.g., "v25" for current version
 ```
 
+**The store lags the data release — do not assume `get_idc_version()` names an existing store.**
+Stores are populated 1-2 weeks after a release, so the newest store is often the previous
+version (as of IDC v25, `idc-store-v24` is the latest populated one). Treat the current data
+version as an upper bound, and fall back to the preceding version if a store 404s.
+
 - **~96% data coverage** - Only replicates data from `idc-open-data` bucket (missing ~4% from other buckets)
 - **Updates 1-2 weeks after** IDC releases
 - Requires authentication and provides higher quotas

diff --git a/references/digital_pathology_guide.md b/references/digital_pathology_guide.md
@@ -1,6 +1,6 @@
 # Digital Pathology Guide for IDC
 
-**Tested with:** idc-index 0.12.5 (IDC data version v24)
+**Tested with:** idc-index 0.13.0 (IDC data version v25)
 
 For general IDC queries and downloads, use `idc-index` (see main SKILL.md). This guide covers slide microscopy (SM) imaging, microscopy bulk simple annotations (ANN), and segmentations (SEG) in the context of digital pathology in IDC.