[dhairyajangir]

Closes #1808.

extend() can write rows while leaving add_fields() with an older read-side metadata snapshot. A subsequent schema change can then rebuild the footer without those rows. This change invalidates the cached footer and page-list envelopes after a successful extend() sync, so add_fields() reloads the current snapshot when needed.

The regressions cover explicit add_fields() and extend(..., accept_new_fields=True), as well as cache invalidation and a rejected extend. The equal-count stale-snapshot test deliberately constructs that state.

Validation:

• The earlier-base revision passed 49 focused schema-evolution, update and flushing tests, plus Ruff, Black and git diff --check.
• On Linux with Python 3.14.10, ROOT and XRootD, the complete tests collection passed: 2,722 passed, 1 skipped (python -m pytest -vv -n auto tests). The skipped test requires a case-insensitive filesystem. This isolated fork run tested PR commit 111c5962a0fa398f3b6efe3b0428e2e25e73a401 integrated with main 8503a78187b11564abbf517628be5558fba274af (tree ac6c92f8e4738ed6aa5fc5d52166c94ae80a6145).
• The same integrated tree passed all 14 applicable pre-commit hooks; one no-files hook was skipped. The hooks job passed even though its separate CPU setup job failed before tests.

The upstream workflows on the new PR head currently await maintainer authorization. These isolated Linux runs are separate from the upstream PR checks and do not cover GPU, WebAssembly, Windows, macOS or other Python versions. They did not measure coverage. Recovery from an I/O failure before the final sync was not verified.

I used Codex to help investigate, implement and validate the change. I personally reviewed and tested this revision.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.04%. Comparing base (39ff721) to head (0fc2738).

Additional details and impacted files

Files with missing lines	Coverage Δ
src/uproot/writing/writable.py	87.65% <100.00%> (ø)

[ariostas]

🤖 AI text below 🤖

Thanks for tracking this down! The fix works, but it detects stale state indirectly, by comparing the cluster-group counts in add_fields(). The real cause is that NTuple.extend() in _cascadentuple.py appends a cluster group to the write-side footer without invalidating the read-side snapshot (_existing_footer / _existing_page_list_envelopes). Any change to the footer that doesn't change the number of groups would get past the count check.

Could we invalidate the snapshot where it actually goes stale instead? For example, at the end of NTuple.extend():

# the write-side footer and page lists changed; the read-side snapshot
# used by add_fields() is now out of date
self.__dict__.pop("_existing_footer", None)
self.__dict__.pop("_existing_page_list_envelopes", None)

Then add_fields() can keep the original if not hasattr(self._cascading, "_existing_footer") check (and its comment about everything already being flushed to disk, which is why reloading is safe). Every other reader of those attributes is covered too, not just this one.

[dhairyajangir]

Thanks for the guidance, @ariostas. In 111c596, extend() now invalidates the cached footer and page-list envelopes after its final sync, and add_fields() uses the original lazy reload. I added tests for snapshot invalidation and a rejected extend; the equal-count stale snapshot is intentionally constructed.

The revised commit integrated with main 8503a781 passed the complete Linux CPU collection with Python 3.14, ROOT and XRootD: 2,722 passed, 1 case-sensitive-filesystem skip (run). All 14 applicable pre-commit hooks passed on the same integration tree (hooks job); its separate CPU setup job failed before tests. These fork runs are separate from upstream PR checks and do not cover GPU or other platforms.

The upstream workflows on the new head are awaiting maintainer authorization; could you enable them when convenient?

[ariostas]

Thank you, @dhairyajangir! I pushed a minor commit to make the test a little bit more robust, but now this is good to go.