Skip to content

fix: correct errors found while testing writing - #1827

Merged
ariostas merged 4 commits into
scikit-hep:mainfrom
ariostas:fix/writing
Nov 12, 2026
Merged

ariostas merged 4 commits into
scikit-hep:mainfrom
ariostas:fix/writing

Conversation

@ariostas

@ariostas ariostas commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Part of the split of #1814 (the list is in #1834). These bugs were found while writing the coverage tests of #1814, and tests/test_1827_fix_writing.py holds their regression tests: every test in it fails on main (for parametrized tests, at least one of their cases).

  • Directories: without cycle numbers, a name stands for its highest cycle that passes the filters, in both ReadOnlyDirectory and WritableDirectory (keys, items, classnames), and WritableDirectory.values() lists each name once, as its highest cycle, like ReadOnlyDirectory.values(). The writable directory also stops listing a name twice and filters subdirectories on their full paths.
  • Keys and streamers
    • A key that moves can become a big key, so keys that move past 2 GB are written correctly.
    • Growing the list of streamers allocates the size it uses and sets the file length.
    • Streamers that aren't TStreamerInfo raise a TypeError.
    • FileSink.tell doesn't reopen a file that is open.
  • TTrees
    • A branch name that is defined twice raises a ValueError. Before, the second one was silently left out, and when it was a field of a record whose flattened name collided with another branch ({"r_x": ..., "r": {"x": ..., "y": ...}}), the TTree could never be extended. A scalar integer branch can still name the counter of a jagged branch before it.
    • Branch types can be any Mapping, and type strings that make NumPy raise a SyntaxError are understood.
    • Categorical strings (an Awkward IndexedArray) can be written.
    • Jagged arrays that can't be written raise a TypeError that names the branch.
    • extend and add_branches refuse trees with branches they can't interpret, instead of leaving those branches behind. These include custom interpretations without a dtype, so Interpretation's base methods now raise NotImplementedError naming the subclass, instead of a bare AssertionError.
  • Histograms and other objects
    • to_TArray picks the class from the dtype's kind and size, so long/long long/intc arrays work on every platform. On Windows with NumPy 1, int32 data was rejected.
    • Histogram-like objects whose values() doesn't take flow can be written: their fallback still called values(flow=False).
    • Tuples of arrays with 0 or more than 3 dimensions raise a TypeError.
    • The as_TGraph error about maxY names maxY.
  • Copying: copy_from doesn't try to make a directory named "" for the root of the copy.

@github-actions github-actions Bot added the type/fix PR title type: fix (set automatically) label Nov 6, 2026
@ariostas
ariostas marked this pull request as ready for review November 6, 2026 14:47
@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.07692% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.76%. Comparing base (b4acc0f) to head (551ded0).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/uproot/writing/_cascadetree.py 89.28% 4 Missing and 2 partials ⚠️
src/uproot/writing/identify.py 50.00% 1 Missing and 1 partial ⚠️
src/uproot/writing/_cascade.py 94.73% 0 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (93.07%) is below the target coverage (98.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
Files with missing lines Coverage Δ
src/uproot/interpretation/__init__.py 96.96% <100.00%> (+18.18%) ⬆️
src/uproot/reading.py 83.18% <100.00%> (+0.12%) ⬆️
src/uproot/sink/file.py 91.66% <100.00%> (+0.82%) ⬆️
src/uproot/writing/interpret.py 72.81% <ø> (+19.41%) ⬆️
src/uproot/writing/writable.py 89.46% <100.00%> (+1.80%) ⬆️
src/uproot/writing/_cascade.py 91.02% <94.73%> (+2.62%) ⬆️
src/uproot/writing/identify.py 82.75% <50.00%> (+8.89%) ⬆️
src/uproot/writing/_cascadetree.py 90.22% <89.28%> (+2.74%) ⬆️

... and 1 file with indirect coverage changes

These were found by the tests of scikit-hep#1814, which are the regression tests here.
The fixes are to keys that move or grow past 2 GB, streamers added after
others, TTree type specifications and categorical strings, TArray classes
from dtypes, histogram conversion, directory listings of names without
cycles, copying directories, and extending TTrees with branches that Uproot
can't interpret, which includes custom interpretations without a dtype: the
methods of the Interpretation base class raise NotImplementedError, naming
the subclass, instead of a bare AssertionError.

Assisted-by: Claude Code:claude-opus-5-5
update_streamers rejected an object that isn't a streamer in the middle of
its loop, after the streamers before it had been recorded but before space
was made for them. A retry then skipped those streamers as already known,
and the file's list of streamers couldn't be read back.

Assisted-by: Claude Code:claude-opus-5-5
WritableDirectory.values() returned every cycle of each name, while
keys(), items() and classnames() without cycles, and the values() of a
ReadOnlyDirectory, return each name once, as its highest cycle.

Assisted-by: Claude Code:claude-opus-5-5
mktree (and the cascade under it) silently left out a branch whose name was
already taken, including fields of records whose flattened names collided
with another branch: {"r_x": ..., "r": {"x": ..., "y": ...}} made a
TTree with only r_x and r_y, which could never be extended. Now a name
defined twice raises a ValueError. A scalar integer branch may still name
the counter of a jagged branch defined before it.

Assisted-by: Claude Code:claude-opus-5-5
ariostas added a commit to ariostas/uproot5 that referenced this pull request Nov 6, 2026
…files

Four tests skipped without ROOT before checking anything with uproot; now
only their ROOT checks need it. The test of updating a file object closes
both files. The test that the first of two branches with the same name was
kept leaves for scikit-hep#1827, which rejects such names instead.

Assisted-by: Claude Code:claude-opus-5-5

@ianna ianna left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ariostas -- Thanks! Looks good. The coverage is low but the subsequent PRs should fix it I think. Let's merge the tests first and update this one to check it. Thanks!

ariostas added a commit to ariostas/uproot5 that referenced this pull request Nov 12, 2026
…files

Four tests skipped without ROOT before checking anything with uproot; now
only their ROOT checks need it. The test of updating a file object closes
both files. The test that the first of two branches with the same name was
kept leaves for scikit-hep#1827, which rejects such names instead.

Assisted-by: Claude Code:claude-opus-5-5
@ariostas
ariostas merged commit 30b7f1f into scikit-hep:main Nov 12, 2026
22 of 23 checks passed
ariostas added a commit to ariostas/uproot5 that referenced this pull request Nov 12, 2026
…files

Four tests skipped without ROOT before checking anything with uproot; now
only their ROOT checks need it. The test of updating a file object closes
both files. The test that the first of two branches with the same name was
kept leaves for scikit-hep#1827, which rejects such names instead.

Assisted-by: Claude Code:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/fix PR title type: fix (set automatically)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants