Repository navigation
Conversation
Add tests for every reachable line and branch on the CPU job, fix the bugs they uncovered, consolidate weak and duplicate tests, mark code that can't be reached with a pragma and its reason, and set the Codecov project and patch targets to 100%. Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
Compare paths in the forward-slash form uproot and fsspec return, build expected reprs with repr() so backslashes are escaped, glob every item of the repeated-match pattern so all matches share separators, and give the RNTuple repr test an explicit int64 dtype. Assisted-by: Claude Code:claude-opus-5-5
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
|
…first cluster gpu_deserialize_decompressed_content indexed the per-cluster buffers, which start at start_cluster_idx, by the absolute cluster index, so an entry range starting after the first cluster read the wrong cluster or raised IndexError. Index them by position instead, and drop the negative-cluster guard and the dtype view step, which this path never needs: the cluster range never starts below zero here, and assigning into the result already converts a narrower representation's values. Cover the rest of the GPU code: optional fields (_cupy_insert), uncompressed pages, and the CuFileSource counters. Assisted-by: Claude Code:claude-opus-5-5
A column stored uncompressed has compression settings 0, which isn't in compression_settings_dict, so the GPU interpreter raised KeyError: 0 for any uncompressed RNTuple. Assisted-by: Claude Code:claude-opus-5-5
Keep main's lookup of GPU cluster buffers by cluster index (scikit-hep#1793) and loop over the non-negative part of the cluster range; drop its empty ``starts`` branch, which can't run because the loop only sees columns of clusters that were read, and cover the negative-start paths with direct GPU calls. Drop the later-cluster cases added to test_1786, which test_1792 now covers. Also compile leaf-title range specs with the filename "<dynamic>", so coverage doesn't warn that it can't parse a source file named after the title. Assisted-by: Claude Code:claude-opus-5-5
Take main's _normalize_jagged_array (scikit-hep#1771), with a branch_name argument so it keeps raising TypeError for arrays extend() can't write instead of the AssertionError main had there, and keep the dead _cascading guard deleted. Update the tests that scikit-hep#1771 made outdated: streamers_named only returns real TStreamerInfos, and a counter is found through its leaf even when the leaf isn't named after its branch. Assisted-by: Claude Code:claude-opus-5-5
ROOT can write a plain TBranch whose counter is a TLeafElement; reading such a tree for update finds no counter for it and records the "counter in another branch type" blocker. Mark the leafless-TBranch arm, which a TBranch without leaves never reaches since it has no interpretation. Assisted-by: Claude Code:claude-opus-5-5
ResourceFuture._run sets the chunk's data before it sends the notification, so read() now waits for each notification with a bounded get() instead of checking the queue size right away. The batching test records each vector_read and checks the batches are [2, 2, 1], and the histogram round trip skips only its ROOT checks when ROOT is missing. Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
Fixes: - A key past the 2 GB boundary kept the copy in its directory's list of keys at its own size (the new Key.location setter made an RNTuple's small anchor key big there, so readers missed its data by 8 bytes). - Histograms without fSumw2 have variances |values|, as ROOT's errors are sqrt(|content|), also for negative bins. - ObjectArray takes negative indexes. - An AsArray with a header ends where its byte count says in AwkwardForth too, not at the end of the entry, and one without a byte count raises a DeserializationError. - arrays([], how="zip") returns an empty record array, not None. - Writable directories list each name once without cycle numbers, as the highest cycle, like ReadOnlyDirectory. - HTTPSource and XRootDSource send no request for an empty list of ranges. - Clearer messages for a TTree with branches update can't write and for library="pd"'s how. Tests: values for the strided TDatime and multiindex-cut checks, a nested Forth AsArray, all_fields with only base classes, a narrower RNTuple representation on the GPU, test_0816 comparing each branch read alone, the ROOT checks of four writing tests after their uproot checks, a timeout on the readline test, and a sturdier conftest. Assisted-by: Claude Code:claude-opus-5-5
_cupy_insert always copies what's left of the array (an empty slice if nothing is), and _reduce_errors raises AssertionError for a which that errors() didn't validate, instead of marking the branch as never taken. Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
|
Thanks for the extensive coverage work and the regression tests accompanying the fixes. This is an AI-assisted review requested by Ianna. I found no actionable correctness issues in the production changes inspected. I reviewed the interpretation, HTTP, writing and RNTuple changes at 6f8d5d2, then inspected the subsequent source changes through e9d65ee, including container byte-count handling, object indexing, histogram variance handling and writable directory iteration. Validation: 575 targeted tests passed (4 optional tests skipped) on the earlier revision. On e9d65ee, the targeted container, interpretation, histogram, HTTP/XRootD and writing suites passed with 620 tests and 33 optional-dependency skips. I did not run GPU or ROOT-dependent tests locally. CI for the latest revision is still pending, so this review does not certify those checks. One nonblocking scope concern: this combines coverage enforcement with many behavior fixes across a large diff. Separating those fixes where practical would make review and regression attribution easier. The added coverage is valuable, but 100% line/branch coverage alone does not establish correctness. |
if/elif chains whose last case is always true now end in an else that raises AssertionError, which coverage excludes, instead of marking the last elif "no branch". Unreachable else blocks raise AssertionError instead of TypeError or NotImplementedError under "no cover", and the "G" case for TLeaf types, which _dtype_to_char never gives, is gone. Assisted-by: Claude Code:claude-opus-5-5
Bug fixes: - RNTuple: decode narrow column representations (zigzag, truncated or quantized floats) in their own dtype before widening to the result's dtype - GPU RNTuple: place each page of a column at its element offset and decide compression per page, which fixes multi-page bit columns; treat algorithm or level 0 as uncompressed, like Compression.from_code_pair - RNTuple writing: let the anchor key be big past the 2 GB boundary, which overflowed past 2**31 bytes - AsStrings.final_array: convert StringArray baskets when a branch mixes them with awkward baskets - AsArray.__eq__: don't compare a dtype with a Model class - update_streamers: reject a non-streamer before recording its name/version - RNTuple split columns with an unsupported bit width raise ValueError Tests: - give each XRootD test server its own admin path, so that pytest-xdist workers don't collide - add tests that fail without the TH2 variance fix, the TProfile variance squaring, the empty-ranges guard in iterate, and the fixes above - replace vacuous assertions, close opened files, fix a leaking monkeypatch cleanup, make hand-made bytes match ROOT's layout, check uproot results before the ROOT-only parts, and restore checks that were narrowed Pragmas: replace the remaining avoidable ones with tests, AssertionError, or by removing dead code. Assisted-by: Claude Code:claude-opus-5-5
to_TArray checked issubclass(dtype.type, numpy.int32) and so on, which rejected distinct C types of the same size, such as numpy.longlong, or the 32-bit long that is NumPy 1's default integer on Windows. Assisted-by: Claude Code:claude-opus-5-5
|
🤖 This comment was written by Claude Code. The PR sets the Codecov project and patch targets to 100% with a threshold of 0%, and adds 43 test files named All 42 changed files under
|
Without cycle numbers, ReadOnlyDirectory and WritableDirectory now both take each name as its highest cycle among the keys that pass the filters, and iterkeys, iteritems and iterclassnames share one generator, so that the items and classnames come from the same key and each name is listed once. ReadOnlyDirectory used the first cycle in the key list, and WritableDirectory yielded the highest cycle even if only a lower one passed filter_classname, and listed every cycle in iterclassnames. In subdirectories, the writable directory now applies filter_name to the full path only, as the read-only one does. An empty objectwise std::map now has the dtypes of the non-empty ones. Assisted-by: Claude Code:claude-opus-5-5
|
🤖 AI text below 🤖 Thanks for the review. Points 1–3 are fixed in 91e9911, and we're keeping the setup for point 4, for the reasons below.
|
With filterwarnings = error, pytest 9.2 stops at startup: anyio's pytest plugin imports the deprecated _pytest.python.CallSpec2, and pytest-asyncio warns that asyncio_default_fixture_loop_scope is unset. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
* test: tighten the existing tests and make their fixtures robust Changes to existing tests from #1814 that don't depend on its fixes: they check values instead of only running, restore checks that had been dropped, close the files they open, and run concurrently (each XRootD server has its own admin path and waits until it accepts connections, and reset_classes puts the original model classes back). Assisted-by: Claude Code:claude-opus-5-5 * test: keep the tests that the new coverage tests replace Six tests (of Future.result timeouts, ThreadPoolExecutor.shutdown, the order of LRUCache, and Library equality) were deleted here in favor of tests in #1829, #1830 and #1832, which are based on this PR; until those are merged nothing else would cover them. They are now deleted together with the tests that replace them. A comment no longer points to a file of #1828. Assisted-by: Claude Code:claude-opus-5-5 * test: iterate over XRootD in one step per file MET_pt is in one TBasket, so with steps of 15 entries the later steps of a file had nothing to request, and the XRootD source then sent an empty vector read, which failed now and then. #1825 stops sending it; until then, these tests iterate in one step per file. Assisted-by: Claude Code:claude-opus-5-5
These were found by the tests of scikit-hep#1814, which are the regression tests here. The fixes are to containers (std::map, C arrays, AsDynamic and AsArray equality and hashing), strided and object arrays, the Awkward and Pandas libraries, string baskets, typename parsing, Float16/Double32 titles, TLeafG, TRef, TBasket.array, and model pickling. The reset_classes fixture now puts the original model classes back after the test, so that tests running after it in the same process (with pytest-xdist) compare against the classes they looked up. Assisted-by: Claude Code:claude-opus-5-5
…1826) * fix: give unweighted histogram bins of negative content a variance Like ROOT's TH1::GetBinError, the variance of a bin without fSumw2 is the absolute value of its content, rather than 0 when the content is negative. TProfile gets _values_variances, which the histogram conversions use. Found by the tests of #1814. Assisted-by: Claude Code:claude-opus-5-5 * fix: take the absolute value of integer bin contents as floats numpy.abs keeps the dtype, and the absolute value of the lowest integer of a dtype doesn't fit in it, so a TH1C bin of -128 had a variance of -128 (and TH1S, TH1I, TH2 and TH3 likewise). Assisted-by: Claude Code:claude-opus-5-5
These were found by the tests of scikit-hep#1814, which are the regression tests here. The fixes are to columns of narrower representations (zigzag, delta and truncated or quantized floats), GPU reads of uncompressed and multi-page columns, split columns of unsupported widths, unknown field roles, field paths, linked attribute sets, anchors past 2 GB, and writing empty and masked arrays. Assisted-by: Claude Code:claude-opus-5-5
…1828) * fix: correct errors found while testing RNTuple reading and writing These were found by the tests of #1814, which are the regression tests here. The fixes are to columns of narrower representations (zigzag, delta and truncated or quantized floats), GPU reads of uncompressed and multi-page columns, split columns of unsupported widths, unknown field roles, field paths, linked attribute sets, anchors past 2 GB, and writing empty and masked arrays. Assisted-by: Claude Code:claude-opus-5-5 * test: declare only the C++ that the RNTuple regression tests use The ROOT fixture also declared writers that only the tests of #1832 use, and if any of them didn't compile with some ROOT version, the tests here would fail too. The fixture also restores gErrorIgnoreLevel if a declaration fails. Assisted-by: Claude Code:claude-opus-5-5 * refactor: remove the unused page_is_compressed field of FieldPayload Each page now decides whether it is compressed, and uncompressed pages are recorded as None, so nothing reads the cluster-wide flag anymore. Assisted-by: Claude Code:claude-opus-5-5
* fix: correct errors found while testing the interpretations These were found by the tests of #1814, which are the regression tests here. The fixes are to containers (std::map, C arrays, AsDynamic and AsArray equality and hashing), strided and object arrays, the Awkward and Pandas libraries, string baskets, typename parsing, Float16/Double32 titles, TLeafG, TRef, TBasket.array, and model pickling. The reset_classes fixture now puts the original model classes back after the test, so that tests running after it in the same process (with pytest-xdist) compare against the classes they looked up. Assisted-by: Claude Code:claude-opus-5-5 * fix: read the range of version 3 TStreamerElements Version 3 of TStreamerElement writes fXmin, fXmax and fFactor after its type name (later versions take the range from the title instead), and reading one stopped before them, so a file with version 3 elements failed to deserialize. The format for them was defined but unused. Assisted-by: Claude Code:claude-opus-5-5 * fix: indent every child of an AwkwardForth node in its str Children after the first were indented two spaces less than the first. Assisted-by: Claude Code:claude-opus-5-5 * test: close the files of the strided-object tests with headers Assisted-by: Claude Code:claude-opus-5-5
These were found by the tests of #1814, which are the regression tests here. The fixes are to HTTP and XRootD sources (closing responses, empty requests, readline at the end of a stream), the deflate decompressor, streamer dependencies, TBranch steps inside already-read baskets, paths into RNTuples, files without streamers, cuts that aren't strings, and grouped TBranches in uproot.dask. Assisted-by: Claude Code:claude-opus-5-5
* test: cover the interpretations New tests from #1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5 * test: tighten an equality check and drop the test of Node.__str__ The check that a sequence isn't equal to a string passed for anything but True. The test of Node.__str__ expected its wrong indentation, which #1824 fixes; it is now there, with the right one. The test of Library equality in test_0016 is deleted here, as test_names_and_equality covers it. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
* test: cover the interpretations New tests from #1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5 * test: tighten an equality check and drop the test of Node.__str__ The check that a sequence isn't equal to a string passed for anything but True. The test of Node.__str__ expected its wrong indentation, which #1824 fixes; it is now there, with the right one. The test of Library equality in test_0016 is deleted here, as test_names_and_equality covers it. Assisted-by: Claude Code:claude-opus-5-5
* fix: correct errors found while testing writing These were found by the tests of #1814, which are the regression tests here. The fixes are to keys that move or grow past 2 GB, streamers added after others, TTree type specifications and categorical strings, TArray classes from dtypes, histogram conversion, directory listings of names without cycles, copying directories, and extending TTrees with branches that Uproot can't interpret, which includes custom interpretations without a dtype: the methods of the Interpretation base class raise NotImplementedError, naming the subclass, instead of a bare AssertionError. Assisted-by: Claude Code:claude-opus-5-5 * fix: check a whole batch of streamers before adding any of them update_streamers rejected an object that isn't a streamer in the middle of its loop, after the streamers before it had been recorded but before space was made for them. A retry then skipped those streamers as already known, and the file's list of streamers couldn't be read back. Assisted-by: Claude Code:claude-opus-5-5 * fix: list the values of a writable directory as its highest cycles WritableDirectory.values() returned every cycle of each name, while keys(), items() and classnames() without cycles, and the values() of a ReadOnlyDirectory, return each name once, as its highest cycle. Assisted-by: Claude Code:claude-opus-5-5 * fix: reject branch names that are defined more than once mktree (and the cascade under it) silently left out a branch whose name was already taken, including fields of records whose flattened names collided with another branch: {"r_x": ..., "r": {"x": ..., "y": ...}} made a TTree with only r_x and r_y, which could never be extended. Now a name defined twice raises a ValueError. A scalar integer branch may still name the counter of a jagged branch defined before it. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
New tests from scikit-hep#1814, apart from the regression tests of its fixes. Assisted-by: Claude Code:claude-opus-5-5
🤖 AI text below 🤖
This PR brings the test suite to 100% line and branch coverage and makes Codecov enforce it (
codecov.ymlproject and patch targets set to 100%, threshold 0%). Branch coverage was already enabled inpyproject.toml.Codecov merges the CPU coverage job with the CUDA 13 GPU job. Together they now cover all 22,667 lines and 3,810 branches;
mainwas at 77.51%. The README gets a coverage badge next to the docs badge.What changed
tests/test_1814_cov_*.pyfiles cover the remaining reachable lines and branches, grouped by area. Existing tests were strengthened where they had no assertions, compared values withoutassert, looped over empty collections, or duplicated others. Duplicates were merged, andtest_1146was folded into the new HTTP tests. Skips went from 17 to 1.tests-cuda/test_1814_cov_gpu_rntuple.pycovers optional fields, uncompressed pages, theCuFileSourcecounters, and direct reads from before the first cluster.# pragma: no cover (reason)or# pragma: no branch (reason).raise AssertionErroris excluded inpyproject.toml, and a note inAGENTS.mddescribes the convention.reset_classesinconftest.pyrestores the registered classes.gc.get_objectstest is monkeypatched.<dynamic>, so coverage.py no longer warns that it can't parse a source file named after each title.AsStridedObjects.final_array).basket_array._to_packedcorrupted masked data on write. Files already written this way stay wrong, which is worth a release note.KeyError: 0(gpu_read_cluster_pages).gpu_deserialize_decompressed_content).fSumw2were 0 for negative bins; they're now |content|, matching ROOT'sGetBinError.source.chunks([])failed with steps inside already-read baskets: HTTP raisedIndexErrorand XRootD had a race._ResponseBuffer.readlinecould hang.AsMapwith non-memberwise strings raised an error.uproot.daskon anAsGroupedbranch raisedAssertionError.interpretation/.update_streamerswrote the wrong file length.FileSink.tellwas wrong.uproot.updatecrashed on files with a branch it can't interpret._cascadewas handled wrongly.cycle=False, all with the highest cycle's object.copy_fromwith an empty directory name failed.to_writablewithNonefailed.reset_classesdroppedTClonesArray.Model_TLeafGwas not registered.TBasket.arrayfailed on jagged branches.TReffailed to read.DynamicModel.__setstate__,MetaData.__getattr__andstreamer_rules=Nonehad bugs.directory["ntuple:field"]failed.all_fieldsraisedIndexError.==was wrong.TypeError.use_threads=Falsecrashed.ObjectArray[-1]read the wrong bytes.AsArraywith a header inside another container read to the end of the entry.arrays([], how="zip")returnedNone.HTTPSource.chunks([])raisedIndexError, andXRootDSource.chunks([])sent an empty vector read.Behaviour changes to review
cutthat is not a string orNoneraisesTypeError.CustomInterpretationandAsBinarycompare by identity.attribute_set_name.Interpretationmethods raiseNotImplementedErrorinstead ofAssertionError.awkward_formraisesTypeError.TStreamerInfoorRawStreamerInfoinupdate_streamersraisesTypeError.uproot.updateskips branches it can't interpret instead of crashing.use_threads=Falsereport 0 workers.extendandadd_branchesraiseTypeErrorfor jagged arrays they can't write, instead ofAssertionError; so does writing an RNTuple field with anEmptyForm.fSumw2report |content| as the variance of negative bins, as ROOT does, instead of 0.keys/itemswithcycle=Falselist each name once, as its highest cycle.AsArrayheader without a byte count raisesDeserializationErrorinstead ofTypeError.Notes for review
Pinned values: many new expected values come from uproot's own output, so they pin current behaviour rather than check it independently.
Changed tests:
test_1063now usesprocesses=Falseand a smaller file.test_0927andtest_0965(3D) are no longer skipped. They passed on Linux with ROOT 6.32 and 6.40, but should be skipped again if the test_0927_dont_assume_uproot_in_global_scope_in_TPython_Eval now raises an exception #1237 segfault comes back.Conflicts with open PRs: fix: TProfile reading, writing and to_hist fixes #1766 and fix: writing weighted, Int64, buffered and old-version histograms #1769 touch
data_noflowinwriting/identify.py.Bugs found but not fixed here, because they need a decision:
Model._to_writableemptiesTStringmembers.Writing a TGraph drops
fFunctionsandfHistogram.Strided branches with no members can't be read.
ROOT can write a wrong
fENDafterupdate_streamers.RNTuple
iterateignorescut.copy_fromcan be passed an empty list of ranges.On Windows, a file pattern that mixes a literal name and a glob, such as
{a,?}.root, can return the same file twice. The glob match uses\and the literal keeps/, so duplicate removal misses it.ReadOnlyDirectory.items(cycle=False)gives a name's first key in the file's order, so for an uproot-written file it's the lowest cycle, whiledirectory[name]is the highest.Whether ROOT writes a header for a container nested in an objectwise
std::map(such asmap<string,vector<int>>) is unverified; uproot reads one.uproot.updatecan't write to a TTree with a jaggedTBranchcounted by aTBranchElementleaf. The tree is protected, but the specific "counter in another branch type" message is never shown, because theTBranchElementcheck comes first.These are listed here for follow-up.
This PR was written with significant help from Claude Code (Claude Opus 5.5), including the tests, the fixes and this description.