[syahra712]

When writing in chunks, a chunk where every entry of a jagged field is empty has Awkward type var * unknown. RNTuple writing fails on it, but the TTree writer accepts it:

import awkward as ak
import uproot

with uproot.recreate("out.root") as file:
    file["ntuple"] = {"muons": ak.Array([[1.5, 2.5], [3.5]])}
    file["ntuple"].extend({"muons": ak.Array([[], []])})
    # ValueError: Data is not compatible with this RNTuple. Expected ... got ...

with uproot.recreate("out.root") as file:
    file["ntuple"] = {"muons": ak.Array([[], []])}
    # AssertionError: This should not have happened. Please report this issue to the Uproot developers.

The same chunks work with mktree + extend. The TTree writer turns the unknown type into float64 (_branch_ak_to_np).

Changes in writing/_cascadentuple.py:

• extend now calls a new _fill_unknown. It walks the data next to the RNTuple's form and replaces each EmptyArray with a zero-length array of the type the form expects at that position (form.length_zero_array()). Nothing else in the data is converted, so data with a different type, such as floats into an int64 field, is still rejected by the existing check.
• _to_packed_form writes an EmptyForm as float64, matching the TTree writer, so an RNTuple can also be created from such a chunk.

tests/test_1812_rntuple_unknown_type_empty_lists.py (named after the expected PR number; I'll rename it if this gets a different one) covers:

• extending with all-empty lists in a plain list field, an optional list field and a list inside a record;
• extending with a different type still raising;
• creating an RNTuple from all-empty lists.

The two new-behaviour tests fail on main and pass with this change. The full test suite passes locally (-m "not network and not xrootd and not distributed": 1170 passed, 253 skipped), and black and ruff at the pre-commit versions are clean.

AI-assisted (Claude). I reviewed the code and ran the tests.

[codecov]

Codecov Report

❌ Patch coverage is 85.18519% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.07%. Comparing base (24f1c3f) to head (fe49721).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/uproot/writing/_cascadentuple.py	85.18%	2 Missing and 2 partials ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
src/uproot/writing/_cascadentuple.py	91.14% <85.18%> (+0.34%)	⬆️

... and 14 files with indirect coverage changes

[ariostas]

🤖 AI text below 🤖

Thanks for this! I think creating an RNTuple and extending one need different handling here.

Creation: raise instead of guessing float64. When the input has var * unknown, any type we pick is a guess, and it becomes the permanent schema. If a later chunk has real data of a different type (e.g. int64), extend fails with a "not compatible, expected float64" error that's hard to trace back to the first chunk. I'd rather raise a clear TypeError at creation when the form contains an EmptyForm, pointing users to mkrntuple with an explicit type, e.g.

file.mkrntuple("ntuple", {"muons": "var * float64"})

So the EmptyForm → float64 case in _to_packed_form would go.

Extend: keep filling from the schema, but simplify. In extend nothing is guessed, because the schema is already fixed and unknown can only become one type. A chunk where every list is empty is ordinary data when writing in chunks, and it has to be accepted for the mkrntuple workaround above to work. Instead of the hand-written _fill_unknown walk, I think a single ak.enforce_type(data, <stored type>) would be simpler and more complete. The current walk misses a few cases:

• tuple RNTuples: _fill_unknown runs before the dict with fields "0", "1", … is turned into a tuple, so the is_tuple check returns early and an all-empty chunk is still rejected;
• UnmaskedArray and UnionArray aren't handled, so empty lists under them aren't filled;
• it rebuilds the whole layout on every extend call, even when there's nothing to fill.

It's also another walk over layout types that has to be kept in sync with _to_packed and _to_packed_form. It would be good to check that enforce_type handles unknown types inside unions and unmasked arrays, and to only call it when the data's form contains an EmptyForm.

A related case: a zero-length chunk with unknown type (ak.Array([])) currently raises "Data is not compatible". It could just be a no-op, like a zero-length chunk with a known type already is, e.g. by returning early when len(data) == 0.

[syahra712]

Thanks @ariostas, that split makes sense. c92b913 does it that way:

• Creation now raises a TypeError when the form has an EmptyForm, pointing to mkrntuple with an explicit type. The EmptyForm → float64 case in _to_packed_form is gone.
• Extend uses ak.enforce_type(data, <stored type>) instead of _fill_unknown, and only when the chunk's form contains an EmptyForm. _fill_unknown is removed. Tuple RNTuples work now (there's a test), and I checked that all-empty lists under an UnmaskedArray extend and read back fine, and that enforce_type fills unknown inside a union.

Two things came up along the way:

1. enforce_type also converts numbers lossily, e.g. ak.enforce_type(ak.Array([[1.5]]), "var * int64") gives [[1]]. So it's only applied when the chunk's form matches the stored form everywhere except the unknown parts; anything else still raises "not compatible" (test_extend_with_incompatible_type_still_fails).
2. For the zero-length chunk, an early return broke test_1786_rntuple_arrays_cluster_boundary, because a zero-length extend with a known type isn't a no-op, it writes an empty cluster. So ak.Array([]) now goes through the same path and also adds an empty cluster, matching the known-type behaviour (the test checks cluster_summaries == [2, 0, 1]).

Locally the full suite passes (1186 passed, 303 skipped), and 4 of the 5 new tests fail on main. The macOS CI failures on this run look unrelated to me (the Pyodide and ROOT jobs failed at plugin loading), but I can't see the macOS logs anymore. Could you re-run them?

Separately, while testing I noticed that extending an RNTuple with a union field re-reads the first chunk, on main too:

f["n"] = {"u": ak.Array([[1, 2], 3.5])}
f["n"].extend({"u": ak.Array([[5], 4.5])})
# reads back [[1, 2], 3.5, [1, 2], 3.5]

Happy to open a separate issue for that if you'd like.

[syahra712]

Ah, never mind about the union issue. I see you already opened #1815 for it.

[syahra712]

I merged main into this branch so CI picks up #1821 (the pytest 9.2 startup fix), which was behind the Pyodide and ROOT job failures. No changes to the PR itself; the full suite still passes locally (1263 passed).

[ariostas]

Thank you, @syahra712!