Skip to content

fix: typetracer correctness issues affecting dask-awkward#4100

Open
henryiii wants to merge 3 commits into
mainfrom
henryiii/fix-typetracer
Open

fix: typetracer correctness issues affecting dask-awkward#4100
henryiii wants to merge 3 commits into
mainfrom
henryiii/fix-typetracer

Conversation

@henryiii

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

This PR fixes several typetracer correctness issues, most of which affect dask-awkward (which runs awkward operations on shape-only typetracer arrays). They were surfaced by an automated multi-agent code review (Claude Code) and each was verified before fixing.

Fixes

  • TypeTracer.min/max/sum(axis=None) infinite recursion (_nplikes/typetracer.py): min recursed with axis=axis (still None) instead of axis=0, causing a RecursionError; max/sum delegated to it. Additionally, sum now reports NumPy's promoted output dtype (e.g. bool/int32 → platform int) rather than preserving the input dtype, matching the concrete backends.
  • TypeTracer.stack wrong shape/dtype (_nplikes/typetracer.py): it treated the prototype tuple (0,) * ndim as data for empty_like, and normalized negative axes over the input ndim instead of the ndim + 1 output. Now the result shape is built symbolically (inserting len(arrays) at the normalized axis) and the dtype uses numpy.result_type. Reachable from _broadcasting.py.
  • searchsorted dtype (_nplikes/typetracer.py): returned the input array's dtype; it returns insertion indices, so it now reports intp, matching np.searchsorted and the convention used by nonzero.
  • asarray dtype handling (_nplikes/typetracer.py): for non-TypeTracerArray inputs it silently dropped the requested dtype and mis-fired the copy=False dtype check when dtype is None. It now honors the requested dtype and only raises for copy=False when an incompatible dtype is explicitly requested.
  • reshape with zero-size dimensions (_nplikes/typetracer.py): rejected any 0, but zero-size reshapes are valid in NumPy and the concrete path. Now permitted (with the -1 placeholder handled consistently for zero size).
  • __repr__ mutating the report (_nplikes/typetracer.py, _nplikes/placeholder.py, _nplikes/virtual.py): __repr__ read the touching .shape property, so merely printing an array mutated the TypeTracerReport. Now uses self._shape, and the impossible if shape is None dead branch is removed in all three.
  • CuPy reducers ignoring keepdims (_nplikes/cupy.py): the five overridden reducers (all/any/min/sum/max) accepted keepdims but never forwarded it. They now forward it, and the cupy#3819 .item() workaround is applied only for axis=None without keepdims so that axis=None, keepdims=True returns the 1×…×1 array NumPy produces.
  • ak.pad_none on 1D typetracer arrays (contents/numpyarray.py): _pad_none with clip=False did target < self.length without an unknown_length guard, raising TypeError under dask-awkward. Now uses the same guard pattern as content.py.
  • ak.combinations(axis=0) on typetracer (contents/content.py): _combinations_axis0 checked size is None, but unknown lengths are unknown_length, never None, so it raised TypeError. Now checks against unknown_length.
  • Dead-code removal (_nplikes/typetracer.py): removed a verified-unreachable integer bounds-check in __getitem__ (the index is always an unknown scalar after asarray), two dead is_unknown_scalar(length_scalar) branches in derive_slice_for_length (only reached when the length is known), and a dead hasattr(x, "shape") loop in broadcast_arrays (every element is asserted to be a TypeTracerArray).

Testing

A regression test file exercises all of the above on typetracer layouts (using to_typetracer(forget_length=True) for the high-level ak.pad_none/ak.combinations cases). The existing typetracer suites and the relevant operation suites pass.

The CuPy change is untested locally (no GPU available); it was made by inspection, mirroring the NumPy base class.

AI assistance

This PR was prepared with Claude Code via an automated multi-agent review. All findings were verified before fixing.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.38%. Comparing base (712dac0) to head (ca3df4a).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/awkward/_nplikes/typetracer.py 83.33% 7 Missing ⚠️

❌ Your patch check has failed because the patch coverage (87.50%) is below the target coverage (98.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/_nplikes/cupy.py 91.45% <100.00%> (ø)
src/awkward/_nplikes/placeholder.py 65.44% <100.00%> (+0.22%) ⬆️
src/awkward/_nplikes/virtual.py 93.30% <100.00%> (+0.41%) ⬆️
src/awkward/contents/content.py 77.61% <100.00%> (+0.16%) ⬆️
src/awkward/contents/numpyarray.py 91.56% <100.00%> (+0.17%) ⬆️
src/awkward/_nplikes/typetracer.py 77.24% <83.33%> (+1.63%) ⬆️

henryiii added 3 commits June 11, 2026 15:40
Fixes several typetracer correctness issues, mostly affecting
dask-awkward, found via an automated multi-agent review:

- typetracer min/max/sum(axis=None) recursed with axis=None instead of
  axis=0, causing a RecursionError; sum now reports NumPy's promoted
  dtype (e.g. bool/int32 -> platform int).
- typetracer.stack built the result shape incorrectly (treated the
  prototype tuple as data, and normalized negative axis over the input
  ndim rather than ndim+1); now builds the shape symbolically and uses
  numpy.result_type for the dtype.
- searchsorted returned the input dtype instead of intp insertion
  indices.
- asarray now honors the requested dtype for non-TypeTracerArray inputs
  and only rejects copy=False when an incompatible dtype is requested.
- reshape now permits zero-size dimensions, matching NumPy and the
  concrete backends.
- TypeTracerArray.__repr__ no longer reads the touching .shape property
  (printing mutated the report); same dead None-shape branch removed in
  PlaceholderArray and VirtualNDArray.
- cupy reducers (all/any/min/sum/max) now forward keepdims, preserving
  the cupy#3819 .item() workaround only for axis=None without keepdims.
- NumpyArray._pad_none and Content._combinations_axis0 now guard against
  unknown_length, fixing ak.pad_none and ak.combinations(axis=0) on
  typetracer layouts.
- Removed verified-dead branches in __getitem__, derive_slice_for_length,
  and broadcast_arrays.

Assisted-by: ClaudeCode:claude-opus-4.8
Assisted-by: ClaudeCode:claude-opus-4.8
Keep one focused test per fix and drop comments that merely restate
the adjacent code.

Assisted-by: ClaudeCode:claude-opus-4-8
@henryiii henryiii force-pushed the henryiii/fix-typetracer branch from c039be1 to ca3df4a Compare June 11, 2026 19:43
@TaiSakuma TaiSakuma added the type/fix PR title type: fix (set automatically) label Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/fix PR title type: fix (set automatically)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants