Skip to content

F-contiguous numpy inputs give wrong output #47

Description

@nnn112358

InferenceSession.run copies Fortran-contiguous (column-major) numpy inputs to the device verbatim, so the NPU reads a transposed tensor and returns wrong results — with no error raised. Affects 0.1.3, both providers (AXCLRTExecutionProvider, AxEngineExecutionProvider).

Full self-contained reproduction (tiny prebuilt axmodel + script, verified on real AX650N):
👉 https://github.com/nnn112358/pyaxengine-fcontiguous-bug

The bug

InferenceSession.run memcpy's each input's raw npy.ctypes.data buffer to the device, but only normalizes the layout when the array is neither C- nor F-contiguous:

# axengine/_axclrt.py:360-362   (AXCLRT, axclrtMemcpy)
# axengine/_axe.py:372-374      (AxEngine, engine_cffi.memmove)
if not (npy.flags.c_contiguous or npy.flags.f_contiguous):
    npy = np.ascontiguousarray(npy)
npy_ptr = ...cast("void *", npy.ctypes.data)   # raw buffer -> device

The device buffer expects C-order (row-major) bytes. An F-contiguous (column-major) array satisfies npy.flags.f_contiguous, so it slips through the guard untouched and its column-major bytes are copied verbatim into the row-major buffer → the NPU reads a transposed tensor → silently wrong output.

This is easy to hit in practice: np.swapaxes(x, -1, -2), x.T, and np.asfortranarray(x) all return F-contiguous arrays. For example a VITS flow input z_p = np.swapaxes(attn @ np.swapaxes(m_p, 1, 2), 1, 2) is F-contiguous, which made an otherwise-correct *.axmodel look badly quantized (output cosine ≈ 0.14 vs the ONNX reference) until the input was made C-contiguous.

Reproduction

The repro feeds the same values as C-contiguous vs F-contiguous and compares NPU outputs. On a real AX650N (AXCLRT):

toggling input 'x' shape=(1, 8, 16)
  values identical (np.array_equal) : True
  C-input  c_contiguous=True  f_contiguous=False
  F-input  c_contiguous=False f_contiguous=True
  cos(out_C , out_F)            = 0.06398   # should be 1.0 (same values!)
  cos(out_C , ascontig(out_F)) = 1.00000   # np.ascontiguousarray fixes it
RESULT: BUG REPRODUCED
expected actual
F-contiguous input, identical values same output as C-contiguous (cos = 1.0) transposed → cos << 1
error on unsupported layout none (silent)

Root cause & fix

F-contiguous is not a safe layout for a raw row-major memcpy/memmove; only C-contiguous is. The or npy.flags.f_contiguous term in the guard is incorrect. Drop it in both providers:

- if not (npy.flags.c_contiguous or npy.flags.f_contiguous):
+ if not npy.flags.c_contiguous:
      npy = np.ascontiguousarray(npy)
  npy_ptr = ...cast("void *", npy.ctypes.data)

np.ascontiguousarray is a zero-copy no-op for already-C-contiguous arrays, so this only adds a copy for the (previously broken) F-order case.
Files: axengine/_axclrt.py (~line 360), axengine/_axe.py (~line 372).

Workaround (caller side): wrap every input in np.ascontiguousarray() before run, and don't pass np.swapaxes / .T / np.asfortranarray views directly.

Environment

  • pyaxengine 0.1.3, numpy 2.4.6, Python 3.12
  • Device: AX650N, AXCL driver V3.6.4, PCIe host (x86_64); the same code path in _axe.py affects the SoC-native AxEngineExecutionProvider.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions