Skip to content

ZFP with Zarr breaks when data is a view #812

@tasansal

Description

@tasansal

With Zarr v3 and using zarr.codecs.ZFPY as serializer, the ZFP compressor logic in numcodecs complains that it is not F-order data. The chunk data neither C/F contiguous when it is a sliced view of an existing array.

Reproducer

import numpy as np
import asyncio
import zarr
from zarr.storage import MemoryStore
from zarr import create_array
from zarr.codecs import numcodecs

zfp = numcodecs.ZFPY(mode=4, tolerance=0.001)
store = MemoryStore()

zfp_arr = create_array(
    store,
    shape=(100, 100, 100),
    chunks=(50, 50, 100),
    dtype="float32",
    serializer=zfp,
    compressors=None
)

rng = np.random.default_rng()
data = rng.random(zfp_arr.shape)
zfp_arr[:] = data

Raises:

ValueError: The zfp codec does not support F order arrays. Your arrays flags were   C_CONTIGUOUS : False
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
.

If we look at the chunk slices' numpy flags:

>>> data[:50, :50, :].flags.c_contiguous
False
>>> data[:50, 50:, :].flags.c_contiguous
False
>>> data[50:, 50:, :].flags.c_contiguous
False
>>> data[50:, :50, :].flags.c_contiguous
False

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions