Skip to content

Cannot write a 0 dimensional array with sharding #3751

@d-v-b

Description

@d-v-b

This:

#!/usr/bin/env python3
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "zarr @ git+https://github.com/zarr-developers/zarr-python.git@main",
# ]
# ///

import zarr

arr = zarr.create_array({}, shape=(), dtype="f4", chunks=(), shards=())
arr[()] = 42.0

Makes this:

(dedl-cube-io) d-v-b@pc ➜  data-model-1 git:(feat/healpix-cli) ✗ uv run zarr_scalar_sharding_bug.py
Traceback (most recent call last):
  File "/home/d-v-b/dev/data-model-1/zarr_scalar_sharding_bug.py", line 13, in <module>
    arr[()] = 42.0
    ~~~^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/array.py", line 2928, in __setitem__
    self.set_basic_selection(cast("BasicSelection", pure_selection), value, fields=fields)
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/array.py", line 3162, in set_basic_selection
    sync(self.async_array._set_selection(indexer, value, fields=fields, prototype=prototype))
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/sync.py", line 159, in sync
    raise return_result
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/sync.py", line 119, in _runner
    return await coro
           ^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/array.py", line 1714, in _set_selection
    return await _set_selection(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/array.py", line 5913, in _set_selection
    await codec_pipeline.write(
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/codec_pipeline.py", line 488, in write
    await concurrent_map(
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/common.py", line 117, in concurrent_map
    return await asyncio.gather(*[asyncio.ensure_future(run(item)) for item in items])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/common.py", line 115, in run
    return await func(*item)
           ^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/codec_pipeline.py", line 347, in write_batch
    await self.encode_partial_batch(
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/codec_pipeline.py", line 247, in encode_partial_batch
    await self.array_bytes_codec.encode_partial(batch_info)
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/abc/codec.py", line 283, in encode_partial
    await concurrent_map(
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/common.py", line 117, in concurrent_map
    return await asyncio.gather(*[asyncio.ensure_future(run(item)) for item in items])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/core/common.py", line 115, in run
    return await func(*item)
           ^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/codecs/sharding.py", line 584, in _encode_partial_single
    shard_dict = shard_reader.to_dict_vectorized(np.asarray(_morton_order(chunks_per_shard)))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/codecs/sharding.py", line 284, in to_dict_vectorized
    starts, ends, valid = self.index.get_chunk_slices_vectorized(chunk_coords_array)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/d-v-b/.cache/uv/environments-v2/zarr-scalar-sharding-bug-4174d1dad5099f58/lib/python3.12/site-packages/zarr/codecs/sharding.py", line 177, in get_chunk_slices_vectorized
    starts = offsets_and_lengths[:, 0]
             ~~~~~~~~~~~~~~~~~~~^^^^^^
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

seems like we are missing some 0D arrays in our sharding test matrix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions