Skip to content

[Unexpected Behaviour] open_group raises AssertionError if array w/o group exists #3114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FabricioArendTorres opened this issue Jun 3, 2025 · 1 comment
Labels
bug Potential issues with the zarr-python library

Comments

@FabricioArendTorres
Copy link

FabricioArendTorres commented Jun 3, 2025

Zarr version

3.0.8

Numcodecs version

0.15.1

Python Version

3.11.11

Operating System

Linux, Dockerfile based on python:3.11-slim

Installation

poetry

Description

I think there is some unexpected behavior that occurs if one

  1. creates an array in the root of a store
  2. attempts to open a group in the root of this store in r or r+ mode.

I would have expected some meaningful error, such as the zarr.errors.ContainsArrayError, or FileNotFoundError, but one just gets a very vague AssertionError.
I would like to avoid catching such a broad error in my code.

Below you can find a reproducible test code (tested with uv run) that highlights the expected and actually occuring behaviour.

Note on Stores:
I originally stumbled upon it when using the object store, and could directly reproduce it with the memory store, so it should be independent of the store.

Steps to reproduce

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
#   "pytest>=8.3",
# ]
# ///
# This script automatically imports the development branch of zarr to check for issues
import pytest
import zarr.errors
import zarr.storage


def test_open_group_in_dir_with_array():
    store = zarr.storage.MemoryStore()

    # this passes
    with pytest.raises(FileNotFoundError):
        zarr.open_group(store, mode="r+")

    # now create an array within root
    zarr.create_array(store, shape=(2,), dtype=float)

    # this somehow raises an assertion error ?
    # I would expect zarr.errors.ContainsArrayError or FileNotFoundError.
    # with pytest.raises(AssertionError):
    with pytest.raises((zarr.errors.ContainsArrayError, FileNotFoundError)):
        zarr.open_group(store, mode="r+")


def test_create_group_in_dir_with_array():
    store = zarr.storage.MemoryStore()

    # this passes
    with pytest.raises(FileNotFoundError):
        zarr.open_group(store, mode="r+")

    # now create an array within root
    zarr.create_array(store, shape=(2,), dtype=float)

    with pytest.raises(zarr.errors.ContainsArrayError):
        tmp = zarr.create_group(store)


if __name__ == "__main__":
    pytest.main([__file__])

Additional output

The test_open_group_in_dir_with_arrray fails with the following stacktrace:

==================================================================== FAILURES ====================================================================
_______________________________________________________ test_open_group_in_dir_with_array ________________________________________________________

    def test_open_group_in_dir_with_array():
        store = zarr.storage.MemoryStore()
    
        # this passes
        with pytest.raises(FileNotFoundError):
            zarr.open_group(store, mode="r+")
    
        # now create an array within root
        zarr.create_array(store, shape=(2,), dtype=float)
    
        # this somehow raises an assertion error ?
        # I would expect zarr.errors.ContainsArrayError or FileNotFoundError.
        # with pytest.raises(AssertionError):
        with pytest.raises((zarr.errors.ContainsArrayError, FileNotFoundError)):
>           zarr.open_group(store, mode="r+")

zarr_bug.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/_compat.py:43: in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/api/synchronous.py:529: in open_group
    sync(
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/sync.py:163: in sync
    raise return_result
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/sync.py:119: in _runner
    return await coro
           ^^^^^^^^^^
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/api/asynchronous.py:828: in open_group
    return await AsyncGroup.open(
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/group.py:583: in open
    return cls._from_bytes_v3(
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/group.py:647: in _from_bytes_v3
    return cls.from_dict(store_path, group_metadata)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/group.py:656: in from_dict
    metadata=GroupMetadata.from_dict(data),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'zarr.core.group.GroupMetadata'>
data = {'attributes': {}, 'chunk_grid': {'configuration': {'chunk_shape': [2]}, 'name': 'regular'}, 'chunk_key_encoding': {'c...ion': {'endian': 'little'}, 'name': 'bytes'}, {'configuration': {'checksum': False, 'level': 0}, 'name': 'zstd'}], ...}

    @classmethod
    def from_dict(cls, data: dict[str, Any]) -> GroupMetadata:
        data = dict(data)
>       assert data.pop("node_type", None) in ("group", None)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AssertionError
@FabricioArendTorres FabricioArendTorres added the bug Potential issues with the zarr-python library label Jun 3, 2025
@FabricioArendTorres FabricioArendTorres changed the title Inconsistent & Unexpected Behaviour: open_group raises AssertionError if Array exists [Unexpected Behaviour] open_group raises AssertionError if array w/o group exists Jun 3, 2025
@FabricioArendTorres FabricioArendTorres changed the title [Unexpected Behaviour] open_group raises AssertionError if array w/o group exists [Unexpected Behaviour] open_group raises AssertionError if array w/o group exists Jun 3, 2025
@d-v-b
Copy link
Contributor

d-v-b commented Jun 4, 2025

agreed that an assertion error here is not very helpful. I'll have a look at this today

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

No branches or pull requests

2 participants