Write spec v3 with zarr-python v3 and tensorstore

[K-Meech]

Write zarr spec version 3 with:

• zarr-python v3
• tensorstore

[ruaridhg]

I'm not entirely sure how to implement for tensorstore as although zarr3 is mentioned in the tensorstore docs and in the zarr-specs docs.

However, when I try to run tox run -e py313-tensorstore -- --benchmark-only --dev-image after adding in zarr_format=3 into the metadata

"metadata": {
                "dtype": image.dtype.str,
                "shape": image.shape,
                "chunks": chunks,
                "compressor": compressor,
                "fill_value": 0,
                "zarr_format": 3,  # v3 spec
            },

I get the following error:
ValueError: Error parsing object member "metadata": Error parsing object member "zarr_format": Expected integer in the range [2, 2], but received: 3 [source locations='tensorstore/internal/json_binding/json_binding.h:873\ntensorstore/internal/json_binding/json_binding.h:830\ntensorstore/internal/json_binding/json_binding.h:873\ntensorstore/internal/json_binding/json_binding.h:830']

Note that "zarr_format": 2, works in the metadata and I updated to the latest version of tensorstore to check it wasn't a new feature we were missing.

There's a potentially related PR that mentions zarr_spec_format = "3" and there's also mention of 'driver': 'zarr3', in tensorstore code which when I tried it gave the following error:
ValueError: Error parsing object member "metadata": Error parsing object member "fill_value": must be specified in conjunction with "data_type" [source locations='tensorstore/driver/zarr3/metadata.cc:391\ntensorstore/internal/json_binding/json_binding.h:873\ntensorstore/internal/json_binding/json_binding.h:830\ntensorstore/internal/json_binding/json_binding.h:388\ntensorstore/internal/json_binding/json_binding.h:873\ntensorstore/internal/json_binding/json_binding.h:830']

[K-Meech]

@ruaridhg - you'll need to change both the driver and the metadata to match. As I understand it, the zarr driver can only write zarr spec version 2 (hence why you're getting an error passing zarr_format=3 to it), and zarr3 should be used for writing version 3.

So you'll need to change the driver line to:

"driver": "zarr3",

and add metadata matching the zarr3 spec which is slightly different e.g. see the example in the tensorstore docs and the full description of each metadata member. You'll see that e.g. chunks -> chunk_grid, compressor -> codecs etc