Zarr v2 to v3: what actually changed

Zarr v3 changed the on-disk metadata layout, not the data model. Arrays are still chunked N-dimensional arrays in a group hierarchy. What moved is where the metadata lives, what the fields are called, and how compression is described.

One metadata file instead of three

Zarr v2 wrote .zarray for array metadata, .zgroup to mark a group, and .zattrs for user attributes, up to three files per node. Zarr v3 consolidates all of it into a single zarr.json per node, with attributes as a nested attributes object.

{
  "zarr_format": 3,
  "node_type": "array",
  "shape": [10000, 10000],
  "data_type": "float32",
  "chunk_grid": {
    "name": "regular",
    "configuration": { "chunk_shape": [64, 10000] }
  },
  "chunk_key_encoding": {
    "name": "default",
    "configuration": { "separator": "/" }
  },
  "fill_value": 0.0,
  "codecs": [
    { "name": "bytes", "configuration": { "endian": "little" } },
    { "name": "blosc", "configuration": { "cname": "zstd", "clevel": 5 } }
  ],
  "attributes": { "units": "kelvin" }
}

The root node stores its metadata at the key zarr.json; a node at path /foo/bar stores it at foo/bar/zarr.json. Group nodes carry only zarr_format, node_type: "group" and optional attributes.

Renamed fields

Chunk keys changed shape as a result. A v2 array with the default separator stored chunk (1, 0) at key 1.0; a v3 array with the default encoding stores it at c/1/0. The v2 chunk key encoding exists for compatibility.

The codec pipeline

v2 had two distinct concepts: filters that transformed the array before compression, and a single compressor. v3 replaces both with one ordered codecs list, typed by what each stage consumes and produces.

1. Zero or more array → array codecs (for example transpose).
2. Exactly one array → bytes codec (for example bytes, which fixes endianness). This is mandatory.
3. Zero or more bytes → bytes codecs (for example blosc, gzip, crc32c).

The typing makes the pipeline verifiable, and it lets a codec declare a partial-decode path. A reader can then decompress a sub-region of a chunk rather than the whole thing, which is not expressible in the v2 model.

Sharding

The sharding_indexed codec addresses the main operational weakness of Zarr on object storage: one object per chunk. It packs many chunks into a single stored object with an index at the end, so the chunk grid stays fine-grained while the object count drops.

{
  "name": "sharding_indexed",
  "configuration": {
    "chunk_shape": [64, 1000],
    "codecs": [{ "name": "bytes" }, { "name": "blosc" }],
    "index_codecs": [{ "name": "bytes" }, { "name": "crc32c" }]
  }
}

Compatibility

The two formats coexist rather than one replacing the other in place. zarr-python 3.x reads and writes both, selected by the zarr_format argument; a store contains one format or the other, distinguished by whether it has zarr.json or .zarray/.zgroup. There is no in-place upgrade: converting means rewriting the metadata, and rewriting the chunks if the key encoding changes.

import zarr

g = zarr.open_group("store.zarr")       # detects v2 or v3
print(g.metadata.zarr_format)

# Write v2 explicitly when a consumer requires it.
zarr.create_array(store="out.zarr", shape=(1000, 1000),
                  chunks=(100, 100), dtype="f4", zarr_format=2)

The Zarr viewer reads both layouts and reports which one a store uses, along with the chunk grid and codec chain, which helps when a store fails to open in a tool that only supports one.