Zarr depth limit rejects NWB chunk paths

[h-mayorquin]

I converted an IBL session while testing hdmf-zarr's v3 migration and tried uploading it to sandbox dandiset 218696. DANDI 0.78.0 stopped at validation:

[dandi_zarr.tree_depth_exceeded] Zarr directory tree more than 7 directories deep

The processed file has 34 files over the limit, including:

processing/pose_estimation/RightCamera/PoseEstimationSeriesTubeTop/data/c/1/0

The NWB array path has five components. Zarr v3 adds c/1/0 for the chunk coordinates. With the usual v2 encoding this would end in data/1.0 and fit under the limit.

This numeric-only example reproduces it:

import numpy as np
import zarr

group = zarr.open_group("deep.zarr", mode="w", zarr_format=3)
array = group.create_array("a/b/c/d/series/data", shape=(2, 2), chunks=(1, 1), dtype="float64")
array[:] = np.arange(4).reshape(2, 2)

Run dandi validate deep.zarr (outside a dandiset it also reports DANDI.NO_DANDISET_FOUND). The IBL upload also hit the string validation error reported in #1918. This example uses only numeric data to isolate the depth error.

I found the same problem in #983 from 2022. Satra pointed out that the nested NGFF chunk paths were legitimate. John asked whether to increase or remove the limit. Satra suggested seven to cover the use cases at the time, and Yarik agreed:

#983 (comment)

The original proposal was concerned about accidentally uploading large directory trees as thousands of assets:

#852 (comment)

Do we still need this restriction inside a recognized Zarr asset? Could we remove it or exclude chunk coordinates from the count?