Skip to content

Use a typesize of 1 in Blosc for zero-itemsize buffers - #870

Open
raashish1601 wants to merge 1 commit into
zarr-developers:mainfrom
raashish1601:fix/516-blosc-zero-itemsize
Open

raashish1601 wants to merge 1 commit into
zarr-developers:mainfrom
raashish1601:fix/516-blosc-zero-itemsize

Conversation

@raashish1601

@raashish1601 raashish1601 commented Oct 7, 2026 •

Copy link
Copy Markdown

Fixes #516.

compress passes the buffer's itemsize to blosc as the typesize. For a dtype with an itemsize of 0, such as a structured dtype with a zero-length field, that is 0, which blosc does not accept:

>>> numcodecs.Blosc().encode(np.zeros((5,), dtype=[("foo", int, (0,))]))
RuntimeError: error during blosc compression: -10   # "`typesize` parameter must be greater than 0!"

(With the older c-blosc in the issue this was a floating point exception that killed the process.)

Such a buffer is always empty, so the typesize does not matter. When the typesize comes from the buffer and is 0, compress now uses 1. An explicit typesize argument is still validated as before.

TODO:

  • Unit tests and/or doctests in docstrings: test_zero_itemsize in tests/test_blosc.py (all four shuffle modes); it fails before the change.
  • Tests pass locally: tests/test_blosc.py (228 passed, 15 skipped), built in a Linux container.
  • Docstrings and API docs for any new/modified user-facing classes and functions (no API change)
  • Changes documented in docs/release.md
  • Docs build locally (not built; only a release note line was added)
  • GitHub Actions CI passes
  • Test coverage to 100% (Codecov passes)

Decoding the result (like any empty Blosc buffer) still fails; that is #831, which #854 addresses, so the test only checks the encoding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Blosc throws floating point exception when encoding NumPy array with zero-itemsize dtype

1 participant