build: replace c-blosc git submodule with meson wrap subprojects

[d-v-b]

🤖 AI text below 🤖

Note

Stacked on #864 (NEON shuffle). The first commit here is #864's; review the last three commits. Draft until #864 lands and the c-blosc tarball is hosted in this repo (see Before merging).

Summary

This replaces the c-blosc git submodule, and the ~170 lines of hand-listed internal-complibs sources in src/numcodecs/meson.build, with meson subprojects:

• zstd, lz4 and zlib come from WrapDB (subprojects/{zstd,lz4,zlib}.wrap). Previously their versions were whatever c-blosc bundled; now they can be bumped independently with meson wrap update. This PR moves zstd 1.5.6 → 1.5.7 and zlib 1.3.1 → 1.3.2; lz4 stays at 1.10.0.
• c-blosc 1.x is not in WrapDB, so subprojects/c-blosc.wrap pins the same commit the submodule pointed to (04c06fb). subprojects/packagefiles/c-blosc/ holds its meson.build (upstream only ships CMake) and numcodecs' patches, currently the NEON shuffle from perf(blosc): use NEON SIMD for byte-shuffle on aarch64 #864. Patching c-blosc no longer requires forking the submodule.
• Trimmed c-blosc tarball. The upstream archive carries its own zstd/lz4/zlib/snappy copies, tests and compat fixtures (~2.4 MB of sdist). tools/make_c_blosc_tarball.py extracts only blosc/, the license files and the README into a reproducible 60 KB tarball, which the wrap points at.
• The system_* options keep their meaning. When a library isn't taken from the system, its subproject is configured first, so every dependency() lookup, including c-blosc's, resolves to the same static build. The build log shows c-blosc reusing the zstd subproject (found: YES 1.5.7 (cached)).
• Packaging. In pyproject.toml, meson-python now skips installing subproject artifacts (--skip-subprojects) and includes their sources in the sdist (--include-subprojects), so the sdist builds offline.
• Licenses. License texts for all bundled C code now live in LICENSES/ and are listed in license-files. This adds zstd's license, which the wheels were missing before.
• CI and docs. The submodules: checkout settings and Read the Docs config are removed, and the contributing guide is updated.

Trade-offs

• A C++ compiler is now required to build from source. zstd's WrapDB meson project declares C++, although libzstd itself is C. Standard CI images and Alpine's build-base include one. Alternatives: a numcodecs-maintained zstd overlay, or an upstream WrapDB change.
• The sdist grows from 6.3 MB (0.17.0) to 8.3 MB. WrapDB's zstd (2.5 MB), zlib (1.6 MB) and lz4 (0.4 MB) are full upstream release archives, where the old sdist only had their lib/ sources via c-blosc. These could be trimmed the same way as c-blosc, at the cost of losing plain meson wrap update bumps.
• Builds from a git checkout, including cibuildwheel, download sources on first build. Offline builders can run meson subprojects download beforehand or build from the sdist.

Before merging

• Merge perf(blosc): use NEON SIMD for byte-shuffle on aarch64 #864 and rebase.
• Host the c-blosc tarball in this repository. source_url in subprojects/c-blosc.wrap currently points at a release asset on the PR author's fork (vendor/c-blosc-04c06fb7339d). To verify it, run python tools/make_c_blosc_tarball.py 04c06fb7339d3df3600b1862077b8699b30a0e1d; the tarball sha256 should be ebadbf11…a7f7a81 and the uncompressed tar sha256 6b7381ba…42b28f4. Then re-upload it here and update the URL.

Testing

Locally on macOS arm64 (Python 3.14):

• The full test suite passes (1,007 passed, 31 skipped).
• A sha256 over Blosc output from 1,452 (typesize × length × blocksize × shuffle × codec) combinations matches the pre-migration build.
• The sdist contains all subproject sources, and a wheel built from it with --wrap-mode=nodownload succeeds.
• The wheel contains only numcodecs' extension modules plus license files; the extensions link only libSystem.

Linux, Windows and cibuildwheel runs are not yet verified locally; this PR's CI covers them.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (1f83681) to head (1879207).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #865   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           27        27           
  Lines          905       905           
=========================================
  Hits           905       905           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mkitti]

How much conda-forge breakage is involved here?

https://github.com/conda-forge/numcodecs-feedstock/blob/main/recipe%2Fmeta.yaml

[d-v-b]

Not much I think

[mkitti]

I think we should just wrap cmake here with https://mesonbuild.com/CMake-module.html

[mkitti]

This might be relevant: Blosc/c-blosc2#690

We should move to blosc2 as soon as possible.

[d-v-b]

This might be relevant: Blosc/c-blosc2#690

We should move to blosc2 as soon as possible.

blosc2 cannot write data that blosc1 reads. so we have to keep blosc1 around.

[mkitti]

c-blosc2 has a blosc1_compress function now as of c-blosc 2.3.0:
https://blosc.org/c-blosc2/reference/blosc1.html#c.blosc1_compress

https://github.com/Blosc/c-blosc2/blob/main/RELEASE_NOTES.md#changes-from-220-to-230

[d-v-b]

it would be a breaking change to blosc1 readers

buffers generated with C-Blosc2 are not format-compatible with C-Blosc1 (i.e. forward compatibility is not supported)