Skip to content

Migrate build system from setuptools to meson-python#826

Open
maxrjones wants to merge 26 commits intozarr-developers:mainfrom
maxrjones:meson
Open

Migrate build system from setuptools to meson-python#826
maxrjones wants to merge 26 commits intozarr-developers:mainfrom
maxrjones:meson

Conversation

@maxrjones
Copy link
Member

@maxrjones maxrjones commented Mar 18, 2026

This PR includes a complete modernization of the build setup for numcodecs. I could break these changes into individual, more easily reviewable PRs, but I wanted to first open a proof of concept with the final state to gauge appetite for this scale of change. The motivation here is to reduce the maintenance burden for numcodecs. I'd like to enable small fixes or releases without developers needing to spend hours futzing with the build/ci setup, which is unfortunately how I spent my afternoon/evening.

Concrete changes included in this modernization:

  • Replace setup.py with meson.build for all 8 C/Cython extensions
  • Move to src/ layout with top-level tests/
  • Add system library support (-Dsystem_blosc, -Dsystem_zstd, -Dsystem_lz4)
  • Fix cross-compilation SIMD detection (meson host_machine vs py-cpuinfo)
  • Update CI workflows and cibuildwheel config
  • Update contributing guide to recommend uv/venv

I tested these changes locally, but will need to check if there's any issues with the ci.

Closes #509
Closes #824
Closes #703

TODO:

  • Unit tests and/or doctests in docstrings
  • Tests pass locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • Docs build locally
  • GitHub Actions CI passes
  • Test coverage to 100% (Codecov passes)

@codecov
Copy link

codecov bot commented Mar 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (06a53c6) to head (a1c9a43).

Additional details and impacted files
@@             Coverage Diff             @@
##             main      #826      +/-   ##
===========================================
+ Coverage   99.33%   100.00%   +0.66%     
===========================================
  Files          26        27       +1     
  Lines         898       902       +4     
===========================================
+ Hits          892       902      +10     
+ Misses          6         0       -6     
Files with missing lines Coverage Δ
src/numcodecs/__init__.py 100.00% <100.00%> (ø)
src/numcodecs/abc.py 100.00% <ø> (ø)
src/numcodecs/astype.py 100.00% <ø> (ø)
src/numcodecs/base64.py 100.00% <ø> (ø)
src/numcodecs/bitround.py 100.00% <ø> (ø)
src/numcodecs/bz2.py 100.00% <ø> (ø)
src/numcodecs/categorize.py 100.00% <ø> (ø)
src/numcodecs/checksum32.py 100.00% <ø> (ø)
src/numcodecs/compat.py 100.00% <ø> (ø)
src/numcodecs/delta.py 100.00% <ø> (ø)
... and 17 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@d-v-b
Copy link
Contributor

d-v-b commented Mar 18, 2026

thanks so much for picking this up max!

@d-v-b
Copy link
Contributor

d-v-b commented Mar 18, 2026

one possibility to put on the table: we could consider putting codecs with orthogonal dependencies in separate sub-projects, and make this project a monorepo. We should only think about this if it makes things simpler of course.

@maxrjones
Copy link
Member Author

Thanks so much for your prompt review @QuLogic! I believe that I addressed all your helpful comments in the latest set of commits.

@maxrjones maxrjones mentioned this pull request Mar 18, 2026
7 tasks
@maxrjones
Copy link
Member Author

one possibility to put on the table: we could consider putting codecs with orthogonal dependencies in separate sub-projects, and make this project a monorepo. We should only think about this if it makes things simpler of course.

I think this could be a separate task. I'm a bit ambivalent about that change, but am really motivated by this PR because it'll help us keep all the checks green.

@d-v-b
Copy link
Contributor

d-v-b commented Mar 18, 2026

one possibility to put on the table: we could consider putting codecs with orthogonal dependencies in separate sub-projects, and make this project a monorepo. We should only think about this if it makes things simpler of course.

I think this could be a separate task. I'm a bit ambivalent about that change, but am really motivated by this PR because it'll help us keep all the checks green.

makes sense! this PR looks great as-is

@maxrjones maxrjones requested a review from a team March 19, 2026 01:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nightly wheels still failing declarative build 0.12.1: numcodecs/tests is part of the installable resources

3 participants