Best practices for Python packaging

Hi all,

I’m interested in using mojo to speed up hot paths in a number of Python libraries that I develop/maintain, both in CPU and GPU code. I have had a great experience with both Mojo itself and the recently added Python-calling-Mojo interop layer, kudos to the team for all of the hard work!

I’d like to understand/figure out best practices for packaging/distributing Python libraries backed by Mojo (both with GPU code and without).

Specific constraints:

  • I’d like to minimize additional maintenance burden (relative to python packages with PyPI only distribution)
  • I’d like to support as wide a range of hardware as possible
  • I don’t mind minimal overhead from JIT compilation

I saw the mention of some work on building wheels in the Q3 roadmap. I’m hoping to avoid making an ad-hoc solution which will likely force complexity on my users at some point so maybe waiting for that is the right call…

In any case, I’d love to understand how people here are thinking about this problem and see how they are tackling it if there are examples out there!

For context:

  • I have some experience packaging for both conda (both conda-forge and other channels) and PyPI
  • I have never built Python modules backed by compiled code
  • I am vaguely aware of cibuildwheel but have never used it
2 Likes