For the past few months I have been working on a creative coding environment leveraging python/Mojo interop. It is called MMMAudio and the repo is found here:
MMMAudio - MMM Means Mojo Audio
This environment is designed the same way that SuperCollider/Max/pd are, in that there is a control environment and an audio environment. In this case the control environment is python and the audio environment is Mojo. What I have made is a way for the two to talk to each other to facilitate audio creativity. I have also built a bunch of dsp that can synthesize and process sound in many ways (I am particularly proud of the Oscillator). There are many great sounding examples to try out and a Getting Started Guide to get you going.
When I heard Chris Lattner describe the problems with AI inference development two years ago, it clicked for me that these are the exact issues in audio, mainly: the audio creative coding environments have their own languages that users use to make instruments, but when you want to write dsp, you have to write in c or c++ and deal with cmake, etc. MMMAudio uses Mojo’s awesome Python Interop to create a development environment where everything exists in one space and the creative instrument building is in the same language as the dsp.
Here are some key features:
-
All levels of dsp from unit generators to synths to graphs are written in Mojo and the user can modify any of the dsp layers at any time. This will encourage innovation in DSP alongside instrument-building, rather than in a totally separate space.
-
Leverages the python ecosystem: loading audio files, getting midi, open sound control, and ai inference can all happen using existing robust python libraries. Plus you get to use python’s numpy, scipy, etc for audio analysis and data processing on the python side. The best tools are already there and ready to use.
-
Use of pytorch and eventually Max allows nn training and nn inference to happen in python or in Mojo, using industry leading ai tools. No need to reinvent the wheel and make new objects. Just use the best stuff in the best languages for AI development.
The two next steps are: integrating @martinvuyk ‘s fft and then building bindings so that the portaudio engine is running in Mojo instead of python.
Thanks to Modular for this awesome language. I hope some people can get some joy out of my project. If anyone wants to contribute, this project is open source and I am waiting for your PRs.
Sam
