For the past few months I have been working on a creative coding environment leveraging python/Mojo interop. It is called MMMAudio and the repo is found here:
This environment is designed the same way that SuperCollider/Max/pd are, in that there is a control environment and an audio environment. In this case the control environment is python and the audio environment is Mojo. What I have made is a way for the two to talk to each other to facilitate audio creativity. I have also built a bunch of dsp that can synthesize and process sound in many ways (I am particularly proud of the Oscillator). There are many great sounding examples to try out and a Getting Started Guide to get you going.
When I heard Chris Lattner describe the problems with AI inference development two years ago, it clicked for me that these are the exact issues in audio, mainly: the audio creative coding environments have their own languages that users use to make instruments, but when you want to write dsp, you have to write in c or c++ and deal with cmake, etc. MMMAudio uses Mojo’s awesome Python Interop to create a development environment where everything exists in one space and the creative instrument building is in the same language as the dsp.
Here are some key features:
All levels of dsp from unit generators to synths to graphs are written in Mojo and the user can modify any of the dsp layers at any time. This will encourage innovation in DSP alongside instrument-building, rather than in a totally separate space.
Leverages the python ecosystem: loading audio files, getting midi, open sound control, and ai inference can all happen using existing robust python libraries. Plus you get to use python’s numpy, scipy, etc for audio analysis and data processing on the python side. The best tools are already there and ready to use.
Use of pytorch and eventually Max allows nn training and nn inference to happen in python or in Mojo, using industry leading ai tools. No need to reinvent the wheel and make new objects. Just use the best stuff in the best languages for AI development.
The two next steps are: integrating @martinvuyk ‘s fft and then building bindings so that the portaudio engine is running in Mojo instead of python.
Thanks to Modular for this awesome language. I hope some people can get some joy out of my project. If anyone wants to contribute, this project is open source and I am waiting for your PRs.
Since I started programming in graphical environments (Max/MSP) over 10 years ago, I’ve worked with PureData and SuperCollider (the one I used most), and I’ve always dreamed of a high-level language (like Python) that would allow low-level work so I could work with audio and utilize the full power of the hardware. I’ve explored several languages in search of this (V, Zig, Rust), and I even tried working with C++ (doing a few things with JUCE) (…spoiler alert: I didn’t like it).
This is the first time I see the light at the end of the tunnel. I truly believe that this time it will be possible. Mojo-Python. I can’t imagine a better combination to finally democratize DSP and multimedia programming.
I hope to be a part of this in one way or another . Thanks to all
I wanted to give a little update on MMMAudio because Ted and I are about to give a paper presentation about it at the International Computer Music Conference and a workshop at the SEAMUS Conference. The Mojo takeover of audio begins!
If you are into academic papers, our ICMC paper is here. As far as these things go, I think this is both informative and entertaining.
MMMAudio alpha-v2:
is on Mojo 26.2. We plan on releasing beta versions with upcoming Mojo betas and 1.0 with Mojo 1.0.
is capable of running multiple instances using Python multiprocessing. I can run 45,000 oscillators on 8 MMMAudio instances my M2 Mac. this is very slick. only SuperCollider can also do this.
has an audio analysis suite that fully integrates with sci-kit learn for data analysis and dimensionality reduction
has polyphonic voice allocation modules that allow flexible automagic polyphonic voicing
has many new examples, including UMAP/KDTree, Dattorro Reverb, Parallel Audio Graphs, and NessStretch. (Try the NessStretch with a beautifully produced pop track. You won’t be disappointed.)
Runs on Raspberry Pi now that Linux Arm is supported (I can confirm this works).
Anyhow, let me know if you have any feedback for us on this thread, on our GitHub page, or just DM me.
. Thanks to all
