My project is written in Python, and it makes extensive use of computationally intensive libraries internally, which results in extremely low performance. I’m currently conducting research on replacing Python with a different programming language and have found that Mojo’s positioning astonishingly aligns with my requirements. When I was reading the documentation, I discovered that when Mojo calls Python libraries, it uses the Python runtime environment (CPython). So, can using Mojo to call Python libraries still bring about a performance improvement in this case?
What you would want to do is first replace the “main” function of your project and move it into mojo, even if it’s just a stub. There isn’t a good way to call Mojo from Python right now, although a team is working on it. Once you have that, convert over your high-level control flow for the program as well, so that you can call into python as much as possible for anything not computationally intensive. Then, you can rewrite just the paths that go directly to the computationally intensive bits, as well as those bits. That should give a reasonable performance boost.
If the computationally intensive library has a C API, you can just talk to that directly and bypass all of the slow python bits.
I can be more helpful if you can describe the application and domain a bit more.
Mojo’s approach to speeding up Python is to make the interoperability between the two languages be as easy as possible, and then make sure that Mojo code is super fast. This means that you can move the parts of your python code that are critical to performance to Mojo to get a speedup.
If the computationally intensive library I’m using is a third-party library, does that mean I have to wait until the third-party library provides a Mojo version, and only then can I switch my high-level code to Mojo to achieve significant performance improvements?In my case, the flow control code part accounts for only 10% of the whole performance, but the computationally intensive libraries account for about 90%. I am afraid I will spend too much effort/time in changing to Mojo, but it may not be valuable at all.
Mojo can’t speed up python code, but if that third-party library is a python wrapper around some C/C++ code, which I would hope any computationally intensive Python library would be, it should be possible to directly use that interface from Mojo. If you’re spending a lot of time doing round-tripping between Python and compiled languages, Mojo can help you because it can optimize that and avoid forcing the C/C++ code to go through the “sanity check the python” work. If the library you need to use is a big pile of pure python, there’s not much that Mojo can do for you outside of a rewrite of that library. If you are using something like Jax where you set up a big computation and then spend the next several minutes in a “run” function, Mojo also probably can’t help you without a rewrite.
Sadly, Mojo is not yet at a place where we have a magic “libpython2mojo” that can make your code 100x faster by converting it to Mojo for you. If that existing library is already making use of numpy, pytorch, jax, etc, Mojo won’t be able to magically give an order of magnitude performance boost because those libraries are fairly fast already. You might get a 2x or a 5x depending on what exactly you’re doing with a full rewrite to MAX, but that’s algorithm and implementation specific.
thanks a lot for replying.