Are there plans for out of tree dialects to be supported in Mojo? Interesting MLIR-based compilers for specific hardware, like TT-MLIR, are starting to pop up. That one in particular should only require turning on the RISC-V LLVM backend since most of the rest can be done in normal Mojo at this point.
Hi Owen! There aren’t plans in the roadmap, but this is absolutely an aspirational goal for Mojo as a native frontend for MLIR.
In principle it’s not hard to do, the standard library uses tools to generate the dialects we use in our compilation stack for example. In practice there are a few big hurdles we’d need to clear first:
- How do we tell the parser how to import your dialect? In principle this isn’t difficult, we just need to pick a way. In practice there’s some lurking complexities in MLIR where it’s not possible in general to link against two different libraries that both compile against MLIR and send objects between them. We would need to provide a general solution to this problem.
- Once you are able to generate your new dialect, how do we make it practical for people to use?
- Your code will likely always translate to a mixture of dialects. Say you use an
ifstatement or a function call; that will be a native Mojoiforcalland not one from a dialect you may want to be using. This may not be an issue in the specific case you mention, but limits its usefulness in general. - Our compiler stack doesn’t know how to lower the custom dialect, so there’s no way for us to translate it down. I think this is realistically the biggest unknown. One could imagine Mojo providing native MLIR metaprogramming tools where you could write your lowerings as a Mojo function and register it with the compiler. With less effort we could allow registering and linking against custom passes in our compiler.
- Your code will likely always translate to a mixture of dialects. Say you use an
Hopefully that gives a taste of some of the problems here. I’d love to get there, but one step at a time!
That’s understandable. I know that bridging that gap is going to be tough, but I was hopeful that enough pdl would more or less work. My thought was that Tenstorrent’s hardware would be lower-hanging fruit than other options since it’s RISC-V with a few custom instructions, not an entirely new backend. That doesn’t seem to be doable so I guess my programming will be in MLIR for now and I’ll lift it to Mojo when the time comes.
Hi, it’s great to see that Mojo is considering being extendable in this way. Is there any update on plans regarding this? The ability to handle custom (or even “just” unregistered) dialects would make Mojo very interesting for us in the quantum computing community. I understand that Mojo allows me to generate MLIR operations, which I could do at comptime and forward to the Mojo compiler. What’s missing from my perspective is the ability to introduce custom dialect elements, and to emit IR text/bitcode for further processing.
I imagine this would be useful long before a more involved integration like linking against custom passes can be realized. (I’m aware of this issue where the answer was “no”, but this was quite a while ago and from this thread and the fact that the compiler will be open sourced soon, I feel like the attitude could have changed?)