Note: this is a question that was submitted for the February 3rd community meeting – we weren’t able to answer this question live so we’re answering it here instead.
What kind of metaprogramming mechanism(s) are planned for Mojo? Specifically, will the metaprogramming be similar to Zig, in that the metaprogramming syntax is the same as the rest of the language?
Here are a few metaprogramming features planned for Mojo:
As an addition to what @Caroline has given as the official answer, there’s a few more details that may be helpful.
First, Zig’s comptime is a very, very good reflection API, but Zig has an “if all you have is a hammer” problem. There are a lot of data structures for which the “shorthand” of List[T: CollectionElement] is sufficient, because you don’t need to do very aggressive transforms. It’s also much easier for most developers to use it this way, and combines in a much better way with an algebraic type system. An idea I have seen raised repeatedly is to try to offer both a Rust-style parameter API that integrates more tightly with algebraic traits, and a Zig-like reflection API. Zig’s approach is, in my opinion, overkill and extra stress on the compiler for data structures and generic functions, but it works very, very well for writing struct transforms and things like serialization/deserialization code. There’s also oppertunities to improve on Zig’s approach by letting you use traits to constrain parameters, then generating functions using those types, which removes a lot of those checks that Zig makes you write, like is_memcpy_safe.
The big downside of Zig’s comptime is that it’s really, really hard to write an LSP for, especially one that is responsive. Andrew has decided that the best path forwards for Zig is to make the compiler fast enough to let you run an entire compile without emitting object files as an LSP. While this is a noble goal, as codebases increase in size this becomes harder and harder to sustain, and Mojo has a much more advanced type system than Zig does, which makes the problem even harder. Minimizing the use of comptime will help a lot with this issue.
Hi,
I originally asked this question via the community Google Form.
Thanks for the detailed feedback.
Much appreciated.
If it’s not to much trouble, could you explain what is meant by:
[…] Rust-style parameter API that integrates more tightly with algebraic traits, and a Zig-like reflection API.
Secondly, regarding the LSP challenges that arise due to Zig’s comptime, would it be a solution if Mojo could output code generated via metaprogramming to special directories, so that the code could be linted for use in LSPs, kind of like what Kotlin’s KSP system does?
I realise this suggestion is a major departure from what has probably already been implemented, but it does seem to address the LSP issue.
Best regards,
Monte.
Zig’s metaprogramming is effectively a very powerful reflection API. Powerful enough that you can use it for most “stamp out this data structure”-like things. What it can’t do is check for whether your type doesn’t contain anything that would be invalid if the instance were moved to another process, or check if you have a special way to serialize the type to JSON. Rust-style traits cover a lot of the basic “Just give me a list of T” stuff, and then if you want to implement generic serialization, want to make the data structure do heavy specialization like store the input data in an odd way, or transform a type to SoA, AoSoA or some other special format, the Zig-style metaprogramming exists as a way to do that.
When used sparingly, Zig-style metaprogramming isn’t a problem, because you can just run the metaprogram. It only encounters problems when you start to have very extensive use of it. Zig sees this problem because it has no other option which can be done more efficiently and which lets the compiler symbolically reason about types. Mojo, if it has both, can keep that cost for the places where it’s necessary. There wouldn’t be a need to write out the extra code in most cases, although it might get implemented in an LSP cache at some point. As far as I am aware, Mojo’s LSP is the mojo compiler used like a library, so it can do all of the things that the normal compiler can do, including interpret stuff entirely in memory.
Hi,
Just putting a link to some updates to the compile.reflection module, for ease of reference if someone is curious.
We can now get the struct name of type parameters.
Seems like a step in the right direction 