Alright, I’ve now had enough time to study @clattner’s proposal and formulate a proper response. Here’s my feedback.
First, I want to make clear that I agree with the vast majority of your proposal Chris. In particular I agree that enums should be more or less “just structs”, and support method declarations and trait conformances. I also agree that enums should introduce brand-new nominal types. Despite my earlier posts, I now agree with others that desugaring enums to Variant is probably not a suitable approach, for a few different reasons.
My main concern about Chris’s proposal (as currently written) is that it leads us away from being able to treat cases as structs. To be more precise: I think we want the ability to treat an entire case as a struct, including its embedded discriminant. Treating just the payload of a case as a struct (where the payload carries no discriminant) closes some doors that IMO we should be leaving open. In the rest of this post I will explain what these “doors” (opportunities) are, and why we might want to pursue them.
But first I’ll show everyone what I think enums should look like. It so happens my design is almost identical to the one that Owen proposed in his most recent post. This is a happy accident: until just now I had no idea what Owen wanted enums to look like; I hadn’t seen his post. I’ll take that as a sign that this design is especially promising.
My proposed design
For “simple” enums where each case is a value without a payload, I am in 100% agreement with Chris’s current proposal. For example, here’s how I’d define a 3-value Color enum:
enum Color:
case Red
case Green
case Blue
Note that I am capitalizing the cases. This is because I am treating them as (singleton) types. Like in Chris’s proposal, I would expect that Color.Red can be used as a value, but I would also like to reserve the possibility of using Color.Red as a type. (Typically as part of a more complicated type expression.) You can think of this as analogous to how None is used in both Python and Mojo: you can write -> None as a return type, but also write return None.
For “complicated” cases that need to carry data, I would like us to adopt the case struct syntax that Owen proposed earlier:
enum Color:
case Red
case Green
case Blue
@fieldwise_init
case struct RGB:
var r: Int32
var g: Int32
var b: Int32
@fieldwise_init
case struct HSL:
var h: Int32
var s: Int32
var l: Int32
The syntax and capabilities of a case struct are identical to those of an ordinary struct: you can add decorators, define fields and methods using the standard syntax, and even (some day) conform to traits:
enum File:
case struct Open(Writer): # Open files can be written to
...
case Closed # You cannot write to a closed file
Notice how we’re using parentheses to delimit trait conformances, as we do for ordinary structs. This syntax is at odds with Chris’s proposal, wherein parens are used to delimit the payload tuple, e.g. case RGB(Int32, Int32, Int32). This is one example of a door that I don’t want us to close unnecessarily. IMO we should reserve the use of parentheses next to a case’s name to list its trait conformances, even if the enum MVP doesn’t support that.
Case structs are constructed using a standard constructor call, with the exception being that the return type should (by default) be inferred as the enum’s type:
var c = Color.RGB(100, 240, 70) # c has type `Color`, not `Color.RGB`
Some day in the distant future, likely when Mojo has class hierarchies and subtyping, we can explore more sophisticated alternatives for how to infer the type of c. For example, maybe it resolves to the most specific type that type-checks. But in the short-to-medium term, the above behaviour seems like the only sensible choice.
One important detail worth noting is that a case struct is not semantically equivalent to the “payload structs” discussed in Chris’s proposal. Case structs have an (implicit) embedded discriminant, whereas Chris’s payload structs do not. More generally, the size and alignment of each case struct matches that of its enclosing enum. I will explain why this distinction is important shortly.
This pretty much summarises what I would like the enum MVP to look like. The biggest difference is that we are modelling enum cases as structs. Apart from that, everything else should work more or less the same as in Chris’s proposal. Pattern matching, for example, would use the struct’s name: case .RGB(r=r, g=0, b=0).
One small benefit of this design change is that Mojo users can reuse their existing knowledge of struct definitions to define enum cases, and to pattern match on them. A case is “just a struct”, so everything you can do with a struct, you can do with a case, more or less. The resultant syntax is also more concise in many situations. Here is some code from Chris’s proposal:
struct SourceRange:
var start: Int
var end: Int
enum Token:
case eof
case source_range(SourceRange)
match some_token:
case .source_range(SourceRange(start=x, end=y)):
...
And here’s what it would look like with case structs:
enum Token:
case EOF
case struct SourceRange:
var start: Int
var end: Int
match some_token:
case .SourceRange(start=x, end=y):
...
The two notable differences are that:
- We were able to define
SourceRangeinside the definition ofToken. - We were able to eliminate the
.source_rangewrapper label in both the enum definition and the pattern match.
Overall, I think this is a nice clean up. I expect the nearest-term benefit of modelling cases as structs will be to help Mojo users remember how to read and write enum definitions, especially enums that have complicated payloads. But more importantly, this design direction opens up a ton of opportunities that we can explore in the future, after this MVP lands. I’ll dive into those next.
Opportunities provided by this design (post-MVP)
I will now explain why modelling cases as structs is actually interesting/useful. There are a lot of opportunities here, and we don’t need to pursue all of them. I will start with features that I think are “easy wins”, and gradually progress to more complex (but powerful) features. You should think of everything below as options enabled by the MVP, rather than a list of features I’d like to see implemented. My intention is to keep the door open to these opportunities, by ensuring that the enum MVP doesn’t preclude them.
Using pattern matching to “downcast” an enum instance
If enum cases are types, then by pattern matching, we should be able to obtain an instance of that type:
enum Tree[T: type]:
@fieldwise_init
case struct Leaf:
var value: T
@fieldwise_init
case struct InteriorNode:
var value: T
var left: Box[Tree[T]]
var right: Box[Tree[T]]
var tree: Tree[Int32]
match tree:
case .Leaf as var x: # x has type `Tree[Int32].Leaf`
...
case .InteriorNode as var x: # x has type `Tree[Int32].InteriorNode`
foo(x.left) # now we can access the fields `.left` and `.right`
Note: The as keyword is part of Python’s pattern matching grammar and is planned for Mojo as well.
What we are doing here is narrowing or refining the matched value, producing a “more specific type” based on the information we have learned. This is a form of subtyping or type refinement. In the short term, Mojo doesn’t need true subtyping to implement this feature, but it’s something to keep in mind as a direction we can go in, to support more advanced use cases.
If we choose our syntax for Mojo’s “if let” feature carefully, we can also do this rebinding within if-statements:
var tree: Tree[Int32]
if tree case .InteriorNode as x:
foo(x.left)
I find this if-let syntax easier to read than the Rust-style syntax if var <pat> as <ident> = tree. And I like the consistency with the syntax of match statements. But that’s just a personal preference.
If Mojo was to support some form of flow-sensitive type narrowing (a la TypeScript), we could make this code even more succinct:
var tree: Tree[Int32]
if tree case .InteriorNode:
foo(tree.left) # in this scope, `tree` now has type `Tree[Int32].InteriorNode`
But this opens up a whole can of worms, so I think for the moment, the first option (binding a new name to narrow a type) is the better one.
Storing and manipulating case structs
Once we have the ability to downcast case structs, we can do all kinds of useful things with them. For example, we can partition a list of enum instances into a list of its cases:
enum Color:
case struct RGB: ...
case struct HSL: ...
var colors: List[Color] = get_colors()
var rgb_colors: List[Color.RGB] = []
var hsl_colors: List[Color.HSL] = []
for color in colors:
match color:
case .RGB as x:
rgb_colors.append(x)
case .HSL as x:
hsl_colors.append(x)
And as Owen mentioned in his last post, we can write useful functions to convert between cases:
def convert_to_hsl(rgb: Color.RGB) -> Color.HSL:
...
This all feels pretty useful. Of course, you can do something similar by defining RGB and HSL as standalone structs that an enum wraps, but this adds boilerplate to both the struct definition and the pattern matching. The mechanics of case structs that I’ve presented so far seem simple enough to be worth implementing, even if just to avoid this boilerplate. But if we’re looking for stronger justification for case structs, continue reading!
Guaranteed compatibility between Pointer[MyEnum] and Pointer[MyEnum.Case]
Earlier I mentioned that a case struct is not “just” a payload type, because it also (implicitly) stores a discriminant, and has the size and alignment of the parent enum. Here is where that distinction comes into play.
Consider the earlier example where we partitioned colors into RGB and HSL. Let’s imagine we want the partitions to store pointers rather than copies of the colors:
var colors: List[Color] = get_colors()
var rgb_colors: List[Pointer[Color.RGB, _]] = []
var hsl_colors: List[Pointer[Color.HSL, _]] = []
Now let’s imagine we’ve defined a function that takes an immutable pointer (or reference) to a color:
def foo(color: Pointer[Color, _]): ...
It would be great if we could safely pass a pointer to an RGB or a HSL color to this function, even though they expect a more general Color pointer. (For a more sophisticated example, imagine passing an arbitrary data structure full of such pointers.) Such a function call is not “safe” unless it’s valid to reinterpret an RGB pointer as a Color pointer. This is where it becomes important that each case struct inherits the discriminant and alignment of its parent enum. If we can guarantee this, then (as long as the caller doesn’t mutate the pointee), an RGB pointer is just a Color pointer where you know the value of the discriminant. The callee can match on that discriminant, rediscovering the information the caller already knew. The RGB pointer is guaranteed to be aligned correctly. Everything plugs together just fine.
In practice, to convert between pointers we would either need to bitcast them:
var p: Pointer[Color.RGB, _]
foo(p.unsafe_bitcast[Color]())
…or Mojo’s type system would need to support subtyping, and the associated type-compatibility rules. Thankfully, it seems like such a feature is already (whether intentionally or not) on the Mojo roadmap, given that Python-style classes (with inheritance) naturally introduce subtyping to any static type system.
There are also some subtleties when it comes to mutability, but that’s worth a post of its own, so for the moment, I’ll move onto other topics. If you know anything about the relationship between mutability and variance in languages with subtyping, that’s the issue at play here.
In summary: we have the opportunity to offer guaranteed “compatibility” between (pointers to) case structs and their enclosing enum type. But all of this requires case structs to carry discriminants and inherit the enum’s alignment, so case structs cannot be replaced by Swift/Rust-style “tagless” payload values. They are a fundamentally different concept, that requires proper treatment at the language level.
Reminder: I’m just discussing opportunities for Mojo’s evolution. None of this needs to be part of the enum MVP! It’s just a door that we can keep open if we design the MVP carefully.
Case unions
I want to talk about one more opportunity that Mojo could pursue some day: case unions.
Python already has a concept of class unions:
x: int | str
The principle that makes class unions work in Python is that of uniform representation: every Python reference is encoded as a pointer to its payload, stored alongside (effectively) a discriminant, that can be used to check whether x is an int or a str at runtime. A Python “union” is therefore exclusively a compile-time concept: it specifies what values the reference can take on at runtime, without influencing the runtime representation of the value.
In Python we can match on x to discover its type:
match x:
case int(): ...
case str(): ...
This is very reminiscent of our enum pattern matching syntax. That raises the question: can Mojo offer language features that facilitate both static typing of classes, and static typing of enums?
I think there’s a lot of promise here. What if we can offer the concept of (enum) case unions, to mirror that of Python class unions?
Here’s how we might declare a color variable whose case is restricted to be an RGB or HSL color:
var c: Color.RGB | Color.HSL
Just as in Python, this “union operator” would not introduce a new class or struct. The type of c would be a subtype of Color, and its runtime representation would be identical to a Color. As in Python, we can then rely on the presence of the discriminant to test which (sub)type the variable holds at runtime:
match c:
case Color.RGB: ...
case Color.HSL: ...
This match would be considered exhaustive, even if the Color enum contains other variants, because the type of c indicates that c does not hold those remaining variants.
It’s important to clarify: this “union operator” is not the same concept as “anonymous sum types”. If that were the case then the type of c would be a brand new type, distinct from the Color enum and its cases, and it would introduce an additional discriminant value to c that makes its runtime representation different to that of a Color. Instead, we are retaining the representation of Color, and providing some extra compile-time knowledge about which colors c will store.
This also implies that it would be nonsense to take the union of two (non-case) structs, e.g. Pointer[X] | Pointer[Y], because such structs do not store a discriminant that can be used to differentiate them at runtime.
This also implies that Color.RGB | Color.RGB should be the same type as Color.RGB. Indeed, this is why “class unions” (and my proposed “case unions”) use the word “union”: the semantics reflects that of a set union.
In summary: by modelling Mojo’s enums as a set of “case structs” we are opening a pathway towards (some day) supporting “case unions” that behave exactly like Python “class unions”. In fact, I would go as far as to say that the two language features can actually be merged into one, with the notion of a “common discriminant” (across the unioned cases) being the unifying principle. This would help to make Mojo’s type system expressive, offering the typing features that Python users expect while keeping the language small.
Conclusion
I hope the intention behind this post makes sense to everyone. To recap: there’s a certain model for enums (that of case structs) that opens up a lot of exciting possibilities for Mojo’s future evolution. We don’t have to pursue all of those possibilities, but it would be good to leave them on the table.
It would be difficult to implement the possibilities I’ve presented above if we were to stabilize Chris’s original proposal as-is, so we really do need to do some advance planning if we would like to keep these doors open. (Or else we’d need to be willing to do a major “enums 2.0” redesign that amongst other things, changes the enum syntax.)
Another benefit of the case struct proposal is that it’s more concise than declaring “out-of-line” payload structs (as discussed early in this post), so that’s an added win. The main downside I can see with my alternative MVP is that case struct: ... is more verbose than case(X, Y, Z). In my opinion this is not a big deal, given the tradeoff we’d be making.
It seems Owen and I have converged on the same core design, for what that’s worth as well. (I wasn’t aware of this until I saw Owen’s post just now.)
That’s all I have to say for now. Please let me know what you all think. Thanks for reading!