Adding a static / comptime Optional to the stdlib

Optional is a great way to achieve dynamism at runtime. However, often the existence of attributes/arguments can be already inferred at comptime.

Consider a function that takes an optional argument, which is only processed if provided:

fn fun(
    owned arg1: Float64, arg2: Float64, optional_arg: Optional[Float64] = None
) -> Float64:
    if Bool(optional_arg):
        arg1 *= optional_arg.value()
    return arg1 + arg2

This works, but requires a runtime if evaluation and reserves memory for the optional_arg even if not needed.

A more efficient solution would be an overload:

fn fun(arg1: Float64, arg2: Float64) -> Float64:
    return arg1 + arg2

fn fun(owned arg1: Float64, arg2: Float64, optional_arg: Float64) -> Float64:
    arg1 *= optional_arg
    return arg1 + arg2

However, this is verbose and can lead to duplicate code. I would like to propose (and beforehand get some feedback) to add a static optional type, which would enable the following features:

  • Zero-cost optional arguments
  • Automatic inference of the presence of the optional at comptime
  • Parametric existence of struct attributes.

The above example would then read as follows:

fn fun[
    has_optional: Bool = False
](
    owned arg1: Float64,
    arg2: Float64,
    optional_arg: ComptimeOptional[Float64, has_optional] = None,
) -> Float64:
    @parameter
    if Bool(optional_arg):
        arg1 *= optional_arg.value()
    return arg1 + arg2

Because I needed this type for Larecs, I have already implemented a version there. As another showcase for the gained ergonomics, consider the following example:

from larecs.comptime_optional import ComptimeOptional
from sys.info import sizeof

struct S[has_value: Bool = False]:
    var _value: ComptimeOptional[Int, has_value]

    fn __init__(out self, value: ComptimeOptional[Int, has_value] = None):
        self._value = value

def main():
    s0 = S()
    s1 = S(1)

    print("Size of s0 =", sizeof[__type_of(s0)]())
    print("Size of s1 =", sizeof[__type_of(s1)]())

    @parameter
    if s0.has_value:
        print(s0._value.value())

    @parameter
    if s1.has_value:
        print(s1._value.value())

Output:

Size of s0 = 0
Size of s1 = 8
1

I’d be very open to renaming the type to StaticOptional (or any other more suitable name) and happy to create a proposal on GH and a corresponding PR based on my implementation in Larecs (some adjustment might be necessary to mirror the behaviour of the builtin Optional). However, I want to keep the repo clean of unwanted feature requests of additional types.

What do people think?

1 Like

Hi.

I’m definitely not a language design expert (although I do find language design very interesting),
I think a static / comptime Optional type is very useful.
This seems like an elegant solution to this kind of problem.

I am unsure about the “has_value” name, but that’s largely just a nit-pick.
Naming things is hard :slight_smile:

Overall, very cool.

Given an Optional is a special case of a tagged union, it might make more sense for Mojo to offer a means to declare that a type variable is “one of N alternatives”, where the alternative is known at comptime.

In my opinion, the simplest solution would be to introduce a special kind of trait bound that means “one of N alternatives”. In fact, Python already offers this feature, see here! The feature was introduced alongside Python’s type-parameter syntax in 2022.

The syntax looks like:

fn foo[MaybeFloat: (Float64, None)](..., optional_arg: MaybeFloat = None):

Personally, I’m not super keen on treating the bare tuple : (A, B) as meaning “one of A or B”. It’s too subtle. Perhaps Mojo could offer a more explicit syntax, using a trait-level parametric alias:

alias OneOf[*T: AnyType]: Trait = <mlir stuff>

fn foo[MaybeFloat: OneOf[Float64, None]](..., optional_arg: MaybeFloat = None):

OP’s ComptimeOptional type could then be defined as follows:

alias ComptimeOptional[T: AnyType]: Trait = OneOf[T, None]

Alternatively, instead of modelling these constraints as traits, perhaps we could model them using the new requires feature that is being added to Mojo:

fn foo[MaybeFloat: AnyType](..., optional_arg: MaybeFloat = None)
  requires MaybeFloat in (Float64, None):
# Or with syntactic sugar:
fn foo[MaybeFloat: AnyType in (Float64, None)](
  ..., optional_arg: MaybeFloat = None
):
1 Like

This would be something equivalent a “Comptime Variant” then, which could be useful indeed. (Though I understand that you would suggest not introducing a new type but rather to add a new language feature for such a trait bound.)

I am not sure if this could model the same things as the suggested static Optional, though. Suppose you have two attributes a and b and you only need b if you have a. a and b could be of arbitrary types. How would you model this with a trait-like solution unless you have conditional vars?

An example:

struct Foo[has_a: Bool = False]:
    var _a: ComptimeOptional[List[Int], has_a]
    var _b: ComptimeOptional[List[Float32], has_a]

    fn __init__(out self, a: ComptimeOptional[List[Int], has_a] = None):
        self._a = a

        @parameter
        if has_a:
            self._b = List[Float32](capacity=len(a.value()))
            for val in a.value():
                self._b.value().append(Float32(val[]) / 2.0)
        else:
            self._b = None

    fn get_half(self) -> ref [self._b.value()] List[Float32]:
        return self._b.value()


def main():
    bar = Foo(List(1, 2, 3))
    print(bar.get_half()[0])

I think what we actually want is sum types that work at compile time.

1 Like

I think this would be a different proposal (though it would likely go in the direction of what @Nick had in mind). The thing that ComptimeOptional is supposed to solve is not that different types can be assumed but that an attribute is present or not based on a parameter. Such a thing as the “parametric var” sought for in a different post.

Yes, if you need one parameter to toggle the existence of multiple arguments/fields (of arbitrary type), then the feature that I suggested probably wouldn’t be sufficient.

I can see the value of the feature you’re suggesting.

Here’s a different idea. Again, my goal is to allow Mojo users to express a comptime equivalent of Optional and Variant.

Python and Mojo already have if-expressions. What if we allow such expressions to return a type?

fn foo[cond: Bool](optional_arg: Float64 if cond):
# Or equivalently:
fn foo(optional_arg: Float64 if _)

struct Thing[cond: Bool = False]:
    var _a: List[Int] if cond
    var _b: List[Float32] if cond

You’ll notice that I’ve omitted the else branch in the above examples. This seems like a clean way to model optionality. This is also consistent with my proposed syntaxes for having comptime control over async and raises:

fn foo() raises if cond:

fn bar() yields if cond:

This syntax can express more than just optionality. To select between two types, one could write:

fn foo[is_integer: Bool](arg: Int64 if is_integer else Float64):

It would be possible to abstract these if-expressions behind parametric aliases:

alias Foo[cond: Bool] = T1 if cond or something() else T2
alias Bar = Thing[X, Y, T1 if blah() else T2]

To be honest, I like this design better than my earlier ideas! It’s simultaneously more concise, more expressive, and easier to teach. Python and Mojo users are already comfortable with if-expressions.

This feature seems like it would subsume @rd4com’s idea for parameterized var declarations.

@clattner, perhaps this expanded set of use cases comes closer to justifying the feature? The current use cases are:

  • optional arguments (whose existence is known at comptime)
  • optional fields (whose existence is known at comptime)
  • optional local vars (whose existence is known at comptime)
  • arguments/fields/locals whose type is selected from 1 of N possibilities, using a comptime Bool, or maybe an Int (in other words, “comptime type selection”)

Yes, this would be a more expressive alternative solution. I am not sure how this would work with multiple types (one could apply an index parameter to a variadic parameter list), but this can be figured out.

I am not sure how likely it is that this feature will make it into the language and how soon this will happen. The proposed ComptimeOptional has the advantage that it can be (and is already) fully implemented with what we have.

Anyway, the intention of my post was to evaluate whether it is worth it creating a feature request issue regarding ComptimeOptional along with a respective PR. So far, my impression is that the community’s interest in this feature is moderate at best and that the chances of ComptimeOptional being accepted into the stdlib are low. So I will refrain from further work in that direction in the short run. This is perfectly fine.