Responding to @Nick’s post, I don’t see it as a step forward, but there are a few other things mixed in here:
Only functions with access rights to the fields of the struct can deinitialize the fields one-by-one. Therefore, this design preserves encapsulation
Mojo doesn’t have access control yet, and won’t for quite some time. Are you proposing we consider this sometime later?
I am proposing that to de-construct a var
I don’t see this as a desirable feature. Just the documentation burden alone would be significant.
fn \__del_\_(var self):
del self.x
del self.y
Just to reiterate an upthread comment - del has a specific defined meaning in Python. Mojo doesn’t have support for it yet, but it presumably will some day. We should not appropriate something like that and give it new meaning.
Zooming out, Nick, you’re an infinitely smart person who thinks about things from the inside out. This is a superpower, but this isn’t how most programmers engage with a programming language. In my opinion/experience, most people apply patterns they see them without understanding them - “this is the way to write a destructor”. They want simple things and also benefit from standardization - “there is one right way to do things” - rather than many similar-but-different options to choose from.
Different languages take different approaches, but Mojo aspires to have “simple things that compose”, not “enable every possible thing” and I’m very comfortable saying “no” to things that are super niche if it keeps the core language smaller and easier to explain.
Your proposal appears to add new capabilities (being able to forget a local var) and “named constructors”. Furthermore it shifts behavior from being active (a keyword in the source code) to implicit (an emergent property of a mode switch that happens as a result of multiple separate statements working together). I think this would lead to significant confusion: if I forgot to deinitialize one field then the behavior would change. This can happen easily, e.g. when I add a new field to a struct but didn’t update the destructor.
Yes, I’m suggesting that we could defer this proposal until we have access control. In the meantime we could stick to deinit or even just __disable_del.
del has a specific defined meaning in Python. We should not appropriate something like that and give it new meaning.
The meaning of del local_var in Python is roughly “deinitialize the variable”, and the meaning of del local_var.x is roughly “deinitialize the attribute”. The way I was using del is consistent with that! (Python stores vars in dictionaries ofc, so there are some implementation differences.)
“named constructors”
Feel free to completely disregard my footnote on constructors. It was just a bonus, and I don’t care much about it.
it shifts behavior from being active (a keyword in the source code) to implicit (…). I think this would lead to significant confusion: if I forgot to deinitialize one field then the behavior would change. This can happen easily, e.g. when I add a new field to a struct but didn’t update the destructor.
I appreciate your detailed reply and insights, but on this key point, are you sure you understood my proposal? It’s impossible for “the behaviour to change” if you forget to deinitialize a subset of the fields. You will always get a compile-time error, which will tell you that you’ve partially deinitialized the variable, and you must either finish the deinitialization, or undo it. This design is therefore very robust to code refactorings. I don’t see anybody getting confused, as long as the compiler has good error messages. (For a concrete example, see struct Foo in my previous post.)
Final word on deinit
If we’re sticking with deinit in the short term (or forever), can I suggest that we at least make the syntax fn foo(deinit var self)? This makes it clear that the argument convention is still var. We haven’t changed the argument convention, we’ve just added a modifier.
I think that having to deinit each field one by one will be intensely annoying. Consider the following struct from an actual raft implementation I wrote in Rust:
Now, imagine you’re reviewing that code in a PR. Can you spot the mistake I added without an LSP or the compiler helping you? Also, this would print a bunch of warnings because I’m trying to move trivially copyable types. This is a fairly complex algorithm, so this might be on the far end of what size a struct may be, but it’s not that different from a hundred other structs in implementations of similar algorithms. Game devs are also likely to have large structs that would suffer from this issue.
This isn’t a realistic example. Your destructor is doing standard fieldwise destruction, so you’d be using the auto-generated implementation; you wouldn’t write it by hand.
Can you spot the mistake I added
You omitted self.next_index^, which as I keep trying to explain, would be a compile-time error, so you can’t ever miss it!
I can also throw a counter-example at you, using the current deinit proposal:
struct Foo:
var a: String
var b: String
var c: String
var d: String
var e: String
var f: String
fn to_list(deinit self) -> List[String]:
return List(self.a^, self.b^, self.c^, self.e^, self.f^)
Can you spot the mistake I added?
With the deinit implementation in nightly, if I forget to transfer one of the fields, the compiler is going to implicitly destroy it, rather than ask you what you want to do with the field. Whereas in my proposal, the compiler will produce the error “self is partially uninitialized. You must either deinitialize {d}, or reinitialize {a, b, c, e, f}.”
Yep, I understand what del does in Python. But I think we can take some creative liberty in Mojo. If del is used on a struct instance, it seems reasonable for it to invoke __del__. We don’t currently have a good syntax for invoking __del__ explicitly, after all!
I acknowledge that my latest proposal isn’t being well received.
My low controversy proposal is to put deinit var self in function signatures. That clarifies to everyone that the argument convention is still var. This is much easier to teach. With the currently implemented proposal, we will have to teach people that :
deinit isn’t an argument convention. When you write fn foo(deinit self), the argument is implicitly var.
That is a pedagogical nightmare!
I would be okay with this:
struct Foo:
fn __del__(deinit var self):
fn __moveinit__(out self, deinit var existing: Self):
Here, we are making it explicit that the arguments are var. And the LSP Intellisense can strip out the deinit modifier, because that’s an implementation detail, and callers don’t need to worry about it.
FYI value^.__del__() works fine, through composition of existing operations the same way that value^.named_dtor() works.
Ironically, the only reason to use this is if you want to EXTEND a value to some point, not because you want to kill off a value: Mojo will already kill off a value ASAP for you, so you shouldn’t ever need to do this. Some people use the _ = x^ idiom for lifetime extension.
To me as a “simple” minded developer, del x would be fine for Mojo to mean “I want x to be explicitly deleted from this point on“ (which implies that the lifetime gets extended to this point). Most of the time I will not need to explicitly delete a value as Mojo deletes a value asap if not used anymore.
I think this is easier to read than _ = x^ or _ = x.y^ and close enough to python semantics.
BTW: I‘d like to introduce the __delitem__ dunder function for collection structs which would be written as
Thanks for giving your perspective on that. If I understand correctly:
You believe it’s desirable to think of it as a declarative mechanism (in the signature rather than as a statement)
You’ve already implemented it that way
You don’t see much value in making it imperative instead
It composes well with ASAP destruction
However, I could say all those things about the imperative approach. I think imperative is desirable, it’s been implemented that way (in both Mojo and Vale) and I don’t see much value in making it declarative. Re composability with ASAP destruction, a statement would also compose:
Notice how ASAP destruction is destructing x, y, and z just after their last uses.
So, that all said, there’s a few more reasons to prefer imperative.
First: __del__(deinit self) is unnecessarily coupled to argument conventions. It raises the question of “is deinit owning, like a var?” These are two orthogonal concepts that should compose, yet we’re raising questions about how they overlap (though, deinit var self addresses that).
Second: Since this is a systems programming language, I believe we want code (deinit/deconstruct)'s effect to be near the code’s generated instructions (the call, or lack thereof, to __del__) whenever possible.
The declarative approach doesn’t have that property; the presence/absence of a deinit keyword causes something to happen at “some point later” that isn’t explicit/clear in the code. (Not trying to argue against ASAP destruction in general, but it is a real drawback.)
To see this drawback in action, try extracting the body of a __del__(deinit self): ... method into a helper function. The user will be confused that they must now change their __del__(deinit self) signature to __del__(var self).
Third: deconstruct is easier to explain than a deinit arg convention IMO.
deconstructtakes ownership of the object (like a move) and gives you ownership of its fields.
deinit lets you take a reference to the object only until the first time you move a field away, and prevent calling __del__ on it.
(I could be convinced on this one by some good Arthur Evans wordsmithing.)
Also, if anyone values deinit self because it’s shorter, it’s good to keep in mind that shorthands can lead to noncomposing solutions (see Java’s extends) and it’s better to have composing features even if more verbose. Also, most destructors are auto-generated; when the user is writing a __del__ manually, it’s because they want more precise control over what’s going on. Systems programming, especially explicit destructors, should value control.
Also, mostly unrelated, but it’s interesting to view __del__ not as a “destructor”. For example, it can receive a var and then just move the instance onto a free-list if it wants to. I’d encourage everyone to think of __del__ as just a poorly named “drop” function, that happens when an object needs to go out of scope (but not necessarily get destroyed per se).
TL;DR: IMO, a deconstruct statement to give ownership of the fields is the right call. It’s composable with var, is near its effects, and easier to explain.
Thanks again to everyone here for bringing up all these great points, these are all aspects I didn’t consider until hearing y’all talk about it.
PS. Still considering deinit var self… I don’t think it’s as good as the deconstruct statement (for lots of the above reasons), but it could be a good mitigation for deinit’s downsides.
Why don’t we give the denit convention a few months to see whether the speculated problems occur in real-world use. Modular writes a lot of Mojo code; I’m sure issues with the current design (if any) will become obvious pretty quickly.
I don’t think the case is as simple as lattner makes it out to be. Unlike rust, mojo requires all fields to be initialized at the end of a function, even if there is no del.
In functions with control flow there is a difference between __disable_del and deinit.
Also, not running the destructor in most cases is fine, since memory gets cleaned up after the program exits. In cases where if matters, e.g. files, I think an attribute could be added to enforce the destructor.
Yes, absolutely, imperative is absolutely possible and (as you say) Mojo used to do it this way.
I agree that ASAP destruction “works” with an imperative approach. I’m arguing that ASAP destruction makes it confusing and problematic to be imperative. There is a good reason to have something be a statement: it’s because the position of that statement in code matters, and you want the developer to manipulate and control that position. This isn’t how disable_del works - the position doesn’t matter.
First: __del__(deinit self) is unnecessarily coupled to argument conventions. It raises the question of “is deinit owning, like a var?” These are two orthogonal concepts that should compose, yet we’re raising questions about how they overlap (though, deinit var self addresses that).
This is an argument against the existing spelling, not an argument against it being declarative. A declarative implementation as a function decorator (@deinit fn foo(var self): ...) doesn’t have this problem and is still declarative. As you say deinit var self is another possible spelling that is still declarative.
But imperative statement doesn’t give you that! ASAP destruction puts the dels where they are supposed to go. This appears to be a key misunderstanding that is consistently discussed in this thread. Simple example:
Notice that the inserted dels have nothing to do with the position of the disable_del.
This is my key point, so I’ll restate it for clarity:
The position of a disable_del statement doesn’t matter, if used, it can always be at the bottom of a function. Given this, IMO making it a statement doesn’t make sense. Imperative statements exist because the position in the function matters.
Thanks! I agree that__disable_del was indeed implemented to not care about the position. It could have been implemented other ways though.
The position of a disable_del statement doesn’t matter, if used, it can always be at the bottom of a function. Given this, IMO making it a statement doesn’t make sense. Imperative statements exist because the position in the function matters.
I could have been clearer here: for the deconstruct I’m talking about, the position actually would matter. It would be after the user is done with self as a whole object, and before they can move the fields away (e.g. self.x^.some_method()).
That way, it can become a simple concept: “it takes ownership of the object, and gives you ownership of its fields.”
I think users will appreciate an easy-to-explain mechanism like that.
I also like it because it maps cleanly from their existing knowledge of Mojo semantics: you can’t use something after it’s moved, and you can’t use something before you have it.
(PS. For the record, I don’t actually think deinit var self is the best approach here, I’m just saying it’s a bit nicer than deinit self because it shows how var and deinit compose cleanly.)
It turns out that the position of deconstruct is actually significant, so maybe this will change your mind r.e. whether it should be a statement.
deconstruct would be usable on a mut argument, not just a var argument. As Evan says, it is best understood as ending the lifetime of the argument’s value, similar to a transfer ^. Just as with a transfer, we can reinitialize the argument afterward.
Example:
struct Foo:
var x: String
var y: String
fn extract_values(mut self, out result: List[String]):
deconstruct self
result = [self.x^, self.y^]
# self is now considered uninitialized; so it can be assigned a new value
self = get_next_value()
In other words, deconstruct is not equivalent to writing __disable_del at the end of the function. It’s much more expressive than that!
It’s worth repeating: deconstruct is not a new syntax for __disable_del. It’s an entirely different feature. In my opinion, this feature is simple, powerful, and will be easy to teach.
Ownership of an value should not mean ownership of destructors in my opinion.
users should not know how an type works just to use it.
Maybe knowing just enough to add an new useful method feature to it. But should know exactly how it works to create an additional destructor.
If the goal of deconstructing is to get rid of the destructor,
quickly recombine parts in an few lines (inline),
how does that help with facilitating team work on an massive scale?
Many of the examples we studied here are about simple structs that seem to be only data,
but often types are more than data, and the teardown/destructor have an huge part.
Keeping custom destructor logics self-contained and well explained helps a lot.
This is why i think deinit is great.
What we program today is fresh in our minds,
but in 5 years we might look uppon some of our work,
and refer to it as old times!
We’d be probably be glad to have custom teardowns well explained in self-contained methods.
Please let’s not intertwine simple implementations with custom destructors.
let’s keep that contained into deinit methods.
And we can always add one with an extension
This is just my opinion, appreciating contributions of all!
I feel like there is enough brilliant people to create new amazing features like ref.
The position of a disable_del statement doesn’t matter, if used, it can always be at the bottom of a function. Given this, IMO making it a statement doesn’t make sense. Imperative statements exist because the position in the function matters.
I could have been clearer here: for the deconstruct I’m talking about, the position actually would matter. It would be after the user is done with self as a whole object, and before they can move the fields away (e.g. self.x^.some_method()).
I think I’m still leaning towards deconstruct x being a mechanism to ask the compiler to not call x.__del__(). We can already move fields away without this keyword, we just need to put them back so that x is in a properly initialized state at the end of the function. We can explain this to users this way:
In Mojo, you are required to initialize every field of a struct in __init__, __moveinit__, and __copyinit__. However, this actually stems from a more general property. In Mojo, functions are required to make sure all fields of their return value and all mutable arguments are initialized before they return. This typically shows up in the initialization functions, since you get an uninitialized self. However, these functions are not special. If you have a mut reference to a value or ownership of it via var or deconstruct, you can move fields away from a struct so long as you replace them, re-initializing that field, if the uninitialized field could be observed by another function. This is done to preserve local reasoning, so you the programmer know that whenever you construct an object or call a function with a mutable reference, the value is always in a valid state when control flow returns to the caller. Additionally, you may not call a function with a partially uninitialized value, avoiding the problem of having invalid arguments. However, when you are deconstructing a value, no other function will see that value again. This means that you can safely leave fields uninitialized, and the compiler will make sure you don’t trip up inside of that deconstructor. You do this via the deconstruct keyword, which stops the compiler from inserting the function call to __del__. The reason you need to re-initialize fields on values you own is because __del__ is a function call inserted by the compiler to clean up values, and the rule that you can’t call a function with a partially initialized value is still enforced. Since you aren’t returning self from a deconstructor, and you aren’t calling any functions with self, all of the compiler’s checks pass and you can leave self uninitalized. This mechanism is also how you deconstruct linear types, which have a special __del__ that causes a compiler error if it’s ever called. Linear types and why they are useful are explained later in the Mojo manual.
IMPORTANT: deconstruct x does not disable the destructors of x’s fields, and the compiler will insert __del__ calls for each of them.
More examples can be threaded throughput, so this shouldn’t be considered a final explanation, but I think that, with this view of deconstruct, where the keyword goes matters much less, and it becomes more a matter of how you like to read code (Do you prefer to know that something is going to be deconstructed up front so you can look at fields being moved away or do you want to it as more of a “I am done moving all of the fields away”).
It does give us a good name for the unsafe escape hatch: __unsafe_deconstruct_recursive.
Regarding syntax, I am curious what your opinion towards field decorators is, i.e. @deinit is. I think this would make mojo more consistent and extensible. Also I don’t think deinit is useful enough, syntactically and functionally, to be hardcoded it into to the compiler. The decorator would just be defined to emit __deinit(self) on the first line.
