How to specify origin for list elements

Mojo
1

I am trying capture the address of a Vector(s) in Container struct as shown below. These Containers I am planning to keep in a List[Container].


struct Container[mut: Bool, //,  origin: Origin[mut], dtype: DType]:
    var address: Pointer[Vector[dtype], origin]

    fn __init__(out self, ref [origin] vector: Vector[dtype]):
        self.address = Pointer(to=vector)

    fn __copyinit__(out self, other: Self):
        self.address = other.address

    fn pointee(self) -> Vector[dtype]:
        return self.address[].copy()

I am able to do things like:


v = Vector[DType.int32](1, 2, 3, 4)
container = Container(v)

But when I want to put the ‘container’ into a list, I am not sure how to go about that.

What I am trying:


l = List[Container[ImmutableAnyOrigin, DType.int32]]()]]  # Not sure  - what should be here! Or how to specify it!
l.append(container)

For the above I get the following error:


error: invalid call to 'append': method argument #0 cannot be converted from 'Container[v, DType.float32]' to 'Container[ImmutableAnyOrigin, DType.float32]'
  l.append(container)

Please enlighten!

2

I explore what you can (almost) do below and explain what is going on. That said, there isn’t a really good way to do this - you’re trying to make a collection that references various things on the stack and the compiler needs to understand that.

This is going to be complicated and annoying, but if you really want to do it, you can "almost” do it today. Let me dive in. Here are your types mocked out so they compile. Note that I made Container copyable and movable so it can work with List:

struct Vector[dt: DType]:
  fn __init__(out self, a: Int, b: Int, c: Int, d: Int): pass

struct Container[mut: Bool, //,  origin: Origin[mut], dtype: DType](Copyable, Movable):
    var address: Pointer[Vector[dtype], origin]

    fn __init__(out self, ref [origin] vector: Vector[dtype]):
        self.address = Pointer(to=vector)

    fn __copyinit__(out self, other: Self):
        self.address = other.address

Ok, now lets figure out how to put multiple things into the list, this looks like this:


fn main():
   # 1)
   v = Vector[DType.int32](1, 2, 3, 4)
   v2 = Vector[DType.int32](5, 6, 7, 8)

   # 2)
   container = Container[origin_of(v,v2)](v)
   container2 = Container[origin_of(v,v2)](v2)

   #3)
   l = List[Container[origin_of(v, v2), DType.int32]]()

   #4)
   l.append(container^)
   l.append(container2^)

#1 declares the two vectors we want to reference from the list.

#3 declares a list containing elements of your Container type, and uses originof to say that the elements can point to either v or v2. This is the literal answer to your question.

However, one annoying thing is that Container[origin_of(v)]is not implicitly convertible to Container[origin_of(v, v2)] - these are different types and Mojo doesn’t “know” they are related. We want this sort of thing to be implicitly convertible (e.g. UnsafePointer has the same issue) and the way to do this is to define an @implicit conversion. I /think/ we have the ability to do this (or the near ability) now that where clauses are coming in, but I haven’t tried it. To work around this, I declared container and container2 with explicitly origin_of clauses in #2. The “ref” argument in the initializer does support this implicit conversion (from an origin to a super set origin) so this approach works.

If you give this code example a try with the current nightly, you’ll see that it “almost” works, in that it type checks. The issue is that the compiler pukes on it due to exclusivity checking:


c.mojo:26:12: error: argument of 'append' call allows writing a memory location previously writable through another aliased argument
   l.append(container^)
           ^
c.mojo:26:12: note: 'v2' memory accessed through reference embedded in value of type 'Container[origin_of(v2, v), DType.int32]'

Mojo is complaining because it knows that “l” may point to v or v2, and that you’re passing in something else that could point to “v” or “v2”, and it doesn’t know whether append is potentially causing an aliasing access. We know it isn’t but Mojo doesn’t.

There is no good way to solve this error with today’s mojo, we need to continue to extend the type system to support this (this is related to the work to remove the @__unsafe_disable_nested_origin_exclusivity hack). In the meantime, I’d recommend not doing this - instead use value semantics instead of references. If you “really really” need to do this, you can hackaround this by using something like ImmutableAnyOrigin.

-Chris

2 Likes
3

Thank you, Chris! This is an incredibly clear explanation and perfectly addresses my confusion. Really appreciate you taking the time to help with this.

1 Like