What replaces public method some_dict.get_ptr(key)?
I see a note in the commit, “This eliminates the extraneous get_ptr() method which is unnecessary now. _find_ref() is enough.”
By convention, the leading underscore signifies a private method that may change in the future, correct? Removal of get_ptr breaks my code in several critical places, so I’d like a fix that does not rely on another interim solution. Essentially, how do I access the actual objects stored in a dictionary (i.e. not a copy)? Thanks!
Discord post: Discord
Github commit: [mojo-stdlib] Simplify some code in dict.mojo and remove fixmes · modular/modular@1e12d81 · GitHub
I think, that it should be added that you want to obtatn reference to an object, no its copy (performance?) if I understand corrextly
1 Like
Exactly!
Essentially, how do I access the actual objects stored in a dictionary (i.e. not a copy)?
Here’s a similar issue with a (temporary) solution using get_ptr method.
Looking at dict.mojo on GitHub, it appears that fn find, fn setdefault, and fn __getitem__ now return a reference to a value stored in a dictionary. This is a change from previous behaviour, correct? Specifically, does fn __getitem__ no longer returns a copy of the value, but a reference? I have not experimented to confirm this yet.
dict.mojo: modular/mojo/stdlib/stdlib/collections/dict.mojo at 1e12d811fb9c38d0c1620be88f47bfea810e8003 · modular/modular · GitHub
struct Dict[...]:
fn _find_ref(
ref self, key: K
) raises -> ref [self._entries[0].value().value] Self.V:
"""Find a value in the dictionary by key.
Args:
key: The key to search for in the dictionary.
Returns:
An optional value containing a reference to the value if it is
present, otherwise an empty Optional.
"""
Yes this is a reference
I need a reference to ‘bar’ from the Dict, not a copy. Public method ‘fn get_ptr’ provided this access to the object in the Dict.
fn main():
foo_dict = Dict[String, Int]()
foo_dict["bar"] = 1
try:
print("1.1) foo_dict['bar'] =", foo_dict["bar"]) # 1 OK
foo_dict["bar"] += 3
print("1.2) foo_dict['bar'] =", foo_dict["bar"]) # 4 OK
# I need a reference to 'bar' from the Dict, not a copy.
# 'get_ptr' provided this access to the object in the Dict.
# Below, 'bar' is a copy of foo_dict["bar"].
bar = foo_dict["bar"]
bar += 5
print("1.3) bar =", bar)
print("1.4) foo_dict['bar'] =", foo_dict["bar"], " <-- KO") # 4 KO (should be 9)
except e:
pass
Output:
1.1) foo_dict['bar'] = 1
1.2) foo_dict['bar'] = 4
1.3) bar = 9
1.4) foo_dict['bar'] = 4 <-- KO
The comment for fn _find_ref in struct Dict says it returns an Optional, but evidence suggests otherwise. I definitely need some help here. TIA!
The assignment bar = foo_dict["bar"] implicitly copies the referenced value into bar. So bar += 5 only modifies the copied value.
1 Like
I’m sorry for breaking you! Instead of d01.get_ptr("ClientA").value() you should be able to just use d01["ClientA"] in a try block.
Hi Chris! I’m not sure it’s returning a reference. That’s why I need fn get_ptr or equivalent in the first place. In the repro below, I expect that the value at 1.3) should be 9. What am I missing?
@clattner
@value
struct Foo:
var value: Int
fn add(mut self, x: Int):
self.value += x
alias foos = Dict[String, Foo]
fn main():
foos = foos()
foos["bar"] = Foo(1)
try:
print("1.1) value =", foos["bar"].value) # 1 OK
foos["bar"].add(3)
print("1.2) value =", foos["bar"].value) # 4 OK
# Below, 'bar' is a copy of foos["bar"].
bar = foos["bar"]
bar.add(5)
print(
"1.3) value =", foos["bar"].value, " <-- KO"
) # 4 KO (should be 9)
except e:
pass
Output:
1.1) value = 1
1.2) value = 4
1.3) value = 4 <-- KO
bar = foos["bar"]
This line copies foos["bar"]. You can see its returning refrence by doing this which correctly modifies the value.In mojo = copies the value not refrence if my understanding is correct.
foos["bar"].add(5)
If you want to „reference“ the original value from another var without copying it, you can use a Pointer: bar = Pointer(to=foos[”bar”]).
To modify the original value: bar[].add(5)
That will probably give immutable pointer. If you want mutable pointer you may need to do this
bar = UnsafePointer(to=foos["bar"]).origin_cast[mut=True]()
bar[].add(5)
@clattner shouldn’t __getitem__ in Dict just return a ref by default? With autoderef this would be more performant. IMO there is no need to return copy of data here and not the data itself and this is more natural to operate on original data
1 Like
Referring to the code below, I expect/need for each add operation to affect the same object in memory, regardless of whether it’s a variable, stored in a List/Dict, or retrieved from a List/Dict and assigned to a variable. No copies, mutable by default.
I use a Dict to store complex structs that, when retrieved, must point to the same object because its internal state changes often as the application runs. This is a fundamental dependency, with memory and performance/speed secondary considerations (for now 
).
from collections import Dict
from testing import assert_true
@value
struct Foo:
var value: Int
fn add(mut self, x: Int):
self.value += x
alias DictFoos = Dict[String, Foo]
alias ListFoos = List[Foo]
alias A = 1
alias B = 3
alias C = 5
fn main():
foo = Foo(1)
dict01 = DictFoos()
dict01["bar"] = foo
list01 = ListFoos()
list01.append(foo)
try:
# Initial values
assert_true(foo.value == A, "Foo")
assert_true(dict01["bar"].value == A, "Dict")
assert_true(list01[0].value == A, "List")
foo.add(B)
# dict01["bar"].add(B) # Equivalent
# list01[0].add(B) # Equivalent
expected = A + B
print("1.1) expected =", expected)
print("1.2) foo value =", foo.value)
print("1.3) dict value =", dict01["bar"].value, "KO")
print("1.4) list value =", list01[0].value, "KO")
assert_true(foo.value == expected, "Foo_1")
assert_true(dict01["bar"].value == expected, "Dict_1")
assert_true(list01[0].value == expected, "List_1")
except e:
print(e)
Output:
1.1) expected = 4
1.2) foo value = 4
1.3) dict value = 1 KO
1.4) list value = 1 KO
Something like this maybe solution for you
from collections import Dict
from testing import assert_true
from memory import UnsafePointer
@value
struct Foo:
var value: Int
fn add(mut self, x: Int):
self.value += x
alias DictFoos = Dict[String, UnsafePointer[Foo]]
alias A = 1
alias B = 3
alias C = 5
fn main():
var foo = Foo(1)
var dict01 = DictFoos()
dict01["bar"] = UnsafePointer(to=foo)
try:
# Initial values
assert_true(foo.value == A, "Foo")
assert_true(dict01["bar"][].value == A, "Dict")
foo.add(B)
# dict01["bar"].add(B) # Equivalent
alias expected = A + B
print("1.1) expected =", expected)
print("1.2) foo value =", foo.value)
print("1.3) dict value =", dict01["bar"][].value)
assert_true(foo.value == expected, "Foo_1")
assert_true(dict01["bar"][].value == expected, "Dict_1")
except e:
print(e)
1 Like
__getitem__ returns a ref. But assigning it to another var makes a copy (this is by design).
Per @christoph_schlumpf’s comment, is the recommended/safe approach to use an ArcPointer as shown in the code below? This way, even if variable foo_ptr goes out of scope and is destroyed, the Dict and List instances each contain a valid/usable pointer to the original Foo object.
from collections import Dict
from testing import assert_true, assert_false
from memory import ArcPointer
@value
struct Foo:
var value: Int
fn add(mut self, x: Int):
self.value += x
alias DictFoos = Dict[String, ArcPointer[Foo]]
alias ListFoos = List[ArcPointer[Foo]]
alias A = 1
alias B = 3
alias C = 5
fn main():
foo_ptr = ArcPointer(Foo(1))
foo = foo_ptr[] # Makes a copy of the `Foo object`
dict01 = DictFoos()
dict01["bar"] = foo_ptr
list01 = ListFoos()
list01.append(foo_ptr)
try:
# Initial values
assert_true(foo.value == A, "Foo")
assert_true(dict01["bar"][].value == A, "Dict")
assert_true(list01[0][].value == A, "List")
# 'foo' is a copy of the original object,
# so 'add' affects only the variable 'foo'.
foo.add(B)
expected = A + B
assert_true(foo.value == expected, "Foo_1")
assert_false(dict01["bar"][].value == expected, "Dict_1")
assert_false(list01[0][].value == expected, "List_1")
print("---------------------------")
print("1.0) foo.add(B)")
print("1.1) expected =", expected)
print("1.2) foo value =", foo.value, "copy, changed")
print("1.3) dict value =", dict01["bar"][].value, "unchanged")
print("1.4) list value =", list01[0][].value, "unchanged")
# 'add' affects objects in both Dict and List.
dict01["bar"][].add(B)
assert_true(foo.value == expected, "Foo_2")
assert_true(dict01["bar"][].value == expected, "Dict_2")
assert_true(list01[0][].value == expected, "List_2")
print("---------------------------")
print("2.0) dict01['bar'][].add(B)")
print("2.1) expected =", expected)
print("2.2) foo value =", foo.value, "copy, unchanged")
print("2.3) dict value =", dict01["bar"][].value, "changed")
print("2.4) list value =", list01[0][].value, "changed")
# 'add' affects objects in both Dict and List.
list01[0][].add(C)
expected = A + B + C
assert_false(foo.value == expected, "Foo_3")
assert_true(dict01["bar"][].value == expected, "Dict_3")
assert_true(list01[0][].value == expected, "List_3")
print("---------------------------")
print("3.0) list01[0][].add(C)")
print("3.1) expected =", expected)
print("3.2) foo value =", foo.value, "copy, unchanged")
print("3.3) dict value =", dict01["bar"][].value, "changed")
print("3.4) list value =", list01[0][].value, "changed")
except e:
print(e)
Output:
1.0) foo.add(B)
1.1) expected = 4
1.2) foo value = 4 copy, changed
1.3) dict value = 1 unchanged
1.4) list value = 1 unchanged
---------------------------
2.0) dict01['bar'][].add(B)
2.1) expected = 4
2.2) foo value = 4 copy, unchanged
2.3) dict value = 4 changed
2.4) list value = 4 changed
---------------------------
3.0) list01[0][].add(C)
3.1) expected = 9
3.2) foo value = 4 copy, unchanged
3.3) dict value = 9 changed
3.4) list value = 9 changed
1 Like
Hi @EzRyder ,
Looks good to me. See Intro to pointers | Modular
The reference (and hence the pointer) that we get this way is immutable (see this issue). First I thought this is just a missing ref keyword, but now I came to guess that returning a mutable ref could cause confusion with __setitem__. If this is the case (and not trivial to fix) I think it would be great to still have some kind of get_ptr…
1 Like