DISCLAIMER: I know that this looks a bit like a bug report, but as far as I can tell it isn’t (yet). Therefore I am posting it here on the forum, because my intention is to first discuss and investigate the problem further, before posting a detailed (hopefully complete) bug report to the
modular/modularrepo. If I should do something differently I am open for your feedback and happy to follow your advise!
While working on Larecs🌲 we found a discrepancy between the behavior of code executed with mojo test and mojo run. We identified that a value passed to a constructor (Query.__init__) had different values right before the constructor was called and inside at the start of the constructor. We observed this behavior only when using mojo test to execute the code, during mojo run everything worked as expected!
Code (abbreviated)
full code can be found here
...
def without_bar():
world = World[Foo, Bar, Zork]()
_ = world.add_entities(Foo(0), count=n())
_ = world.add_entities(Foo(0), Bar(0), count=n())
query = world.query[Foo]().without[Bar]()
count = 0
for _ in query:
count += 1
assert_equal(count, n())
def without_zork():
world = World[Foo, Bar, Zork]()
_ = world.add_entities(Foo(0), count=n())
_ = world.add_entities(Foo(0), Zork(0), count=n())
_ = world.add_entities(Foo(0), Bar(0), Zork(0), count=n())
query = world.query[Foo]().without[Zork]()
count = 0
for _ in query:
count += 1
assert_equal(count, n())
for _ in range(n()):
_ = world.add_entity(Foo(0), Zork(0))
count = 0
for _ in query:
count += 1
assert_equal(count, 2 * n())
def execute():
print("Run without_bar()...")
without_bar()
print("Run without_zork()...")
without_zork()
def test_execute():
print(compile.compile_info[execute, emission_kind="llvm-opt"]())
execute()
...
Test execution command
pixi run mojo test -D ASSERT=all -I src/ test/mac_bug_test.mojo
Observed output (without LLVM dump…
)
Run without_bar()...
Query.__iter__ <before constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
_EntityIterator.__init__ <inside constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
Run without_zork()...
Query.__iter__ <before constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
_EntityIterator.__init__ <inside constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
Unhandled exception caught during execution: At /Volumes/Programming/tree-d/larecs/test/mac_bug_test.mojo:49:17: AssertionError: `left == right` comparison failed:
left: 20
right: 10
As can be seen, the - Mask (a bitmask with 256 bits SIMD[UInt8, 32]) is different before the call to _EntityIterator.__init__ and inside the called constructor.
Code calling _EntityIterator.__init__
# src/larecs/query.mojo:84
struct BitMask:
var value: SIMD[UInt8, 32]
struct Query[has_without_mask: Bool]:
var _mask: BitMask
var _without_mask: StaticOptional[BitMask, has_value=has_without_mask]
...
fn __iter__(self, out iterator: _EntityIterator[...]) raises:
print("Query.__iter__ <before constructor call>")
print("+ Mask " + self._mask.__str__())
@parameter
if Self.has_without_mask:
print("- Mask " + self._without_mask[].__str__())
else:
print("- Mask <None>")
iterator = self._world[].Iterator(
Pointer(to=self._world[]._archetypes),
Pointer(to=self._world[]._locks),
self._mask,
self._without_mask,
)
Start of __EntityIterator.__init__
# src/larecs/query.mojo:491
struct _EntityIterator:z
fn __init__(
out self,
archetypes: Pointer[List[Self.Archetype], archetype_origin],
lock_ptr: Pointer[LockMask, lock_origin],
owned mask: BitMask,
owned without_mask: StaticOptional[BitMask, has_without_mask] = None,
owned start_indices: Self.StartIndices = None,
) raises:
print("_EntityIterator.__init__ <inside constructor call>")
print("+ Mask " + mask.__str__())
@parameter
if has_without_mask:
print("- Mask " + without_mask[].__str__())
else:
print("- Mask <None>")
...
Minimal reproducer
- Unfortunately I could not produce a small isolated reproducer of the bug. All my attempts lead to eventually depending on code from Larecs.
- This is the minimal code I could come up with, that shows the buggy behavior: larecs/test/mac_bug_test.mojo at a898ccee0a15c06fcb80bf9b2ed92f9cd2bdaccf · samufi/larecs · GitHub
- Some learnings from trying to reduce the code:
- There must be two
Querys, which both have awithout_mask. - They must be constructed in to different functions (don’t need to be different test cases, just different functions).
- One
Queryneeds to be iterated more than once (in this case theQueryfromwithout_zork). - The execution order of the two functions
without_zorkandwithout_baris not important.without_zorkis always the failing function!
- There must be two
Investigations done so far
- Comparison of bitmask value through print statements
- Execution of
test/mac_bug_test.mojowith different commands:pixi run mojo test -D ASSERT=all -I src/ test/mac_bug_test.mojopixi run mojo test -I src/ test/mac_bug_test.mojopixi run mojo run -I src/ test/mac_bug_test.mojo
- Inspection of generated LLVM IR (by using
compile.compile_info()) - Tested on Linux – everything looked fine there and worked as expected🤷
Observations
Different execution contexts
| Command | BitMask mutated | Test Outcome |
|---|---|---|
mojo test -D ASSERT=all |
yes |  |
mojo test |
no |  |
mojo run |
no | |
LLVM IR inspection
I am by no means a compiler engineer or claim in any way to be an expert for LLVM. Out of curiosity and hope, that I’d find a hint for the causes of the previously described problem, I looked a bit into the optimized LLVM IR produced by the compile module. Maybe this is completely wrong and this IR isn’t at all what the mojo test command executes, but here are my findings anyway.
My conclusion was, that all the calls of the constructor _EntityIterator.__init__ receive the BitMask (that is somehow mutated while passed to the constructor) as a literal?!
(The full LLVM IR dump is attached at the end)
; bug.ll:2252
%1056 = call { i1, { ptr, i64 }, { ptr, ptr, i8, i64, i64, i64, { { [0 x { ptr, i64, i64 }] } }, { { [0 x i64] } }, { ptr, <8 x i32>, <32 x i8>, { { [1 x <32 x i8>] } }, i64, i64, i64 } } } @"larecs::query::_EntityIterator::__init__(::Pointer[$0, ::List[larecs::archetype::Archetype[$3, $4], ::Bool(False)], $1, ::AddressSpace(::Int(0))],::Pointer[::Bool(True), larecs::lock::LockMask, $2, ::AddressSpace(::Int(0))],larecs::bitmask::BitMask,larecs::static_optional::StaticOptional[larecs::bitmask::BitMask, $5],larecs::static_optional::StaticOptional[::List[::UInt, ::Bool(True)], $6])_REMOVED_ARG,ComponentTypes=[[typevalue<#kgen.instref<\1B\22mac_bug_test::Foo\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Bar\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Zork\22>>, struct<(scalar<ui8>) memoryOnly>]],has_without_mask=1,has_start_indices=0"(ptr nonnull %46, ptr nonnull %45, <32 x i8> <i8 1, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, { { [1 x <32 x i8>] } } { { [1 x <32 x i8>] } { [1 x <32 x i8>] [<32 x i8> <i8 4, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>] } })
; bug.ll:3474
%1635 = call { i1, { ptr, i64 }, { ptr, ptr, i8, i64, i64, i64, { { [0 x { ptr, i64, i64 }] } }, { { [0 x i64] } }, { ptr, <8 x i32>, <32 x i8>, { { [1 x <32 x i8>] } }, i64, i64, i64 } } } @"larecs::query::_EntityIterator::__init__(::Pointer[$0, ::List[larecs::archetype::Archetype[$3, $4], ::Bool(False)], $1, ::AddressSpace(::Int(0))],::Pointer[::Bool(True), larecs::lock::LockMask, $2, ::AddressSpace(::Int(0))],larecs::bitmask::BitMask,larecs::static_optional::StaticOptional[larecs::bitmask::BitMask, $5],larecs::static_optional::StaticOptional[::List[::UInt, ::Bool(True)], $6])_REMOVED_ARG,ComponentTypes=[[typevalue<#kgen.instref<\1B\22mac_bug_test::Foo\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Bar\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Zork\22>>, struct<(scalar<ui8>) memoryOnly>]],has_without_mask=1,has_start_indices=0"(ptr nonnull %46, ptr nonnull %45, <32 x i8> <i8 1, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, { { [1 x <32 x i8>] } } { { [1 x <32 x i8>] } { [1 x <32 x i8>] [<32 x i8> <i8 4, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>] } })
; bug.ll:17816
%417 = call { i1, { ptr, i64 }, { ptr, ptr, i8, i64, i64, i64, { { [0 x { ptr, i64, i64 }] } }, { { [0 x i64] } }, { ptr, <8 x i32>, <32 x i8>, { { [1 x <32 x i8>] } }, i64, i64, i64 } } } @"larecs::query::_EntityIterator::__init__(::Pointer[$0, ::List[larecs::archetype::Archetype[$3, $4], ::Bool(False)], $1, ::AddressSpace(::Int(0))],::Pointer[::Bool(True), larecs::lock::LockMask, $2, ::AddressSpace(::Int(0))],larecs::bitmask::BitMask,larecs::static_optional::StaticOptional[larecs::bitmask::BitMask, $5],larecs::static_optional::StaticOptional[::List[::UInt, ::Bool(True)], $6])_REMOVED_ARG,ComponentTypes=[[typevalue<#kgen.instref<\1B\22mac_bug_test::Foo\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Bar\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Zork\22>>, struct<(scalar<ui8>) memoryOnly>]],has_without_mask=1,has_start_indices=0"(ptr nonnull %32, ptr nonnull %31, <32 x i8> <i8 1, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, { { [1 x <32 x i8>] } } { { [1 x <32 x i8>] } { [1 x <32 x i8>] [<32 x i8> <i8 2, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>] } })
Conclusions / Hypothesis
mojo testandmojo rundiffer in compilation options (optimization levels, test instrumentation, linking) which could change codegen.- Test-runner instrumentation or different runtime in
testmode introduces aliasing or unintended writes. - macOS-specific codegen/runtime behavior/ABI issue.
- A bug in the optimizer that is only triggered for the test build configuration on macOS.
Open Questions / Next Steps?
- Do you share our conclusions so far?
- What are the exact differences between
mojo runandmojo testin terms of compilation, optimization levels, and runtime instrumentation? Are different optimization strategies used by default? - Does the test runner perform extra transformations or link with different runtime support that could lead to mutation or aliasing differences?
- Are there known macOS-specific codegen/runtime issues that could explain this behavior?
- Would maintainers like me to:
- produce assembly diffs for
testvsrun, - run
mojo testwith optimizations disabled (e.g.,-O0),
- produce assembly diffs for
- Any other recommended immediate steps I should try?
Attachements
Environment
- Hardware: MacBook Air M4 2025
- OS: macOS 15.6.1 (24G90)
- Mojo version: 25.05
Exact steps to reproduce
- Clone or create repo: GitHub - samufi/larecs: Larecs🌲 – a performance-oriented archetype-based ECS
- Check out branch
mac-bug - Run (test):
mojo test -D ASSERT=all -I src/ test/mac_bug_test.mojo
- Observe behavior
- Optional: Run
mojo runfor comparison:
mojo run -I src/ test/mac_bug_test.mojo