Unexpected argument mutation with `mojo test` — not under `mojo run`

moseschmiedel · August 29, 2025, 7:51pm

DISCLAIMER: I know that this looks a bit like a bug report, but as far as I can tell it isn’t (yet). Therefore I am posting it here on the forum, because my intention is to first discuss and investigate the problem further, before posting a detailed (hopefully complete) bug report to the modular/modular repo. If I should do something differently I am open for your feedback and happy to follow your advise!

While working on Larecs🌲 we found a discrepancy between the behavior of code executed with mojo test and mojo run. We identified that a value passed to a constructor (Query.__init__) had different values right before the constructor was called and inside at the start of the constructor. We observed this behavior only when using mojo test to execute the code, during mojo run everything worked as expected!

Code (abbreviated)

full code can be found here

...

def without_bar():
    world = World[Foo, Bar, Zork]()
    _ = world.add_entities(Foo(0), count=n())
    _ = world.add_entities(Foo(0), Bar(0), count=n())

    query = world.query[Foo]().without[Bar]()
    count = 0
    for _ in query:
        count += 1
    assert_equal(count, n())


def without_zork():
    world = World[Foo, Bar, Zork]()
    _ = world.add_entities(Foo(0), count=n())
    _ = world.add_entities(Foo(0), Zork(0), count=n())
    _ = world.add_entities(Foo(0), Bar(0), Zork(0), count=n())

    query = world.query[Foo]().without[Zork]()
    count = 0
    for _ in query:
        count += 1
    assert_equal(count, n())

    for _ in range(n()):
        _ = world.add_entity(Foo(0), Zork(0))

    count = 0
    for _ in query:
        count += 1
    assert_equal(count, 2 * n())


def execute():
    print("Run without_bar()...")
    without_bar()
    print("Run without_zork()...")
    without_zork()


def test_execute():
    print(compile.compile_info[execute, emission_kind="llvm-opt"]())
    execute()

...

Test execution command

pixi run mojo test -D ASSERT=all -I src/ test/mac_bug_test.mojo

Observed output (without LLVM dump…)

Run without_bar()...
Query.__iter__ <before constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
_EntityIterator.__init__ <inside constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
Run without_zork()...
Query.__iter__ <before constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
_EntityIterator.__init__ <inside constructor call>
+ Mask [1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
- Mask [0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]
Unhandled exception caught during execution: At /Volumes/Programming/tree-d/larecs/test/mac_bug_test.mojo:49:17: AssertionError: `left == right` comparison failed:
   left: 20
  right: 10

As can be seen, the - Mask (a bitmask with 256 bits SIMD[UInt8, 32]) is different before the call to _EntityIterator.__init__ and inside the called constructor.

Code calling `_EntityIterator.init`

Permalink

# src/larecs/query.mojo:84

struct BitMask:
	var value: SIMD[UInt8, 32]

struct Query[has_without_mask: Bool]:
	var _mask: BitMask
	var _without_mask: StaticOptional[BitMask, has_value=has_without_mask]
	
	...
	
	fn __iter__(self, out iterator: _EntityIterator[...]) raises:
		print("Query.__iter__ <before constructor call>")
		print("+ Mask " + self._mask.__str__())
		
		@parameter
		if Self.has_without_mask:
			print("- Mask " + self._without_mask[].__str__())
		else:
			print("- Mask <None>")
		iterator = self._world[].Iterator(
			Pointer(to=self._world[]._archetypes),
			Pointer(to=self._world[]._locks),
			self._mask,
			self._without_mask,
		)

Start of `EntityIterator.init__`

Permalink

# src/larecs/query.mojo:491

struct _EntityIterator:z
    fn __init__(
        out self,
        archetypes: Pointer[List[Self.Archetype], archetype_origin],
        lock_ptr: Pointer[LockMask, lock_origin],
        owned mask: BitMask,
        owned without_mask: StaticOptional[BitMask, has_without_mask] = None,
        owned start_indices: Self.StartIndices = None,
    ) raises:
		        print("_EntityIterator.__init__ <inside constructor call>")
        print("+ Mask " + mask.__str__())

        @parameter
        if has_without_mask:
            print("- Mask " + without_mask[].__str__())
        else:
            print("- Mask <None>")
        ...

Minimal reproducer

Unfortunately I could not produce a small isolated reproducer of the bug. All my attempts lead to eventually depending on code from Larecs.
This is the minimal code I could come up with, that shows the buggy behavior: larecs/test/mac_bug_test.mojo at a898ccee0a15c06fcb80bf9b2ed92f9cd2bdaccf · samufi/larecs · GitHub
Some learnings from trying to reduce the code:
- There must be two Querys, which both have a without_mask.
- They must be constructed in to different functions (don’t need to be different test cases, just different functions).
- One Query needs to be iterated more than once (in this case the Query from without_zork).
- The execution order of the two functions without_zork and without_bar is not important. without_zork is always the failing function!

Investigations done so far

Comparison of bitmask value through print statements
Execution of test/mac_bug_test.mojo with different commands:
- pixi run mojo test -D ASSERT=all -I src/ test/mac_bug_test.mojo
- pixi run mojo test -I src/ test/mac_bug_test.mojo
- pixi run mojo run -I src/ test/mac_bug_test.mojo
Inspection of generated LLVM IR (by using compile.compile_info())
Tested on Linux – everything looked fine there and worked as expected🤷

Observations

Different execution contexts

Command	BitMask mutated	Test Outcome
`mojo test -D ASSERT=all`	yes
`mojo test`	no
`mojo run`	no

LLVM IR inspection

I am by no means a compiler engineer or claim in any way to be an expert for LLVM. Out of curiosity and hope, that I’d find a hint for the causes of the previously described problem, I looked a bit into the optimized LLVM IR produced by the compile module. Maybe this is completely wrong and this IR isn’t at all what the mojo test command executes, but here are my findings anyway.

My conclusion was, that all the calls of the constructor _EntityIterator.__init__ receive the BitMask (that is somehow mutated while passed to the constructor) as a literal?!
(The full LLVM IR dump is attached at the end)

; bug.ll:2252

%1056 = call { i1, { ptr, i64 }, { ptr, ptr, i8, i64, i64, i64, { { [0 x { ptr, i64, i64 }] } }, { { [0 x i64] } }, { ptr, <8 x i32>, <32 x i8>, { { [1 x <32 x i8>] } }, i64, i64, i64 } } } @"larecs::query::_EntityIterator::__init__(::Pointer[$0, ::List[larecs::archetype::Archetype[$3, $4], ::Bool(False)], $1, ::AddressSpace(::Int(0))],::Pointer[::Bool(True), larecs::lock::LockMask, $2, ::AddressSpace(::Int(0))],larecs::bitmask::BitMask,larecs::static_optional::StaticOptional[larecs::bitmask::BitMask, $5],larecs::static_optional::StaticOptional[::List[::UInt, ::Bool(True)], $6])_REMOVED_ARG,ComponentTypes=[[typevalue<#kgen.instref<\1B\22mac_bug_test::Foo\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Bar\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Zork\22>>, struct<(scalar<ui8>) memoryOnly>]],has_without_mask=1,has_start_indices=0"(ptr nonnull %46, ptr nonnull %45, <32 x i8> <i8 1, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, { { [1 x <32 x i8>] } } { { [1 x <32 x i8>] } { [1 x <32 x i8>] [<32 x i8> <i8 4, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>] } })

; bug.ll:3474

%1635 = call { i1, { ptr, i64 }, { ptr, ptr, i8, i64, i64, i64, { { [0 x { ptr, i64, i64 }] } }, { { [0 x i64] } }, { ptr, <8 x i32>, <32 x i8>, { { [1 x <32 x i8>] } }, i64, i64, i64 } } } @"larecs::query::_EntityIterator::__init__(::Pointer[$0, ::List[larecs::archetype::Archetype[$3, $4], ::Bool(False)], $1, ::AddressSpace(::Int(0))],::Pointer[::Bool(True), larecs::lock::LockMask, $2, ::AddressSpace(::Int(0))],larecs::bitmask::BitMask,larecs::static_optional::StaticOptional[larecs::bitmask::BitMask, $5],larecs::static_optional::StaticOptional[::List[::UInt, ::Bool(True)], $6])_REMOVED_ARG,ComponentTypes=[[typevalue<#kgen.instref<\1B\22mac_bug_test::Foo\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Bar\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Zork\22>>, struct<(scalar<ui8>) memoryOnly>]],has_without_mask=1,has_start_indices=0"(ptr nonnull %46, ptr nonnull %45, <32 x i8> <i8 1, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, { { [1 x <32 x i8>] } } { { [1 x <32 x i8>] } { [1 x <32 x i8>] [<32 x i8> <i8 4, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>] } })

; bug.ll:17816

%417 = call { i1, { ptr, i64 }, { ptr, ptr, i8, i64, i64, i64, { { [0 x { ptr, i64, i64 }] } }, { { [0 x i64] } }, { ptr, <8 x i32>, <32 x i8>, { { [1 x <32 x i8>] } }, i64, i64, i64 } } } @"larecs::query::_EntityIterator::__init__(::Pointer[$0, ::List[larecs::archetype::Archetype[$3, $4], ::Bool(False)], $1, ::AddressSpace(::Int(0))],::Pointer[::Bool(True), larecs::lock::LockMask, $2, ::AddressSpace(::Int(0))],larecs::bitmask::BitMask,larecs::static_optional::StaticOptional[larecs::bitmask::BitMask, $5],larecs::static_optional::StaticOptional[::List[::UInt, ::Bool(True)], $6])_REMOVED_ARG,ComponentTypes=[[typevalue<#kgen.instref<\1B\22mac_bug_test::Foo\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Bar\22>>, struct<(scalar<ui8>) memoryOnly>], [typevalue<#kgen.instref<\1B\22mac_bug_test::Zork\22>>, struct<(scalar<ui8>) memoryOnly>]],has_without_mask=1,has_start_indices=0"(ptr nonnull %32, ptr nonnull %31, <32 x i8> <i8 1, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, { { [1 x <32 x i8>] } } { { [1 x <32 x i8>] } { [1 x <32 x i8>] [<32 x i8> <i8 2, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>] } })

Conclusions / Hypothesis

mojo test and mojo run differ in compilation options (optimization levels, test instrumentation, linking) which could change codegen.
Test-runner instrumentation or different runtime in test mode introduces aliasing or unintended writes.
macOS-specific codegen/runtime behavior/ABI issue.
A bug in the optimizer that is only triggered for the test build configuration on macOS.

Open Questions / Next Steps?

Do you share our conclusions so far?
What are the exact differences between mojo run and mojo test in terms of compilation, optimization levels, and runtime instrumentation? Are different optimization strategies used by default?
Does the test runner perform extra transformations or link with different runtime support that could lead to mutation or aliasing differences?
Are there known macOS-specific codegen/runtime issues that could explain this behavior?
Would maintainers like me to:
- produce assembly diffs for test vs run,
- run mojo test with optimizations disabled (e.g., -O0),
Any other recommended immediate steps I should try?

Attachements

Environment

Hardware: MacBook Air M4 2025
OS: macOS 15.6.1 (24G90)
Mojo version: 25.05

Exact steps to reproduce

Clone or create repo: GitHub - samufi/larecs: Larecs🌲 – a performance-oriented archetype-based ECS
Check out branch mac-bug
Run (test):

mojo test -D ASSERT=all -I src/ test/mac_bug_test.mojo

Observe behavior
Optional: Run mojo run for comparison:

mojo run -I src/ test/mac_bug_test.mojo

LLVM IR

gist.github.com

https://gist.github.com/moseschmiedel/0b152896a1cc17bcf76a2659569dd810

bug.ll

; Testing Time: 4.726s

; Total Discovered Tests: 1

; Passed : 0 (0.00%)
; Failed : 1 (100.00%)
; Skipped: 0 (0.00%)

; ******************** Failure: '/Volumes/Programming/tree-d/larecs/test/mac_bug_test.mojo::test_execute()' ********************

This file has been truncated. show original

npanchen · September 18, 2025, 6:34pm

Small update while I’m building old compiler I can debug: the problem seems to unrelated to mojo test or mojo run and is more related to use of -D ASSERT=all.

Regardless how test is executed mojo run, mojo test or mojo build + execute, result is incorrect if assertions are enabled.

It also seems that root of the problem is in invalid IR as with IR verification enabled compiler crashes with

Assertion failed: (isa<To>(Val) && "cast<Ty>() argument of incompatible type!"), function cast

That said, it looks like a compiler bug. Still need to check if it has been fixed or needs to be fixed.

billyzhu · September 19, 2025, 7:09pm

Hey, looks like unfortunately you ran into the same issue as reported here, which gets triggered if there are inferred parameters on a struct method while the struct has variadic parameters (see Archetype.get_component). The proper fix for this might take some time (feel free to follow the linked issue), but in the meantime, some possible workarounds I can think of are to either have the IndexType not be inferred, or write the method as a standalone function. Hopefully this helps to at least unblock while we work on fixing it.

Topic		Replies	Views
Bugs: Mojo Test - Crashes when non-test file contains a struct with a parameter Mojo discussion	12	158	October 20, 2025
I have discovered a suspect efficiency anomaly in the mojo compiler, how to proceed? Mojo discussion , mojo-compiler , 25_1	20	334	March 8, 2025
Proposal: deprecating `mojo test` Mojo	8	400	October 30, 2025
Mojo test command not working Mojo discussion	4	90	August 20, 2025
Exploring Metaprogramming in Mojo Content	1	557	May 28, 2025