`Indexer` and `Int` vs `UInt`

[owenhilyard]

Int and UInt operations don’t perform overflow/underflow checks; UInt will wrap around, while Int follows 2’s complement overflow/underflow semantics. These checks would be impractical for core numeric types (Int, UInt, or SIMD) in performance sensitive code.

While I agree with this for the “speed of the hardware” code in kernels, I can think of many situations where the correctness of the program is worth the performance penalty of turning on overflow/underflow checks. This can also be helpful for debugging kernels as a way to help root out unwanted overflow/underflow.

Perhaps we could make the default controlled by how the user chooses to compile the code (O3/release means no checks, O0/1 and debug symbols on means checks), with an explicit switch to set it of course, and then add ways to do explicit wrapping/saturating/checked operations for code that should never do anything else under any circumstances? I have a lot of Rust code where -1 and 2 ^ 64 - 1 are both equally wrong, but being able to have the compiler assist in detecting underflow means that I don’t have to litter my code with manual checks since I can turn on debug mode to figure out where the problem happened. If it’s all time-of-use checks, then I need to go do a “reverse value flow analysis” to trace where the problem may have happened.

var idx = 1
var my_list = [“a”, “foo”, “bar”]
print(my_list[idx])

While it’s is possible that in the future Mojo will have special syntax for different literal types, we feel strongly that the only reasonable default for idx above is Int, and therefore core types must be indexable with Int.

I think the current solution of IntLiteral as a dependent type is a better idea, since it should enable us to safely implicitly cast it to any scalar which can store the value, which means that it can safely implicitly cast to UInt for any positive integer.

The only situation where 32bit systems would run into problems using Int as an index type is when a collection is larger than ~2GB, and we want to index into it (and even then, only if it’s a collection of bytes). This can only happen if an app has a single array that covers more than half of the available address space of the process. Such an extreme use-case can use other specialized types (e.g. specific adapters of List) if it arises.

I agree that 8-bit and 16-bit processors are largely obsolete at this point, and that it’s not worthwhile to sacrifice anything to support them. 32-bit processors, however, represent the majority of ML accelerators on the planet in the form of Intel’s and Qualcomm’s NPUs, which have the majority market shares for laptops and smartphones. Supporting these would mean that DeviceBuffer and friends would need to support indexing using UInt since I don’t think that >2 GB tensors are out of the question for use in the future, likely via batched input data.

To put it concisely, kernel APIs should almost never use UInt, and all thread and block indexing primitives (e.g. thread_idx.x) will also be Int. It seems the main reason UInt was chosen in many kernels to model indices is to ensure division and modulo operations happen using unsigned integers. To make this easier, we can provide wrappers for performing unsigned div/mod on Int arguments.

Are we sure that converting the likely size_t-typed *_idx equivalents to Int is free on all GPUs?

Other concerns

FFI

I feel like the FFI problem hasn’t been well addressed. C and C++ code that is desirable to interoperate with, such as MPI implementations, both of the popular libraries for RDMA, userspace device drivers, standard POSIX interfaces and more all want to use size_t (UInt), and I think this will add a lot of friction as Mojo attempts to integrate with existing, hard to replace code. Additionally, I think the suggestion that integer literals become Int is a very bad idea because of this, since I could easily see myself needing to construct an array of offsets for scatter/gather or some other operation and writing code like [UInt(0), UInt(1), UInt(3), UInt(5), UInt(7)].

Type correctness

There has been a recent trend in programming of moving towards using the most specific type possible. Using UInt provides a clear signal to the compiler and callers of a function that it is incorrect for the value to ever go below 0. This can be replaced with a copy/pasted comment about the allowed value range for an Int not including any negative values, but that seems silly to me.

Is ecosystem fragmentation solvable with a rename?

Pointing back to the data I gathered much earlier in the thread, Rust still has a perfectly manageable amount of casts despite not even allowing the safe casts to be implicit, which should cause the worst possible version of ecosystem fragmentation to occur. As someone who is currently primarily a Rust developer, I see no such issue in the ecosystem. Based on some old discussions in the Rust discourse forum, Rust ran into issues with people defaulting to ssize_t when they called it Int and decided to change the name to isize, which people do not immediately default to so they are forced to consider what type the integer is best represented by. I think that forcing people to break the habit of defaulting to int for everything helps fix some of the problems that Swift saw with the problem.