# GPU tensor creation?

**URL:** <https://forum.modular.com/t/gpu-tensor-creation/1421>\
**Category:** GPU Programming\
**Created:** [May 10, 2025, 7:45pm UTC](https://forum.modular.com/t/gpu-tensor-creation/1421 "2025-05-10T19:45:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![alpha21164enthusiast](https://avatars.discourse-cdn.com/v4/letter/a/e99b99/32.png) [@alpha21164enthusiast](https://forum.modular.com/u/alpha21164enthusiast)\
**Post date:** [May 10, 2025, 7:45pm UTC](https://forum.modular.com/t/gpu-tensor-creation/1421/1 "2025-05-10T19:45:48Z")

</div>

Hi!

I’m super new to mojo and I’ve been looking through some of the examples, but I can’t quite figure out how to create new tensors on the GPU. Basically, I’m looking for the equivalent of:

torch.zeros(shape, device=“cuda”) or torch.randn(shape, device=“cuda”)

in mojo.

I can see that in examples such as `naive_matrix_multiplication`, we can allocate buffers on the GPU and then overlay them with `LayoutTensor`s, but I’m not sure if this is the intended way to create tensors on the GPU. This actually begs a further question: how can I create LayoutTensors with dynamic shapes? I see that `RuntimeLayout` is a thing, but there don’t seem to be many examples that show me how to use it.

Sorry for the kind of vague questions, I’m clearly very confused 🙂

Thanks!

---

<div class="post-metadata">

**Author:** ![arthur](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/arthur/32/139_2.png) [@arthur](https://forum.modular.com/u/arthur)\
**Post date:** [May 10, 2025, 11:27pm UTC](https://forum.modular.com/t/gpu-tensor-creation/1421/2 "2025-05-10T23:27:51Z")

</div>

There are a couple of ways to create tensors on the GPU. Yes, you can create a device buffers (which lives in the GPU’s global memory) and create a `LayoutTensor` which is a view of that memory. (You can then use various methods like `LayoutTensor.tile()` to get different views of that memory, subsets, etc.)

If you want to copy data to shared memory or local (register) memory, you can use the [`stack_allocation()`](https://docs.modular.com/mojo/kernels/layout/layout_tensor/LayoutTensor/#stack_allocation) static method to create a statically-sized tensor in the appropriate memory space. You can also do this with the [`LayoutTensorBuild`](https://forum.modular.com/mojo/kernels/layout/tensor_builder/LayoutTensorBuild/) struct, which provides a means for defining and allocating tensors in different memory spaces.

You can see an example of this in Kernel 3 of the matrix multiplication walkthrough: [Custom Operations: Optimizing Matrix Multiplication Recipe | MAX Builds](https://builds.modular.com/recipes/custom-ops-matrix-multiplication)

(Or if you want to look at the full source of those kernels, here: [max-recipes/custom-ops-matrix-multiplication/operations/matrix\_multiplication.mojo at main · modular/max-recipes · GitHub](https://github.com/modular/max-recipes/blob/main/custom-ops-matrix-multiplication/operations/matrix_multiplication.mojo#L237))

Here’s an example of using `LayoutTensor.stack_allocation()` to create a tensor tile in local (register) memory:

> <https://github.com/modular/modular/blob/0bfe79b5bb8e5333203166540668eb6bdf104f9c/max/kernels/src/linalg/matmul_sm90.mojo#L1009>

You can create dynamic layouts with `LayoutTensorBuild` or directly with `LayoutTensor`. (I just posted a simple example of creating a dynamic layout tensor in another thread on this forum.)

If you look through the MAX kernel library, you’ll see a lot of code using either the standalone `stack_allocation()` function or using `NDBuffer.stack_allocation()`. These are all basically doing the same thing. (We’re trying to phase out `NDBuffer` in favor of `LayoutTensor`, but right now there’s still a lot of code using `NDBuffer`.)

Hope that helps!