# NVIDIA hardware support in MAX 24.6 **URL:** \ **Category:** MAX\ **Tags:** discussion, 24\_6\ **Created:** [December 17, 2024, 9:33pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340 "2024-12-17T21:33:30Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![BradLarson](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/bradlarson/32/62_2.png) [@BradLarson](https://forum.modular.com/u/BradLarson)\ **Post date:** [December 17, 2024, 9:33pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/1 "2024-12-17T21:33:30Z")
With the support for NVIDIA GPUs launching in MAX 24.6, a number of people have already asked what hardware is compatible with MAX in this release. This thread is intended to be a reference for our officially-supported hardware, as well as a place for community discussions about hardware compatibility. Our officially-supported NVIDIA GPUs are listed in the Linux tab of our [system requirements](https://docs.modular.com/max/faq/#system-requirements). At present, these are the datacenter-class NVIDIA GPU models of **A10** , **A100** , **L4** , and **L40**. These are the GPUs that Modular regularly tests MAX against, and if you encounter issues with these please let us know. Unofficially, we’ve had reports that other Ampere or Ada Lovelace family GPUs may work with MAX in 24.6, but only the ones listed above are known to work. Note that a GPU with 24 GB of RAM is required to serve our flagship Llama 3.1 8B pipeline using bfloat16 weights. ---
**Author:** ![owenhilyard](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/owenhilyard/32/41_2.png) [@owenhilyard](https://forum.modular.com/u/owenhilyard)\ **Post date:** [December 17, 2024, 11:11pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/2 "2024-12-17T23:11:29Z")
Older Nvidia drivers may be an issue, make sure you are on a current version! I have 560 working on my 4090, and the current “production level” release is 550. 535 has caused issues for at least one person. ---
**Author:** ![dmeaux](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/dmeaux/32/143_2.png) [@dmeaux](https://forum.modular.com/u/dmeaux)\ **Post date:** [December 18, 2024, 9:53am UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/3 "2024-12-18T09:53:52Z")
For others reading this, I have an NVIDIA A2000 in my laptop. MAX recognizes the GPU (i.e. accelerator\_count() == 1), but the calculations take slightly longer than running on the CPU (EDIT: using the nightly build to run the addition sample in custom\_ops). Whether that’s because MAX isn’t optimized for the processor or it’s falling back to the CPU, I’ll leave that up to the Modular team members to answer, if they wish to do so. That aside, I’m shocked, in a good way, about how easy it was to write a custom kernel that could so easily be used for either device and how smoothly it just ran. 🎉 🎉 🎉 🙏Thank you for the great work! ---
**Author:** ![joshpeterson](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/joshpeterson/32/14_2.png) [@joshpeterson](https://forum.modular.com/u/joshpeterson)\ **Post date:** [December 18, 2024, 12:00pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/4 "2024-12-18T12:00:37Z")
If you pass the `--use-gpu` flag to `magic run` MAX will execute on the GPU. That code doesn’t try to make a decision about which is “better”, CPU or GPU, at this point. ---
**Author:** ![ivellapillil](https://avatars.discourse-cdn.com/v4/letter/i/eada6e/32.png) [@ivellapillil](https://forum.modular.com/u/ivellapillil)\ **Post date:** [December 18, 2024, 12:40pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/5 "2024-12-18T12:40:05Z")
Does anyone know when the Apple GPU support comes? ---
**Author:** ![joshpeterson](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/joshpeterson/32/14_2.png) [@joshpeterson](https://forum.modular.com/u/joshpeterson)\ **Post date:** [December 18, 2024, 1:34pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/6 "2024-12-18T13:34:01Z")
That is definitely something on out minds, but we don’t have an ETA for it yet. ---
**Author:** ![dmeaux](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/dmeaux/32/143_2.png) [@dmeaux](https://forum.modular.com/u/dmeaux)\ **Post date:** [December 18, 2024, 1:43pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/7 "2024-12-18T13:43:27Z")
I get the following error if I try that: error: unexpected argument ‘–use-gpu’ found FWIW, it’s a laptop GPU, not a datacenter-grade GPU on the list of supported GPUs, so I’m not expecting it to be supported. ---
**Author:** ![joshpeterson](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/joshpeterson/32/14_2.png) [@joshpeterson](https://forum.modular.com/u/joshpeterson)\ **Post date:** [December 18, 2024, 1:55pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/8 "2024-12-18T13:55:10Z")
What is your full command line? It sounds like there is an argument parsing error, which might be unexpected. ---
**Author:** ![dmeaux](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/dmeaux/32/143_2.png) [@dmeaux](https://forum.modular.com/u/dmeaux)\ **Post date:** [December 18, 2024, 2:12pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/9 "2024-12-18T14:12:16Z")
> magic run --use-gpu addition where addition is set in in the mojoproject.toml file as: > [tasks] > addition = { cmd=“mojo package kernels/ -o kernels.mojopkg && python addition.py”, env={ MODULAR\_ONLY\_USE\_NEW\_EXTENSIBILITY\_API=“true” }} ---
**Author:** ![joshpeterson](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/joshpeterson/32/14_2.png) [@joshpeterson](https://forum.modular.com/u/joshpeterson)\ **Post date:** [December 18, 2024, 2:31pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/10 "2024-12-18T14:31:51Z")
Ahh , thanks! Sorry, I’ve confused things here. The `--use-gpu` option is a specific command line option for the llama3 pipeline, as documented here: [Deploy Llama 3 on GPU with MAX Serve | Modular Docs](https://docs.modular.com/max/tutorials/max-serve-local-to-cloud). However, it is not a command line option for `magic`, which is why this error occurs (correctly). I was getting mixed up! The custom op example chooses the device based on this [code](https://github.com/modularml/max/blob/nightly/examples/custom_ops/addition.py#L56): ```python devices=[CPU() if accelerator_count() == 0 else CUDA()], ``` To be sure it is running on GPU, you could remove the `if` completely, and just used `CUDA()`. You could also print the result of the `accelerator_count()` call to see if your GPU is correctly detected by MAX. And one more note - we will be renaming `CUDA()` to the more general `Accelerator` in a nightly build soon! ---
**Author:** ![dmeaux](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/dmeaux/32/143_2.png) [@dmeaux](https://forum.modular.com/u/dmeaux)\ **Post date:** [December 18, 2024, 2:34pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/11 "2024-12-18T14:34:58Z")
Yeah, I did that. I’m responding to you and Brad in Discord. Let’s pick up the conversation there. ---
**Author:** ![owenhilyard](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/owenhilyard/32/41_2.png) [@owenhilyard](https://forum.modular.com/u/owenhilyard)\ **Post date:** [December 18, 2024, 5:21pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/12 "2024-12-18T17:21:42Z")
Is this setting an environment variable somewhere that we can use without magic? ---
**Author:** ![BradLarson](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/bradlarson/32/62_2.png) [@BradLarson](https://forum.modular.com/u/BradLarson)\ **Post date:** [December 18, 2024, 6:28pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/13 "2024-12-18T18:28:16Z")
I’ve created a new thread about the custom Mojo GPU operation examples, if people want to carry on discussions about those in that topic: [[Experimental] Examples of custom CPU / GPU operations in Mojo](https://forum.modular.com/t/experimental-examples-of-custom-cpu-gpu-operations-in-mojo/348) ---
**Author:** ![system](https://us1.discourse-cdn.com/flex001/uploads/modular/original/1X/2751e0fbdc595a99718b216730957e9db4448cfd.jpeg) [@system](https://forum.modular.com/u/system)\ **Post date:** [June 16, 2025, 6:28pm UTC](https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340/14 "2025-06-16T18:28:37Z")
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.