Hi there, apologies for sending this message here but I think I’ve read all the docs, watched few youtube interviews and I’m still not fully sure if I can run or if there are plans to support Mojo and/or MAX on my Intel Ultra 7 258V GPU or NPU?
I tried to ask it on discord as well and referred here. Please make it more prominent if this is the case on the first page of your docs/guides/web page or if there is a plan to support them.
I’m trying/experimenting trying to compile Intels DPC++/llvm compiler now locally as we speak to my NixOS system, so I can package oneapi/SYCL based programs in NixOS/flake.nix, sadly no Intel engineer has done this, despite their massive good work on their hardware and software. I managed to get whisper.cpp working on the GPU on NixOS for example but I can’t package it since the mkDerivation sandbox can’t use my manually installed(without nixpkgs) OneAPI Base Toolkit(which has DPC++ icx/icpx compiler). I can compile whisper.cpp SYCL on my Intel Lunar Lake GPU manually with $ nix develop and then $ make (with sycl flags), just packaging it is pain/not working yet, here is a very advanced/hacky flake.nix: Before-refactor whisper.cpp flake.nix · GitHub I can see the NPU device on openvino python environment as well, so I bet I can run inferencing on my NPU in python with openvino. (setting up the npu driver on NixOS was another hack).
Now question is can I just avoid all these python/or other language cruft and just run everything on my NPU and/or GPU with MAX on Mojo?
Would like to know if Modular will eventually support edge inferencing with Intel NPU laptops as well in near future(and ideally “how near” if they will). Otherwise I’ll have to resort to custom solutions with openvino/VLLM or something rust based, sadly
Welcome Izel. I’m not on the Modular team. I just wanted to confirm if this is the processor you are talking about?
I too am curious about this. Modular has skipped CUDA and went straight to ctx, ditto for ROCm, I imagine they’d go straight to Level Zero as well.
@BradLarson
1 Like
To answer your question about current support, we have a tiered list of what’s supported in MAX and Mojo today. Our CI and internal tests check regularly against the tier 1 GPUs, datacenter-class NVIDIA and AMD GPUs, due to these being the highest-demand GPUs among our partners. We have a number of popular NVIDIA consumer GPUs in tier 2, where we don’t test regularly but know them to work well with Mojo and MAX models. Tier 3 contains some of our more bleeding-edge consumer GPUs, such as AMD RDNA GPUs, where Mojo GPU functions work but not yet full MAX models.
When we first released GPU support in December, we had one supported architecture (NVIDIA Ampere GPUs) and have significantly expanded that list over the last few months. We’re progressively adding support for new accelerator architectures as they are both feasible from an engineering perspective and would unlock new opportunities for Modular’s platform. We have a lot that we’re working on, so we have to be deliberate about how we build out new features and hardware support.
All that is to say that we do not currently support Intel GPUs or NPUs, and unfortunately do not have them on our roadmap for the next few months. We will have more to say about new hardware pretty soon that I think people will be excited about, but that will be for a different manufacturer. As someone who has tinkered with different iterations of the Neural Compute Stick, I hear you on the software stack, but it may be a little while before MAX and Mojo can support these devices.
1 Like
Thats precisely what I’ve seen as well and still made me doubt whether Modular might have supported Intel Lunar Lake or has plans to support it given how huge the market is and they offer the best value for these so-called "AI PC"s. I found their work on OpenVINO/DPC++ to be quite massive undertaking as well, the hardware specs and software efforts certainly surprised me.
Thank you for your quick and detailed response @BradLarson, I hope Modular reconsider and prioritize Intel Ultra Series GPU/NPU support given the market is massive(it will be if it isn’t already) so I hope to be able to use Mojo on the edge with my new great laptop. Given significant open-source work has been done on these chips, supporting these chips should be trivial and hope Modular will take this low-hanging fruit as soon as possible as well. I’m rooting for you guys and hope to use Mojo/MAX soon(especially once there is Intel support ;))