# MAX **URL:** https://forum.modular.com/c/max/6.md [Latest](https://forum.modular.com/latest.md) · [Categories](https://forum.modular.com/categories.md) · [Tags](https://forum.modular.com/tags.md) --- ## [About the MAX category](https://forum.modular.com/t/about-the-max-category/18)
**Author:** [@Caroline](https://forum.modular.com/u/Caroline)\ **Replies:** 0\ **Last updated:** [December 4, 2024, 9:54pm UTC](https://forum.modular.com/t/about-the-max-category/18 "2024-12-04T21:54:39Z")
Questions and discussion related to the Modular Accelerated Xecution (MAX) platform: a unified set of APIs and tools that help you build and deploy high-performance AI pipelines. --- ## [MAX vs vLLM for HPC inference farm](https://forum.modular.com/t/max-vs-vllm-for-hpc-inference-farm/3464)
**Author:** [@Curious](https://forum.modular.com/u/Curious)\ **Replies:** 5\ **Last updated:** [October 13, 2026, 10:21am UTC](https://forum.modular.com/t/max-vs-vllm-for-hpc-inference-farm/3464 "2026-10-13T10:21:34Z")
I am looking to create an inference farm on a Docker HPC host with 4 nodes with 4 H100s each. I asked Gemini and it said MAX loses to vLLM for more users and is more geared towards local deployments. I took this person… --- ## [Official Modular training and inference library in mojo?](https://forum.modular.com/t/official-modular-training-and-inference-library-in-mojo/3461)
**Author:** [@timo.david](https://forum.modular.com/u/timo.david)\ **Replies:** 1\ **Last updated:** [September 2, 2026, 12:27pm UTC](https://forum.modular.com/t/official-modular-training-and-inference-library-in-mojo/3461 "2026-09-02T12:27:27Z")
Are there any plans to have a mojo library for both training and inference, like pytorch or tensorflow? Now that mojo reached 1.0 and that we can write gpu kernels in pure mojo, this looks like a natural next step. I kn… --- ## [How to build SLM based on agent sessions/traces](https://forum.modular.com/t/how-to-build-slm-based-on-agent-sessions-traces/3420)
**Author:** [@aljosa](https://forum.modular.com/u/aljosa)\ **Replies:** 0\ **Last updated:** [August 18, 2026, 7:36pm UTC](https://forum.modular.com/t/how-to-build-slm-based-on-agent-sessions-traces/3420 "2026-08-18T19:36:02Z")
I’d like to explore whether the Modular ecosystem can support most of an automated teacher–student model pipeline, rather than using MAX only as the final inference server. The idea is to use a model such as GLM-5.2 thr… --- ## [Is there a MAX roadmap?](https://forum.modular.com/t/is-there-a-max-roadmap/3403)
**Author:** [@Curious](https://forum.modular.com/u/Curious)\ **Replies:** 4\ **Last updated:** [August 17, 2026, 12:36am UTC](https://forum.modular.com/t/is-there-a-max-roadmap/3403 "2026-08-17T00:36:42Z")
Is there a MAX equivalent of the Mojo Roadmap or is MAX’s faith linked to Mojo so we do not need one? --- ## [Does MAX distribute inference accross PBS nodes](https://forum.modular.com/t/does-max-distribute-inference-accross-pbs-nodes/3400)
**Author:** [@Curious](https://forum.modular.com/u/Curious)\ **Replies:** 4\ **Last updated:** [August 15, 2026, 12:09am UTC](https://forum.modular.com/t/does-max-distribute-inference-accross-pbs-nodes/3400 "2026-08-15T00:09:43Z")
Suppose I qsub 5 compute nodes each having CPUs and GPUs, I now use MPI to distribute my job accross them but can custom ops and MAX do the same or is there a Mojo std.distributed or something of that sort? --- ## [Congrats on Mojo 1.0 and Max releases](https://forum.modular.com/t/congrats-on-mojo-1-0-and-max-releases/3393)
**Author:** [@mseritan](https://forum.modular.com/u/mseritan)\ **Replies:** 1\ **Last updated:** [August 11, 2026, 10:45pm UTC](https://forum.modular.com/t/congrats-on-mojo-1-0-and-max-releases/3393 "2026-08-11T22:45:09Z")
Of course Mojo 1.0 is fantastic but I also want to give a shout to his Max brother. The new driver increased my M2 Pro tok/s by a factor of 3, in my Millfolio app. Thanks! --- ## [First-class autodiff/training in MAX: timeline, and supersede-or-compose with community autograd libraries?](https://forum.modular.com/t/first-class-autodiff-training-in-max-timeline-and-supersede-or-compose-with-community-autograd-libraries/3152)
**Author:** [@Gles](https://forum.modular.com/u/Gles)\ **Replies:** 4\ **Last updated:** [August 10, 2026, 5:56am UTC](https://forum.modular.com/t/first-class-autodiff-training-in-max-timeline-and-supersede-or-compose-with-community-autograd-libraries/3152 "2026-08-10T05:56:11Z")
Hi there. I’m considering investing serious time in a pure-Mojo autograd / tensor-NN library, and before I commit I’d like to align with Modular’s direction rather than guess at it. Here’s what I’ve found so far (happy… --- ## [Issue in starting gemma 4 models using max serve](https://forum.modular.com/t/issue-in-starting-gemma-4-models-using-max-serve/3360)
**Author:** [@sandkuma](https://forum.modular.com/u/sandkuma)\ **Replies:** 3\ **Last updated:** [August 3, 2026, 8:00am UTC](https://forum.modular.com/t/issue-in-starting-gemma-4-models-using-max-serve/3360 "2026-08-03T08:00:23Z")
I wanted to run gemma 4 models using modular on a machine that has a h200 gpu and was not successful i tried various steps with copilot and a complete summary generated by copilot is as under please help in figuring out … --- ## [MAX models can now run on Apple silicon GPUs](https://forum.modular.com/t/max-models-can-now-run-on-apple-silicon-gpus/3283)
**Author:** [@BradLarson](https://forum.modular.com/u/BradLarson)\ **Replies:** 4\ **Last updated:** [July 14, 2026, 3:53pm UTC](https://forum.modular.com/t/max-models-can-now-run-on-apple-silicon-gpus/3283 "2026-07-14T15:53:25Z")
For the last several months, we’ve been progressively improving support for Mojo and MAX on Apple silicon GPUs, first unlocking the ability to program them via Mojo, then enabling basic MAX graphs to run on these GPUs. W… --- ## [Optimized Kernels for Blackwell -- do they work on GB10](https://forum.modular.com/t/optimized-kernels-for-blackwell-do-they-work-on-gb10/3265)
**Author:** [@gchauhan](https://forum.modular.com/u/gchauhan)\ **Replies:** 2\ **Last updated:** [June 22, 2026, 4:46pm UTC](https://forum.modular.com/t/optimized-kernels-for-blackwell-do-they-work-on-gb10/3265 "2026-06-22T16:46:54Z")
I am seeing very low perf numbers for model inference on Nvidia DGX Spark GB10 with Modular mojo / max serve. What parts from the Blackwell kernel series ( Modular: Matrix Multiplication on Blackwell ) are available for… --- ## [Why were FBCache and TeaCache not carried over to Flux2Executor?](https://forum.modular.com/t/why-were-fbcache-and-teacache-not-carried-over-to-flux2executor/3094)
**Author:** [@junu](https://forum.modular.com/u/junu)\ **Replies:** 0\ **Last updated:** [May 19, 2026, 7:51am UTC](https://forum.modular.com/t/why-were-fbcache-and-teacache-not-carried-over-to-flux2executor/3094 "2026-05-19T07:51:33Z")
Hi, I was looking at the recent Flux2Executor migration( \[Pipelines\] Remove flux2\_modulev3 architecture and TeaCache support · modular/modular@f767446 · GitHub ) and wanted to check my understanding. The flux2\_modulev3 … --- ## [Notebook format for the max-llm-book](https://forum.modular.com/t/notebook-format-for-the-max-llm-book/3084)
**Author:** [@dunnoconnor](https://forum.modular.com/u/dunnoconnor)\ **Replies:** 0\ **Last updated:** [May 15, 2026, 1:50pm UTC](https://forum.modular.com/t/notebook-format-for-the-max-llm-book/3084 "2026-05-15T13:50:58Z")
The max-llm-book has gotten a couple of big updates in the past month. We trimmed the python steps that (while nicely broken up to pace the tutorial) made it harder to picture the whole architecture you were building tow… --- ## [I burned some Codex quota testing MAX training](https://forum.modular.com/t/i-burned-some-codex-quota-testing-max-training/3074)
**Author:** [@Ethan](https://forum.modular.com/u/Ethan)\ **Replies:** 1\ **Last updated:** [May 13, 2026, 6:50pm UTC](https://forum.modular.com/t/i-burned-some-codex-quota-testing-max-training/3074 "2026-05-13T18:50:45Z")
I had some Codex quota left this week, so I used it on a small experiment: can MAX be made to support PyTorch-like training behavior, and how fast is a compiled train step? I built a prototype standalone package, max\_tr… --- ## [How can I build a minimal Docker image for MAX inference](https://forum.modular.com/t/how-can-i-build-a-minimal-docker-image-for-max-inference/3044)
**Author:** [@ficapy](https://forum.modular.com/u/ficapy)\ **Replies:** 1\ **Last updated:** [May 8, 2026, 8:09pm UTC](https://forum.modular.com/t/how-can-i-build-a-minimal-docker-image-for-max-inference/3044 "2026-05-08T20:09:20Z")
Background I have some models that may only be called a few times per day, so keeping a dedicated GPU server running for them is not cost-effective. GPU serverless is very appealing for this kind of low-frequency inferen… --- ## [Help with max serve performance on H100](https://forum.modular.com/t/help-with-max-serve-performance-on-h100/2965)
**Author:** [@cwp](https://forum.modular.com/u/cwp)\ **Replies:** 6\ **Last updated:** [April 14, 2026, 12:00pm UTC](https://forum.modular.com/t/help-with-max-serve-performance-on-h100/2965 "2026-04-14T12:00:31Z")
Hi, I’m just getting into the modular world, so not sure where I’m going wrong. I just deployed MAX on an AWS p5.48xlarge, running GPT-OSS. I was going to do some load testing, but I’m getting really bad performance fro… --- ## [Overhead of Multi‑Dimensional Reduction](https://forum.modular.com/t/overhead-of-multi-dimensional-reduction/2828)
**Author:** [@minkyu](https://forum.modular.com/u/minkyu)\ **Replies:** 1\ **Last updated:** [March 10, 2026, 5:03pm UTC](https://forum.modular.com/t/overhead-of-multi-dimensional-reduction/2828 "2026-03-10T17:03:18Z")
Hi, Modular team. I’d like to report a slowdown in MAX when using a single reduction over all dimensions (mean(..., axis=None)), compared to a multi‑step reduction workaround. For example, this code is slow. mean\_a = … --- ## [Constraint failed: copy\_from\_async only allows 4, 8, 16 bytes element](https://forum.modular.com/t/constraint-failed-copy-from-async-only-allows-4-8-16-bytes-element/2768)
**Author:** [@vishnoianil](https://forum.modular.com/u/vishnoianil)\ **Replies:** 0\ **Last updated:** [February 25, 2026, 6:58am UTC](https://forum.modular.com/t/constraint-failed-copy-from-async-only-allows-4-8-16-bytes-element/2768 "2026-02-25T06:58:43Z")
I was experimenting with tiled GEMM kernel written in Mojo. Here is code . I am running this kernel on nvidia A100. It works fine with float32 DType, but fails with bfloat16/float16, because it seems like max layout te… --- ## [Help compiling MLP in MAX](https://forum.modular.com/t/help-compiling-mlp-in-max/2760)
**Author:** [@AndPotap](https://forum.modular.com/u/AndPotap)\ **Replies:** 4\ **Last updated:** [February 23, 2026, 11:43pm UTC](https://forum.modular.com/t/help-compiling-mlp-in-max/2760 "2026-02-23T23:43:09Z")
Hi, I went over the really good content on the MAX LLM book but I can’t seem to get the model to compile. To simplify the problem right now I’m just trying to get the MLP part of the model to compile but without any suc… --- ## [Initial support for running MAX models on AMD RDNA GPUs](https://forum.modular.com/t/initial-support-for-running-max-models-on-amd-rdna-gpus/2744)
**Author:** [@BradLarson](https://forum.modular.com/u/BradLarson)\ **Replies:** 0\ **Last updated:** [February 17, 2026, 11:04pm UTC](https://forum.modular.com/t/initial-support-for-running-max-models-on-amd-rdna-gpus/2744 "2026-02-17T23:04:11Z")
Pretty excited to say that as of the last nightly (commit), we now have the ability in MAX to run some text LLMs on AMD RDNA GPUs! This has been a progressive effort over time, with some of the background explained in th… --- ## [MAX models can now use customized Mojo kernels and standard library](https://forum.modular.com/t/max-models-can-now-use-customized-mojo-kernels-and-standard-library/2742)
**Author:** [@BradLarson](https://forum.modular.com/u/BradLarson)\ **Replies:** 0\ **Last updated:** [February 17, 2026, 8:13pm UTC](https://forum.modular.com/t/max-models-can-now-use-customized-mojo-kernels-and-standard-library/2742 "2026-02-17T20:13:49Z")
In December, we enabled the capability for customizing Python MAX models and layers within your local modular checkout, and then using Bazel to run tests against your local changes. Over the last couple of months, we’ve … --- ## [Question on register custom ops for max on different device](https://forum.modular.com/t/question-on-register-custom-ops-for-max-on-different-device/2717)
**Author:** [@knightyangduke](https://forum.modular.com/u/knightyangduke)\ **Replies:** 6\ **Last updated:** [February 13, 2026, 1:40am UTC](https://forum.modular.com/t/question-on-register-custom-ops-for-max-on-different-device/2717 "2026-02-13T01:40:18Z")
Hi folks, I am new to modular and looking at some demos/documentsions on MAX compiler. Ideally, I want to define an op that can be “overrided” with different implementations on different target. In the documented exam… --- ## [Failed to resolve module path for \`MOGGKernelAPI\`](https://forum.modular.com/t/failed-to-resolve-module-path-for-moggkernelapi/2707)
**Author:** [@AndPotap](https://forum.modular.com/u/AndPotap)\ **Replies:** 1\ **Last updated:** [February 10, 2026, 4:15pm UTC](https://forum.modular.com/t/failed-to-resolve-module-path-for-moggkernelapi/2707 "2026-02-10T16:15:36Z")
I get the following error related to MOGGKernelAPI python3 addition.py Traceback (most recent call last): File "/home/ubu/code\_examples/modular/forum/.pixi/envs/default/lib/python3.13/site-packages/max/engine/api.py",… --- ## [Build an LLM in MAX from scratch 📖](https://forum.modular.com/t/build-an-llm-in-max-from-scratch/2470)
**Author:** [@dunnoconnor](https://forum.modular.com/u/dunnoconnor)\ **Replies:** 10\ **Last updated:** [February 6, 2026, 5:59pm UTC](https://forum.modular.com/t/build-an-llm-in-max-from-scratch/2470 "2026-02-06T17:59:32Z")
The MAX Experimental API is a model-building framework we’ve been developing to help you create custom models in the MAX Framework. You can now get hands-on with this experimental API by following our MAX LLM tutorial. I… --- ## [All MAX API tests can now be run via Bazel in the \`modular\` repository](https://forum.modular.com/t/all-max-api-tests-can-now-be-run-via-bazel-in-the-modular-repository/2538)
**Author:** [@BradLarson](https://forum.modular.com/u/BradLarson)\ **Replies:** 7\ **Last updated:** [February 6, 2026, 5:24pm UTC](https://forum.modular.com/t/all-max-api-tests-can-now-be-run-via-bazel-in-the-modular-repository/2538 "2026-02-06T17:24:42Z")
As a follow-on to our recent open-sourcing of the remainder of the MAX Python API, you can now run all MAX Python API unit and integration tests in the modular repository. This means that you can make additions or modifi… --- ## [Mypy type checking workflow for new model development](https://forum.modular.com/t/mypy-type-checking-workflow-for-new-model-development/2630)
**Author:** [@byungchul](https://forum.modular.com/u/byungchul)\ **Replies:** 2\ **Last updated:** [January 23, 2026, 3:22pm UTC](https://forum.modular.com/t/mypy-type-checking-workflow-for-new-model-development/2630 "2026-01-23T15:22:37Z")
Hi, I’m developing support for a new model architecture in the MAX framework. My current iterative development workflow is as follows: Make code changes Run ./bazelw run format to ensure code consistency Build with ./b… --- ## [Using @F.functional decorator for forward method](https://forum.modular.com/t/using-f-functional-decorator-for-forward-method/2623)
**Author:** [@minkyu](https://forum.modular.com/u/minkyu)\ **Replies:** 2\ **Last updated:** [January 23, 2026, 8:20am UTC](https://forum.modular.com/t/using-f-functional-decorator-for-forward-method/2623 "2026-01-23T08:20:59Z")
Hi, while working on defining models using module\_v3, there are some forward methods decorated with @F.functional. (example) Upon some comments related to this, it seems that it allows possible fusion of operations that… --- ## [MAX Graph Python API built-in ops fail to compile for GPU - what's the correct pattern?](https://forum.modular.com/t/max-graph-python-api-built-in-ops-fail-to-compile-for-gpu-whats-the-correct-pattern/2614)
**Author:** [@abhisheksreesaila](https://forum.modular.com/u/abhisheksreesaila)\ **Replies:** 3\ **Last updated:** [January 19, 2026, 3:04am UTC](https://forum.modular.com/t/max-graph-python-api-built-in-ops-fail-to-compile-for-gpu-whats-the-correct-pattern/2614 "2026-01-19T03:04:36Z")
Hi! I’m trying to run MAX graphs on GPU but getting compilation failures for even simple operations. Looking for guidance on the correct API pattern. Environment: MAX version: modular-26.1.0.dev2026011405 (nightly) … --- ## [Model Correctness Check](https://forum.modular.com/t/model-correctness-check/2573)
**Author:** [@as0909024](https://forum.modular.com/u/as0909024)\ **Replies:** 1\ **Last updated:** [January 14, 2026, 4:16pm UTC](https://forum.modular.com/t/model-correctness-check/2573 "2026-01-14T16:16:39Z")
I am currently exploring the Modular repository and have understood several aspects of the MAX pipeline so far. However, I am stuck at one specific point and would appreciate some clarification. Integration test models … --- ## [Tensor Creation Allocates Memory on All GPUs and Fails with CUDA\_VISIBLE\_DEVICES](https://forum.modular.com/t/tensor-creation-allocates-memory-on-all-gpus-and-fails-with-cuda-visible-devices/2586)
**Author:** [@pei0033](https://forum.modular.com/u/pei0033)\ **Replies:** 4\ **Last updated:** [January 14, 2026, 2:49am UTC](https://forum.modular.com/t/tensor-creation-allocates-memory-on-all-gpus-and-fails-with-cuda-visible-devices/2586 "2026-01-14T02:49:40Z")
Package Version max 26.1.0.dev2026010820 max-core 26.1.0.dev2026010820 max-mojo-libs 26.1.0.dev2026010820 max-shmem-libs … [Next page](https://forum.modular.com/c/max/6.md?page=1)