I’d like to explore whether the Modular ecosystem can support most of an automated teacher–student model pipeline, rather than using MAX only as the final inference server.
The idea is to use a model such as GLM-5.2 through Modular Cloud, automatically collect application sessions and execution traces, and use that data to periodically fine-tune or distill a smaller language model. Each new version of the smaller model would then be deployed with MAX and evaluated in parallel with the original model before receiving production traffic.
I understand that parts of this could be built with existing non-Modular tools. However, my main goal is to understand how far it makes sense to take this using Modular Cloud, MAX, and Mojo specifically.
Is there a coherent direction within the Modular ecosystem for building this workflow? In particular:
-
Which parts are already supported or are natural use cases for Modular Cloud, MAX, or Mojo?
-
Which parts would currently require custom development within the Modular ecosystem?
-
Are there parts—such as trace collection, dataset preparation, fine-tuning, evaluation orchestration, model versioning, or shadow deployment—that would not make sense to implement with Mojo/MAX and should remain external?
-
Could Mojo reasonably be used for any of the training or data-processing components, or is its practical role here mainly custom model architecture, inference kernels, and performance optimization?
-
Which specific Modular concepts, APIs, examples, or roadmap areas should I study before attempting this?
I’m already comfortable building the surrounding infrastructure with conventional tools. I’m specifically trying to determine whether this could become a meaningful Modular-native reference architecture, where the ecosystem has clear advantages, or whether some parts would be forcing Mojo/MAX into roles they are not intended to fill.