Hi, I’m new to modular and I’m trying to run offline inference code from Quickstart | Modular . from max.entrypoints.llm import LLM from max.pipelines import PipelineConfig def main(): model_path = "modularai/Llama-3.1-8B-Instruct-GGUF" pipeline_config = PipelineConfig(model_path=model_pat…

Glad it was helpful. Considering we’re seeing multiple people report the same problem I’m working to get the docs and code example updated.

Error running offline_inference.py

zetwhite May 29, 2025, 12:17am 2

Ah, I found this post Quickstart Docs and giving setting to LLM fixed my problem!

Topic		Replies	Views
Max Local Server Hanging with Message: instrument is None for maxserve.pipeline_load MAX discussion	5	196	July 2, 2025
MAX on CPU doubt and request MAX discussion , feature-request , 24_6	2	255	January 10, 2025
Inference performance issue MAX	6	440	October 31, 2025
Max llm tutorial bug General debugging	4	121	June 13, 2026
Gemma-3-1b-it having trouble with long promts, unlike VLLM MAX	4	365	June 18, 2025