Hi, I’m new to modular and I’m trying to run offline inference code from Quickstart | Modular . from max.entrypoints.llm import LLM from max.pipelines import PipelineConfig def main(): model_path = "modularai/Llama-3.1-8B-Instruct-GGUF" pipeline_config = PipelineConfig(model_path=model_pat…

Error running offline_inference.py

zetwhite May 29, 2025, 12:17am 2

Ah, I found this post Quickstart Docs and giving setting to LLM fixed my problem!

2 Likes