Issue running the llama3 example in MAX 24.6

I am following the getting started instructions for MAX24.6 on a MacOS. After pulling the repo I run the magic command with the following result.

Many thanks for your help!

❯ magic run llama3 --prompt "What is the meaning of life?"
  × default: error installing/updating PyPI dependencies
  ├─▶ Failed to prepare distributions
  ├─▶ Failed to download and build `faster-fifo==1.4.7`
  ╰─▶ Build backend failed to build wheel through `build_wheel` (exit status: 1)
      
      [stdout]
      running bdist_wheel
      running build
      running build_py
      creating build/lib.macosx-11.0-arm64-cpython-312/faster_fifo_reduction
      copying faster_fifo_reduction/__init__.py -> build/lib.macosx-11.0-arm64-cpython-312/
      faster_fifo_reduction
      creating build/lib.macosx-11.0-arm64-cpython-312/cpp_faster_fifo
      copying cpp_faster_fifo/__init__.py -> build/lib.macosx-11.0-arm64-cpython-312/cpp_faster_fifo
      creating build/lib.macosx-11.0-arm64-cpython-312/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/comparison_tests.py -> build/lib.macosx-11.0-arm64-cpython-312/
      cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/test_faster_fifo.py -> build/lib.macosx-11.0-arm64-cpython-312/
      cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/__init__.py -> build/lib.macosx-11.0-arm64-cpython-312/
      cpp_faster_fifo/tests
      running build_ext
      building 'faster_fifo' extension
      creating build/temp.macosx-11.0-arm64-cpython-312/cpp_faster_fifo/cpp_lib
      clang++ -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -Wall -fPIC -O2
      -isystem /Users/foo/dev/modularml/max/pipelines/python/.magic/envs/default/include -arch
      arm64 -fPIC -O2 -isystem /Users/foo/dev/modularml/max/pipelines/python/.magic/envs/default/
      include -arch arm64 -Icpp_faster_fifo/cpp_lib -I/Users/foo/Library/Caches/rattler/cache/uv-
      cache/builds-v0/.tmp4pb1SS/include -I/Users/foo/dev/modularml/max/pipelines/python/.magic/
      envs/default/include/python3.12 -c cpp_faster_fifo/cpp_lib/faster_fifo.cpp -o build/temp.macosx-
      11.0-arm64-cpython-312/cpp_faster_fifo/cpp_lib/faster_fifo.o -std=c++11

It looks like this is happen during the build of the faster-fifo library, which is not part of the MAX code specifically, but seems to be a Python dependency.

I wonder if Xcode or Xcode Command Line Tools are installed correctly on this machine. You can see the system requirements for MAX here: MAX FAQ | Modular Docs

You can run this command to check for an Xcode installation:

xcode-select -p

If Xcode is not installed, you can install it from the Mac App Store. If you have Xcode installed but not the Xcode Command Line Tools, you can install them with:

xcode-select --install

I can’t reproduce this unfortunately! looks like a local toolchain issue as Josh said compiling the dependency which uses Cython. Could you try magic clean and then magic add --pypi Cython prior to running llama3?

1 Like

We had some folks hit this on Tuesday too. faster-fifo is a C++ based python lib but doesn’t release binary wheels, only a source package, so it requires that you have a local compatible C++ toolchain (like from xcode) setup.

I sent a PR upstream to the package developer to contribute code to build and release binary versions of the package, and I’m hoping they accept to make the installation a little easier. Build/publish binary wheels with cibuildwheel by zbowling · Pull Request #52 · alex-petrenko/faster-fifo · GitHub

Updating xcode might help, but running on macos 11 it may be a challenge to get a compatible C++ toolchain and may still run into issues there. I recommend updating to macos 14 or 15.2 at least.

4 Likes

Good eye to see the toolchain was for macOS 11, I did not notice that @Zac! Our minimum requirements indicate MAX needs macOS 13 at least, so even if we get this to work, something might fail later on macOS 11.

Thanks for all the suggestions!

I have MacOS Sequoia 15.1.1. I am not sure why the path has macOS-11.0.

I have the Xcode tools already

❯ xcode-select --install
xcode-select: note: Command line tools are already installed. Use "Software Update" in System Settings or the softwareupdate command line interface to install updates

It looks like there are updates to both the OS and Xcode cli, installing those now.

I have a C++ compiler installed

❯ c++ --version
Apple clang version 16.0.0 (clang-1600.0.26.4)
Target: arm64-apple-darwin24.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I cannot add Cython, running magic add --pypi Cython fails with the same C++ compilation error.

I wonder if something is wrong with the SDK path, and Xcode is trying to use an older (11.0) SDK version. Can you try this command?

xcrun --show-sdk-path

That should display the SDK path. That path is likely a symlink. On my machine is is /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk. Then you can run this command, where the argument is the SDK path:

ls -al /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

My machine reports /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk ⇒ MacOSX15.1.sdk. I wonder if your path is pointing to the 11.0 SDK for some reason.

1 Like

It looks like it points to 15.2

ls -ltr `xcrun --show-sdk-path`
lrwxr-xr-x  1 root  wheel  14 Dec 19 20:43 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -> MacOSX15.2.sdk

FWIW the same C++ compilation issue happens in a brand new python project with just this dependency.

Also this issue does not happen on an Ubuntu environment.

Thanks for checking! Is there any other part of that error message? I wonder if we can somehow see the actual error from the C++ compiler. Maybe it gets put in a log file somewhere.

Could you try

  • Python 3.10 instead of 3.12 which means in pixi.toml use this instead
[dependencies]
python = "==3.10"
  • brew install cmake if you haven’t
export CFLAGS="-I$(brew --prefix)/include"
export LDFLAGS="-L$(brew --prefix)/lib"

Setting this env variable made the compilation work, finally :-/

I really appreciate all the help Modular team, Happy Holidays!

export CPLUS_INCLUDE_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/
1 Like

Great! thanks for letting us know.

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.