MEF Documentation

MAX
1

I have been using a combination of modal + modular to do some custom inference work for a bit, but I’ve been having issues persisting the compiled execution graphs to my serverless environment.

After doing some digging on my local macOS env, i managed to find the file path where the MEF files are stored, (/Users/{username}/.local/share/modular/.max_cache/mof/mef) then I just manually set the volume to be what i expected the path would be on my linux env(/root/.local/share/modular/.max_cache/mof/mef). That seems to have worked (no more slow building + compiling on boot).

However, it would be great to get deeper documentation on the MEF paradigm as a whole. Took longer than it should have to get things working.

2 Likes
2

So generally the MEF location is controlled by the modular.cfg. in particular:

[max]
cache_dir = <path>

In conda world, we set that to be inside the environment:

cache_dir = $ENV/share/max/.max_cache

You can also override with MODULAR_MAX_CACHE_DIR like we do in the mojo entrypoint script in Python world: modular/mojo/python/mojo/run.py at main · modular/modular · GitHub

"MODULAR_MAX_CACHE_DIR": str(root / "share" / "max" / ".max_cache"),
1 Like
3

Caveat is the cache may not be work in the serverless env such as modal that users don’t have control over the CPU arch.

4

makes sense. Beyond my current strategy, which seems to be holding up for now, is there anything on the roadmap to make the caching simpler across diff archs? I know there isn’t MEF portability at the moment, but I’m surprised to see a technical choice that is CPU arch dependent. That is rare ime

5

No, we don’t have any near term plans to change this. I agree, the current behavior is not ideal though. I suspect we will improve it at some point, but this is not a target feature we expect to work on soon.