Modular has acquired BentoML! Ask us anything

Modular Official Announcements
21
  1. BentoML services scaling in Mammoth?
  2. BentoML SDK for Mojo? (for example, serve Mojo code, use Python client, but scale it with Ray?)
  3. BentoML and OpenLLM inference with MAX backend?
  4. BentoML and OpenLLM license changes?
22

Awesome :+1: :smiling_face_with_sunglasses: we are getting max engine models into industrial scale. We’ll deploy them on BentoML. I appreciate all your contributions on the questions and we’ve effectively learned how BentoML works at industrial scale deployment.

It was more like a tutorial event​:fire::grin:

Thanks Chris

23

As mentioned above, Mammoth won’t go extinct! :wink: Mammoth is a key part of our technology stack for large scale distributed inference. It works great within BentoML already and we’ll be tightening the integration. Our vision is a unified Modular Cloud product that makes it super simple to scale and deploy with full control and performance for your workloads.

We definitely plan to implement increased support for both Mojo and MAX within BentoML! We’re still determining exactly what that looks like. We’re excited to share more concrete details as they develop.

The license for BentoML and OpenLLM won’t change. It will remain Apache 2.0.

1 Like