# Ask Steffi anything about efficient coroutine implementation in MLIR **URL:** \ **Category:** Community Showcase\ **Tags:** modular-content, llvm, ask-me-anything\ **Created:** [December 6, 2024, 4:22pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39 "2024-12-06T16:22:19Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Modular](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/modular/32/17_2.png) [@Modular](https://forum.modular.com/u/Modular)\ **Post date:** [December 6, 2024, 4:22pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/1 "2024-12-06T16:22:19Z")
[![](https://us1.discourse-cdn.com/flex001/uploads/modular/original/1X/1fae948e00e3ccac3da92065bb24b783546b9b7d.jpeg "2024 LLVM Dev Mtg - Efficient Coroutine Implementation in MLIR") ](https://www.youtube.com/watch?v=ILjuvj13EpQ) ---
**Author:** ![owenhilyard](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/owenhilyard/32/41_2.png) [@owenhilyard](https://forum.modular.com/u/owenhilyard)\ **Post date:** [December 9, 2024, 8:38pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/2 "2024-12-09T20:38:17Z")
Reposted from the discord. It seems like there isn’t any room in that design to support arena allocating the frames, nor any place for handling the allocation of a coroutine frame failing. This is somewhat concerning to me because while being able to move to stack allocations is nice, being able to grab a right-sized allocation from an arena allocator is nicer, especially in the context of ensuring you have enough memory for the coroutine. For frequently allocated coroutines (consider the `handle_request` top-level function of an HTTP server), this means that instead of going through all of the machinery in tcmalloc you may be performing a dequeue operation on a ring buffer of free frames, substantially faster. Would it be possible to have the coroutine take an `alloc: Allocator[CoroutineFrameType] = DefaultMojoAllocator` parameter in some way or otherwise inject an allocator into the coroutine? I’m still thinking over how I would want custom allocators to behave, but I know that this is a feature I and others will want. For my specialty of databases, not being able to handle allocation failures (because the database is likely the largest memory consumer on any system it is on and typically has a lot of caching, so it can actually do something about allocation failures), means that you can’t use the feature in production code because it could lead to unnecessary crashes. One other question was that it didn’t look like the frame was a tagged union as I would expect. Is it represented that way at the level we would see in MLIR reflection? ---
**Author:** ![Caroline](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/caroline/32/8_2.png) [@Caroline](https://forum.modular.com/u/Caroline)\ **Post date:** [December 12, 2024, 3:02pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/3 "2024-12-12T15:02:28Z")
---
**Author:** ![ivellapillil](https://avatars.discourse-cdn.com/v4/letter/i/eada6e/32.png) [@ivellapillil](https://forum.modular.com/u/ivellapillil)\ **Post date:** [December 12, 2024, 4:02pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/4 "2024-12-12T16:02:11Z")
What is the roadmap for the async implementation? ---
**Author:** ![taylorpool](https://avatars.discourse-cdn.com/v4/letter/t/ce73a5/32.png) [@taylorpool](https://forum.modular.com/u/taylorpool)\ **Post date:** [December 12, 2024, 5:13pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/5 "2024-12-12T17:13:02Z")
Your last slide touched on function coloring. How does Modular plan to address that problem, if at all? ---
**Author:** ![Brian-M-J](https://avatars.discourse-cdn.com/v4/letter/b/54ee81/32.png) [@Brian-M-J](https://forum.modular.com/u/Brian-M-J)\ **Post date:** [December 12, 2024, 5:20pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/6 "2024-12-12T17:20:07Z")
Have you had time to look at Rust’s implementation of coroutines? Can we get a comparison b/w Mojo’s coroutine implementation and Rust’s? ---
**Author:** ![a2svior](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/a2svior/32/90_2.png) [@a2svior](https://forum.modular.com/u/a2svior)\ **Post date:** [December 12, 2024, 7:01pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/7 "2024-12-12T19:01:17Z")
Are there docs or examples of how one could use the current async implementation in community libraries? Or is it too early for this ---
**Author:** ![Steffi](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@Steffi](https://forum.modular.com/u/Steffi)\ **Post date:** [December 12, 2024, 10:34pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/8 "2024-12-12T22:34:25Z")
Hi @owenhilyard, thanks for the question! There is room to support custom allocation techniques. The coroutine lowering does depend on an allocator, but we have the power to specify which allocator we use. That specification is not yet exposed in the language. The plan was to migrate from using a malloc call to a bump pointer allocator. The allocator would be created when invoking a coroutine from a synchronous context and passed down. I don’t see why we could not expose this in mojo to allow for user customization. This would only apply to memory coroutines and is separate from memory promotion, which can be skipped using a flag. Re: handling the allocation of a coroutine frame failing, Mojo async is under early development and that case is not currently handled. However we do have a place to handle that situation. Async functions can be throwing functions and coroutines already have error slots so any failure to create a child coroutine can fail in the resume and propagate up the chain via the error slot. ---
**Author:** ![Steffi](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@Steffi](https://forum.modular.com/u/Steffi)\ **Post date:** [December 12, 2024, 10:36pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/9 "2024-12-12T22:36:28Z")
Hey @ivellapillil 👋 Mojo Async is under early development and paused to address some higher priorities but I expect that sometime early to mid next year we will have a production quality version to release. ---
**Author:** ![Steffi](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@Steffi](https://forum.modular.com/u/Steffi)\ **Post date:** [December 12, 2024, 10:37pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/10 "2024-12-12T22:37:30Z")
Hi @taylorpool! Thanks for asking. At this time, we don’t have plans to address function coloring. ---
**Author:** ![Steffi](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@Steffi](https://forum.modular.com/u/Steffi)\ **Post date:** [December 12, 2024, 10:38pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/11 "2024-12-12T22:38:57Z")
Hi @Brian-M-J 👋 I have not yet examined Rust’s implementation but I would like to and upon release we can include a side by side comparison. ---
**Author:** ![Steffi](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@Steffi](https://forum.modular.com/u/Steffi)\ **Post date:** [December 12, 2024, 11:55pm UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/12 "2024-12-12T23:55:26Z")
Hi @a2svior, great question. We don’t have docs or examples yet, but we’re hoping to do this in the new year. ---
**Author:** ![owenhilyard](https://sea1.discourse-cdn.com/flex001/user_avatar/forum.modular.com/owenhilyard/32/41_2.png) [@owenhilyard](https://forum.modular.com/u/owenhilyard)\ **Post date:** [December 13, 2024, 1:43am UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/13 "2024-12-13T01:43:29Z")
A few more questions since it’s been a bit and I’ve spent some time talking about async with @Nick. How is the async scheduler designed right now? While work stealing is great for some workloads, it causes a lot of headaches for others with either cache misses or requiring thread safety bounds (like `Send + Sync + 'static` in Rust), due to the inability to determine whether a task will end up on another thread. This causes issues for things like io\_uring, which is designed to either have one thread do all of the io, have an “IO lock”, or create an io ring per core. Thread per core isn’t as great at work sharing, but tends to make these issues go away and align better with shared-nothing designs. Has any attention been paid to being generic over “asyncness”? Rust ran into issues with this essentially forcing everyone who talks to a database or does things with HTTP into using async. ---
**Author:** ![Brian-M-J](https://avatars.discourse-cdn.com/v4/letter/b/54ee81/32.png) [@Brian-M-J](https://forum.modular.com/u/Brian-M-J)\ **Post date:** [December 13, 2024, 9:03am UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/14 "2024-12-13T09:03:56Z")
> [@Steffi](#): > > upon release we can include a side by side comparison. Now that’s interesting. Is this side by side comparison only going to be between the stackless coroutines of C++, Rust and Mojo or are you considering other concurrency models as well (like Go and [Hylo’s](https://accu.org/journals/overload/32/181/teodorescu/) stackful coroutines, futures/promises, C++26’s [senders and receivers](https://wg21.link/p2300) etc)? ---
**Author:** ![system](https://us1.discourse-cdn.com/flex001/uploads/modular/original/1X/2751e0fbdc595a99718b216730957e9db4448cfd.jpeg) [@system](https://forum.modular.com/u/system)\ **Post date:** [June 11, 2025, 9:03am UTC](https://forum.modular.com/t/ask-steffi-anything-about-efficient-coroutine-implementation-in-mlir/39/15 "2025-06-11T09:03:58Z")
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.