Web development support in Mojo

Let’s start a community-driven effort to collect resources and discuss web development in Mojo. Although the Modular team is currently focused on refining the language for GPU programming and AI use cases, there’s a growing interest among community members to explore web development with Mojo.

Inspired by projects like arewewebyet.org (which showcases the Rust ecosystem), I’d love to see if we can create a similar static page for Mojo. To get started, let’s gather resources, share knowledge, and discuss the possibilities. What resources do we need to get started? Are there any existing libraries, frameworks, or projects that we can learn from?

Essential Components of a Modern Web Framework

Core Infrastructure

  1. Lower Level Network Stack

    • TCP/IP
    • Sockets
    • Network protocols
  2. Async I/O and Concurrency

    • Threads
    • Coroutines
    • Event loops
    • Async/await support
  3. HTTP Layer

    • HTTP Clients (similar to requests, httpx)
    • HTTP/2 and HTTP/3 support
    • WebSocket support
    • Server-Sent Events (SSE)

Data Layer

  1. Database Integration

    • SQL drivers (PostgreSQL, MySQL)
    • NoSQL drivers (MongoDB, Redis)
    • Connection pooling
  2. Object-Relational Mapping (ORM)

    • Model definition and relationships
    • Query building and execution
    • Migration management
    • Schema validation
    • Lazy loading
    • Eager loading
    • Transaction management
    • Connection pooling
    • Model hooks and events
    • Inheritance mapping
  3. Serialization & Data Formats

    • JSON, YAML, TOML
    • Binary formats (Protocol Buffers, MessagePack)
    • CSV, XML processing
    • Base64 and other encodings

Security

  1. Authentication & Authorization

    • OAuth 2.0
    • JWT, PASETO
    • Session management
    • Role-based access control
  2. Cryptography

    • Hashing algorithms
    • Encryption/Decryption
    • Digital signatures
    • Security best practices

Development & Testing

  1. Testing Framework

    • Unit testing (pytest)
    • Integration testing
    • Browser testing (Selenium)
    • API testing
  2. Logging & Monitoring

    • Structured logging
    • Tracing
    • Telemetry
    • Debugging tools

Deployment & Operations

  1. Deployment Tools

    • Docker/Containerization support
    • Single binary packaging
    • Environment management
    • Configuration handling
  2. Cloud Integration

    • AWS services
    • Google Cloud Platform
    • Azure services
    • Cloud-native features

Web Features

  1. Templating Engines

    • Server-side templates (Jinja)
    • Template inheritance
    • Component-based templating
  2. Internationalization (i18n)

    • Multi-language support
    • Locale handling
    • Text translation
    • RTL support
  3. Web Utilities

    • Cookie management
    • Session handling
    • MIME types
    • Time/date handling

Communication

  1. Email Support

    • SMTP integration
    • IMAP support
    • Email templates
    • Attachments handling
  2. Content Syndication

    • RSS feeds
    • Atom feeds
    • Content aggregation
    • Feed generation
4 Likes

Threads and coroutines are kinda just working, async await implemented.

In terms of Cryptography, that’s something we can already do with what the language has to offer right now, I think that it’s a good place to start while we wait on networking

I’m sorry, but this feels like a copy paste. How does AWS support fit into here?
Or telemetry, for example? That’s in no way an essential component of a browser, and I reckon Google Chrome has decieved you here :slight_smile:
Edit: Post seems oddly structured, as if an AI model were to write the body) Or maybe you are online much :wink:

A network stack isn’t really workable until we get trait objects and parametric traits. While threads exist, the runtime already spawns a thread per core (obeying cgroup restrictions), which is what a web server should want, so we don’t need to do much extra there.

Nick! and I have spent the last few weeks arguing back and forth on async design, and at a minimum the current solution doesn’t work very well for high throughput due to the allocations it creates. We’re moving more in a structured concurrency direction to deal with some problems that come from work stealing async executors.

Until we can solidify those layers, working on anything else that list is difficult to impossible.

We also haven’t nailed down what the basic IO traits will look like, so anything that wants to do IO that assumes anything more capable than an iterator of bytes may end up with issues. Those are likely to get broken several times, so building anything big that relies on behavior that looks more like POSIX IO will probably go poorly.

Here’s what I think can be productively worked on right now:

  • HTTP: Sans-io HTTP implementations which take a buffer as input and return a zero-copy
  • Data format parsers: Again, for now take a byte iterator as input.
  • Basic cryptography, but that probably needs to be written by an actual cryptographer if it is designed to go into std. Right now I’m not actually sure Mojo has the ability to write sound cryptographic code, so calling out to openssl may be a better idea.
  • Templating engines should be doable.
  • Date/Time handling is also good to go, but there be dragons

As an additional note, Mojo already has a unit test framework.

3 Likes

Hey @sazid, this feels a bit AI-generated – just a reminder to please not post AI-generated questions and help us keep this forum a genuinely fun place to engage :slight_smile: AI-generated questions/content won’t count towards the swag challenges.

Is there already the event loop implemented? I didn’t see one. Probably I didn’t look hard enough?

1 Like

An event loop is just a while loop, a switch statement (currently a bunch of ifs, match once mojo gets it), and some way to pull from an event queue. Epoll is what made them a headache to implement, but we’re mostly free from that now with all major OSes having completion-based IO mechanisms.

1 Like

Hey everyone, let me clear up the concerns:

this feels a bit AI-generated

Yes! However, the list of the topics you see - I did hand pick them - put them all together in a single list and then let AI just reorganize it into proper headings. If that’s not allowed, I can try bringing my original list back.

How does AWS support fit into here? Or telemetry, for example?

Mostly related to deployment of web services - although not core/fundamental to web development itself. Thought I would just put a mention of cloud services here. But I was skeptical of whether I should be putting this category or not.

Telemetry - logging, tracing, etc. are necessary I believe when things go into production. However, they are honestly at the bottom of the priority list until we have a web stack up and running.

1 Like

For the IO traits - I really like how go handles IO with its different combinations of IO interfaces. The Reader and Writer interfaces are clean and universal - allowing any type to automatically become a reader/writer if they implement those methods. However:

  • I wish it allowed something like a context manager to auto clean up resources. Although there’s the defer keyword and Closer family of interfaces.
  • Enforce better error handling - exhaustive checks for errors with tagged unions/sum types. Right now, you need to read the docs to understand what type of errors can be expected, what if a dev forgot to mention a variant of error?

Take a look at the docs for the Reader interface:

There’s tons of conditions - you as a programmer need to be aware of. If these could all be enforced as part of the type system - sort of like a contract - so you’re forced to handle the cases or neglect explicitly, it would be great.

Go’s Reader and Writer are not actually universal, since they neglect:

  • Vectored IO
  • IO which gives out the buffer (io_uring, kernel bypass networking)
  • Block size requirements for things like Direct IO

io_uring-style APIs will probably win since that is where the state of the art is heading.

Yes, we will get some types at some point, and probably checked exceptions as well.

Automatic resource cleanup can be handled in type destructors.

3 Likes

@Caroline, why is there a policy against AI-generated content? I’ve found AI tools to be valuable writing assistants that can help create well-structured content more efficiently. As long as users carefully review and verify the output matches their intended message, this seems beneficial for both the writer and the readers. Additionally, given Modular’s focus on advancing AI technology, discouraging AI-assisted content feels somewhat contradictory.

1 Like

Hey @elliotwaite, great question. There’s no issue with folks using AI to edit/supplement their own content or responses – the concern is focused more on community members copy/pasting AI-generated responses directly. For this thread, it sounds like @sazid just used AI to supplement their own content, so no issues. Let me know if that makes sense or if you have any other questions!

I would say a big framework would be JavaScript for sure.

@Caroline, thanks for the reply. Perhaps the policy could be updated to encourage responsible use instead of banning it. I’ve started a separate topic for further discussion here.

1 Like