I’d like to avoid libc for debug/compat reasons.
What would be the mojo equivalent for a simple mmap call requesting 1 page?
void *addr = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
I don’t think Mojo currently has the ability to select the “syscall” ABI so I’m not sure if you really can without doing a pile of inline assembly. On macOS, Windows and BSD, as far as I am aware there is no stable API to the kernel except libc.
Only macos really has a guaranteed globally linked libc.
Whereas on Windows syscalls are done through DLL wrappers like kernel32.
Many libc functions have cursed semantics
Not all syscalls have libc wrappers, e.g. it is not possible to call perf_event_open.
Wouldn’t you need assembly clobbers to make syscalls?
I know that implementing the functionality that libc provides requires a significant investment. However, I think it is finally time to take Linux seriously, as I expect its adoption to increase significantly.
The Linux kernel is generally in good shape because of significant corporate investment.
It is used in Android, servers, AI, embedded systems, and much more—it runs the world.
The Linux desktop gets this for free.
As is usually the case in areas without significant corporate investment, the state of the Linux desktop is dire.
The Linux desktop is massively underfunded and has significant technical debt.
The only positive is that limited resources have led to a modular design, which drives competition and innovation.
Thus, you can replace one part of the Linux desktop, and doing so is just a matter of a few million dollars.
New Linux desktop environments are not taken seriously in the Linux community.
They start with great mockups and promises, but then they get massively delayed and exceed the budget. Then, they are either canceled or stripped of features.
Existing desktop environments have problems, as seen with Gnome Shell, but KDE is doing a similar thing.
Gnome-shell is written in C and JavaScript. Most of its code exists within a single-threaded JavaScript execution context. These languages either don’t support multithreading or don’t provide sufficient higher-level abstractions or safety guarantees.
Gnome-shell is effectively a single monolith. Extensions are injected into the JavaScript context and modify the shell’s objects directly.
Thus, bugs in an extension can affect other extensions and the entire session.
Even worse, one JavaScript exception can crash the entire session.
Since the extensions are not isolated, they regularly break when the internals change in a new version of Gnome Shell.
This design also does not offer great security or sandboxing.
Another approach is the aggressive use of processes and IPC.
This solves the problems of Gnome Shell. , it also drastically increases implementation complexity in the compositor and for plug-ins like the dock and workspace view.
This approach has already resulted in significant technical debt and an inability to implement planned features.
This approach also requires hacks, such as sub-compositors. Additionally, the memory usage is much higher. Twenty of these applets that do nothing but sit idle in the background can easily consume 1 GB of memory. Resources cannot be shared effectively; for example, each applet queries the system and initializes its own font rendering engine.
The compositor is one level below the kernel and knows everything about the session.
It is much easier to implement features in the compositor. However, this leads to the dilemma that, while features are significantly easier to implement in the compositor, they have low modularity and code reuse and create a giant monolith.
Another approach is to use WASM plugins in the compositor.
This provides WASM-level isolation, solving all these problems.
You don’t have to build a gigantic monolith, yet you can still provide a robust user experience.
Plugins can also be accurately sandboxed on a per-plugin level.
Thus, permissions can be configured on a per-plugin basis, and communication between the plugin and compositor can be as simple as a WASM function call.
How to Build a Great Desktop Environment
Avoid technical debt and implementation complexity without making the compositor a giant monolith.
Use WASM modules for isolation while drastically reducing implementation complexity.
A strong focus is required.
Don’t focus on apps that aren’t critical to the core desktop; only the settings and welcome apps are relevant. Defer developing other apps to a later stage.
Don’t focus on design; you can easily add some polish later.
Focus on delivering a great compositor, not something half-baked.
Have a vertical stack and prove that the design works before implementing features.
Don’t hinder yourself by choosing a toolkit just because it’s functional and fancy.
Furthermore, I expect new languages to emerge in the future that have better tooling and allow for highly generic, readable code. This would greatly reduce the CAPEX required to build a great desktop environment.
Not linking libc by default and making direct SYSCALLS has worked great for Zig and Go.
I encourage Mojo to take the same approach, since the goal should be to provide low-level abstractions at the software level as well as the hardware level.
The Linux kernel is the only thing all Linux distributions have in common.
It’s not just about compatibility with distributions that don’t link libc globally; it’s also about correctness and efficiency.
getaddrinfo calls getenv, which causes problems in multi-threaded programs (see Proposal: make environments a non-global resource · Issue #25962 · ziglang/zig · GitHub).
Since Mojo does not have global variables, the solution is to simply use global variables in libc.
I agree with you leb that I’d much rather write the code myself and avoid C-interop and the 50 someodd years of cruft that comes with it. I’ve written linux threading and some allocators here. If you’re just interested in mmap, that’s here
I’m actively working on this area as I find mojo code to be far cleaner and less hastle than dealing with jthread or something. Let me know if you have any questions.
Direct syscalls only work on Linux, every other OS demands you use libc, and we have the issue of not really having access to NVIDIA GPUs if you don’t like glibc. Perhaps once more stuff is open source it will be more doable to work on that, but first we’d need to figure out what doesn’t work when you do that and put it behind a libc feature flag (after adding feature flags of course).
My 2c is that libc should be an avenue of last resort. Libc has serious design and performance problems and treating OS like they’re homogenous means we end up losing out on major optimization features like gigantic pages or RSEQ. These are small details that could have 20-30% performance implications for a large ML system if ignored/inaccessible. Although I’m less worried about linux-first and more worried about libc leaking bad design and non-vectorized code into Mojo’s currently very clean interfaces/abstractions.
This is still not true. On Windows, you need to install a separate toolchain.
We could have a compatibility layer where most operations are direct system calls, but globals would be initialized to match the behavior of libc if libc were called.