I’ve been building Pion: a key-value and vector database that speaks the Redis protocol (RESP2/3), written in Mojo from the socket up. Its main job is to be a local model’s memory. It keeps a prompt’s K/V cache across processes and restarts, so mlx-lm skips a long prefill it has already done once.
The engine is Apache-2.0. One part is not: the tuned 1536-dim vector-search kernels ship as a free, closed static library. An open reference implementation of the same algorithms is in the repo, and a differential test checks on every release that the two return identical results. pixi run build-open builds everything from source; vector search at 1536 dims is 22–34% slower that way, and nothing else changes. I’d rather say that here than have you find it.
Numbers (M4 Mac mini; the scripts that produce them are in the repo):
- Prompt cache, Llama-3.2-1B-4bit with a 2,049-token prefix and a short question: time to first token goes from 1,242 ms cold to 61.9 ms in the same process (20×), and to 73.9 ms (17×) from a separate process over the wire. A shorter prefix saves less: from a separate process, 1.5× at 34 tokens, 4.5× at 268 and 11× at 1,035.
- KV on one worker, `valkey-benchmark -P 10`: GET 2.1M and SET 2.2M requests/s.
- Vector search (VectorDBBench, 50K OpenAI 1536-d vectors, ef=150): about 9.4K QPS at recall@100 0.960.
Most of the code was written by Claude Code working to my direction. Every change goes through the same gate before it’s pushed: about 130 test files, a differential run against a real Redis, and benchmark floors that never move to make a run pass. The gate is no substitute for users, though: nobody but me has used Pion yet, so there may well be a lot of bugs, because the tests only cover what I thought to test. The two compiler bugs below surfaced that way, as a crash that happened only at -O3 and a decode that disagreed with its reference.
Mojo things that might be useful to you:
- a zero-allocation request path: RESP parsed in place, a 32-byte value type with 23-byte inline strings, replies written into a preallocated buffer;
- workers as raw pthreads with explicit 16 MB stacks (
parallelizemoved tomax, and -O0 frames overflowed the default pool stacks); - a Swiss table that probes 16 control bytes per SIMD compare;
- SDOT / VNNI / SSE int8 kernels behind one
sdot_int8, and what it cost when the SSE arm was quietly unsigned-by-signed; - two compiler behaviours worth knowing, with minimal repros in
tools/mojo_repros/: a pointer into astack_allocationbuffer passed to an out-of-line call is emitted as atail call, and at -O3 the caller’s stores into that buffer disappear (filed as modular/modular#7241, and still there in the October 4 nightly); and an int8 → uint8 → uint64 cast chain that sign-extended (fixed in 1.1, modular/modular#6935).
Repo: github.com/pavelhorak/pion site: pion.pavelhorak.com