DuckDB.mojo: Mojo bindings for DuckDB

I finally got the bandwidth to continue working on my Mojo bindings for DuckDB, a fast analytical columnar database, so I’d like to add it here.

The project not completely new. I actually started working on it last summer, when the community showcases were still in discord as the forum didn’t exist yet. I also presented it at a community meeting about one year ago.

I took a break then, as I didn’t have the bandwidth and was stuck with an issue around passing structs by value into c , but kept watching Mojo mature and improve at an amazing speed.

Recently I started picking up the code again, and I’m making good progress towards a more usable package now:

  • Updated to Mojo 0.25.6
  • Found a workaround for the issue that had blocked me by adding a small c wrapper so I can pass the struct by pointer from Mojo.
  • Created a proper build using Pixi with the Mojo backend, as well as automatically building the c wrapper as a source dependency (this can probably be simplified).

I’ve now started to work on adding support for writing scalar DuckDB functions in Mojo, as I think it could be a very good fit, using it to write highly performant vectorized functions, leveraging SIMD or even GPUs.

I used these around 6 months ago! Awesome project, having access to duckdb as a swiss army knife for working with data is huge.

Amazing!! Thanks for sharing :slight_smile: We can highlight this in our next community blog!

I got some bandwidth to improve the library with one significant feature update: Writing fast DuckDB UDFs in Mojo with minimal boilerplate is now a thing.

From the Readme:

Scalar Functions

Register Mojo functions as DuckDB scalar functions (UDFs) that operate on table
columns. There are several convenience levels:

Stdlib math functions (zero boilerplate)

Pass Mojo stdlib math functions directly — types and SIMD vectorization are
handled automatically:

import math
from duckdb import *
from duckdb.scalar_function import ScalarFunction

var conn = DuckDB.connect(":memory:")

# Register stdlib math functions as SQL scalar functions — one line each
ScalarFunction.from_simd_function["mojo_sqrt", DType.float64, math.sqrt](conn)
ScalarFunction.from_simd_function["mojo_sin",  DType.float64, math.sin](conn)
ScalarFunction.from_simd_function["mojo_cos",  DType.float64, math.cos](conn)
ScalarFunction.from_simd_function["mojo_exp",  DType.float64, math.exp](conn)
ScalarFunction.from_simd_function["mojo_log",  DType.float64, math.log](conn)

# Binary stdlib functions work too
ScalarFunction.from_simd_function["mojo_atan2", DType.float64, math.atan2](conn)

# Now use them in SQL
var result = conn.execute("SELECT mojo_sqrt(x), mojo_sin(x) FROM my_table")

Custom SIMD functions

Write your own SIMD-vectorized kernels for fused computations:

fn sin_plus_cos[w: Int](x: SIMD[DType.float64, w]) -> SIMD[DType.float64, w]:
    return math.sin(x) + math.cos(x)

# Register — processes data in hardware-optimal SIMD batches automatically
ScalarFunction.from_simd_function[
    "mojo_sin_plus_cos", DType.float64, DType.float64, sin_plus_cos
](conn)

Row-at-a-time functions

For simple per-row logic without manual SIMD:

fn add_one(x: Int32) -> Int32:
    return x + 1

ScalarFunction.from_function["add_one", DType.int32, DType.int32, add_one](conn)

And if my benchmark is correct, we can get some nice speedups there, even without fusing ops:

======================================================================
Math Function Benchmark: Mojo SIMD vs DuckDB Builtins
======================================================================
Rows:         100000000
Type:         FLOAT (float32)
SIMD Width:   16 (auto-detected)
Max Iters:    50

Setting up database...
Generating test data...
  math_data rows: 100000000

Registering Mojo scalar functions...
  Unary:  mojo_sqrt, mojo_sin, mojo_cos, mojo_exp, mojo_log, mojo_abs
  Fused:  mojo_sin_plus_cos, mojo_hypot1, mojo_gauss
  Binary: mojo_hypot, mojo_atan2

Validating correctness (first row)...
  sqrt: std=4.774824 mojo=4.774824
  sin:  std=-0.722767 mojo=-0.722767
  Aggregate check — sqrt: True, sin: True, cos: True

======================================================================
UNARY FUNCTIONS
======================================================================

  sqrt(x)
  -------
    Standard DuckDB:  33.45217866666667 ms
    Mojo SIMD:        11.59015825 ms
    Speedup:          2.886257283559236x faster

  sin(x)
  ------
    Standard DuckDB:  141.4621541111111 ms
    Mojo SIMD:        63.32670477777778 ms
    Speedup:          2.2338467571859533x faster

  cos(x)
  ------
    Standard DuckDB:  140.48050944444446 ms
    Mojo SIMD:        62.691406111111114 ms
    Speedup:          2.24082562760618x faster

  exp(x) [clamped]
  ----------------
    Standard DuckDB:  64.79612444444444 ms
    Mojo SIMD:        22.534778499999998 ms
    Speedup:          2.8753832412616993x faster

  log(x)
  ------
    Standard DuckDB:  62.02466966666667 ms
    Mojo SIMD:        14.127817499999999 ms
    Speedup:          4.390251336886725x faster

  abs(x)
  ------
    Standard DuckDB:  23.4139548 ms
    Mojo SIMD:        23.3906847 ms
    Speedup:          1.0009948447554422x faster

======================================================================
FUSED COMPUTATIONS (single Mojo function vs DuckDB expression)
======================================================================

  sin(x) + cos(x)
  ---------------
    Standard DuckDB:  282.3374547777778 ms
    Mojo SIMD:        70.57823455555555 ms
    Speedup:          4.000347367084342x faster

  sqrt(x*x + 1)
  -------------
    Standard DuckDB:  55.04654788888889 ms
    Mojo SIMD:        11.883272230769231 ms
    Speedup:          4.632271887734546x faster

  exp(-x*x) [Gaussian]
  --------------------
    Standard DuckDB:  78.87852133333334 ms
    Mojo SIMD:        22.2693132 ms
    Speedup:          3.5420275706272495x faster

======================================================================
BINARY FUNCTIONS
======================================================================

  sqrt(x*x + y*y) [hypot]
  -----------------------
    Standard DuckDB:  67.68933611111112 ms
    Mojo SIMD:        18.2338536 ms
    Speedup:          3.7122891077238394x faster

  atan2(y, x)
  -----------
    Standard DuckDB:  151.972219 ms
    Mojo SIMD:        140.45342233333335 ms
    Speedup:          1.0820115058451867x faster

======================================================================
Benchmark complete.
======================================================================

Another update thanks to the amazingly evolving Mojo type-level programming and reflection support.

A lot of the design was stolen from inspired by EmberJson (thanks @bgreni).

Reflection-based typed API

Type mapping between Mojo and DuckDB types is now derived automatically at compile time via struct reflection. The old Col[T] approach is removed in favor of get[T] methods taking the actual target type as parameters, including NULL handling.

get[T] expects non-null values and raises at runtime if a NULL is encountered. You can use get[Optional[T]] when nulls are expected:

var chunk = result.fetch_chunk()
var name = chunk.get[String](col=0, row=0) # raises on NULL
var maybe = chunk.get[Optional[String]](col=0, row=0)  # None on NULL
var ages = chunk.get[Int64](col=1)  # whole column as List[Int64]
var ages = chunk.get[Optional[Int64]](col=1)  # whole column as List[Optional[Int64]]

Type mismatches (e.g. get[Int64] on a VARCHAR column) are caught and raise with a descriptive error.

Supported types so far: Bool, Int8Int64, UInt8UInt64, Float32, Float64, String, Date, Time, Timestamp, Interval, List[T], Optional[T], and nested combinations.

Struct decoding

You can deserialize full rows into Mojo structs — fields are matched to columns by position:

@fieldwise_init
struct User(Copyable, Movable):
    var name: String
    var age: Int64

var user = chunk.get[User](row=0)       # single row
var users = chunk.get[User]()           # all rows as List[User]

The NULL handling also applies to struct fields — non-Optional fields raise on NULL, Optional fields allow it:

@fieldwise_init
struct Record(Copyable, Movable):
    var id: Int64                  # must not be NULL
    var nickname: Optional[String] # can be NULL

Recursive decoding of nested types

Nested DuckDB structs, lists , and mixed nesting all work with the typed API:

con = DuckDB.connect(":memory:")
_ = con.execute("CREATE TABLE t (nums INTEGER[])")
_ = con.execute("INSERT INTO t VALUES ([10, 20]), ([30])")
result = con.execute("SELECT nums FROM t")
var chunk = result.fetch_chunk()

var lists = chunk.get[List[Optional[Int32]]](col=0)

assert_equal(len(lists), 2)

var row0 = lists[0].copy()
assert_equal(len(row0), 2)
assert_equal(row0[0].value(), 10)
assert_equal(row0[1].value(), 20)

var row1 = lists[1].copy()
assert_equal(len(row1), 1)
assert_equal(row1[0].value(), 30)

Nested Mojo structs:

@fieldwise_init
struct Point(Copyable, Movable):
    """Simple 2D point with two Float64 fields."""
    var x: Float64
    var y: Float64

@fieldwise_init
struct LabeledPoint(Copyable, Movable):
    var label: String
    var pt: Point

con = DuckDB.connect(":memory:")
_ = con.execute(
    "CREATE TABLE t (label VARCHAR, pt STRUCT(x DOUBLE, y DOUBLE))"
)
_ = con.execute("INSERT INTO t VALUES ('origin', {'x': 0.0, 'y': 0.0})")
_ = con.execute(
    "INSERT INTO t VALUES ('offset', {'x': 1.5, 'y': 2.5})"
)
var result = con.execute("SELECT label, pt FROM t")
var chunk = result.fetch_chunk()

var row0 = chunk.get[LabeledPoint](row=0)
assert_equal(row0.label, "origin")
assert_equal(row0.pt.x, 0.0)
assert_equal(row0.pt.y, 0.0)

var row1 = chunk.get[LabeledPoint](row=1)
assert_equal(row1.label, "offset")
assert_equal(row1.pt.x, 1.5)
assert_equal(row1.pt.y, 2.5)

Nice use of static reflection, this is really cool!

Great to see progress on the duckdb bindings! I’m also working on db bindings (sqlite3), and there’s plenty I can learn from your implementation :slight_smile:

Time for another update. Since the last post the project moved to Mojo 1.0 and DuckDB 1.5.5, gained prepared statements, a more mature extension story, and an experimental GPU acceleration path based on MAX.

Prepared statements and a more Python-like API

Parameterized queries: bind positionally (? / $1) or by name ($name), and Optional[T] binds SQL NULL for None. executemany prepares once and re-binds per row, and con.prepare(...) returns a statement you can re-bind and re-run.

The rest of the client API also aligns more closely with DuckDB’s Python API: DB-API-style fetchone[T]/fetchmany[T] on results, read_csv and friends, a module-level duckdb.sql, and a lazy, composable Relation API that mirrors DuckDB’s Python relational API (con.sql(...).filter(...).aggregate(...) only runs at a terminal like show or get[T]).

DuckDB 1.5.5

Updating DuckDB is now simpler. The Mojo FFI layer is auto-generated from the same declarative JSON descriptors that DuckDB uses to generate its duckdb.h header (the C API function definitions plus the stable/unstable extension API descriptors).

The typed API gains the new VARIANT and GEOMETRY types (Map support landed in the typed API and the appender since the last update).

Mojo 1.0

The update to Mojo 1.0 brought type-level and lifecycle improvements that the typed API benefits from, and much better FFI. The abi("C") function effect fixed struct-by-value passing, so the C shim the project originally needed is removed, and the generated FFI wrappers now track pointer origins instead of erasing them.

Lifecycle improvements: Data access on chunks and vectors is parametric over origin and mutability (following the stdlib’s List.unsafe_ptr pattern), so a pointer into a Chunk can no longer outlive it, that is a compile error now, and writing requires a mutable binding. This means no every unsafe_mut_cast[True]() in the codebase. For UDF authors the visible change is that callbacks now take mut output: Chunk.

Extensions written in Mojo

The extension development path is now more than a POC. An extension is a Mojo init function that receives a Connection and registers your functions, plus a small C entry point:

fn add_numbers(a: Int64, b: Int64) -> Int64:
    return a + b

fn init(conn: Connection[ApiLevel.EXT_STABLE]) raises:
    ScalarFunction.from_function[
        "mojo_add", DType.int64, DType.int64, DType.int64, add_numbers
    ](conn)

@export("my_ext_init_c_api")
fn my_ext_init_c_api(
    info: duckdb_extension_info,
    access: UnsafePointer[duckdb_extension_access, MutExternalOrigin],
) abi("C") -> Bool:
    return Extension.run[init](info, access)

Built as a shared library, it loads with the usual LOAD (with allow_unsigned_extensions for local builds). DuckDB’s Extension C API is split into a stable and an unstable level. In this exampleExtension.run targets the stable struct, which is append-only, so a compiled extension stays forward-compatible with future DuckDB releases of the same API major version, but you can also target unstable or (with shims), the C++ API which allows you to access more parts of DuckDB at the cost of being tied to a single DuckDB version.

This is what the accelerators below are built on: they are themselves extensions written in Mojo.

Experimental GPU acceleration

In addition to the existing (and improved) SIMD support, DuckDB operators can now be accelerated on the GPU. The compute kernels are written in Mojo on top of MAX (max.gpu, max.algorithm.parallelize) and hooked in via a DuckDB OptimizerExtension. Supported plans, like aggregation over filter/join and vector-search top-k, route to the GPU transparently, so unchanged SQL just gets faster, and EXPLAIN shows the GPU operator. Anything unsupported, or any GPU runtime error, falls back to stock DuckDB with decimal-exact results. Tested on NVIDIA and Apple GPUs.

The SIMD path got a similar extension: it rewrites builtins in place (sqrt/sin/cos/ln/exp, sum/avg/min/max, vector distances, and a fused one-pass kernel for sum/avg of transcendentals), so existing queries speed up without renaming functions. It links the kernels straight in, so its .so is self-contained with no Mojo runtime dependency.

Conda packaging

The bindings are now a conda package (duckdb-mojo), with the SIMD kernels precompiled in. Submitting it to the modular-community channel is the next step.