How to use path or git dependencies - migrating to pixi-build-rattler-build

[duck_tape (Seth Stadick)]

tl;dr: Follow this tutorial in order to use the rattler-build backend for your package so it can be relied on via path/git dependencies. This is a PR for a mojo package to do that.

Motivation

pixi supports using git and path dependencies when the workspace.preview = "pixi-build" feature is enabled.

This is the error message you might currently be getting:

❯ pixi run build                                                                                                                                                                                                           
Error:   × source dependencies are not allowed without enabling pixi-build
    ╭─[/Users/u103330/dev/mojo_adt_example/pixi.toml:18:13]
 17 │ modular = ">25.3.0"
 18 │ extramojo = { git = "https://github.com/ExtraMojo/ExtraMojo.git" }
    ·             ───────────────────────────┬──────────────────────────
    ·                                        ╰── source dependency specified here
 19 │ zlib = ">=1.3.1,<2"
    ╰────
  help: Add `workspace.preview = ["pixi-build"]` to enable pixi build support

You can update your pixi.toml file to use the pixi.build so it should look something like this:

[workspace]
authors = ["duck_tape <duck_tape@email.com>"]
channels = [
    "https://conda.modular.com/max-nightly",
    "https://repo.prefix.dev/modular-community",
    "conda-forge",
]
name = "example"
platforms = ["osx-arm64"]
preview = ["pixi-build"]
version = "0.1.0"


[tasks]
build = "pixi run mojo build -D MOJO_LOG_LEVEL=info -I . ./main.mojo -o example"

[dependencies]
modular = ">25.3.0"
extramojo = { git = "https://github.com/ExtraMojo/ExtraMojo.git" }

However, the next issue you will run into is that that the package you are trying to install needs to have a package section, to specify how to build it. Example error message when I try to add Kelvin:

❯ pixi run build
ERROR error setting up build frontend: the pixi.toml does not describe a package
Error:   × failed to extract metadata for package 'kelvin'
  ├─▶   × the initialization of the build backend for 'https://github.com/bgreni/Kelvin.git@4c2e78d5df2d3ec838d322b9224f447302a8a5c1' failed
  │   
  ╰─▶   × the pixi.toml does not describe a package
        help: A [package] section is missing in the manifest

Solution

In addition to using the preview = ["pixi-build"], if we specify our build system as pixi-build-rattler-build, we can then install the packages via git/path references. Here is the PR migrating ExtraMojo to using the rattler-build backend: feat: switch to rattler-build backend by sstadick · Pull Request #27 · ExtraMojo/ExtraMojo · GitHub. And a pixi tutorial that walks through setting it up as well: Advanced Building Using rattler-build - Pixi by prefix.dev.

This means adds some duplication between your pixi.toml and your recipe.yaml. But if you were publishing packages to modular-community you already needed a recipe.yaml for rattler-build there.

Unanswered Questions

• Can we rely on multiple layers of git deps in pixi tomls? I think you can maybe run pixi install inside of your recipe.yaml to fetch things for you. So maybe? Seems gross though.
• How does this work with nightly / lock files? I’m not sure yet. i.e. as long as you have an appropriately defined max/modular dep that is something like >25.stable, and your code hasn’t been broken by nighly, I think this is pulling and building for your local source and should match version.