Mojo native version of Snappy compression

Community Showcase
1

Hi all,

I’d like to introduce mojo-snappy, a mojo-native implementation of the snappy compression algorithm. I have been working for a while on a bigger project to read / write Apache parquet files in pure mojo. Snappy compression is one of the supported algorithms by parquet ( Compression | Parquet ). Out of the collection of compression algorithms, this seemed like the most straight-forward to extract into a small library.

The reference implementation against which development, benchmarking and profiling was done is Google’s cpp implementation. https://github.com/google/snappy

Mojo-snappy is not quite as fast or as efficient as Google’s implementation, but it is close enough that it is usable. Correctness has been tested extensively, but I’ll look for other snappy implementations for further evaluation and improvement.

The only dependency is Mojo 1.0.0

I would like to get this package onto the modular-community - prefix.dev .

2

Hey @StrGenIx, thanks so much for sharing this! Looks like you were able to get it added to the community channel :partying_face: Have you checked out related project marrow?

3

Hi @Caroline , yes I was able to figure it out thanks to the documentation on the forum :slight_smile: .
I already knew about marrow. At some point I doubted making the arrow columnar format the data back-end for pyroquet itself. Ultimately I went with NuMojo for now because it’s more general.