BtrBlocks - Efficient Columnar Compression for Data Lakes

• Paper (two-column version)
• Video (SIGMOD 2023 presentation)

Usage

After building the library, follow one of the examples to get started.

Components

• btrblocks/: the compression library, schemes, utilities, ...
• btrfiles/: helper library for binary files and yaml schema information
• tools/: various conversion, measurement and benchmarking tools
• test/: rudimentary tests for the library

Building

We currently require an x86 platform. This library was built and tested on linux only.

mkdir build 
cd build
cmake ..

Then, depending on your usecase, build only the library or any of the tools:

• build everything: make
• install static library and headers on your system: sudo make install
• build the compression library only: make btrblocks
• build the tests make tester
• build the in-memory decompression speed benchmark: make decompression_speed
• ...

For a list of all valid targets, run make help.

Library was built and tested on Linux (x86, ARM) and MacOS (ARM).

Contributors

Adnan Alhomssi David Sauerwein Maximilian Kuschewski

License

MIT - See License File