Skip to content

BtrBlocks: Efficient Columnar Compression for Data Lakes (SIGMOD 2023 Paper)

License

Notifications You must be signed in to change notification settings

maxi-k/btrblocks

Repository files navigation

BtrBlocks - Efficient Columnar Compression for Data Lakes

Paper Build

  • Paper (two-column version)
  • Video (SIGMOD 2023 presentation)

Usage

After building the library, follow one of the examples to get started.

Components

  • btrblocks/: the compression library, schemes, utilities, ...
  • btrfiles/: helper library for binary files and yaml schema information
  • tools/: various conversion, measurement and benchmarking tools
  • test/: rudimentary tests for the library

Dependency Graph

Building

We currently require an x86 platform. This library was built and tested on linux only.

mkdir build 
cd build
cmake ..

Then, depending on your usecase, build only the library or any of the tools:

  • build everything: make
  • install static library and headers on your system: sudo make install
  • build the compression library only: make btrblocks
  • build the tests make tester
  • build the in-memory decompression speed benchmark: make decompression_speed
  • ...

For a list of all valid targets, run make help.

Library was built and tested on Linux (x86, ARM) and MacOS (ARM).

Contributors

Adnan Alhomssi David Sauerwein Maximilian Kuschewski

License

MIT - See License File

About

BtrBlocks: Efficient Columnar Compression for Data Lakes (SIGMOD 2023 Paper)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •