Skip to content

ShyftNetwork/RedisBloom

Repository files navigation

GitHub issues CircleCI Dockerhub codecov

RedisBloom: Probabilistic Data Structures for Redis

Forum Discord

logo

Overview

RedisBloom adds a set of probabilistic data structures to Redis, including Bloom filter, Cuckoo filter, Count-min sketch, Top-K, and t-digest. Using this capability, you can query streaming data without needing to store all the elements of the stream. Probabilistic data structures each answer the following questions:

  • Bloom filter and Cuckoo filter:
    • Did value v already appear in the data stream?
  • Count-min sketch:
    • How many times did value v appear in the data stream?
  • Top-k:
    • What are the k most frequent values in the data stream?
  • t-digest:
    • Which fraction of the values in the data stream are smaller than a given value?
    • How many values in the data stream are smaller than a given value?
    • Which value is smaller than p percent of the values in the data stream? (What is the p-percentile value?)
    • What is the mean value between the p1-percentile value and the p2-percentile value?
    • What is the value of the nᵗʰ smallest/largest value in the data stream? (What is the value with [reverse] rank n?)

Answering each of these questions accurately can require a huge amount of memory, but you can lower the memory requirements drastically at the cost of reduced accuracy. Each of these data structures allows you to set a controllable trade-off between accuracy and memory consumption. In addition to having a smaller memory footprint, probabilistic data structures are generally much faster than accurate algorithms.

Redis Bloom is part of Redis Stack.

Getting started

  1. Launch RedisBloom with Docker
  2. Use RedisBloom with redis-cli or with RedisInsight

Note: You can also build and load the module yourself.

1. Launch RedisBloom with Docker

docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest

2. Use RedisBloom with redis-cli

docker exec -it redis/redis-stack-server bash

# redis-cli
# 127.0.0.1:6379>

Create a new bloom filter by adding a new item:

# 127.0.0.1:6379> BF.ADD newFilter foo
(integer) 1

Find out whether an item exists in the filter:

# 127.0.0.1:6379> BF.EXISTS newFilter foo
(integer) 1

In this case, 1 means that the foo is most likely in the set represented by newFilter. But recall that false positives are possible with Bloom filters.

# 127.0.0.1:6379> BF.EXISTS newFilter bar
(integer) 0

A value 0 means that bar is definitely not in the set. Bloom filters do not allow for false negatives.

Building and Loading RedisBloom

To build RedisBloom, ensure you have the proper git submodules, and afterwards run make in the project's directory.

git submodule update --init --recursive
make

If the build is successful, you'll have a shared library called redisbloom.so.

To load the library, pass its path to the loadmodule directive when starting redis-server:

$ redis-server --loadmodule /path/to/redisbloom.so

Client libraries

Project Language License Author Stars Package Comment
jedis Java MIT Redis Stars Maven
redis-py Python MIT Redis Stars pypi
node-redis Node.JS MIT Redis Stars npm
nredisstack .NET MIT Redis Stars nuget
redisbloom-go Go BSD Redis Stars GitHub
rueidis Go Apache License 2.0 Rueian Stars GitHub
rebloom JavaScript MIT Albert Team Stars GitHub
phpredis-bloom PHP MIT Rafa Campoy Stars GitHub
phpRebloom PHP MIT Alessandro Balasco Stars GitHub
vertx-redis-client Java Apache License 2.0 Eclipse Vert.x Stars GitHub
rustis Rust MIT Dahomey Technologies Stars GitHub

Documentation

Documentation and full command reference at redisbloom.io.

Mailing List / Forum

Got questions? Feel free to ask at the RedisBloom mailing list.

License

RedisBloom is licensed under the Redis Source Available License 2.0 (RSALv2) or the Server Side Public License v1 (SSPLv1).