They work as window functions, and they also support creating roll-up tables, e.g. you have a table storing a sketch of daily active users and can use that to get weekly and monthly active users.
Same for quantiles (e.g. median and tail latencies) and next up top-K (e.g. most popular content, highest spenders, etc).
> Unsure uses a blocked Bloom filter. The bit array is divided into 512-bit blocks, each matching the size of a CPU cache line. When you add or check an item, it is hashed once: part of the hash selects a block, and the rest selects k bits within that block.
The article also has a good quality list of references.
I'm in the process of adding a few of these to my Go SQLite driver.
https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/approx
They work as window functions, and they also support creating roll-up tables, e.g. you have a table storing a sketch of daily active users and can use that to get weekly and monthly active users.
Same for quantiles (e.g. median and tail latencies) and next up top-K (e.g. most popular content, highest spenders, etc).
I'm kind of sad there's no mention of Cuckoo filters. I've generally found them preferable to Bloom filters
A related CppCon talk: Fantastic Algorithms and Where To Find Them & the related GitHub Repo:
- https://youtu.be/YA-nB2wjVcI
- https://github.com/CppCon/CppCon2017
Now I've learnt about blocked Bloom filters:
> Unsure uses a blocked Bloom filter. The bit array is divided into 512-bit blocks, each matching the size of a CPU cache line. When you add or check an item, it is hashed once: part of the hash selects a block, and the rest selects k bits within that block.
The article also has a good quality list of references.
My introduction to probabilistic data structures was Pugh's skip list, which is probabilistic in structure and performance, but precise with answers.
Note a lot of the diagrams seem broken
Seem fixed now