Wrote this project to learn about HyperLogLog, a random algorithm for estimating the cardinality of very large datasets using only a constant amount of memory (while introducing some small error). While writing the post, I've thought about optimizing the algorithm with SIMD, which ended up being a very interesting rabbit hole. I also benchmarked the implementation against some other Go, Rust, and Python.
No prior knowledge of either HyperLogLog or SIMD is required; any feedback on the post/code would be welcome!
Wrote this project to learn about HyperLogLog, a random algorithm for estimating the cardinality of very large datasets using only a constant amount of memory (while introducing some small error). While writing the post, I've thought about optimizing the algorithm with SIMD, which ended up being a very interesting rabbit hole. I also benchmarked the implementation against some other Go, Rust, and Python.
No prior knowledge of either HyperLogLog or SIMD is required; any feedback on the post/code would be welcome!