Our S3 cache is full of holes. We put them there. — Berserk
The data store Berserk built an S3 cache utilizing sparse files and hole punching to maintain fast queries while reclaiming disk space. Query workers coordinate reads across process boundaries using a shared bitmap and counters to safely read directly from disk or fetch missing blocks from S3. Open article →