{
  "id": 15672033,
  "title": "One Hash Bloom Filters",
  "url": "https://urgent.news/2026/10/19/one-hash-bloom-filters-15672033",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-19T20:50:08.000Z",
  "source": {
    "name": "Hacker News",
    "slug": "hacker-news",
    "url": "https://calabro.io/ohbf"
  },
  "original_language": "en",
  "account": "One Hash Bloom Filters: A Breakthrough in Efficiency\n\nProbabilistic data structures known as bloom filters enable the swift and economical verification of whether a specific key, X, is present within a set. These filters strike a balance between memory consumption and computational efficiency, making them a popular choice in high-performance data systems. Utilized by Bluesky, these filters have notably accelerated query processing, cutting disk IO and enabling the handling of millions of queries per second.\n\nBloom filters consist of an array of bits and k hashing functions. A key is added to the set by executing it through the k hash functions, subsequently marking the corresponding bits in the array. During verification, the key is hashed k times, and the resulting positions are inspected. If any bits are found to be zero, the key was not incorporated into the set. Conversely, if all bits are set, it is probable that the key is within the set.\n\nHowever, the traditional two-hash approach can be computationally expensive in high-performance scenarios. In 2008, Kirsch and Mitzenmacher introduced a technique demonstrating that two hashes could yield similar results. They utilized the formula (hash_1 + i*hash_2) % n to determine the i-th location, maintaining a comparable false positive rate with a reduced computational cost.\n\nBluesky adopted the double-hash bloom filter implementation from Daniel Lemire's repo, \"bits-and-blooms/bloom,\" which runs two 128-bit murmur3 hashes to produce four 64-bit values, subsequently amalgamating them to generate all k bit locations. The Bluesky backend routinely loads data into these filters upon process initialization, with an additional real-time atproto firehose consumer populating the filters during process startup. However, this implementation lacked thread-safety, necessitating mutex implementation to prevent data races.\n\nJasmine forked the library, creating \"atomic-bloom,\" an implementation that harnesses atomic CPU instructions to execute the same logic without mutexes. This optimization led to a significant reduction in mutex contention, resulting in faster deploys from 90 minutes to approximately 15 minutes. While Amdahl's Law indicated no further speedup due to contention, the overall impact was still substantial.\n\nIn pursuit of further optimization, the Bluesky team explored a single-hash bloom filter technique. They developed \"jcalabro/gloom,\" a one-hash bloom filter (OHBF) implementation in go that has been operational in the Bluesky backend for nine months and remains stable. Additionally, this filter has been integrated into \"jetstream,\" Bluesky's high-performance sync service.\n\nThis exploration into one-hash bloom filters showcases the potential for enhanced performance and efficiency in handling probabilistic data structures, providing valuable insights for those interested in low-level engineering details.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": "https://urgent.news/ill/15672033.png",
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}