{
  "id": 2978446,
  "title": "Building certgrep.sh: a free certificate transparency search engine",
  "url": "https://urgent.news/2026/08/24/building-certgrep-sh-a-free-certificate-transparency-search-engine",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-24T08:40:45.000Z",
  "source": {
    "name": "Lobsters",
    "slug": "lobsters",
    "url": "https://haveibeensquatted.com/blog/building-certgrep"
  },
  "original_language": "en",
  "account": "Certificate transparency offers a valuable dataset for security professionals. Every certificate issued by a trusted certificate authority is recorded in an append-only log, often before the certificate is even used. This provides early indicators of malicious infrastructure. However, managing access to this dataset can be challenging. Third-party search tools may be unreliable, slow, or expensive. To address these issues, Have I Been Squatted developed certgrep.sh, a free, open-source tool for searching certificates with full regular expression support.\n\nInitially, certgrep.sh was built as an internal tool to handle the high query volume needed for their detection pipeline. They faced issues with existing public search tools and costly commercial APIs. Their approach revolved around the fact that certificate metadata is secondary to domain occurrences. They focused on efficient domain name lookups, particularly using regular expressions for typosquatting and phishing patterns.\n\nThe solution involved using finite state transducers (FSTs), specifically the Rust fst crate. FSTs are deterministic automata that efficiently handle lookups by key length, store data as a flat byte sequence, and allow for fast pattern matching using regular expressions. By compiling regular expressions to DFAs and intersecting them with the FST, certgrep.sh can quickly determine if a domain pattern matches any certificate in the logs.\n\nDomain names, being short and highly repetitive, compress well, making the FST an ideal solution. Once compiled, the FST is immutable, ensuring the data remains consistent. This architecture allows certgrep.sh to provide fast, free access to certificate transparency data while minimizing storage costs and operational overhead.",
  "summary": null,
  "key_points": [
    "certgrep.sh is a free, open-source certificate transparency search tool",
    "Uses finite state transducers for efficient domain name lookups",
    "Provides fast access to certificate transparency data without cost"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}