{
  "id": 7983378,
  "title": "Why we stopped using one Elasticsearch index per tenant and built our own",
  "url": "https://urgent.news/2026/09/17/why-we-stopped-using-one-elasticsearch-index-per-tenant-and-built-our",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-17T08:13:38.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/seacloud-labs/why-we-stopped-using-one-elasticsearch-index-per-tenant-and-built-our-own-1a7c"
  },
  "original_language": "en",
  "account": "We abandoned using one Elasticsearch index per tenant and instead developed our own solution called SeaSearch. This decision was driven by the limitations of replicating data across indexes, which caused operational complexity and scalability issues. SeaSearch tackles these problems by storing index data in S3 and managing ownership instead of the data itself.\n\nThe primary challenge is the cost of each index, which requires at least one shard, much like a mini Lucene database. As the number of indexes increases, so does the strain on the master node, leading to sluggish performance. Rather than building a search engine from scratch, the team chose ZincSearch as a foundation, which provided most of the necessary runtime footprint and migration capabilities.\n\nTo address the issue of unbounded index counts, SeaSearch stores indexes in S3 and routes ownership through etcd and a cluster manager. Compute nodes handle reads and writes, all communicating with S3-compatible storage. The proxy (gateway) routes client requests to the appropriate node, which then retrieves the required segments from S3 as needed. This approach eliminates the need for redundant data copies and simplifies failover processes.\n\nOne potential drawback is the speed of S3 compared to local NVMe storage. To mitigate this, SeaSearch employs a cache system on compute nodes, utilizing immutable segments for read-only data. This allows compute nodes to serve indexes larger than their local disks by continuously cycling through segments. Parallel warm-up and distributed query execution further enhance performance, distributing cache pressure and accelerating query processing across multiple nodes.",
  "summary": "TL;DR: We open-sourced SeaSearch , an Elasticsearch-compatible search engine that keeps index data in S3 and routes ownership instead of replicating data, so tenant count isn't capped by cluster state. The tax on one index per tenant If you've built multi-tenant search on Elasticsearch, you've had this argument with yourself. The clean approach to multi-tenant search is one Elasticsearch index…",
  "key_points": [
    "Abandoned one Elasticsearch index per tenant due to operational complexity and scalability issues.",
    "Developed SeaSearch to store index data in S3 and manage ownership instead of data.",
    "SeaSearch improves performance by caching immutable segments and distributing query execution."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}