Hand Off Your Vector Search POC: A README and RECREATE.md for Amazon OpenSearch Service
The proof of concept worked in my account. The real test was whether the customer's team could build it again in theirs. A customer evaluating Amazon OpenSearch Service as a vector store asked for two things: proof that it could serve 300 queries per second against 4,096-dimension vectors with server-side latency under 50 milliseconds, and an easy path to load those vectors. I built a proof of…
The proof of concept for Amazon OpenSearch Service as a vector search store was successful, demonstrating that the system could handle 300 queries per second against 4,096-dimensional vectors with server-side latency under 50 milliseconds. The project involved creating a README and RECREATE.md file for the customer to easily rebuild the setup in their own environment. The key considerations were meeting the throughput and latency targets, as well as providing a straightforward loading path for the vectors.
The solution consisted of a single CloudFormation template generated using AWS CDK in Python. This template deployed an OpenSearch Service domain running on OpenSearch 3.7, with r8gd.8xlarge.search data nodes, dedicated cluster manager nodes, and three Availability Zones for zone awareness. The domain was secured within a VPC with fine-grained access control and encryption in transit and at rest.
The vector index utilized Faiss HNSW in on_disk mode with 8x compression, and full-precision vectors were stored on local NVMe for rescoring if needed. GPU acceleration was employed to build HNSW graphs for large segments during post-load force merge operations, while the data nodes dedicated their CPU resources to indexing and search.
The team developed an ingestion pipeline that read vectors from Amazon S3, parsed each line into a document, and wrote them to the OpenSearch domain over HTTPS using SigV4 authentication. The pipeline automatically created an index template and scaled its OpenSearch Compute Units (OCUs) based on demand. The pipeline was designed to avoid custom loader hosts and provide an automated loading process.
To achieve the desired throughput and latency, the team formulated three rules to guide node sizing. First, they determined that each primary shard should hold between 50 and 100 GB of data due to recovery and relocation costs. For an index containing 10 million vectors, this resulted in approximately 30 primaries. Next, they considered the query handling capacity, with each data node handling shard searches proportional to the node count and shard count.
The final factor was the memory and disk requirements, which were easily met at the determined node and shard configurations.
The benchmark involved generating 10 million synthetic 4,096-dimensional vectors and loading them into three primary shards on six data nodes. This setup allowed the team to simulate a production-like environment. The benchmark showed that server-side latency remained below 50 milliseconds across the range of queries tested, with a p50 of 9 ms and a p99 of 15 ms. This performance exceeded the target while providing ample headroom for production use cases.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.