Infino: fastest VectorDB at a million, still cheaper at a billion

Infino, the fastest vector database, offers a unique solution that bridges the gap between fast lookups and handling massive datasets. Developed by the same team behind this article, Infino unifies both the fast and large stores into a single engine, eliminating the need for separate systems. This innovative approach addresses a common challenge faced by teams that adopt vector search, where the fast in-memory indexes struggle when dealing with large datasets, necessitating a shift to slower distributed, disk-based systems.

On the benchmark VectorDBBench, Infino outperforms all other engines at every recall point for datasets of up to 1 million vectors. At 10 million vectors, Infino remains ahead of rivals, with the exception of a hosted cloud service at the highest recall. The main advantage of Infino lies in its memory usage, as it keeps hot data in RAM in a graph index like HNSW, while cold data is served from an object storage index.

This design choice significantly reduces costs, making it cheaper to serve billions of vectors compared to keeping them entirely in RAM.

The performance difference between the two modes is evident in the query breakdown. Approximately 60% of the time is spent on navigating the graph, while only a small fraction is dedicated to re-ranking the candidates. The benchmark, however, measures the entire process through a server-client connection, which adds a few hundred microseconds to the total time. Despite this, Infino's design allows it to maintain its edge in performance, even with the overhead of the server connection.