From Memory-Hungry HNSW to Quantized SPANN: The Technical Evolution of Pinterest's Manas Platform

Pinterest's engineering team transformed its distributed search platform, Manas, to manage the scaling burdens of billions of embeddings. By employing quantization techniques such as Scalar Quantization (SQ) and Product Quantization (PQ), Manas dramatically reduced memory footprints. PQ compressed original vector representations into compact byte codes, shrinking HNSW indices by 74% and Inverted File (IVF) indices by 93%, while maintaining recall rates of 70–80%.

SQ compressed vector components into lower-bit integers, cutting HNSW indices by 59% and IVF indices by 75%, while preserving over 90% recall. Offline evaluations showed that while SQ resulted in a 50 GB index with a Recall@100 of 92.92% at 305.2 QPS, PQ achieved a 6.8 GB index with a Recall@100 of 76.00% at 1747.9 QPS. Incorporating both PQ and SQ reduced the index size to 25 GB, delivering a Recall@100 of 95.71% at 1588.8 QPS.

To address the CPU bottleneck of decoding low-bit representations, the team introduced Linear Scaling SQ using SIMD intrinsics, achieving 10–15% savings in query compute resources. Performance tests demonstrated that SQ and PQ delivered 20–30% reductions in serving costs across production workloads. For storage on SSDs, Pinterest compared DiskANN and SPANN, finding that SPANN with PQ boasted 3x the QPS, 1/3 the latency, and only a 5% recall drop.

SPANN, with its centroid index in memory and large posting lists on SSDs, optimized IOPS and ensured efficient search operations. In a test indexing over 5 billion embeddings, SPANN saved over 40% of CPU time for production queries compared to traditional in-memory HNSW. Pinterest is now advancing beyond single-vector two-tower models, exploring multi-vector Late Interaction models like ColBERT, which utilize Sum of MaxSim scoring for fine-grained relevance matching.

This transition necessitated updating the query parser to break multi-token queries into multiple vector embeddings and running simultaneous ANN searches across indices. An internal pilot with a client team is currently testing the feasibility of these advanced multi-embedding queries in live production environments.