Your embeddings are 32x bigger than they need to be. Maybe.

Every OpenAI embedding stored in Postgres takes up 6 KB, which means a hundred million documents would consume 600 GB of vectors before building an index. Storing fewer bits per number could save substantial storage space and reduce database costs. pgvector offers halfvec (16 bits) and binary_quantize (1 bit) compression methods, while VectorChord provides RaBitQ.

TurboQuant, developed by the author, is another compression technique that reduces the size of embeddings. The key finding is that the effectiveness of compression depends on the embedding model, not the codec used. The author compared the compression methods on three public datasets and found that binary quantization with rerank for OpenAI embeddings yields almost half the size (32x) while only losing 1% of recall.

However, this approach fails for other datasets like SIFT, where the wrong results are returned 95% of the time. The author emphasizes that comparing vendor benchmarks or monitoring metrics will not reveal such issues, as they only become apparent when users become dissatisfied with the answers. The article also discusses the trade-offs between different compression techniques and provides practical recommendations for choosing the right method based on collection size, data characteristics, and embedding model changes.