Float Bloat: vector serialization gone wrong

Bonsai has uncovered a widespread issue with vector search across the entire ecosystem, affecting millions of implementations. This problem is present in official vendor SDKs, documentation, tutorials, and articles, with many clients casting and serializing vectors as float64 instead of the default float32. This results in a two-fold precision increase, doubling disk and network costs without adding any accuracy. Bonsai estimates that this unnecessary disk storage overhead could be over 20 Petabytes globally.

The issue arises when a vector from a model, embeddings API, or inference provider is cast and serialized to float64. This side effect of floating point conversion, known as widening, adds meaningless digits that take up storage space and bandwidth. The problem is prevalent across various tiers of vector search clusters, from sandbox to enterprise levels. It is also found in the main branch of popular embedding vendor SDKs and in the public documentation of large cloud companies.

The problem is not intentional but is the default behavior in several popular languages used for vector search. To fix the issue, the cast must be done explicitly, or the up-cast to float64 must be deleted, allowing the native encoder to see the original float32. Every fix is lossless and can significantly reduce the unnecessary overhead. Bonsai has released a new agent skill, bonsai-fix-float-bloat, to help find and fix this problem.