DynamoDB vector search without embeddings
Serverless Advocate #85 asks Lee Harding which AWS service he's most excited about, and he picks DynamoDB vector indexes. Not for embeddings, though. His point is that most people hear "vector" and think semantic search, when a vector index is "a general-purpose tool for any domain where you need to find 'nearest neighbours' in some n-dimensional space". Geometry, topology, geography, sensor…
Lee Harding, a Serverless Advocate, expressed excitement over AWS's DynamoDB vector indexes. He explained that while people often associate "vector" with semantic search, vector indexes serve a broader purpose in finding nearest neighbors in n-dimensional space. This includes domains like geometry, topology, geography, and sensor fusion.
For a store locator example, he used real places in Leeds and York with their latitude and longitude coordinates. Each place was represented by a DynamoDB item with a partition key, name, latitude, longitude, and opening hours. However, these coordinates alone couldn't be searched by distance, so additional steps were needed.
To enable distance search, Lee projected the latitude and longitude onto a unit sphere using trigonometric functions. This transformation converted the coordinates into three numbers (x, y, and z) that could be indexed as a vector attribute called "position." With this vector index, the shops could be searched as points on a sphere, allowing for accurate great-circle distance calculations.
The distance calculation between two points on the sphere is computed as 2 * sin(theta / 2), where theta is the angle between the two points at the center of the sphere. This provides an accurate measurement of the straight-line distance between the points.
The author also compared this approach to using geohashes for proximity queries in DynamoDB. While geohashes are efficient for prefix reads and can quickly retrieve nearby places, they can sometimes include distant places in the same cell, leading to inaccurate results. In contrast, the vector search method consistently provided accurate results, even when the nearest shop was located in a different partition.
Lee emphasized the importance of choosing a consistent projection method for the vector index and keeping it up to date when the projection changes. He concluded by providing a link to a full write-up on the store locator example, encouraging readers to explore the topic further.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
