Build a Serverless Search Engine with DynamoDB Vector Search and AWS CDK

In the context of building a serverless search engine, the project aims to enhance search functionality by incorporating embeddings and semantic similarity. This approach addresses limitations of traditional lexical and prefix-based matching, which often fail to capture the intended meaning behind queries. AWS introduced vector indexes in DynamoDB, enabling the implementation of a serverless search engine that combines DynamoDB vector search, Amazon Bedrock, AWS Lambda, and API Gateway.

All infrastructure is provisioned using AWS CDK in TypeScript. To perform semantic searches, the solution leverages Amazon Titan Text Embeddings V2 to generate 512-dimensional embeddings for products, ensuring consistency in model, dimensions, and normalization settings. DynamoDB stores this data alongside product information, with an approximate nearest neighbor (ANN) index performing the final vector similarity search.

The workflow includes generating embeddings for products and queries, and using DynamoDB's SearchVectors API to retrieve the closest matching vectors based on cosine distance. The architecture consists of separate indexing and querying flows, with API Gateway serving as the entry point for requests. A Lambda function processes these requests, leveraging DynamoDB's on-demand capacity to efficiently handle searches.

This implementation demonstrates how AWS services can be leveraged to create a sophisticated search engine without requiring extensive machine learning expertise.