I Replaced My Vector Database With Boring Algorithms
I recently removed a vector database from a few parts of my software. Not because vector databases are bad. Not because embeddings stopped being useful. And definitely not because I suddenly decided that AI infrastructure is overhyped. The reason was much more boring: I was using the wrong tool for some of the jobs. One of those jobs was source-code retrieval for RAG. The original version worked…
A software engineer recently removed a vector database from their system, not because vector databases are inherently flawed or AI infrastructure is overhyped, but because the tool was not the best fit for certain tasks. Specifically, the engineer was using the vector database for source-code retrieval in a Retrieval-Augmented Generation (RAG) system.
While this initial approach worked reasonably well, the engineer observed that many queries were actually not semantic but rather structural, such as "Where is this function defined?" or "Which class owns this method?" These questions could be answered directly from the source code, which already contained the explicit information in the syntax tree, import graph, file names, symbols, and references.
Adding the extra step of converting this information into embeddings did not significantly improve the system's performance and often led to slower, more expensive, and harder-to-debug results. The engineer's experience led them to reconsider the assumption that an AI-heavy product must also have AI-heavy infrastructure for every component.
They now recognize that many software problems are better addressed using traditional tools such as databases, parsers, indexes, graphs, rules, and algorithms. The key distinction they drew was between the LLM's ability to handle ambiguity and the software's capacity for precision. While LLMs excel at interpreting vague, incomplete, semantic, or natural language inputs, traditional software is more effective when dealing with problems that have structure, explicit rules, known identifiers, or well-defined algorithms.
This led the engineer to propose a new architectural approach: Human request - LLM - interpret ambiguity - traditional software - execute precisely - LLM - explain the result. In this setup, the LLM acts as a translator between the human and the software, cleaning up messy human language and providing clear explanations. The engineer argues that this approach not only results in simpler, more predictable systems but also leads to more effective AI systems, as the model is used where it truly adds value rather than compensating for poorly chosen infrastructure.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.