Why I Stopped Using Vector RAG for Coding Agents (And Used Git Markdown Instead)

If you regularly use tools like Cursor, Claude Code or Windsurf, you may have found yourself in a frustrating situation. After spending 45 minutes detailing your project architecture, API contracts and reasoning against certain coding patterns, the AI agent often forgets the advice the next time you start a new session. It feels like you're constantly working with a junior engineer who has no memory of prior discussions.

Two common solutions have been proposed to this problem: creating massive dumps of all your project rules in a single document, or building vector databases to store past chats and code snippets. However, both approaches have significant drawbacks. The first option is expensive in terms of token usage, while the second leads to context dilution - the AI cannot distinguish between different versions of the same concept.

A more effective solution is to treat the agent's memory like a documentation system. You can set up a structured markdown folder (opencontext/) containing files for architecture, API contracts, state management, and rejected approaches. When the agent queries this index, it pulls only the relevant sections needed for the current task. If the architectural approach changes, the markdown files are directly updated by the agent, eliminating versioning issues.

The three key principles behind this approach are:

1. Limit context size by only sending the index (~100 tokens) instead of massive dumps

2. Avoid appending new data infinitely - instead, update existing markdown files

3. Use Git as the universal sync layer, keeping all context in plain text files within the repository

This method provides deterministic access to memory, requires no external APIs or subscriptions, and allows for easy cross-agency sharing of context through version control systems. Setting up a local MCP server takes just a few seconds and can be integrated with popular AI coding assistants like Claude Code or Cursor. This approach has proven effective in maintaining consistent context across sessions, making it a simple yet powerful alternative to more complex vector-based solutions.