How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms
How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms If your coding agent has ever stalled for tens of seconds on "what's in this repo?" — or burned hundreds of tokens re-reading a file after a failed edit — this is the story of why that happens and how we fixed it. TL;DR — we rebuilt code tooling for agents that have no hands, no eyes, and no memory: repo_map in 98ms (was tens of…
The article discusses how the repo-wide symbol indexing for LLM agents was optimized from taking tens of seconds to just 98 milliseconds. The main issue was that every query required the system to re-parse the entire codebase, which became a bottleneck. To address this, the authors implemented a three-layer solution.
The first layer is a Rust parse daemon that handles all CPU-bound AST (Abstract Syntax Tree) work. This daemon communicates with Node.js over a Unix socket. The Rust code is designed to be zero-copy, meaning it avoids per-node N-API boundary crossings, enabling true parallelism using libraries like rayon.
The second layer involves creating a SQLite index. Once the parse results are obtained, they are stored directly in SQLite (using WAL mode and a per-workspace setup). This allows subsequent queries to be resolved through point lookups, bypassing the need for re-parsing. Incremental self-heal is also employed, where the system only re-extracts changed files when the workspace's timestamp or dirty flags indicate a modification. If the Rust binary's SHA256 changes, the entire database is marked dirty and rebuilt automatically.
The third layer addresses the limitations of human-centric tools, which assume users have hands, eyes, and memory. For an LLM agent, these assumptions don't hold. To compensate, the authors introduced atomic operations, ensuring every edit is all-or-nothing. This includes an undo journal that records changes for rollback purposes, even if a process is terminated prematurely with kill -9.
The system also generates structured output rather than prose, allowing the LLM to consume JSON instead of having to parse human-readable text. Errors are handled in a way that provides executable suggestions, ensuring that failed operations don't lead to silent corruption or dead ends. edit_transaction is used to maintain consistency, where every write is version-anchored, preventing silent loss of work.
Ultimately, the toolkit aims to make the tools for LLM agents fast, safe, and cheap. By separating parsing, indexing, and writing operations, the system avoids the pitfalls of re-parsing the entire codebase with every query, thereby significantly improving the efficiency and reliability of LLM agent operations.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
