I Replaced grep-Based Code Review with a Knowledge Graph + MCP. Here Are 3 Bugs Vector Search Missed.

The AI code review setup previously involved AI grepping for related code within a PR, but this approach had limitations. When bugs arose that didn't show up in grep checks, they were shipped to production. The issue was not the model, but the retrieval process. Vector search and keyword grep are proficient at finding files mentioning specific code, but they struggle with identifying files that depend on the code through multiple layers, such as imports, event buses, and decorators.

This is where the bugs typically reside. To address this, the retrieval layer was rewired with a code knowledge graph (KG) integrated through MCP (Model Context Protocol). Within a week of implementing the graph, three bugs that vector search had overlooked surfaced.

The setup used code-review-graph, an open-source tool that constructs a property graph of the codebase and exposes it as an MCP server. Nodes represented files, classes, functions, and tests, while edges depicted relationships like imports, function calls, inheritance, decorators, event listeners, and tests. Once integrated, the AI could call MCP tools to analyze the codebase differently.

For instance, blast_radius(file) would list every file dependent on the input file, flow_trace(func) would show a function's output flow, semantic_search(query) would perform a hybrid search combining vector and graph proximity, and community_detect() could identify tightly-coupled modules. Additionally, risk_score(diff) provided a numerical risk assessment for code changes, and dead_code() would highlight unreachable code from entry points.

The cost of using the graph was minimal. Before the graph, the AI reviewer received context from a PR diff of auth.py plus one additional file, with around 150,000 tokens. After implementing the graph, the context expanded to auth.py plus two related files, with approximately 18,000 tokens. The reviewer could complete the review in seconds, whereas it previously took over 30 minutes to achieve the same result through manual grepping.

Bug 1 involved a silent contract change in an event handler. The diff was minor, with the addition of a device_id field to a login event payload. While vector search retrieved obvious neighboring files like login.py and auth_test.py, the graph also identified event_handlers/audit_log.py, which was responsible for serializing the payload to a fixed schema in S3.

This new field broke the schema validator, causing production issues shortly after the merge. The graph revealed the structural path from auth.py to audit_log.py, a relationship not apparent through keyword searching.

Bug 2 concerned a transitive change caused by a teammate's modification of the @with_retry decorator. The decorator was updated to include a backoff parameter, but existing callers remained unaffected due to unchanged default values. However, the graph revealed that 19 functions applied the decorator, which in turn were called by 31 files.

One of these files contained a payments webhook handler that experienced an additional 800ms delay under load due to the modified retry behavior, causing silent webhook timeouts. The graph exposed the structural propagation of the decorator's effect across multiple functions and callsites, while grep only identified files with explicit imports.

Bug 3 involved an orphan test that slipped through CI because it had been moved out of the tests/ directory and the CI config hadn't been updated. The PR included a test asserting a new hash for an ID, but since the test file was no longer executed, it passed CI erroneously. The graph detected that the test was no longer part of the CI pipeline, revealing the false confidence in the test's correctness. This bug resulted in corrupted migration data, with the first 8,000 rows being compromised.