Graph RAG: where it actually breaks

When someone asks about which companies are exposed to semiconductor export restrictions, the system must decide whether to perform graph traversal, vector search, or a combination of both. Getting the routing wrong does not result in an obvious error; instead, it returns a response that cites real sources and sounds confident yet misses crucial context present in the graph.

One notable case involved a query about a Korean chipmaker's supplier network that produced news results instead of the desired answer. The issue lay in the system not routing the query to the correct part of the graph, where supplier relationships were already stored. Once the problem was identified, fixing it required only two lines of code.

The second major area where things broke down concerned entity quality. This issue is more subtle and gradual. Initially, the entity resolution process creates a set of merged entities. However, as source data continues to change, companies may be acquired, leading to new parent company names appearing alongside old subsidiary names. This results in the graph accumulating both nodes representing the same underlying entity, causing discrepancies in the results returned during graph traversal.

Rather than a binary state, entity quality degrades continuously. Periodic audits of merged pairs help maintain accuracy by checking whether the alias-to-entity mapping remains relevant for recently active entities. The code to identify stale merges is provided, detecting entities that have gone stale beyond a certain threshold.

However, the final decision on how to handle flagged pairs remains a manual process. The author's work, er-api, is a multilingual entity resolution service supporting Korean, Japanese, Chinese, and English corporate data.