Compile the Knowledge at Index Time

On Monday and Thursday, when a document store is asked the same question, it embeds the query, retrieves matching chunks, and hands them to a model that determines which sentences in those chunks answer the question. This repeated process is what RAGFlow 1.0 aims to eliminate. Instead, the open-source engine, InfiniFlow, positions itself as a supply layer for retrieval agents, focusing on knowledge compilation.

This involves moving the understanding from query time to index time by compiling once when a document arrives and storing the result as artifacts the agent can use directly. Compilation generates several artifact types, including a navigation tree, entity graph, timelines, wikis, and atomic propositions. A proposition is a one-sentence fact extracted from the source, paired with a verbatim quote, which must match the source word for word.

The compilation process stores lineage, documenting the document and passage the evidence came from, and removing it when a document is deleted. When a compiled hit is sufficient to answer, the agent uses the evidence and skips reading the raw chunk entirely, reducing query tokens by 91% in cases where the evidence suffices. This approach is cheaper and less costly when a miss occurs, as the system checks the evidence before starting the model's search.

Compilation pays off when it is used occasionally and loses cheaply, such as in long-tail archives with cold traffic. However, the compilation process has its limits, as it is a bet on reuse and can lead to stale data if the source changes frequently. It also relies on the quality of the extraction process, which is checked through testing the compiled store against real questions.