I gave an AI assistant a memory of PubMed - and learned why "forgetting" is the hard part

I wanted to test whether an AI assistant could keep up with rapid developments in a specialized field of cancer research. To do this, I focused on a specific question: how do KRAS G12C lung and colorectal tumors develop resistance to sotorasib and adagrasib? As part of Mergetober, I created a PubMed connector for cognee, an open-source memory layer for AI agents.

In this article, I'll explain how to use this connector, share my findings when I pointed it at actual PubMed data, and discuss three unexpected aspects of NCBI's API.

Resistance to KRAS G12C inhibitors is a rapidly evolving area of research, with new case reports and preprints appearing monthly. Training a model on a frozen dataset of literature is not scalable and data disappears after each session. What I desired was a memory system: a repository that gathers relevant literature, keeps it updated independently, and converts it into a format that an AI agent can reason over. This is precisely what cognee achieves with documents.

To integrate PubMed into cognee, I used a custom connector that requires just one line of code to ingest literature from a PubMed search. The connector is a dlt source that can be handed to cognee.remember. By specifying a PubMed query, you choose what to ingest. In my case, the query targeted KRAS G12C tumors and their resistance to sotorasib and adagrasib.

Using cognee.remember, I ingested 155 relevant PubMed records added since January 2024. The next day, when I queried the assistant about secondary KRAS mutations causing resistance to G12C inhibitors, it provided a comprehensive response based on the ingested knowledge.

In the process of setting up this literary memory layer, I encountered three challenges related to NCBI's API. First, the ESearch function caps results at 10,000 PMIDs; when this limit is reached, the search must be split into multiple queries. Second, Entrez assigns publication dates using US Eastern time, which can result in missed articles if the search is conducted shortly after midnight UTC.

Finally, retracted papers do not change their Entrez date, requiring additional steps to ensure the memory layer can "forget" outdated information. After addressing these obstacles, I successfully created a working memory system that enables an AI assistant to effectively learn and retain knowledge from PubMed articles.