An Expert-Curated Benchmark for Frontier and Local LLMs Querying a Biomedical Knowledge Graph
We combine GeneNetwork's 1TB structured database with its metadata as an RDF graph and an associated literature corpus. We benchmark frontier and local LLMs querying it through four architectures: document, graph and agentic retrieval-augmented generation (RAG), and a combination of the three. We evaluate them on 40 domain specific expert-curated questions with a new metric: the satisfaction…
We haven't written up this one. bioRxiv has the full story — the link below goes straight to it.