'The GPU never reads the same text twice': Belgian startup Corbenic wants to turn your server's SSD into AI memory, but its 14x speedup owes more to reading less
Galahad promises exact, encrypted KV cache reuse for vLLM, SGLang, and Llama.cpp, but LMCache, Nvidia, and Phison got to the round table first.
Belgian startup Corbenic AI has developed a technology called Galahad that allows servers to reuse AI models' attention states stored on local NVMe SSDs, rather than having the GPU reread the same text multiple times. Founder Sietse Schelpe explained that the GPU never needs to reprocess the same text, effectively turning the server's SSD into part of the AI's working memory.
The technology saves a model's key-value cache, which contains the attention state built while the model reads a prompt, to encrypted local storage and reloads it when the same text appears again. This is achieved through a small library called Taliesin, which writes the state to disk under a fingerprint of the exact input bytes, model, and tenant, and loads it when those bytes reappear.
Galahad's approach is slightly different from other solutions in the market, such as Nvidia's Dynamo and Phison's offerings, as it focuses on reading less text rather than just loading saved work. The company claims to have achieved an impressive 14x speedup in their tests, with a test recalling 100 facts from a 96,726-token list taking only 0.59 seconds on a vLLM model.
However, Corbenic's technology faces competition from established players like Nvidia, Phison, and open-source projects like LMCache, which already offer similar functionality.
Written by urgent.news from TechRadar's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
