These startups are chasing the next big thing in LLMs

In the realm of large language models (LLMs), a new wave of startups is emerging, aiming to revolutionize the technology that underpins them—transformers. The foundation of transformers lies in dense attention, a method of encoding the meaning of text by comparing every word with every other word. However, this approach becomes increasingly inefficient as the text grows longer, leading to high computational costs and limiting the context window.

Four startups are pioneering alternative methods to address these challenges. Subquadratic, a Miami-based AI startup, has developed a sparse attention mechanism called SubQ, which selectively focuses on relevant word pairings, significantly reducing computational needs. Power retention, a mechanism developed by Manifest AI in San Francisco, stores only the most pertinent information in an LLM's context window, essentially creating a rolling summary that eliminates the need to process all data.

By adapting transformer models to use power retention, Manifest AI has created PowerCoder and Brumby, which they claim rival some versions of open-source LLMs. These startups acknowledge the potential for failure but see immense upside in pushing the boundaries of LLM technology, with the potential to make them faster, more efficient, and potentially smarter.