Representing biological concepts as directions in foundation model embeddings
Biological foundation models encode rich information about protein structure, function, and cellular state in high-dimensional embeddings, but accessing specific biological properties typically relies on additional, task-specific prediction models. Here, we show that diverse biological properties can be mapped to interpretable, reusable directions in embedding space. In the Concept Activation…
Biological foundation models capture complex data about proteins, cells, and related biological systems within high-dimensional embedding spaces. However, accessing specific biological attributes usually requires extra, specialized prediction models. A new study demonstrates that various biological traits can be represented as clear, reusable vectors within these embeddings.
The Concept Activation Vector (CAV) technique pinpoints examples tied to a concept and distinguishes them from a neutral dataset. The resulting weight vector then forms the CAV for that concept.
When applied to protein embeddings, CAVs effectively retrieve Gene Ontology and Enzyme Commission classifications, identify Pfam domains, expose novel structural connections, and encapsulate DNA-binding and protein-interaction specificity. In single-cell embedding analyses, CAVs isolate cancer-related expression shifts, and the genetic variations along these axes align closely with traditional differential-expression analysis, despite being a fundamentally different computational method.
By recognizing concept directions as a universal representation, this work allows diverse biological features to be compared, pinpointed, and manipulated within a common framework.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
