Building from the ground up: augmentation strategies to improve protein language model performance on protein sets
An important underlying requirement for deep learning models is the availability of large datasets, however, in sciences such as chemistry and biology, generating experimental data of sufficient size represents a significant bottleneck. In this work, we probed the extent and nature of limited input data, characterize how several protein language models (pLMs) learned on suboptimal datasets, and…
Deep learning models depend heavily on large datasets for optimal performance, but generating enough experimental data for scientific fields like chemistry and biology can be prohibitively challenging. This study examined the impact of limited input data on protein language models (pLMs) and explored augmentation techniques to enhance their predictive capabilities.
The researchers discovered that restricting the variety of protein-protein complexes in a small dataset can significantly reduce training requirements, assuming the model's purpose is to generalize to similar data. Additionally, active learning (AL) demonstrated promising results, with both in- and out-of-distribution Matthews Correlation Coefficient (MCC) values experiencing slight but consistent improvements when the training set was augmented with protein complexes generated by ESM-3, another generative pLM.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
