Prot-LAMBDA: Explicit Distance Learning Enhances Structural Reasoning in Protein Language Models

Protein language models (PLMs) traditionally encode evolutionary relationships from extensive sequence data, but fail to capture three-dimensional structural information. To address this limitation, researchers have developed Prot-LAMBDA, a PLM that explicitly includes spatial relationships by combining residue embeddings with inter-residue contacts.

This new model demonstrates significant improvements across various structure-related tasks, such as contact prediction, secondary structure identification, backbone geometry estimation, solvent accessibility evaluation, and protein fold prediction.

Prot-LAMBDA exhibits a twofold increase in long-range contact recall and an 11.7% decrease in the prediction error of {psi}-angles compared to the ESM2-3B model, despite possessing only around five times fewer parameters. Remarkably, when coupled with the same structure-prediction module, Prot-LAMBDA enhances 3D structure prediction by 5-7% in TM-score, outperforming the ESM2-3B model by a noticeable margin.

In addition to its performance gains, Prot-LAMBDA serves as the foundation for LambdaFold, a lightweight distance-guided structure prediction framework capable of achieving performance similar to ESMFold on proteins distinct from the training data. Furthermore, the integration of structurally-retrieved templates through retrieval-augmented learning methods leads to a substantial improvement in mean TM-score for targets with a high degree of template coverage, even rescuing several cases of incorrect protein folds.

These findings collectively underscore the importance of explicitly incorporating spatial constraints in PLMs, enabling efficient and generalizable structural representation learning and accurate protein structure prediction.