Leakage-controlled benchmarking reveals generalization limits of deep learning for protein-ligand binding affinity prediction

A new study has unveiled the limitations of deep learning models when it comes to predicting protein-ligand binding affinity. To tackle the issues of data leakage and inconsistent evaluation, researchers have introduced PLABench, a benchmarking framework that ensures standardized comparisons among various sequence- and structure-based models.

The benchmarking was conducted on nine deep learning methods across blind CASP16 targets, leakage-controlled ChEMBL35 sets, and standard datasets like Davis and KIBA. Under strict data split settings and standardized structural input using AlphaFold3, the study found that highly pretrained structure-based models, while achieving the highest accuracy, exhibit significant target-dependent variability.

Furthermore, increasing the fidelity of structural inputs from predicted to experimental conformations did not lead to consistent improvements. On the other hand, sequence-based approaches outperformed some structure-based methods on specific targets. The researchers also discovered that protein family-level evaluations showed uneven performance across different families, with substantial inter-model complementarity often hidden by aggregate metrics.

These findings underscore the fact that merely scaling up training and using structural input alone are insufficient to ensure cross-target generalization in protein-ligand binding affinity predictions. The study highlights the necessity for context-aware interaction modeling. PLABench, the open-source platform developed for this purpose, aims to support further advancements in this area.