Redefining the task of Bioactivity Prediction

Small molecules are vital to modern medicine, and accurately predicting their bioactivity against protein targets is crucial for therapeutic discovery and development. However, current machine learning models often rely on spurious features, leading to biased outcomes. Notably, a simple pocket-only baseline can achieve results comparable to, and sometimes better than, more complex models that incorporate both the protein pockets and the small molecules. This phenomenon arises from insufficient training data and an improper evaluation process, which is typically conducted at the pocket level rather than the small molecule level. To address these issues, we redefine the bioactivity prediction task by introducing the SIU dataset-a million-scale Structural small molecule-protein Interaction dataset for Unbiased bioactivity prediction task, which is 50 times larger than the widely used PDBbind. The bioactivity labels in SIU are derived from wet experiments and organized by label types, ensuring greater accuracy and comparability. The complexes in SIU are constructed using a majority vote from three commonly used docking software programs, enhancing their reliability. Additionally, the structure of SIU allows for multiple small molecules to be associated with each protein pocket, enabling the redefinition of evaluation metrics like Pearson and Spearman correlations across different small molecules targeting the same protein pocket. Experimental results demonstrate that this new task provides a more challenging and meaningful benchmark for training and evaluating bioactivity prediction models, ultimately offering a more robust assessment of model performance. Dataset and Code are available at: https://github.com/bowen-gao/SIU . Recently, various 3D machine learning models have been proposed in this direction (Townshend et al., 2020; Zhou et al., 2022; Gao et al., 2023a; Luo et al., 2023) , achieving significant advancements. These methods utilize the structural information of small molecules and protein targets as inputs to learn a mapping function between these inputs and bioactivity labels. This methodology is inherently sound and explainable, as biological insights suggest that the biological effect of a small * Equal contirbution † Work was done while Yanwen Huang was an intern at AIR.