Table Info

Table 8.

Metrics computed on the CASF 2016

Method	Trained on PDBbind	N	R_p↑	SD↓	MAE↓	RMSE↓
GoldScore^a [13]	-	285	0.42	1.99	-	-
ChemScore^b [13]	-	285	0.59	1.76	-	-
X-Score^HM [13]	-	285	0.61	1.73	-	-
Pafnucy [4]	2016	290	0.78	1.37	1.13	1.42
Pafnucy^* [4]	2020	285	0.76	-	-	1.46
OnionNet [7]	2016	290	0.82	1.26	0.98	1.28
AGL-Score^c [8]	2007 (refined)	285	0.83	-	-	1.73
IGN [10]	2016	262	0.84	-	0.94	1.22
ECIF6::LD-GBT^c [11]	2016 (refined)	285	0.87	-	-	-
HGScore	-	285	0.85	1.14	0.90	1.14

The scoring function was used with the software ^a Gold and ^b Sybyl. ^c a problem for comparison, either the dataset is not fully tested, or the test set is also used for training or validation. ↑ indicates that the larger the value, the better; ↓ indicates that the smaller the value, the better. N is the number of tested complexes, some methods have been tested only on a subset of the benchmark, methods with N of 290 have been tested on a former CASF 2016 version. -: not applicable; *: retrained on the same split as HGScore

Declarations

Acknowledgments

The authors thank the supercomputer center ROMEO for providing CPU-GPU times.

Author contributions

KC: Conceptualization, Software, Writing—original draft, Writing—review & editing. AG: Software, Writing— review & editing, Supervision. XV, SB, and LAS: Writing—review & editing, Supervision. All authors read and approved the submitted version.

Conflicts of interest

The authors declare that they have no conflicts of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent to publication

Not applicable.

Availability of data and materials

The datasets analyzed for this study can be found in the PDBbind website http://pdbbind.org.cn/.

Funding

This work was supported by the Association Nationale de la Recherche et de la Technologie (ANRT) through a CIFRE grant [2020/0691]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright

References

Crampon K, Giorkallos A, Deldossi M, Baud S, Steffenel LA. Machine-learning methods for ligand-protein molecular docking. Drug Discov Today. 2022;27:151–64. [DOI] [PubMed]

Ballester PJ, Mitchell JBO. A machine learning approach to predicting protein-ligand binding affinity with applications to molecular docking. Bioinformatics. 2010;26:1169–75. [DOI] [PubMed] [PMC]

Wallach I, Dzamba M, Heifets A. AtomNet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery. arXiv:1510.02855 [Preprint]. 2015 [cited 2023 Jan 27]. Available from: http://arxiv.org/abs/1510.02855 [DOI]

Stepniewska-Dziubinska MM, Zielenkiewicz P, Siedlecki P. Development and evaluation of a deep learning model for protein-ligand binding affinity prediction. Bioinformatics. 2018;34:3666–74. [DOI] [PubMed] [PMC]

Zhang H, Liao L, Saravanan KM, Yin P, Wei Y. DeepBindRG: a deep learning based method for estimating effective protein-ligand affinity. PeerJ. 2019;7:e7362. [DOI] [PubMed] [PMC]

He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Boult TE, Wong A, Ferryman J, Siva P, Christensen GE, Rudd EM, et al., editors. IEEE 2016: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27–30; Las Vegas, NV, USA. IEEE; 2016. pp. 770–8. [DOI]

Zheng L, Fan J, Mu Y. OnionNet: a multiple-layer intermolecular-contact-based convolutional neural network for protein-ligand binding affinity prediction. ACS Omega. 2019;4:15956–65. [DOI] [PubMed] [PMC]

Nguyen DD, Wei GW. AGL-Score: algebraic graph learning score for protein-ligand binding scoring, ranking, docking, and screening. J Chem Inf Model. 2019;59:3291–304. [DOI] [PubMed] [PMC]

Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. 2nd ed. New York: Springer New York; 2009. [DOI]

10.

Jiang D, Hsieh CY, Wu Z, Kang Y, Wang J, Wang E, et al. InteractionGraphNet: a novel and efficient deep graph representation learning framework for accurate protein-ligand interaction predictions. J Med Chem. 2021;64:18209–32. [DOI] [PubMed]

11.

Sánchez-Cruz N, Medina-Franco JL, Mestres J, Barril X. Extended connectivity interaction features: improving binding affinity prediction through chemical description. Bioinformatics. 2021;37:1376–82. [DOI] [PubMed]

12.

Liu Z, Su M, Han L, Liu J, Yang Q, Li Y, et al. Forging the basis for developing protein-ligand interaction scoring functions. Acc Chem Res. 2017;50:302–9. [DOI] [PubMed]

13.

Su M, Yang Q, Du Y, Feng G, Liu Z, Li Y, et al. Comparative assessment of scoring functions: the CASF-2016 update. J Chem Inf Model. 2019;59:895–913. [DOI] [PubMed]

14.

Wójcikowski M, Zielenkiewicz P, Siedlecki P. Open Drug Discovery Toolkit (ODDT): a new open-source player in the drug discovery field. J Cheminform. 2015;7:26. [DOI] [PubMed] [PMC]

15.

Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS. A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learning Syst. 2021;32:4–24. [DOI] [PubMed]

16.

Fey M, Lenssen JE. Fast graph representation learning with PyTorch Geometric. arXiv:1903.02428 [Preprint]. 2019 [cited 2023 Feb 8]. Available from: https://arxiv.org/abs/1903.02428 [DOI]

17.

Xiong Z, Wang D, Liu X, Zhong F, Wan X, Li X, et al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. J Med Chem. 2020;63:8749–60. [DOI] [PubMed]

18.

Veličković P, Cucurull G, Casanova Å, Romero Å, Liò P, Bengio Y. Graph attention networks. arXiv:1710.10903 [Preprint]. 2018 [cited 2023 Feb 8]. Available from: https://arxiv.org/abs/1710.10903 [DOI]

19.

Brody S, Alon U, Yahav E. How attentive are graph attention networks? [Internet]. Credit to ICLR; [cited 2023 Feb 8]. Available from: https://iclr.cc/virtual/2022/poster/6366

20.

Cho K, van Merrienboer B, Bahdanau D, Bengio Y. On the properties of neural machine translation: encoder-decoder approaches. arXiv:1409.1259 [Preprint]. 2014 [cited 2023 Jan 27]. Available from: http://arxiv.org/abs/1409.1259 [DOI]

21.

Spearman C. The proof and measurement of association between two things. Int J Epidemiol. 2010;39:1137–50. [DOI] [PubMed]

22.

Knight WR. A computer method for calculating Kendall’s tau with ungrouped data. J Am Stat Assoc. 1966;61:436–9. [DOI]

23.

Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: a next-generation hyperparameter optimization framework. In: KDD ’19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; 2019 Aug 4–8; Anchorage, USA. New York: Association for Computing Machinery; 2019. [DOI]

24.

Li Y, Su M, Liu Z, Li J, Liu J, Han L, et al. Assessing protein-ligand interaction scoring functions with the CASF-2013 benchmark. Nat Protoc. 2018;13:666–80. [DOI] [PubMed]