Information-Theoretic Limits of Transfer Learning Across Chemical Spaces in Materials GNNs

iamrp.net-TJZN-1791491020-j743528385 1791491020 Institute for Advanced Materials Research Press info@iamrp.net Institute for Advanced Materials Research Press Journal of Computational and Data-Driven Materials Engineering J. Comput. Data-Driven Mater. Eng. 3149-9368 2023 2 2 Information-Theoretic Limits of Transfer Learning Across Chemical Spaces in Materials GNNs Maria Silva Joao Pereira 2023 10.68159/j743528385 https://iamrp.net/pub/journal/2/article/j743528385 Chen C, Ye W, Zuo Y, Zheng C, Ong SP. Graph networks as a universal machine learning framework for molecules and crystals. Chem Mater. 2019;31(9):3564-72. 10.1021/acs.chemmater.9b01294 Batzner S, Musaelian A, Sun L, Geiger M, Mailoa JP, Kornbluth M, et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat Commun. 2022;13(1):2453. 10.1038/s41467-022-29939-5 Blanchard G, Deshmukh AA, Dogan U, Lee G, Scott C. Domain generalization by marginal transfer learning. J Mach Learn Res. 2021;22(2):1-55. Chen S, Sahinidis NV, Gao C. Transfer learning in information criteria-based feature selection. J Mach Learn Res. 2022;23(134):1-105. Schütt KT, Sauceda HE, Kindermans PJ, Tkatchenko A, Müller KR. SchNet - a deep learning architecture for molecules and materials. J Chem Phys. 2018;148(24):241722. 10.1063/1.5019779 Xu K, Hu W, Leskovec J, Jegelka S. How powerful are graph neural networks? arXiv:1810.00826 [Preprint]. 2018. Shen X, Pan S, Choi KS, Zhou X. Domain-adaptive message passing graph neural network. Neural Netw. 2023;164:439-54. 10.1016/j.neunet.2023.04.038 Gerace F, Saglietti L, Sarao Mannelli S, Saxe A, Zdeborová L. Probing transfer learning with a model of synthetic correlated datasets. Mach Learn Sci Technol. 2022;3(1):015030. 10.1088/2632-2153/ac4f3f Mallillin LLD. Different domains in learning and the academic performance of the students. Journal of Educational System. 2020;4(1):1-11. Fang Z, Lu J, Liu A, Liu F, Zhang G. Learning bounds for open-set learning. In: Proceedings of the 38th International Conference on Machine Learning; 2021 Jul 18-24; Virtual. PMLR; 2021. p. 3122-32. Li X, Zhang W. Deep learning-based partial domain adaptation method on intelligent machinery fault diagnostics. IEEE Trans Ind Electron. 2021;68(5):4351-61. 10.1109/TIE.2020.2984968 Choudhary K, DeCost B, Major L, Butler K, Thiyagalingam J, Tavazza F. Unified graph neural network force-field for the periodic table: Solid state applications. Digit Discov. 2023;2(2):346-55. 10.1039/D2DD00096B Chen G, Song Z, Qi Z, Sundmacher K. Generalizing property prediction of ionic liquids from limited labeled data: A one-stop framework empowered by transfer learning. Digit Discov. 2023;2(3):591-601. 10.1039/D3DD00040K Hoffmann N, Schmidt J, Botti S, Marques MAL. Transfer learning on large datasets for the accurate prediction of material properties. Digit Discov. 2023;2(5):1368-79. 10.1039/D3DD00030C Zhu W, Luo J, White AD. Federated learning of molecular properties with graph neural networks in a heterogeneous setting. Patterns (N Y). 2022;3(6):100521. 10.1016/j.patter.2022.100521 Reiser P, Neubert M, Eberhard A, Torresi L, Zhou C, Shao C, et al. Graph neural networks for materials science and chemistry. Commun Mater. 2022;3(1):93. 10.1038/s43246-022-00315-6 Weinberger N. Generalization bounds and algorithms for learning to communicate over additive noise channels. IEEE Trans Inf Theory. 2022;68(3):1886-921. 10.1109/TIT.2022.3176371 Di X, Yu P, Bu R, Sun M. Mutual information maximization in graph neural networks. In: 2020 International Joint Conference on Neural Networks (IJCNN); 2020 Jul 19-24; Glasgow, UK. IEEE; 2020. p. 1-7. 10.1109/IJCNN48605.2020.9207076 Peng Z, Huang W, Luo M, Zheng Q, Rong Y, Xu T, et al. Graph representation learning via graphical mutual information maximization. In: Proceedings of The Web Conference 2020; 2020 Apr 20-24; Taipei, Taiwan. New York: ACM; 2020. p. 259-70. 10.1145/3366423.3380112 Pandeva T, Bakker T, Naesseth CA, Forré P. E-valuating classifier two-sample tests [Preprint]. arXiv; 2022. arXiv:2210.13027. Wu X, Manton JH, Aickelin U, Zhu J. Information-theoretic analysis for transfer learning. In: 2020 IEEE International Symposium on Information Theory (ISIT); 2020 Jun 21-26. IEEE; 2020. p. 2819-24. 10.1109/ISIT44484.2020.9173989 Jose ST, Simeone O. Information-theoretic bounds on transfer generalization gap based on Jensen-Shannon divergence. In: 2021 29th European Signal Processing Conference (EUSIPCO); 2021 Aug 23-27; Dublin, Ireland. IEEE; 2021. p. 1461-5. 10.23919/EUSIPCO54536.2021.9616270 Esposito AR, Gastpar M, Issa I. Generalization error bounds via Rényi-, f-divergences and maximal leakage. IEEE Trans Inf Theory. 2021;67(8):4986-5004. 10.1109/TIT.2021.3085190 Zhou R, Tian C, Liu T. Individually conditional individual mutual information bound on generalization error. IEEE Trans Inf Theory. 2022;68(5):3304-16. 10.1109/TIT.2022.3144615 Chen Q, Marchand M. Algorithm-dependent bounds for representation learning of multi-source domain adaptation. In: Proceedings of the 26th International Conference on Artificial Intelligence and Statistics; 2023 Apr 25-27; Valencia, Spain. PMLR; 2023. p. 10368-94. Rojas-Carulla M, Schölkopf B, Turner R, Peters J. Invariant models for causal transfer learning. J Mach Learn Res. 2018;19(36):1-34. Park GY, Lee SW. Information-theoretic regularization for multi-source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2021 Oct; Virtual. IEEE; 2021. p. 9214-23. 10.1109/ICCV48922.2021.00908 Lv S. Generalization bounds for graph convolutional neural networks via Rademacher complexity [Preprint]. arXiv; 2021. arXiv:2102.10234 Ahn SS, Hu S, Dai Z, Damianou A, Lawrence N. Mutual information guided distillation for transfer learning. In: 32nd Conference on Neural Information Processing Systems; 2018 Dec 3-8; Montréal, Canada.