1
|
Jiang C, Huang Z, Pedapati T, Chen PY, Sun Y, Gao J. Network properties determine neural network performance. Nat Commun 2024; 15:5718. [PMID: 38977665 PMCID: PMC11231255 DOI: 10.1038/s41467-024-48069-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 04/17/2024] [Indexed: 07/10/2024] Open
Abstract
Machine learning influences numerous aspects of modern society, empowers new technologies, from Alphago to ChatGPT, and increasingly materializes in consumer products such as smartphones and self-driving cars. Despite the vital role and broad applications of artificial neural networks, we lack systematic approaches, such as network science, to understand their underlying mechanism. The difficulty is rooted in many possible model configurations, each with different hyper-parameters and weighted architectures determined by noisy data. We bridge the gap by developing a mathematical framework that maps the neural network's performance to the network characters of the line graph governed by the edge dynamics of stochastic gradient descent differential equations. This framework enables us to derive a neural capacitance metric to universally capture a model's generalization capability on a downstream task and predict model performance using only early training results. The numerical results on 17 pre-trained ImageNet models across five benchmark datasets and one NAS benchmark indicate that our neural capacitance metric is a powerful indicator for model selection based only on early training results and is more efficient than state-of-the-art methods.
Collapse
Affiliation(s)
- Chunheng Jiang
- Network Science and Technology Center, Rensselaer Polytechnic Institute, Troy, NY, USA
- Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Zhenhan Huang
- Network Science and Technology Center, Rensselaer Polytechnic Institute, Troy, NY, USA
- Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA
| | | | - Pin-Yu Chen
- IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA
| | - Yizhou Sun
- Department of Computer Science, University of California, Los Angeles, CA, USA
| | - Jianxi Gao
- Network Science and Technology Center, Rensselaer Polytechnic Institute, Troy, NY, USA.
- Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA.
| |
Collapse
|
2
|
Xue X, Wu JJ, Huo BB, Xing XX, Ma J, Li YL, Zheng MX, Hua XY, Xu JG. Age-related alterations of brain metabolic network based on [18F]FDG-PET of rats. Aging (Albany NY) 2022; 14:923-942. [PMID: 35077393 PMCID: PMC8833125 DOI: 10.18632/aging.203851] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2021] [Accepted: 01/14/2022] [Indexed: 11/25/2022]
Abstract
Using animal models to study the underlying mechanisms of aging will create a critical foundation from which to develop new interventions for aging-related brain disorders. Aging-related reorganization of the brain network has been described for the human brain based on functional, metabolic and structural connectivity. However, alterations in the brain metabolic network of aging rats remain unknown. Here, we submitted young and aged rats to [18F]fluorodeoxyglucose with positron emission tomography (18F-FDG PET) and constructed brain metabolic networks. The topological properties were detected, and the network robustness against random failures and targeted attacks was analyzed for age-group comparison. Compared with young rats, aged rats showed reduced betweenness centrality (BC) in the superior colliculus and a decreased degree (D) in the parietal association cortex. With regard to network robustness, the brain metabolic networks of aged rats were more vulnerable to simulated damage, which showed significantly lower local efficiency and clustering coefficients than those of the young rats against targeted attacks and random failures. The findings support the idea that aged rats have similar aging-related changes in the brain metabolic network to the human brain and can therefore be used as a model for aging studies to provide targets for potential therapies that promote healthy aging.
Collapse
Affiliation(s)
- Xin Xue
- School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China
| | - Jia-Jia Wu
- Department of Rehabilitation Medicine, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China
| | - Bei-Bei Huo
- School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China
| | - Xiang-Xin Xing
- School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China.,Department of Rehabilitation Medicine, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China
| | - Jie Ma
- School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China
| | - Yu-Lin Li
- School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China
| | - Mou-Xiong Zheng
- Department of Rehabilitation Medicine, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China.,Department of Traumatology and Orthopedics, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China
| | - Xu-Yun Hua
- Department of Rehabilitation Medicine, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China.,Department of Traumatology and Orthopedics, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China
| | - Jian-Guang Xu
- School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China.,Department of Rehabilitation Medicine, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai 200437, China.,Engineering Research Center of Traditional Chinese Medicine Intelligent Rehabilitation, Ministry of Education, Shanghai 201203, China
| |
Collapse
|