1
|
Morales-Galicia AE, Rincón-Sánchez MN, Ramírez-Mejía MM, Méndez-Sánchez N. Outcome prediction for cholangiocarcinoma prognosis: Embracing the machine learning era. World J Gastroenterol 2025; 31:106808. [DOI: 10.3748/wjg.v31.i21.106808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/07/2025] [Revised: 04/15/2025] [Accepted: 05/12/2025] [Indexed: 06/06/2025] Open
Abstract
We read with great interest the study by Huang et al. Cholangiocarcinoma (CC) is the second most common type of primary liver tumor worldwide. Although surgical resection remains the primary treatment for this disease, almost 50% of patients experience relapse within 2 years after surgery, which negatively affects their prognosis. Key predictors can be used to identify several factors (e.g., tumor size, tumor location, tumor stage, nerve invasion, the presence of intravascular emboli) and their correlations with long-term survival and the risk of postoperative morbidity. In recent years, artificial intelligence (AI) has become a new tool for prognostic assessment through the integration of multiple clinical, surgical, and imaging parameters. However, a crucial question has arisen: Are we ready to trust AI with respect to clinical decisions? The study by Huang et al demonstrated that AI can predict preoperative textbook outcomes in patients with CC and highlighted the precision of machine learning algorithms using useful prognostic factors. This letter to the editor aimed to explore the challenges and potential impact of AI and machine learning in the prognostic assessment of patients with CC.
Collapse
Affiliation(s)
| | - Mariana N Rincón-Sánchez
- Faculty of Medicine “Dr. Jose Sierra Flores,” Northeastern University, Tampico 89337, Tamaulipas, Mexico
| | - Mariana M Ramírez-Mejía
- Plan of Combined Studies in Medicine, Faculty of Medicine, National Autonomous University of Mexico, Mexico City 04360, Mexico
- Liver Research Unit, Medica Sur Clinic and Foundation, Mexico City 14050, Mexico
| | - Nahum Méndez-Sánchez
- Faculty of Medicine, National Autonomous University of Mexico, Mexico City 04360, Mexico
- Liver Research Unit, Medica Sur Clinic and Foundation, Mexico City 14050, Mexico
| |
Collapse
|
2
|
Colin TR, Ikink I, Holroyd CB. Distributed Representations for Cognitive Control in Frontal Medial Cortex. J Cogn Neurosci 2025; 37:941-969. [PMID: 39785682 DOI: 10.1162/jocn_a_02285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2025]
Abstract
In natural and artificial neural networks, modularity and distributed structure afford complementary but competing benefits. The former allows for hierarchical representations that can flexibly recombine modules to address novel problems, whereas the latter can benefit from less constrained training, potentially uncovering fruitful statistical regularities. Here, we investigate these competing demands in the context of human sequential behavior. First, we explore this setting by comparing the properties of several recurrent neural network models. We find that explicit hierarchical structure by itself fails to provide a critical performance advantage when compared with a "flat" model that does not incorporate hierarchical structure. However, hierarchy appears to facilitate cognitive control processes that support nonroutine behaviors and behaviors that are carried out under computational stress. Second, we compare these models against fMRI data using representational similarity analysis. We find that a model that incorporates so-called wiring costs in the cost function, which produces a hierarchically organized gradient of representational structure across the hidden layer of the neural network, best accounts for fMRI data collected from human participants in a previous study [Holroyd, C. B., Ribas-Fernandes, J. J. F., Shahnazian, D., Silvetti, M., & Verguts, T., Human midcingulate cortex encodes distributed representations of task progress. Proceedings of the National Academy of Sciences, U.S.A., 115, 6398-6403, 2018]. The results reveal that the ACC encodes distributed representations of sequential task context along a rostro-caudal gradient of abstraction: Rostral ACC encodes relatively abstract and temporally extended patterns of activity compared with those encoded by caudal ACC. These results provide insight into the role of ACC in motivation and cognitive control.
Collapse
|
3
|
Failor SW, Carandini M, Harris KD. Visual experience orthogonalizes visual cortical stimulus responses via population code transformation. Cell Rep 2025; 44:115235. [PMID: 39888718 DOI: 10.1016/j.celrep.2025.115235] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2024] [Revised: 09/26/2024] [Accepted: 01/06/2025] [Indexed: 02/02/2025] Open
Abstract
Sensory and behavioral experience can alter visual cortical stimulus coding, but the precise form of this plasticity is unclear. We measured orientation tuning in 4,000-neuron populations of mouse V1 before and after training on a visuomotor task. Changes to single-cell tuning curves appeared complex, including development of asymmetries and of multiple peaks. Nevertheless, these complex tuning curve transformations can be explained by a simple equation: a convex transformation suppressing responses to task stimuli specifically in cells responding at intermediate levels. The strength of the transformation varies across trials, suggesting a dynamic circuit mechanism rather than static synaptic plasticity. The transformation results in sparsening and orthogonalization of population codes for task stimuli. It cannot improve the performance of an optimal stimulus decoder, which is already perfect even for naive codes, but it improves the performance of a suboptimal decoder model with inductive bias as might be found in downstream readout circuits.
Collapse
Affiliation(s)
- Samuel W Failor
- UCL Queen Square Institute of Neurology, University College London, London WC1N 3BG, UK.
| | - Matteo Carandini
- UCL Institute of Ophthalmology, University College London, London EC1V 9EL, UK
| | - Kenneth D Harris
- UCL Queen Square Institute of Neurology, University College London, London WC1N 3BG, UK.
| |
Collapse
|
4
|
Dube AR, Ambrose AJH, Velez G, Jadhav M. Real concerns, artificial intelligence: Reality testing for psychiatrists. Int Rev Psychiatry 2025; 37:33-38. [PMID: 40035377 DOI: 10.1080/09540261.2024.2363374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2024] [Accepted: 05/29/2024] [Indexed: 03/05/2025]
Abstract
The use of augmented or artificial intelligence (AI) in healthcare promises groundbreaking advancements, from increasing diagnostic accuracy and minimizing clinical errors to personalized treatment plans and automated clinical decision-making. Its use may allow us to transition from phenomenological categories of psychiatric illness to one driven by underlying etiology and realize the Research Domain Criteria (RDoC) model proposed by the (U.S.) National Institutes of Mental Health (NIMH), which today remains difficult to apply clinically and is accessible primarily to researchers. AI may facilitate the transition to a more syncretic framework of understanding psychiatric illness that accounts for disruptions, all the way from the cellular level to the level of social systems. Yet, despite immense possibilities, there are also associated risks. In this article, we explore the challenges and opportunities associated with the use of AI in psychiatry, focusing on the potential ethical and health equity considerations in vulnerable populations, especially in child and adolescent psychiatry.
Collapse
Affiliation(s)
- Anish R Dube
- Riverside University Health System/Loma Linda University, Moreno Valley, CA, USA
| | - Adrian Jacques H Ambrose
- Department of Psychiatry, Columbia University Irving Medical Center, Vagelos College of Physicians & Surgeons, New York City, NY, USA
| | - German Velez
- Weill Cornell Medicine, The Columbia College of Physicians and Surgeons, New York City, NY, USA
| | - Mandar Jadhav
- National Association of Community Health Centers, Bethesda, MD, USA
| |
Collapse
|
5
|
Zerlaut Y, Tzilivaki A. Interneuronal modulations as a functional switch for cortical computations: mechanisms and implication for disease. Front Cell Neurosci 2025; 18:1479579. [PMID: 39916937 PMCID: PMC11799556 DOI: 10.3389/fncel.2024.1479579] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2024] [Accepted: 12/27/2024] [Indexed: 02/09/2025] Open
Abstract
Understanding cortical inhibition and its diverse roles remains a key challenge in neurophysiological research. Traditionally, inhibition has been recognized for controlling the stability and rhythmicity of network dynamics, or refining the spatiotemporal properties of cortical representations. In this perspective, we propose that specific types of interneurons may play a complementary role, by modulating the computational properties of neural networks. We review experimental and theoretical evidence, mainly from rodent sensory cortices, that supports this view. Additionally, we explore how dysfunctions in these interneurons may disrupt the network's ability to switch between computational modes, impacting the flexibility of cortical processing and potentially contributing to various neurodevelopmental and psychiatric disorders.
Collapse
Affiliation(s)
- Yann Zerlaut
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France
| | - Alexandra Tzilivaki
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Neuroscience Research Center, Berlin, Germany
- Einstein Center for Neurosciences, Chariteplatz, Berlin, Germany
- NeuroCure Cluster of Excellence, Chariteplatz, Berlin, Germany
| |
Collapse
|
6
|
Yang S, Sun M, Shi C, Liu Y, Guo Y, Liu Y, Lu Z, Huang Y, Pu X. Data-Quality-Navigated Machine Learning Strategy with Chemical Intuition to Improve Generalization. J Chem Theory Comput 2024; 20:10633-10648. [PMID: 39589234 DOI: 10.1021/acs.jctc.4c00969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2024]
Abstract
Generalizing real-world data has been one of the most difficult challenges for application of machine learning (ML) in practice. Most ML works focused on improvements in algorithms and feature representations. However, the data quality, as the foundation of ML, has been largely overlooked, also leading to the absence of data evaluation and processing methods in ML fields. Motivated by the challenge and need, we selected an important but difficult reorganization energy (RE) prediction task as a test platform, which is an important parameter for the charge mobility of organic semiconductors (OSCs), to propose a data-quality-navigated strategy with chemical intuition. We developed a data diversity evaluation based on structure characteristics of OSC molecules, a reliability evaluation method based on prediction accuracy, a data filtering method based on the uncertainty of K-fold division, and a data split technique by clustering and stratified sampling based on four molecular descriptor-associated REs. Consequently, a representative RE data set (15,989 molecules) with high reliability and diversity can be obtained. For the feature representation, a complementary strategy is proposed by considering the chemical nature of REs and the structure characteristics of OCS molecules as well as the model algorithm. In addition, an ensemble framework consisting of two deep learning models is constructed to avoid the risk of local optimization of the single model. The robustness and generalization of our model are strongly validated against different OSC-like molecules with diverse structures and a wide range of REs and real OSC molecules, greatly outperforming eight adversarial controls. Collectively, our work not only provides a quick and reliable tool to screen efficient OSCs but also offers methodological guidelines for improving the generalization of ML.
Collapse
Affiliation(s)
- Songran Yang
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Ming Sun
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Chaojie Shi
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Yiran Liu
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Yanzhi Guo
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Yijing Liu
- College of Computer Science, Sichuan University, Chengdu 610064, China
| | - Zhiyun Lu
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Yan Huang
- College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Xuemei Pu
- College of Chemistry, Sichuan University, Chengdu 610064, China
| |
Collapse
|
7
|
Sun M, Fu C, Su H, Xiao R, Shi C, Lu Z, Pu X. Enhancing chemistry-intuitive feature learning to improve prediction performance of optical properties. Chem Sci 2024:d4sc02781g. [PMID: 39381129 PMCID: PMC11457255 DOI: 10.1039/d4sc02781g] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Accepted: 09/22/2024] [Indexed: 10/10/2024] Open
Abstract
Emitters have been widely applied in versatile fields, dependent on their optical properties. Thus, it is of great importance to explore a quick and accurate prediction method for optical properties. To this end, we have developed a state-of-the-art deep learning (DL) framework by enhancing chemistry-intuitive subgraph and edge learning and coupling this with prior domain knowledge for a classic message passing neural network (MPNN) which can better capture the structural features associated with the optical properties from a limited dataset. Benefiting from technical advantages, our model significantly outperforms eight competitive ML models used in five different optical datasets, achieving the highest accuracy to date in predicting four important optical properties (absorption wavelength, emission wavelength, photoluminescence quantum yield and full width at half-maximum), showcasing its robustness and generalization. More importantly, based on our predicted results, one new deep-blue light-emitting molecule PPI-2TPA was successfully synthesized and characterized, which exhibits close consistency with our predictions, clearly confirming the application potential of our model as a quick and reliable prediction tool for the optical properties of diverse emitters in practice.
Collapse
Affiliation(s)
- Ming Sun
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| | - Caixia Fu
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| | - Haoming Su
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| | - Ruyue Xiao
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| | - Chaojie Shi
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| | - Zhiyun Lu
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| | - Xuemei Pu
- College of Chemistry, Sichuan University Chengdu 610064 People's Republic of China
| |
Collapse
|
8
|
Zhang R, Pitkow X, Angelaki DE. Inductive biases of neural network modularity in spatial navigation. SCIENCE ADVANCES 2024; 10:eadk1256. [PMID: 39028809 PMCID: PMC11259174 DOI: 10.1126/sciadv.adk1256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 06/14/2024] [Indexed: 07/21/2024]
Abstract
The brain may have evolved a modular architecture for daily tasks, with circuits featuring functionally specialized modules that match the task structure. We hypothesize that this architecture enables better learning and generalization than architectures with less specialized modules. To test this, we trained reinforcement learning agents with various neural architectures on a naturalistic navigation task. We found that the modular agent, with an architecture that segregates computations of state representation, value, and action into specialized modules, achieved better learning and generalization. Its learned state representation combines prediction and observation, weighted by their relative uncertainty, akin to recursive Bayesian estimation. This agent's behavior also resembles macaques' behavior more closely. Our results shed light on the possible rationale for the brain's modularity and suggest that artificial systems can use this insight from neuroscience to improve learning and generalization in natural tasks.
Collapse
Affiliation(s)
- Ruiyi Zhang
- Tandon School of Engineering, New York University, New York, NY, USA
| | - Xaq Pitkow
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
- Department of Machine Learning, Carnegie Mellon University, Pittsburgh, PA, USA
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, USA
| | - Dora E. Angelaki
- Tandon School of Engineering, New York University, New York, NY, USA
- Center for Neural Science, New York University, New York, NY, USA
| |
Collapse
|
9
|
Poli F, Li YL, Naidu P, Mars RB, Hunnius S, Ruggeri A. Toddlers strategically adapt their information search. Nat Commun 2024; 15:5780. [PMID: 38987261 PMCID: PMC11237003 DOI: 10.1038/s41467-024-48855-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 05/14/2024] [Indexed: 07/12/2024] Open
Abstract
Adaptive information seeking is essential for humans to effectively navigate complex and dynamic environments. Here, we developed a gaze-contingent eye-tracking paradigm to examine the early emergence of adaptive information-seeking. Toddlers (N = 60, 18-36 months) and adults (N = 42) either learnt that an animal was equally likely to be found in any of four available locations, or that it was most likely to be found in one particular location. Afterwards, they were given control of a torchlight, which they could move with their eyes to explore the otherwise pitch-black task environment. Eye-movement data and Markov models show that, from 24 months of age, toddlers become more exploratory than adults, and start adapting their exploratory strategies to the information structure of the task. These results show that toddlers' search strategies are more sophisticated than previously thought, and identify the unique features that distinguish their information search from adults'.
Collapse
Affiliation(s)
- Francesco Poli
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands.
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.
| | - Yi-Lin Li
- Wellcome Centre for Integrative Neuroimaging, Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Pravallika Naidu
- Wellcome Centre for Integrative Neuroimaging, Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Rogier B Mars
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| | - Sabine Hunnius
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| | - Azzurra Ruggeri
- Max Planck Research Group iSearch, Max Planck Institute for Human Development, Berlin, Germany.
- School of Social Sciences and Technology, Department of Education, Technical University Munich, Munich, Germany.
- Department of Cognitive Science, Central European University, Vienna, Austria.
| |
Collapse
|
10
|
Jahani Yekta MM. Biological direct-shortcut deep residual learning for sparse visual processing. Sci Rep 2024; 14:14066. [PMID: 38890361 PMCID: PMC11189431 DOI: 10.1038/s41598-024-62756-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 05/21/2024] [Indexed: 06/20/2024] Open
Abstract
We show, based on the following three grounds, that the primary visual cortex (V1) is a biological direct-shortcut deep residual learning neural network (ResNet) for sparse visual processing: (1) We first highlight that Gabor-like sets of basis functions, which are similar to the receptive fields of simple cells in the primary visual cortex (V1), are excellent candidates for sparse representation of natural images; i.e., images from the natural world, affirming the brain to be optimized for this. (2) We then prove that the intra-layer synaptic weight matrices of this region can be reasonably first-order approximated by identity mappings, and are thus sparse themselves. (3) Finally, we point out that intra-layer weight matrices having identity mapping as their initial approximation, irrespective of this approximation being also a reasonable first-order one or not, resemble the building blocks of direct-shortcut digital ResNets, which completes the grounds. This biological ResNet interconnects the sparsity of the final representation of the image to that of its intra-layer weights. Further exploration of this ResNet, and understanding the joint effects of its architecture and learning rules, e.g. on its inductive bias, could lead to major advancements in the area of bio-inspired digital ResNets. One immediate line of research in this context, for instance, is to study the impact of forcing the direct-shortcuts to be good first-order approximations of each building block. For this, along with the ℓ 1 -minimization posed on the basis function coefficients the ResNet finally provides at its output, another parallel one could e.g. also be posed on the weights of its residual layers.
Collapse
|
11
|
Nikiruy K, Perez E, Baroni A, Reddy KDS, Pechmann S, Wenger C, Ziegler M. Blooming and pruning: learning from mistakes with memristive synapses. Sci Rep 2024; 14:7802. [PMID: 38565677 PMCID: PMC10987678 DOI: 10.1038/s41598-024-57660-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 03/20/2024] [Indexed: 04/04/2024] Open
Abstract
Blooming and pruning is one of the most important developmental mechanisms of the biological brain in the first years of life, enabling it to adapt its network structure to the demands of the environment. The mechanism is thought to be fundamental for the development of cognitive skills. Inspired by this, Chialvo and Bak proposed in 1999 a learning scheme that learns from mistakes by eliminating from the initial surplus of synaptic connections those that lead to an undesirable outcome. Here, this idea is implemented in a neuromorphic circuit scheme using CMOS integrated HfO2-based memristive devices. The implemented two-layer neural network learns in a self-organized manner without positive reinforcement and exploits the inherent variability of the memristive devices. This approach provides hardware, local, and energy-efficient learning. A combined experimental and simulation-based parameter study is presented to find the relevant system and device parameters leading to a compact and robust memristive neuromorphic circuit that can handle association tasks.
Collapse
Affiliation(s)
- Kristina Nikiruy
- Micro- and Nanoelectronic Systems, Department of Electrical Engineering and Information Technology, TU Ilmenau, Ilmenau, Germany.
| | - Eduardo Perez
- IHP - Leibniz-Institut fuer innovative Mikroelektronik, Frankfurt/Oder, Germany
- BTU Cottbus-Senftenberg, Cottbus, Germany
| | - Andrea Baroni
- IHP - Leibniz-Institut fuer innovative Mikroelektronik, Frankfurt/Oder, Germany
| | | | - Stefan Pechmann
- Chair of Micro- and Nanosystems Technology, Technical University of Munich, Munich, Germany
| | - Christian Wenger
- IHP - Leibniz-Institut fuer innovative Mikroelektronik, Frankfurt/Oder, Germany
- BTU Cottbus-Senftenberg, Cottbus, Germany
| | - Martin Ziegler
- Micro- and Nanoelectronic Systems, Department of Electrical Engineering and Information Technology, TU Ilmenau, Ilmenau, Germany
- Institute of Micro- and Nanotechnologies MacroNano, TU Ilmenau, Ilmenau, Germany
| |
Collapse
|
12
|
Vallée R, Gomez T, Bourreille A, Normand N, Mouchère H, Coutrot A. Influence of training and expertise on deep neural network attention and human attention during a medical image classification task. J Vis 2024; 24:6. [PMID: 38587421 PMCID: PMC11008746 DOI: 10.1167/jov.24.4.6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Accepted: 11/19/2023] [Indexed: 04/09/2024] Open
Abstract
In many different domains, experts can make complex decisions after glancing very briefly at an image. However, the perceptual mechanisms underlying expert performance are still largely unknown. Recently, several machine learning algorithms have been shown to outperform human experts in specific tasks. But these algorithms often behave as black boxes and their information processing pipeline remains unknown. This lack of transparency and interpretability is highly problematic in applications involving human lives, such as health care. One way to "open the black box" is to compute an artificial attention map from the model, which highlights the pixels of the input image that contributed the most to the model decision. In this work, we directly compare human visual attention to machine visual attention when performing the same visual task. We have designed a medical diagnosis task involving the detection of lesions in small bowel endoscopic images. We collected eye movements from novices and gastroenterologist experts while they classified medical images according to their relevance for Crohn's disease diagnosis. We trained three state-of-the-art deep learning models on our carefully labeled dataset. Both humans and machine performed the same task. We extracted artificial attention with six different post hoc methods. We show that the model attention maps are significantly closer to human expert attention maps than to novices', especially for pathological images. As the model gets trained and its performance gets closer to the human experts, the similarity between model and human attention increases. Through the understanding of the similarities between the visual decision-making process of human experts and deep neural networks, we hope to inform both the training of new doctors and the architecture of new algorithms.
Collapse
Affiliation(s)
- Rémi Vallée
- Nantes Université, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes, France
| | - Tristan Gomez
- Nantes Université, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes, France
| | - Arnaud Bourreille
- CHU Nantes, Institut des Maladies de l'Appareil Digestif, CIC Inserm 1413, Université de Nantes, Nantes, France
| | - Nicolas Normand
- Nantes Université, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes, France
| | - Harold Mouchère
- Nantes Université, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes, France
| | - Antoine Coutrot
- Nantes Université, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes, France
- Univ Lyon, CNRS, INSA Lyon, UCBL, LIRIS, UMR5205, Lyon, France
| |
Collapse
|
13
|
Noda K, Soda T, Yamashita Y. Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models. Front Neurosci 2024; 18:1330512. [PMID: 38298912 PMCID: PMC10828047 DOI: 10.3389/fnins.2024.1330512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 01/02/2024] [Indexed: 02/02/2024] Open
Abstract
Introduction Associating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills. Methods We employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks. Results Multimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks. Discussion Our novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions.
Collapse
Affiliation(s)
| | | | - Yuichi Yamashita
- Department of Information Medicine, National Institute of Neuroscience, National Center of Neurology and Psychiatry, Kodaira, Japan
| |
Collapse
|
14
|
Jung JH, Wang Y, Rashid AJ, Zhang T, Frankland PW, Josselyn SA. Examining memory linking and generalization using scFLARE2, a temporally precise neuronal activity tagging system. Cell Rep 2023; 42:113592. [PMID: 38103203 PMCID: PMC10842737 DOI: 10.1016/j.celrep.2023.113592] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/26/2023] [Accepted: 11/30/2023] [Indexed: 12/18/2023] Open
Abstract
How memories are organized in the brain influences whether they are remembered discretely versus linked with other experiences or whether generalized information is applied to entirely novel situations. Here, we used scFLARE2 (single-chain fast light- and activity-regulated expression 2), a temporally precise tagging system, to manipulate mouse lateral amygdala neurons active during one of two 3 min threat experiences occurring close (3 h) or further apart (27 h) in time. Silencing scFLARE2-tagged neurons showed that two threat experiences occurring at distal times are dis-allocated to orthogonal engram ensembles and remembered discretely, whereas the same two threat experiences occurring in close temporal proximity are linked via co-allocation to overlapping engram ensembles. Moreover, we found that co-allocation mediates memory generalization applied to a completely novel stimulus. These results indicate that endogenous temporal evolution of engram ensemble neuronal excitability determines how memories are organized and remembered and that this would not be possible using conventional immediate-early gene-based tagging methods.
Collapse
Affiliation(s)
- Jung Hoon Jung
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada
| | - Ying Wang
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5S 1A8, Canada
| | - Asim J Rashid
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada
| | - Tao Zhang
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada
| | - Paul W Frankland
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5S 1A8, Canada; Department of Psychology, University of Toronto, Toronto, ON M5S 3G3, Canada; Child & Brain Development Program, Canadian Institute for Advanced Research (CIFAR), Toronto, ON M5G 1M1, Canada
| | - Sheena A Josselyn
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5S 1A8, Canada; Department of Psychology, University of Toronto, Toronto, ON M5S 3G3, Canada.
| |
Collapse
|
15
|
Khan S, Wong A, Tripp B. Modeling the Role of Contour Integration in Visual Inference. Neural Comput 2023; 36:33-74. [PMID: 38052088 DOI: 10.1162/neco_a_01625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 09/08/2023] [Indexed: 12/07/2023]
Abstract
Under difficult viewing conditions, the brain's visual system uses a variety of recurrent modulatory mechanisms to augment feedforward processing. One resulting phenomenon is contour integration, which occurs in the primary visual (V1) cortex and strengthens neural responses to edges if they belong to a larger smooth contour. Computational models have contributed to an understanding of the circuit mechanisms of contour integration, but less is known about its role in visual perception. To address this gap, we embedded a biologically grounded model of contour integration in a task-driven artificial neural network and trained it using a gradient-descent variant. We used this model to explore how brain-like contour integration may be optimized for high-level visual objectives as well as its potential roles in perception. When the model was trained to detect contours in a background of random edges, a task commonly used to examine contour integration in the brain, it closely mirrored the brain in terms of behavior, neural responses, and lateral connection patterns. When trained on natural images, the model enhanced weaker contours and distinguished whether two points lay on the same versus different contours. The model learned robust features that generalized well to out-of-training-distribution stimuli. Surprisingly, and in contrast with the synthetic task, a parameter-matched control network without recurrence performed the same as or better than the model on the natural-image tasks. Thus, a contour integration mechanism is not essential to perform these more naturalistic contour-related tasks. Finally, the best performance in all tasks was achieved by a modified contour integration model that did not distinguish between excitatory and inhibitory neurons.
Collapse
Affiliation(s)
- Salman Khan
- Centre for Theoretical Neuroscience, Department of System Design Engineering
- Vision and Image Processing Group, Department of System Design Engineering
- Waterloo Artificial Intelligence Institute: University of Waterloo, Waterloo, ON, Canada, N2L 3G1
| | - Alexander Wong
- Vision and Image Processing Group, Department of System Design Engineering
- Waterloo Artificial Intelligence Institute: University of Waterloo, Waterloo, ON, Canada, N2L 3G1
| | - Bryan Tripp
- Centre for Theoretical Neuroscience, Department of System Design Engineering
- Vision and Image Processing Group, Department of System Design Engineering
- Waterloo Artificial Intelligence Institute: University of Waterloo, Waterloo, ON, Canada, N2L 3G1
| |
Collapse
|
16
|
Constantinidis C, Ahmed AA, Wallis JD, Batista AP. Common Mechanisms of Learning in Motor and Cognitive Systems. J Neurosci 2023; 43:7523-7529. [PMID: 37940591 PMCID: PMC10634576 DOI: 10.1523/jneurosci.1505-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 08/29/2023] [Accepted: 08/31/2023] [Indexed: 11/10/2023] Open
Abstract
Rapid progress in our understanding of the brain's learning mechanisms has been accomplished over the past decade, particularly with conceptual advances, including representing behavior as a dynamical system, large-scale neural population recordings, and new methods of analysis of neuronal populations. However, motor and cognitive systems have been traditionally studied with different methods and paradigms. Recently, some common principles, evident in both behavior and neural activity, that underlie these different types of learning have become to emerge. Here we review results from motor and cognitive learning, relying on different techniques and studying different systems to understand the mechanisms of learning. Movement is intertwined with cognitive operations, and its dynamics reflect cognitive variables. Training, in either motor or cognitive tasks, involves recruitment of previously unresponsive neurons and reorganization of neural activity in a low dimensional manifold. Mapping of new variables in neural activity can be very rapid, instantiating flexible learning of new tasks. Communication between areas is just as critical a part of learning as are patterns of activity within an area emerging with learning. Common principles across systems provide a map for future research.
Collapse
Affiliation(s)
| | - Alaa A Ahmed
- Department of Mechanical Engineering, University of Colorado Boulder, Boulder Colorado 80309
| | - Joni D Wallis
- Department of Psychology, University of California Berkeley, Berkeley, California 94720
| | - Aaron P Batista
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15213
| |
Collapse
|
17
|
Ukita J, Ohki K. Adversarial attacks and defenses using feature-space stochasticity. Neural Netw 2023; 167:875-889. [PMID: 37722983 DOI: 10.1016/j.neunet.2023.08.022] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2022] [Revised: 05/27/2023] [Accepted: 08/13/2023] [Indexed: 09/20/2023]
Abstract
Recent studies in deep neural networks have shown that injecting random noise in the input layer of the networks contributes towards ℓp-norm-bounded adversarial perturbations. However, to defend against unrestricted adversarial examples, most of which are not ℓp-norm-bounded in the input layer, such input-layer random noise may not be sufficient. In the first part of this study, we generated a novel class of unrestricted adversarial examples termed feature-space adversarial examples. These examples are far from the original data in the input space but adjacent to the original data in a hidden-layer feature space and far again in the output layer. In the second part of this study, we empirically showed that while injecting random noise in the input layer was unable to defend these feature-space adversarial examples, they were defended by injecting random noise in the hidden layer. These results highlight the novel benefit of stochasticity in higher layers, in that it is useful for defending against these feature-space adversarial examples, a class of unrestricted adversarial examples.
Collapse
Affiliation(s)
- Jumpei Ukita
- Department of Physiology, The University of Tokyo School of Medicine, 7-3-1, Hongo, Bunkyo-ku, 113-0033, Tokyo, Japan.
| | - Kenichi Ohki
- Department of Physiology, The University of Tokyo School of Medicine, 7-3-1, Hongo, Bunkyo-ku, 113-0033, Tokyo, Japan; International Research Center for Neurointelligence (WPI-IRCN), 7-3-1, Hongo, Bunkyo-ku, 113-0033, Tokyo, Japan; Institute for AI and Beyond, 7-3-1, Hongo, Bunkyo-ku, 113-0033, Tokyo, Japan.
| |
Collapse
|
18
|
Qin X, Ran T, Chen Y, Zhang Y, Wang D, Zhou C, Zou D. Artificial Intelligence in Endoscopic Ultrasonography-Guided Fine-Needle Aspiration/Biopsy (EUS-FNA/B) for Solid Pancreatic Lesions: Opportunities and Challenges. Diagnostics (Basel) 2023; 13:3054. [PMID: 37835797 PMCID: PMC10572518 DOI: 10.3390/diagnostics13193054] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 09/06/2023] [Accepted: 09/06/2023] [Indexed: 10/15/2023] Open
Abstract
Solid pancreatic lesions (SPLs) encompass a variety of benign and malignant diseases and accurate diagnosis is crucial for guiding appropriate treatment decisions. Endoscopic ultrasonography-guided fine-needle aspiration/biopsy (EUS-FNA/B) serves as a front-line diagnostic tool for pancreatic mass lesions and is widely used in clinical practice. Artificial intelligence (AI) is a mathematical technique that automates the learning and recognition of data patterns. Its strong self-learning ability and unbiased nature have led to its gradual adoption in the medical field. In this paper, we describe the fundamentals of AI and provide a summary of reports on AI in EUS-FNA/B to help endoscopists understand and realize its potential in improving pathological diagnosis and guiding targeted EUS-FNA/B. However, AI models have limitations and shortages that need to be addressed before clinical use. Furthermore, as most AI studies are retrospective, large-scale prospective clinical trials are necessary to evaluate their clinical usefulness accurately. Although AI in EUS-FNA/B is still in its infancy, the constant input of clinical data and the advancements in computer technology are expected to make computer-aided diagnosis and treatment more feasible.
Collapse
Affiliation(s)
| | | | | | | | | | - Chunhua Zhou
- Department of Gastroenterology, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200025, China; (X.Q.); (T.R.); (Y.C.); (Y.Z.); (D.W.)
| | - Duowu Zou
- Department of Gastroenterology, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200025, China; (X.Q.); (T.R.); (Y.C.); (Y.Z.); (D.W.)
| |
Collapse
|
19
|
Gebicke-Haerter PJ. The computational power of the human brain. Front Cell Neurosci 2023; 17:1220030. [PMID: 37608987 PMCID: PMC10441807 DOI: 10.3389/fncel.2023.1220030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 07/05/2023] [Indexed: 08/24/2023] Open
Abstract
At the end of the 20th century, analog systems in computer science have been widely replaced by digital systems due to their higher computing power. Nevertheless, the question keeps being intriguing until now: is the brain analog or digital? Initially, the latter has been favored, considering it as a Turing machine that works like a digital computer. However, more recently, digital and analog processes have been combined to implant human behavior in robots, endowing them with artificial intelligence (AI). Therefore, we think it is timely to compare mathematical models with the biology of computation in the brain. To this end, digital and analog processes clearly identified in cellular and molecular interactions in the Central Nervous System are highlighted. But above that, we try to pinpoint reasons distinguishing in silico computation from salient features of biological computation. First, genuinely analog information processing has been observed in electrical synapses and through gap junctions, the latter both in neurons and astrocytes. Apparently opposed to that, neuronal action potentials (APs) or spikes represent clearly digital events, like the yes/no or 1/0 of a Turing machine. However, spikes are rarely uniform, but can vary in amplitude and widths, which has significant, differential effects on transmitter release at the presynaptic terminal, where notwithstanding the quantal (vesicular) release itself is digital. Conversely, at the dendritic site of the postsynaptic neuron, there are numerous analog events of computation. Moreover, synaptic transmission of information is not only neuronal, but heavily influenced by astrocytes tightly ensheathing the majority of synapses in brain (tripartite synapse). At least at this point, LTP and LTD modifying synaptic plasticity and believed to induce short and long-term memory processes including consolidation (equivalent to RAM and ROM in electronic devices) have to be discussed. The present knowledge of how the brain stores and retrieves memories includes a variety of options (e.g., neuronal network oscillations, engram cells, astrocytic syncytium). Also epigenetic features play crucial roles in memory formation and its consolidation, which necessarily guides to molecular events like gene transcription and translation. In conclusion, brain computation is not only digital or analog, or a combination of both, but encompasses features in parallel, and of higher orders of complexity.
Collapse
Affiliation(s)
- Peter J. Gebicke-Haerter
- Institute of Psychopharmacology, Central Institute of Mental Health, Faculty of Medicine, University of Heidelberg, Mannheim, Germany
| |
Collapse
|
20
|
Huang B, Zhang Y, Mao Q, Ju Y, Liu Y, Su Z, Lei Y, Ren Y. Deep learning-based prediction of H3K27M alteration in diffuse midline gliomas based on whole-brain MRI. Cancer Med 2023; 12:17139-17148. [PMID: 37461358 PMCID: PMC10501256 DOI: 10.1002/cam4.6363] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 07/02/2023] [Accepted: 07/08/2023] [Indexed: 09/15/2023] Open
Abstract
BACKGROUND H3K27M mutation status significantly affects the prognosis of patients with diffuse midline gliomas (DMGs), but this tumor presents a high risk of pathological acquisition. We aimed to construct a fully automated model for predicting the H3K27M alteration status of DMGs based on deep learning using whole-brain MRI. METHODS DMG patients from West China Hospital of Sichuan University (WCHSU; n = 200) and Chengdu Shangjin Nanfu Hospital (CSNH; n = 35) who met the inclusion and exclusion criteria from February 2016 to April 2022 were enrolled as the training and external test sets, respectively. To adapt the model to the human head MRI scene, we use normal human head MR images to pretrain the model. The classification and tumor segmentation tasks are naturally related, so we conducted cotraining for the two tasks to enable information interaction between them and improve the accuracy of the classification task. RESULTS The average classification accuracies of our model on the training and external test sets was 90.5% and 85.1%, respectively. Ablation experiments showed that pretraining and cotraining could improve the prediction accuracy and generalization performance of the model. In the training and external test sets, the average areas under the receiver operating characteristic curve (AUROCs) were 94.18% and 87.64%, and the average areas under the precision-recall curve (AUPRC) were 93.26% and 85.4%. CONCLUSIONS The developed model achieved excellent performance in predicting the H3K27M alteration status in DMGs, and its good reproducibility and generalization were verified in the external dataset.
Collapse
Affiliation(s)
- Bowen Huang
- Department of NeurosurgeryWest China Hospital of Sichuan UniversityChengduChina
| | - Yuekang Zhang
- Department of NeurosurgeryWest China Hospital of Sichuan UniversityChengduChina
| | - Qing Mao
- Department of NeurosurgeryWest China Hospital of Sichuan UniversityChengduChina
| | - Yan Ju
- Department of NeurosurgeryWest China Hospital of Sichuan UniversityChengduChina
| | - Yanhui Liu
- Department of NeurosurgeryWest China Hospital of Sichuan UniversityChengduChina
| | - Zhengzheng Su
- Department of PathologyWest China Hospital of Sichuan UniversityChengduChina
| | - Yinjie Lei
- College of Electronics and Information EngineeringSichuan UniversityChengduChina
| | - Yanming Ren
- Department of NeurosurgeryWest China Hospital of Sichuan UniversityChengduChina
| |
Collapse
|
21
|
Tsvetkov C, Malhotra G, Evans BD, Bowers JS. The role of capacity constraints in Convolutional Neural Networks for learning random versus natural data. Neural Netw 2023; 161:515-524. [PMID: 36805266 DOI: 10.1016/j.neunet.2023.01.011] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 11/17/2022] [Accepted: 01/11/2023] [Indexed: 02/05/2023]
Abstract
Convolutional neural networks (CNNs) are often described as promising models of human vision, yet they show many differences from human abilities. We focus on a superhuman capacity of top-performing CNNs, namely, their ability to learn very large datasets of random patterns. We verify that human learning on such tasks is extremely limited, even with few stimuli. We argue that the performance difference is due to CNNs' overcapacity and introduce biologically inspired mechanisms to constrain it, while retaining the good test set generalisation to structured images as characteristic of CNNs. We investigate the efficacy of adding noise to hidden units' activations, restricting early convolutional layers with a bottleneck, and using a bounded activation function. Internal noise was the most potent intervention and the only one which, by itself, could reduce random data performance in the tested models to chance levels. We also investigated whether networks with biologically inspired capacity constraints show improved generalisation to out-of-distribution stimuli, however little benefit was observed. Our results suggest that constraining networks with biologically motivated mechanisms paves the way for closer correspondence between network and human performance, but the few manipulations we have tested are only a small step towards that goal.
Collapse
Affiliation(s)
- Christian Tsvetkov
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK.
| | - Gaurav Malhotra
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK.
| | - Benjamin D Evans
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK; Department of Informatics, School of Engineering and Informatics, University of Sussex, Falmer, Brighton, BN1 9RH, UK.
| | - Jeffrey S Bowers
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK.
| |
Collapse
|
22
|
Zador A, Escola S, Richards B, Ölveczky B, Bengio Y, Boahen K, Botvinick M, Chklovskii D, Churchland A, Clopath C, DiCarlo J, Ganguli S, Hawkins J, Körding K, Koulakov A, LeCun Y, Lillicrap T, Marblestone A, Olshausen B, Pouget A, Savin C, Sejnowski T, Simoncelli E, Solla S, Sussillo D, Tolias AS, Tsao D. Catalyzing next-generation Artificial Intelligence through NeuroAI. Nat Commun 2023; 14:1597. [PMID: 36949048 PMCID: PMC10033876 DOI: 10.1038/s41467-023-37180-x] [Citation(s) in RCA: 55] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2022] [Accepted: 03/03/2023] [Indexed: 03/24/2023] Open
Abstract
Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities - inherited from over 500 million years of evolution - that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.
Collapse
Affiliation(s)
- Anthony Zador
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, 11724, USA.
| | - Sean Escola
- Department of Psychiatry, Columbia University, New York, NY, 10027, USA
| | - Blake Richards
- Mila, Montréal, QC, H2S 3H1, Canada
- School of Computer Science, McGill University, Montreal, Canada
- Montreal Neurological Institute, McGill University, Montreal, Canada
- Department of Neurology & Neurosurgery, McGill University, Montreal, Canada
- Learning in Machines and Brains Program, CIFAR, Toronto, Canada
| | - Bence Ölveczky
- Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA, 02138, USA
| | | | - Kwabena Boahen
- Department of Bioengineering, Stanford University, Stanford, CA, 94305, USA
| | | | | | - Anne Churchland
- Department of Neurobiology, University of California Los Angeles, Los Angeles, CA, 90095, USA
| | - Claudia Clopath
- Department of Bioengineering, Imperial College London, London, SW7 2BW, UK
| | - James DiCarlo
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, 02139, USA
| | - Surya Ganguli
- Department of Applied Physics, Stanford University, Stanford, CA, 94305, USA
| | | | - Konrad Körding
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Alexei Koulakov
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, 11724, USA
| | - Yann LeCun
- Meta, Menlo Park, CA, 94025, USA
- Department of Electrical and Computer Engineering, NYU, Brooklyn, NY, 11201, USA
| | | | | | - Bruno Olshausen
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA, 94720, USA
| | - Alexandre Pouget
- Department of Basic Neurosciences, University of Geneva, Genève, 1211, Switzerland
| | - Cristina Savin
- Center for Neural Science, NYU, New York, NY, 10003, USA
| | | | - Eero Simoncelli
- Departments of Neural Science, Mathematics, and Psychology, NYU, New York, NY, 10003, USA
| | - Sara Solla
- Department of Physiology, Northwestern University, Chicago, IL, 60611, USA
| | - David Sussillo
- Meta, Menlo Park, CA, 94025, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA
| | - Andreas S Tolias
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, 77030, USA
| | - Doris Tsao
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA, 94720, USA
| |
Collapse
|
23
|
Li Z, Ortega Caro J, Rusak E, Brendel W, Bethge M, Anselmi F, Patel AB, Tolias AS, Pitkow X. Robust deep learning object recognition models rely on low frequency information in natural images. PLoS Comput Biol 2023; 19:e1010932. [PMID: 36972288 PMCID: PMC10079058 DOI: 10.1371/journal.pcbi.1010932] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2022] [Revised: 04/06/2023] [Accepted: 02/06/2023] [Indexed: 03/29/2023] Open
Abstract
Machine learning models have difficulty generalizing to data outside of the distribution they were trained on. In particular, vision models are usually vulnerable to adversarial attacks or common corruptions, to which the human visual system is robust. Recent studies have found that regularizing machine learning models to favor brain-like representations can improve model robustness, but it is unclear why. We hypothesize that the increased model robustness is partly due to the low spatial frequency preference inherited from the neural representation. We tested this simple hypothesis with several frequency-oriented analyses, including the design and use of hybrid images to probe model frequency sensitivity directly. We also examined many other publicly available robust models that were trained on adversarial images or with data augmentation, and found that all these robust models showed a greater preference to low spatial frequency information. We show that preprocessing by blurring can serve as a defense mechanism against both adversarial attacks and common corruptions, further confirming our hypothesis and demonstrating the utility of low spatial frequency information in robust object recognition.
Collapse
Affiliation(s)
- Zhe Li
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
| | - Josue Ortega Caro
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
| | | | | | | | - Fabio Anselmi
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
| | - Ankit B Patel
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, Texas, United States of America
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, United States of America
| | - Andreas S Tolias
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, Texas, United States of America
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, United States of America
| | - Xaq Pitkow
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, Texas, United States of America
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, United States of America
| |
Collapse
|
24
|
Beiran M, Meirhaeghe N, Sohn H, Jazayeri M, Ostojic S. Parametric control of flexible timing through low-dimensional neural manifolds. Neuron 2023; 111:739-753.e8. [PMID: 36640766 PMCID: PMC9992137 DOI: 10.1016/j.neuron.2022.12.016] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2021] [Revised: 09/23/2022] [Accepted: 12/08/2022] [Indexed: 01/15/2023]
Abstract
Biological brains possess an unparalleled ability to adapt behavioral responses to changing stimuli and environments. How neural processes enable this capacity is a fundamental open question. Previous works have identified two candidate mechanisms: a low-dimensional organization of neural activity and a modulation by contextual inputs. We hypothesized that combining the two might facilitate generalization and adaptation in complex tasks. We tested this hypothesis in flexible timing tasks where dynamics play a key role. Examining trained recurrent neural networks, we found that confining the dynamics to a low-dimensional subspace allowed tonic inputs to parametrically control the overall input-output transform, enabling generalization to novel inputs and adaptation to changing conditions. Reverse-engineering and theoretical analyses demonstrated that this parametric control relies on a mechanism where tonic inputs modulate the dynamics along non-linear manifolds while preserving their geometry. Comparisons with data from behaving monkeys confirmed the behavioral and neural signatures of this mechanism.
Collapse
Affiliation(s)
- Manuel Beiran
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Ecole Normale Superieure - PSL University, 75005 Paris, France; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA
| | - Nicolas Meirhaeghe
- Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Institut de Neurosciences de la Timone (INT), UMR 7289, CNRS, Aix-Marseille Université, Marseille 13005, France
| | - Hansem Sohn
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Srdjan Ostojic
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Ecole Normale Superieure - PSL University, 75005 Paris, France.
| |
Collapse
|
25
|
Neural networks: Explaining animal behavior with prior knowledge of the world. Curr Biol 2023; 33:R138-R140. [PMID: 36854269 DOI: 10.1016/j.cub.2023.01.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2023]
Abstract
Animal behavior is both facilitated and constrained by innate knowledge and previous experience of the world. A new study, exploiting the power of recurrent neural networks, has revealed the existence of such structural priors and their impact on animal behavior.
Collapse
|
26
|
Grote T, Berens P. Uncertainty, Evidence, and the Integration of Machine Learning into Medical Practice. THE JOURNAL OF MEDICINE AND PHILOSOPHY 2023; 48:84-97. [PMID: 36630292 DOI: 10.1093/jmp/jhac034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023] Open
Abstract
In light of recent advances in machine learning for medical applications, the automation of medical diagnostics is imminent. That said, before machine learning algorithms find their way into clinical practice, various problems at the epistemic level need to be overcome. In this paper, we discuss different sources of uncertainty arising for clinicians trying to evaluate the trustworthiness of algorithmic evidence when making diagnostic judgments. Thereby, we examine many of the limitations of current machine learning algorithms (with deep learning in particular) and highlight their relevance for medical diagnostics. Among the problems we inspect are the theoretical foundations of deep learning (which are not yet adequately understood), the opacity of algorithmic decisions, and the vulnerabilities of machine learning models, as well as concerns regarding the quality of medical data used to train the models. Building on this, we discuss different desiderata for an uncertainty amelioration strategy that ensures that the integration of machine learning into clinical settings proves to be medically beneficial in a meaningful way.
Collapse
|
27
|
Bordelon B, Pehlevan C. Population codes enable learning from few examples by shaping inductive bias. eLife 2022; 11:e78606. [PMID: 36524716 PMCID: PMC9839349 DOI: 10.7554/elife.78606] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 12/15/2022] [Indexed: 12/23/2022] Open
Abstract
Learning from a limited number of experiences requires suitable inductive biases. To identify how inductive biases are implemented in and shaped by neural codes, we analyze sample-efficient learning of arbitrary stimulus-response maps from arbitrary neural codes with biologically-plausible readouts. We develop an analytical theory that predicts the generalization error of the readout as a function of the number of observed examples. Our theory illustrates in a mathematically precise way how the structure of population codes shapes inductive bias, and how a match between the code and the task is crucial for sample-efficient learning. It elucidates a bias to explain observed data with simple stimulus-response maps. Using recordings from the mouse primary visual cortex, we demonstrate the existence of an efficiency bias towards low-frequency orientation discrimination tasks for grating stimuli and low spatial frequency reconstruction tasks for natural images. We reproduce the discrimination bias in a simple model of primary visual cortex, and further show how invariances in the code to certain stimulus variations alter learning performance. We extend our methods to time-dependent neural codes and predict the sample efficiency of readouts from recurrent networks. We observe that many different codes can support the same inductive bias. By analyzing recordings from the mouse primary visual cortex, we demonstrate that biological codes have lower total activity than other codes with identical bias. Finally, we discuss implications of our theory in the context of recent developments in neuroscience and artificial intelligence. Overall, our study provides a concrete method for elucidating inductive biases of the brain and promotes sample-efficient learning as a general normative coding principle.
Collapse
Affiliation(s)
- Blake Bordelon
- John A Paulson School of Engineering and Applied Sciences, Harvard UniversityCambridgeUnited States
- Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Cengiz Pehlevan
- John A Paulson School of Engineering and Applied Sciences, Harvard UniversityCambridgeUnited States
- Center for Brain Science, Harvard UniversityCambridgeUnited States
| |
Collapse
|
28
|
Gifford AT, Dwivedi K, Roig G, Cichy RM. A large and rich EEG dataset for modeling human visual object recognition. Neuroimage 2022; 264:119754. [PMID: 36400378 PMCID: PMC9771828 DOI: 10.1016/j.neuroimage.2022.119754] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Revised: 09/14/2022] [Accepted: 11/14/2022] [Indexed: 11/16/2022] Open
Abstract
The human brain achieves visual object recognition through multiple stages of linear and nonlinear transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a large and rich dataset of high temporal resolution EEG responses to images of objects on a natural background. This dataset includes 10 participants, each with 82,160 trials spanning 16,740 image conditions. Through computational modeling we established the quality of this dataset in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the high number of conditions as well as the trial repetitions of the EEG dataset contribute to the trained models' prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output EEG responses for arbitrary input images. We release this dataset as a tool to foster research in visual neuroscience and computer vision.
Collapse
Affiliation(s)
- Alessandro T Gifford
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany; Einstein Center for Neurosciences Berlin, Charité - Universitätsmedizin Berlin, Berlin, Germany; Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany.
| | - Kshitij Dwivedi
- Department of Computer Science, Goethe Universität, Frankfurt am Main, Germany
| | - Gemma Roig
- Department of Computer Science, Goethe Universität, Frankfurt am Main, Germany
| | - Radoslaw M Cichy
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany; Einstein Center for Neurosciences Berlin, Charité - Universitätsmedizin Berlin, Berlin, Germany; Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany; Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
29
|
Rohlfs C. A descriptive analysis of olfactory sensation and memory in Drosophila and its relation to artificial neural networks. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.10.068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
30
|
Pouncy T, Gershman SJ. Inductive biases in theory-based reinforcement learning. Cogn Psychol 2022; 138:101509. [PMID: 36152355 DOI: 10.1016/j.cogpsych.2022.101509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 07/16/2022] [Accepted: 08/23/2022] [Indexed: 11/03/2022]
Abstract
Understanding the inductive biases that allow humans to learn in complex environments has been an important goal of cognitive science. Yet, while we have discovered much about human biases in specific learning domains, much of this research has focused on simple tasks that lack the complexity of the real world. In contrast, video games involving agents and objects embedded in richly structured systems provide an experimentally tractable proxy for real-world complexity. Recent work has suggested that key aspects of human learning in domains like video games can be captured by model-based reinforcement learning (RL) with object-oriented relational models-what we term theory-based RL. Restricting the model class in this way provides an inductive bias that dramatically increases learning efficiency, but in this paper we show that humans employ a stronger set of biases in addition to syntactic constraints on the structure of theories. In particular, we catalog a set of semantic biases that constrain the content of theories. Building these semantic biases into a theory-based RL system produces more human-like learning in video game environments.
Collapse
Affiliation(s)
- Thomas Pouncy
- Department of Psychology and Center for Brain Science, Harvard University, United States of America.
| | - Samuel J Gershman
- Department of Psychology and Center for Brain Science, Harvard University, United States of America; Center for Brains, Minds and Machines, MIT, United States of America
| |
Collapse
|
31
|
Jiao C, Li M, Hu D. The neurons in mouse V1 show different degrees of spatial clustering. Brain Res Bull 2022; 190:62-68. [PMID: 36122802 DOI: 10.1016/j.brainresbull.2022.09.011] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Revised: 08/05/2022] [Accepted: 09/14/2022] [Indexed: 11/02/2022]
Abstract
Higher mammals' primary visual cortex exhibits columnar organization, where neurons with similar response preferences are clustered. In contrast, rodents are presumed to lack this fine-scale organization; their neurons appear to be randomly arranged, described as a salt-and-pepper map. However, recent studies suggested a weak but significant spatial clustering of tuning in the salt-and-pepper map, similar to columnar organization. Thus, the salt-and-pepper map possesses the characteristics of both columnar organization and random arrangement. This raises the question about whether this mixed organization is attributed to different types of neurons. Here, we examined the tuning of primary visual cortical neurons in awake mice with a two-photon calcium imaging dataset, which were released by Allen Institute MindScope Program. First, we demonstrated that neurons with similar response preferences were clustered by showing that neighboring neurons tended to have similar orientation and temporal frequency preferences. Then, we compared the clustering of tuning between simple cells and complex cells and found the clustering of tuning among simple cells was significantly more prominent than that among complex cells. Furthermore, the simple/complex cell classification correlated with the stability of neuronal response. Neurons with stable responses were arranged independent of their tuning similarity, whereas unstable neurons were clustered according to their tuning similarity. These findings might represent a balance between efficiency and robustness: relatively independent tuning among stable neurons represents visual information efficiently, whereas unstable neurons with similar response preferences are clustered to obtain a robust representation with population codes.
Collapse
Affiliation(s)
- Chong Jiao
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya road, Changsha 410073, Hunan, China
| | - Ming Li
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya road, Changsha 410073, Hunan, China.
| | - Dewen Hu
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya road, Changsha 410073, Hunan, China
| |
Collapse
|
32
|
Mill RD, Hamilton JL, Winfield EC, Lalta N, Chen RH, Cole MW. Network modeling of dynamic brain interactions predicts emergence of neural information that supports human cognitive behavior. PLoS Biol 2022; 20:e3001686. [PMID: 35980898 PMCID: PMC9387855 DOI: 10.1371/journal.pbio.3001686] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 05/24/2022] [Indexed: 11/21/2022] Open
Abstract
How cognitive task behavior is generated by brain network interactions is a central question in neuroscience. Answering this question calls for the development of novel analysis tools that can firstly capture neural signatures of task information with high spatial and temporal precision (the "where and when") and then allow for empirical testing of alternative network models of brain function that link information to behavior (the "how"). We outline a novel network modeling approach suited to this purpose that is applied to noninvasive functional neuroimaging data in humans. We first dynamically decoded the spatiotemporal signatures of task information in the human brain by combining MRI-individualized source electroencephalography (EEG) with multivariate pattern analysis (MVPA). A newly developed network modeling approach-dynamic activity flow modeling-then simulated the flow of task-evoked activity over more causally interpretable (relative to standard functional connectivity [FC] approaches) resting-state functional connections (dynamic, lagged, direct, and directional). We demonstrate the utility of this modeling approach by applying it to elucidate network processes underlying sensory-motor information flow in the brain, revealing accurate predictions of empirical response information dynamics underlying behavior. Extending the model toward simulating network lesions suggested a role for the cognitive control networks (CCNs) as primary drivers of response information flow, transitioning from early dorsal attention network-dominated sensory-to-response transformation to later collaborative CCN engagement during response selection. These results demonstrate the utility of the dynamic activity flow modeling approach in identifying the generative network processes underlying neurocognitive phenomena.
Collapse
Affiliation(s)
- Ravi D. Mill
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, New Jersey, United States of America
| | - Julia L. Hamilton
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, New Jersey, United States of America
| | - Emily C. Winfield
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, New Jersey, United States of America
| | - Nicole Lalta
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, New Jersey, United States of America
| | - Richard H. Chen
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, New Jersey, United States of America
- Behavioral and Neural Sciences Graduate Program, Rutgers University, Newark, New Jersey, United States of America
| | - Michael W. Cole
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, New Jersey, United States of America
| |
Collapse
|
33
|
Lessons from infant learning for unsupervised machine learning. NAT MACH INTELL 2022. [DOI: 10.1038/s42256-022-00488-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
34
|
Armeni K, Güçlü U, van Gerven M, Schoffelen JM. A 10-hour within-participant magnetoencephalography narrative dataset to test models of language comprehension. Sci Data 2022; 9:278. [PMID: 35676293 PMCID: PMC9177538 DOI: 10.1038/s41597-022-01382-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Accepted: 05/10/2022] [Indexed: 11/13/2022] Open
Abstract
Recently, cognitive neuroscientists have increasingly studied the brain responses to narratives. At the same time, we are witnessing exciting developments in natural language processing where large-scale neural network models can be used to instantiate cognitive hypotheses in narrative processing. Yet, they learn from text alone and we lack ways of incorporating biological constraints during training. To mitigate this gap, we provide a narrative comprehension magnetoencephalography (MEG) data resource that can be used to train neural network models directly on brain data. We recorded from 3 participants, 10 separate recording hour-long sessions each, while they listened to audiobooks in English. After story listening, participants answered short questions about their experience. To minimize head movement, the participants wore MEG-compatible head casts, which immobilized their head position during recording. We report a basic evoked-response analysis showing that the responses accurately localize to primary auditory areas. The responses are robust and conserved across 10 sessions for every participant. We also provide usage notes and briefly outline possible future uses of the resource.
Collapse
Affiliation(s)
- Kristijan Armeni
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Umut Güçlü
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Marcel van Gerven
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Jan-Mathijs Schoffelen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands.
| |
Collapse
|
35
|
Johnson M, Albizri A, Harfouche A, Fosso-Wamba S. Integrating human knowledge into artificial intelligence for complex and ill-structured problems: Informed artificial intelligence. INTERNATIONAL JOURNAL OF INFORMATION MANAGEMENT 2022. [DOI: 10.1016/j.ijinfomgt.2022.102479] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
36
|
Lin CHS, Garrido MI. Towards a cross-level understanding of Bayesian inference in the brain. Neurosci Biobehav Rev 2022; 137:104649. [PMID: 35395333 DOI: 10.1016/j.neubiorev.2022.104649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Revised: 02/28/2022] [Accepted: 03/29/2022] [Indexed: 10/18/2022]
Abstract
Perception emerges from unconscious probabilistic inference, which guides behaviour in our ubiquitously uncertain environment. Bayesian decision theory is a prominent computational model that describes how people make rational decisions using noisy and ambiguous sensory observations. However, critical questions have been raised about the validity of the Bayesian framework in explaining the mental process of inference. Firstly, some natural behaviours deviate from Bayesian optimum. Secondly, the neural mechanisms that support Bayesian computations in the brain are yet to be understood. Taking Marr's cross level approach, we review the recent progress made in addressing these challenges. We first review studies that combined behavioural paradigms and modelling approaches to explain both optimal and suboptimal behaviours. Next, we evaluate the theoretical advances and the current evidence for ecologically feasible algorithms and neural implementations in the brain, which may enable probabilistic inference. We argue that this cross-level approach is necessary for the worthwhile pursuit to uncover mechanistic accounts of human behaviour.
Collapse
Affiliation(s)
- Chin-Hsuan Sophie Lin
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia; Australian Research Council for Integrative Brain Function, Australia.
| | - Marta I Garrido
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia; Australian Research Council for Integrative Brain Function, Australia
| |
Collapse
|
37
|
Kim J, Orhan E, Yoon K, Pitkow X. Two-Argument Activation Functions Learn Soft XOR Operations Like Cortical Neurons. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2022; 10:58071-58080. [PMID: 36339794 PMCID: PMC9634945 DOI: 10.1109/access.2022.3178951] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Neurons in the brain are complex machines with distinct functional compartments that interact nonlinearly. In contrast, neurons in artificial neural networks abstract away this complexity, typically down to a scalar activation function of a weighted sum of inputs. Here we emulate more biologically realistic neurons by learning canonical activation functions with two input arguments, analogous to basal and apical dendrites. We use a network-in-network architecture where each neuron is modeled as a multilayer perceptron with two inputs and a single output. This inner perceptron is shared by all units in the outer network. Remarkably, the resultant nonlinearities often produce soft XOR functions, consistent with recent experimental observations about interactions between inputs in human cortical neurons. When hyperparameters are optimized, networks with these nonlinearities learn faster and perform better than conventional ReLU nonlinearities with matched parameter counts, and they are more robust to natural and adversarial perturbations.
Collapse
Affiliation(s)
- Juhyeon Kim
- Department of Electronic Engineering, Hanyang University, Seoul 04763, South Korea
| | - Emin Orhan
- Center for Data Science, New York University, New York, NY 10011, USA
| | - Kijung Yoon
- Department of Electronic Engineering, Hanyang University, Seoul 04763, South Korea
| | - Xaq Pitkow
- Department of Neuroscience, Baylor College of Medicine, Houston, TX 77030, USA
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, USA
| |
Collapse
|
38
|
Maier M, Blume F, Bideau P, Hellwich O, Abdel Rahman R. Knowledge-augmented face perception: Prospects for the Bayesian brain-framework to align AI and human vision. Conscious Cogn 2022; 101:103301. [DOI: 10.1016/j.concog.2022.103301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Revised: 11/27/2021] [Accepted: 01/04/2022] [Indexed: 11/03/2022]
|
39
|
Dellaferrera G, Woźniak S, Indiveri G, Pantazi A, Eleftheriou E. Introducing principles of synaptic integration in the optimization of deep neural networks. Nat Commun 2022; 13:1885. [PMID: 35393422 PMCID: PMC8989917 DOI: 10.1038/s41467-022-29491-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Accepted: 03/15/2022] [Indexed: 11/17/2022] Open
Abstract
Plasticity circuits in the brain are known to be influenced by the distribution of the synaptic weights through the mechanisms of synaptic integration and local regulation of synaptic strength. However, the complex interplay of stimulation-dependent plasticity with local learning signals is disregarded by most of the artificial neural network training algorithms devised so far. Here, we propose a novel biologically inspired optimizer for artificial and spiking neural networks that incorporates key principles of synaptic plasticity observed in cortical dendrites: GRAPES (Group Responsibility for Adjusting the Propagation of Error Signals). GRAPES implements a weight-distribution-dependent modulation of the error signal at each node of the network. We show that this biologically inspired mechanism leads to a substantial improvement of the performance of artificial and spiking networks with feedforward, convolutional, and recurrent architectures, it mitigates catastrophic forgetting, and it is optimally suited for dedicated hardware implementations. Overall, our work indicates that reconciling neurophysiology insights with machine intelligence is key to boosting the performance of neural networks.
Collapse
Affiliation(s)
- Giorgia Dellaferrera
- IBM Research - Zurich, Rüschlikon, Switzerland.
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland.
| | | | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | | | - Evangelos Eleftheriou
- IBM Research - Zurich, Rüschlikon, Switzerland
- Axelera AI, High Tech Campus 5, 5656 AE, Eindhoven, Netherlands
| |
Collapse
|
40
|
Raj R, Dahlen D, Duyck K, Yu CR. Maximal Dependence Capturing as a Principle of Sensory Processing. Front Comput Neurosci 2022; 16:857653. [PMID: 35399919 PMCID: PMC8989953 DOI: 10.3389/fncom.2022.857653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Accepted: 02/15/2022] [Indexed: 11/13/2022] Open
Abstract
Sensory inputs conveying information about the environment are often noisy and incomplete, yet the brain can achieve remarkable consistency in recognizing objects. Presumably, transforming the varying input patterns into invariant object representations is pivotal for this cognitive robustness. In the classic hierarchical representation framework, early stages of sensory processing utilize independent components of environmental stimuli to ensure efficient information transmission. Representations in subsequent stages are based on increasingly complex receptive fields along a hierarchical network. This framework accurately captures the input structures; however, it is challenging to achieve invariance in representing different appearances of objects. Here we assess theoretical and experimental inconsistencies of the current framework. In its place, we propose that individual neurons encode objects by following the principle of maximal dependence capturing (MDC), which compels each neuron to capture the structural components that contain maximal information about specific objects. We implement the proposition in a computational framework incorporating dimension expansion and sparse coding, which achieves consistent representations of object identities under occlusion, corruption, or high noise conditions. The framework neither requires learning the corrupted forms nor comprises deep network layers. Moreover, it explains various receptive field properties of neurons. Thus, MDC provides a unifying principle for sensory processing.
Collapse
Affiliation(s)
- Rishabh Raj
- Stowers Institute for Medical Research, Kansas City, MO, United States
| | - Dar Dahlen
- Stowers Institute for Medical Research, Kansas City, MO, United States
| | - Kyle Duyck
- Stowers Institute for Medical Research, Kansas City, MO, United States
| | - C. Ron Yu
- Stowers Institute for Medical Research, Kansas City, MO, United States
- Department of Anatomy and Cell Biology, University of Kansas Medical Center, Kansas City, KS, United States
| |
Collapse
|
41
|
Poom L, Fällmar D. Spotting the difference between pairs of nearly identical Perlin images: Influences of presentation formats. PLoS One 2022; 17:e0264621. [PMID: 35213676 PMCID: PMC8880654 DOI: 10.1371/journal.pone.0264621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 02/14/2022] [Indexed: 11/18/2022] Open
Abstract
We investigated human performance in speed and precision of detecting a deviating visual target embedded in one of two otherwise identical non-figurative Perlin-noise images (i.e. a spot-the-difference task). The image-pairs were presented in four different presentation formats: spatially separated in horizontal or vertical direction while simultaneously presented, or sequentially separated on the same location with a brief delay or without any delay. In the two spatial conditions failure to detect the target within 30 sec (change blindness) occurred in about 6–7% of the trials, and with the brief delay 2.4% of the trials. Fast error-free detection (i.e. pop out) was obtained using the sequential format with no delay. Average detection time when target was detected was about 9 sec for the two spatial formats. Detection time was faster, about 6 sec, for the brief delay condition. In trials where detection was reported, the precision of locating the target was equal in the horizontal and brief delay conditions, and better than in the vertical condition. Misses obtained in the horizontal and brief delay conditions were also more strongly correlated than correlations between misses in the vertical and horizontal, and between the vertical and brief delay conditions. Some individuals’ performances when comparing images in the vertical direction were at chance level. This suggests influences of known poorer precision when making saccades in the vertical compared to horizontal direction. The results may have applications for radiologists since the stimuli and task is similar to radiologists’ task when detecting deviations between radiological images.
Collapse
Affiliation(s)
- Leo Poom
- Department of Psychology, Uppsala University, Uppsala, Sweden
- * E-mail:
| | - David Fällmar
- Department of Surgical Sciences, Radiology, Uppsala University, Uppsala, Sweden
| |
Collapse
|
42
|
Xie Y, Liu YH, Constantinidis C, Zhou X. Neural Mechanisms of Working Memory Accuracy Revealed by Recurrent Neural Networks. Front Syst Neurosci 2022; 16:760864. [PMID: 35237134 PMCID: PMC8883483 DOI: 10.3389/fnsys.2022.760864] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Accepted: 01/18/2022] [Indexed: 11/17/2022] Open
Abstract
Understanding the neural mechanisms of working memory has been a long-standing Neuroscience goal. Bump attractor models have been used to simulate persistent activity generated in the prefrontal cortex during working memory tasks and to study the relationship between activity and behavior. How realistic the assumptions of these models are has been a matter of debate. Here, we relied on an alternative strategy to gain insights into the computational principles behind the generation of persistent activity and on whether current models capture some universal computational principles. We trained Recurrent Neural Networks (RNNs) to perform spatial working memory tasks and examined what aspects of RNN activity accounted for working memory performance. Furthermore, we compared activity in fully trained networks and immature networks, achieving only imperfect performance. We thus examined the relationship between the trial-to-trial variability of responses simulated by the network and different aspects of unit activity as a way of identifying the critical parameters of memory maintenance. Properties that spontaneously emerged in the artificial network strongly resembled persistent activity of prefrontal neurons. Most importantly, these included drift of network activity during the course of a trial that was causal to the behavior of the network. As a consequence, delay period firing rate and behavior were positively correlated, in strong analogy to experimental results from the prefrontal cortex. These findings reveal that delay period activity is computationally efficient in maintaining working memory, as evidenced by unbiased optimization of parameters in artificial neural networks, oblivious to the properties of prefrontal neurons.
Collapse
Affiliation(s)
- Yuanqi Xie
- Department of Computer Science, Vanderbilt University, Nashville, TN, United States
| | - Yichen Henry Liu
- Department of Computer Science, Vanderbilt University, Nashville, TN, United States
| | - Christos Constantinidis
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, United States
- Neuroscience Program, Vanderbilt University, Nashville, TN, United States
- Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Xin Zhou
- Department of Computer Science, Vanderbilt University, Nashville, TN, United States
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, United States
- Data Science Institute, Vanderbilt University, Nashville, TN, United States
| |
Collapse
|
43
|
Abstract
The application of machine-learning technologies to medical practice promises to enhance the capabilities of healthcare professionals in the assessment, diagnosis, and treatment, of medical conditions. However, there is growing concern that algorithmic bias may perpetuate or exacerbate existing health inequalities. Hence, it matters that we make precise the different respects in which algorithmic bias can arise in medicine, and also make clear the normative relevance of these different kinds of algorithmic bias for broader questions about justice and fairness in healthcare. In this paper, we provide the building blocks for an account of algorithmic bias and its normative relevance in medicine.
Collapse
|
44
|
Tang H, Riley MR, Singh B, Qi XL, Blake DT, Constantinidis C. Prefrontal cortical plasticity during learning of cognitive tasks. Nat Commun 2022; 13:90. [PMID: 35013248 PMCID: PMC8748623 DOI: 10.1038/s41467-021-27695-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 11/30/2021] [Indexed: 11/30/2022] Open
Abstract
Training in working memory tasks is associated with lasting changes in prefrontal cortical activity. To assess the neural activity changes induced by training, we recorded single units, multi-unit activity (MUA) and local field potentials (LFP) with chronic electrode arrays implanted in the prefrontal cortex of two monkeys, throughout the period they were trained to perform cognitive tasks. Mastering different task phases was associated with distinct changes in neural activity, which included recruitment of larger numbers of neurons, increases or decreases of their firing rate, changes in the correlation structure between neurons, and redistribution of power across LFP frequency bands. In every training phase, changes induced by the actively learned task were also observed in a control task, which remained the same across the training period. Our results reveal how learning to perform cognitive tasks induces plasticity of prefrontal cortical activity, and how activity changes may generalize between tasks.
Collapse
Affiliation(s)
- Hua Tang
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, NC, 27157, USA
- Center for Neuropsychiatric Diseases, Institute of Life Science, Nanchang University, Nanchang, 330031, Jiangxi, China
- Laboratory of Neuropsychology, National Institutes of Mental Health, NIH, Bethesda, MD, 20892, USA
| | - Mitchell R Riley
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, NC, 27157, USA
| | - Balbir Singh
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, NC, 27157, USA
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37235, USA
| | - Xue-Lian Qi
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, NC, 27157, USA
| | - David T Blake
- Department of Neuroscience and Regenerative Medicine, Medical College of Georgia, Augusta University, Augusta, GA, 30912, USA
| | - Christos Constantinidis
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, NC, 27157, USA.
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37235, USA.
- Neuroscience Program, Vanderbilt University, Nashville, TN, 37235, USA.
- Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center, Nashville, TN, 37232, USA.
| |
Collapse
|
45
|
Allen EJ, St-Yves G, Wu Y, Breedlove JL, Prince JS, Dowdle LT, Nau M, Caron B, Pestilli F, Charest I, Hutchinson JB, Naselaris T, Kay K. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nat Neurosci 2022; 25:116-126. [PMID: 34916659 DOI: 10.1038/s41593-021-00962-x] [Citation(s) in RCA: 151] [Impact Index Per Article: 50.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 10/12/2021] [Indexed: 11/09/2022]
Abstract
Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. Here we present the Natural Scenes Dataset (NSD), in which high-resolution functional magnetic resonance imaging responses to tens of thousands of richly annotated natural scenes were measured while participants performed a continuous recognition task. To optimize data quality, we developed and applied novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we used NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality and breadth, NSD opens new avenues of inquiry in cognitive neuroscience and artificial intelligence.
Collapse
Affiliation(s)
- Emily J Allen
- Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, USA
- Department of Psychology, University of Minnesota, Minneapolis, MN, USA
| | - Ghislain St-Yves
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Yihan Wu
- Graduate Program in Cognitive Science, University of Minnesota, Minneapolis, MN, USA
| | - Jesse L Breedlove
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
- Department of Psychology, University of Minnesota, Minneapolis, MN, USA
| | - Jacob S Prince
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA
- Department of Psychology, Harvard University, Cambridge, MA, USA
| | - Logan T Dowdle
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Minneapolis, MN, USA
- Department of Neurosurgery, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Minneapolis, MN, USA
| | - Matthias Nau
- National Institute of Mental Health (NIMH), Bethesda MD, USA
| | - Brad Caron
- Program in Neuroscience, Indiana University, Bloomington IN, USA
- Program in Vision Science, Indiana University, Bloomington IN, USA
| | - Franco Pestilli
- Department of Psychology, University of Texas at Austin, Austin, TX, USA
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
- Institute for Neuroscience, University of Texas at Austin, Austin, TX, USA
| | - Ian Charest
- Center for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- cerebrUM, Département de Psychologie, Université de Montréal, Montréal QC, Canada
| | | | - Thomas Naselaris
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Kendrick Kay
- Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
46
|
Biological convolutions improve DNN robustness to noise and generalisation. Neural Netw 2021; 148:96-110. [PMID: 35114495 DOI: 10.1016/j.neunet.2021.12.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Revised: 11/11/2021] [Accepted: 12/07/2021] [Indexed: 11/19/2022]
Abstract
Deep Convolutional Neural Networks (DNNs) have achieved superhuman accuracy on standard image classification benchmarks. Their success has reignited significant interest in their use as models of the primate visual system, bolstered by claims of their architectural and representational similarities. However, closer scrutiny of these models suggests that they rely on various forms of shortcut learning to achieve their impressive performance, such as using texture rather than shape information. Such superficial solutions to image recognition have been shown to make DNNs brittle in the face of more challenging tests such as noise-perturbed or out-of-distribution images, casting doubt on their similarity to their biological counterparts. In the present work, we demonstrate that adding fixed biological filter banks, in particular banks of Gabor filters, helps to constrain the networks to avoid reliance on shortcuts, making them develop more structured internal representations and more tolerance to noise. Importantly, they also gained around 20-35% improved accuracy when generalising to our novel out-of-distribution test image sets over standard end-to-end trained architectures. We take these findings to suggest that these properties of the primate visual system should be incorporated into DNNs to make them more able to cope with real-world vision and better capture some of the more impressive aspects of human visual perception such as generalisation.
Collapse
|
47
|
Debellemanière G, Dubois M, Gauvin M, Wallerstein A, Brenner LF, Rampat R, Saad A, Gatinel D. The PEARL-DGS Formula: The Development of an Open-source Machine Learning-based Thick IOL Calculation Formula. Am J Ophthalmol 2021; 232:58-69. [PMID: 33992611 DOI: 10.1016/j.ajo.2021.05.004] [Citation(s) in RCA: 69] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2021] [Revised: 04/30/2021] [Accepted: 05/02/2021] [Indexed: 11/19/2022]
Abstract
PURPOSE To describe an open-source, reproducible, step-by-step method to design sum-of-segments thick intraocular lens (IOL) calculation formulas, and to evaluate a formula built using this methodology. DESIGN Retrospective, multicenter case series METHODS: A set of 4242 eyes implanted with Finevision IOLs (PhysIOL, Liège, Belgium) was used to devise the formula design process and build the formula. A different set of 677 eyes from the same center was kept separate to serve as a test set. The resulting formula was evaluated on the test set as well as another independent data set of 262 eyes. RESULTS The lowest standard deviation (SD) of prediction errors on Set 1 were obtained with the PEARL-DGS formula (±0.382 D), followed by K6 and Olsen (±0.394 D), EVO 2.0 (±0.398 D), RBF 3.0, and BUII (±0.402 D). The formula yielding the lowest SD on Set 2 was the PEARL-DGS (±0.269 D), followed by Olsen (±0.272 D), K6 (±0.276 D), EVO 2.0 (±0.277 D), and BUII (±0.301 D). CONCLUSION Our methodology achieved an accuracy comparable to other state-of-the-art IOL formulas. The open-source tools provided in this article could allow other researchers to reproduce our results using their own data sets, with other IOL models, population settings, biometric devices, and measured, rather than calculated, posterior corneal radius of curvature or sum-of-segments axial lengths.
Collapse
Affiliation(s)
- Guillaume Debellemanière
- From the Department of Ophthalmology, Rothschild Foundation Hospital, Paris, France (D.G., D.M., R.R., S.A.,G.D.)
| | - Mathieu Dubois
- From the Department of Ophthalmology, Rothschild Foundation Hospital, Paris, France (D.G., D.M., R.R., S.A.,G.D.)
| | - Mathieu Gauvin
- Department of Ophthalmology and Visual Sciences, McGill University, Montreal, Quebec, Canada (G.M., W.A.); LASIK MD, Montreal, Quebec, Canada (G.M., W.A.)
| | - Avi Wallerstein
- Department of Ophthalmology and Visual Sciences, McGill University, Montreal, Quebec, Canada (G.M., W.A.); LASIK MD, Montreal, Quebec, Canada (G.M., W.A.)
| | | | - Radhika Rampat
- From the Department of Ophthalmology, Rothschild Foundation Hospital, Paris, France (D.G., D.M., R.R., S.A.,G.D.)
| | - Alain Saad
- From the Department of Ophthalmology, Rothschild Foundation Hospital, Paris, France (D.G., D.M., R.R., S.A.,G.D.)
| | - Damien Gatinel
- From the Department of Ophthalmology, Rothschild Foundation Hospital, Paris, France (D.G., D.M., R.R., S.A.,G.D.).
| |
Collapse
|
48
|
Hennig JA, Oby ER, Losey DM, Batista AP, Yu BM, Chase SM. How learning unfolds in the brain: toward an optimization view. Neuron 2021; 109:3720-3735. [PMID: 34648749 PMCID: PMC8639641 DOI: 10.1016/j.neuron.2021.09.005] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Revised: 08/25/2021] [Accepted: 09/02/2021] [Indexed: 12/17/2022]
Abstract
How do changes in the brain lead to learning? To answer this question, consider an artificial neural network (ANN), where learning proceeds by optimizing a given objective or cost function. This "optimization framework" may provide new insights into how the brain learns, as many idiosyncratic features of neural activity can be recapitulated by an ANN trained to perform the same task. Nevertheless, there are key features of how neural population activity changes throughout learning that cannot be readily explained in terms of optimization and are not typically features of ANNs. Here we detail three of these features: (1) the inflexibility of neural variability throughout learning, (2) the use of multiple learning processes even during simple tasks, and (3) the presence of large task-nonspecific activity changes. We propose that understanding the role of these features in the brain will be key to describing biological learning using an optimization framework.
Collapse
Affiliation(s)
- Jay A Hennig
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA.
| | - Emily R Oby
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Darby M Losey
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Aaron P Batista
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Byron M Yu
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Steven M Chase
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| |
Collapse
|
49
|
Chakraborty B, Mukhopadhyay S. Characterization of Generalizability of Spike Timing Dependent Plasticity Trained Spiking Neural Networks. Front Neurosci 2021; 15:695357. [PMID: 34776837 PMCID: PMC8589121 DOI: 10.3389/fnins.2021.695357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Accepted: 09/29/2021] [Indexed: 11/30/2022] Open
Abstract
A Spiking Neural Network (SNN) is trained with Spike Timing Dependent Plasticity (STDP), which is a neuro-inspired unsupervised learning method for various machine learning applications. This paper studies the generalizability properties of the STDP learning processes using the Hausdorff dimension of the trajectories of the learning algorithm. The paper analyzes the effects of STDP learning models and associated hyper-parameters on the generalizability properties of an SNN. The analysis is used to develop a Bayesian optimization approach to optimize the hyper-parameters for an STDP model for improving the generalizability properties of an SNN.
Collapse
Affiliation(s)
- Biswadeep Chakraborty
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United States
| | | |
Collapse
|
50
|
Liu YH, Zhu J, Constantinidis C, Zhou X. Emergence of prefrontal neuron maturation properties by training recurrent neural networks in cognitive tasks. iScience 2021; 24:103178. [PMID: 34667944 PMCID: PMC8506971 DOI: 10.1016/j.isci.2021.103178] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Revised: 06/16/2021] [Accepted: 09/22/2021] [Indexed: 01/14/2023] Open
Abstract
Working memory and response inhibition are functions that mature relatively late in life, after adolescence, paralleling the maturation of the prefrontal cortex. The link between behavioral and neural maturation is not obvious, however, making it challenging to understand how neural activity underlies the maturation of cognitive function. To gain insights into the nature of observed changes in prefrontal activity between adolescence and adulthood, we investigated the progressive changes in unit activity of recurrent neural networks as they were trained to perform working memory and response inhibition tasks. These included increased delay period activity during working memory tasks and increased activation in antisaccade tasks. These findings reveal universal properties underlying the neuronal computations behind cognitive tasks and explicate the nature of changes that occur as the result of developmental maturation. Properties of RNN networks during training offer insights in prefrontal maturation Fully trained networks exhibit higher levels of activity in working memory tasks Trained networks also exhibit higher activation in antisaccade tasks Partially trained RNNs can generate accurate predictions of immature PFC activity
Collapse
Affiliation(s)
- Yichen Henry Liu
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA
| | - Junda Zhu
- Neuroscience Program, Vanderbilt University, Nashville, TN 37235, USA
| | - Christos Constantinidis
- Neuroscience Program, Vanderbilt University, Nashville, TN 37235, USA.,Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37235, USA.,Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center, Nashville, TN 37232, USA
| | - Xin Zhou
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA.,Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37235, USA.,Data Science Institute, Vanderbilt University, Nashville, TN 37235, USA
| |
Collapse
|