Biological underpinnings for lifelong learning machines

CEL: A Continual Learning Model for Disease Outbreak Prediction by Leveraging Domain Adaptation via Elastic Weight Consolidation

Continual learning is the ability of a model to learn over time without forgetting previous knowledge. Therefore, adapting new data in dynamic fields like disease outbreak prediction is paramount. Deep neural networks are prone to error due to catastrophic forgetting. This study introduces a novel CEL model for Continual Learning by leveraging domain adaptation via Elastic weight consolidation (EWC). This model aims to mitigate the catastrophic forgetting phenomenon in a domain incremental setting. The Fisher information matrix (FIM) is constructed with EWC to develop a regularization term that penalizes changes to essential parameters. We conducted experiments on three distinct diseases, influenza, mpox, and measles, with customized metrics. The high R-squared values during evaluation and reevaluation outperform the other state-of-the-art models in several contexts. The results indicate that CEL adapts well to incremental data. CEL's robustness emphasizes its minimal 65% forgetting rate and 18% higher memory stability compared to existing benchmark studies. This study highlights CEL's versatility in disease outbreak prediction by addressing evolving data with temporal patterns. It offers a valuable model for proactive disease control with accurate and timely predictions.

A Dynamic Systems Approach to Modeling Human-Machine Rhythm Interaction

Rhythm is an inherent aspect of human behavior, present from infancy and embedded in cultural practices. At the core of rhythm perception lies meter anticipation, a spontaneous process in the human brain that typically occurs before actual beats. This anticipation can be framed as a time series prediction problem. From the perspective of human embodied system behavior, although many models have been developed for time series prediction, most prioritize accuracy over biological realism, contrasting with the natural imprecision of human internal clocks. Neuroscientific evidence, such as infants' natural meter synchronization, underscores the need for biologically plausible models. Therefore, we propose a neuron oscillator-based dynamic system that simulates human behavior during meter perception. The model introduces two tunable parameters for local and global adjustments, fine-tuning the oscillation combinations to emulate human-like rhythmic behavior. The experiments are conducted under three common scenarios encountered during human-machine interaction, demonstrating that the proposed model can exhibit human-like reactions. Additionally, experiments involving human-machine and interhuman interactions show that the model successfully replicates real-world rhythmic behavior, advancing toward more natural and synchronized human-machine rhythm interaction.

Input-driven dynamics for robust memory retrieval in Hopfield networks

The Hopfield model provides a mathematical framework for understanding the mechanisms of memory storage and retrieval in the human brain. This model has inspired decades of research on learning and retrieval dynamics, capacity estimates, and sequential transitions among memories. Notably, the role of external inputs has been largely underexplored, from their effects on neural dynamics to how they facilitate effective memory retrieval. To bridge this gap, we propose a dynamical system framework in which the external input directly influences the neural synapses and shapes the energy landscape of the Hopfield model. This plasticity-based mechanism provides a clear energetic interpretation of the memory retrieval process and proves effective at correctly classifying mixed inputs. Furthermore, we integrate this model within the framework of modern Hopfield architectures to elucidate how current and past information are combined during the retrieval process. Last, we embed both the classic and the proposed model in an environment disrupted by noise and compare their robustness during memory retrieval.

Plasticity in inhibitory networks improves pattern separation in early olfactory processing

Distinguishing between nectar and non-nectar odors is challenging for animals due to shared compounds and varying ratios in complex mixtures. Changes in nectar production throughout the day and over the animal's lifetime add to the complexity. The honeybee olfactory system, containing fewer than 1000 principal neurons in the early olfactory relay, the antennal lobe (AL), must learn to associate diverse volatile blends with rewards. Previous studies identified plasticity in the AL circuits, but its role in odor learning remains poorly understood. Using a biophysical computational model, tuned by in vivo electrophysiological data, and live imaging of the honeybee's AL, we explored the neural mechanisms of plasticity in the AL. Our findings revealed that when trained with a set of rewarded and unrewarded odors, the AL inhibitory network suppresses responses to shared chemical compounds while enhancing responses to distinct compounds. This results in improved pattern separation and a more concise neural code. Our calcium imaging data support these predictions. Analysis of a graph convolutional neural network performing an odor categorization task revealed a similar mechanism for contrast enhancement. Our study provides insights into how inhibitory plasticity in the early olfactory network reshapes the coding for efficient learning of complex odors.

Curriculum is more influential than haptic feedback when learning object manipulation

Dexterous manipulation remains an aspirational goal for autonomous robotic systems, particularly when learning to lift and rotate objects against gravity with intermittent finger contacts. We use model-free reinforcement learning to compare the effect of curriculum (i.e., combinations of lift and rotation tasks) and haptic information (i.e., no-tactile versus 3D-force) on learning with a simulated three-finger robotic hand. In addition, a novel curriculum-based learning rate scheduler accelerates convergence. We demonstrate that the choice of curriculum biases the progression of learning for dexterous manipulation across objects with different weights, sizes, and shapes-underscoring the robustness of our learning approach. Unexpectedly, learning is achieved even in the absence of haptic information. This challenges conventional thinking about task "complexity" and the necessity of haptic information for dexterous manipulation for this task. This work invites the analogy of curriculum learning as a malleable developmental process from a pluripotent state driven by the nature of the learning experience.

Learning from Octopuses: Cutting-Edge Developments and Future Directions

This paper reviews the research progress of bionic soft robot technology learned from octopuses. The number of related research papers increased from 760 in 2021 to 1170 in 2024 (Google Scholar query), with a growth rate of 53.95% in the past five years. These studies mainly explore how humans can learn from the physiological characteristics of octopuses for sensor design, actuator development, processor architecture optimization, and intelligent optimization algorithms. The tentacle structure and nervous system of octopus have high flexibility and distributed control capabilities, which is an important reference for the design of soft robots. In terms of sensor technology, flexible strain sensors and suction cup sensors inspired by octopuses achieve accurate environmental perception and interaction. Actuator design uses octopus muscle fibers and movement patterns to develop various driving methods, including pneumatic, hydraulic and electric systems, which greatly improves the robot's motion performance. In addition, the distributed nervous system of octopuses inspires multi-processor architecture and intelligent optimization algorithms. This paper also introduces the concept of expected functional safety for the first time to explore the safe design of soft robots in failure or unknown situations. Currently, there are more and more bionic soft robot technologies that draw on octopuses, and their application areas are constantly expanding. In the future, with further research on the physiological characteristics of octopuses and the integration of artificial intelligence and materials science, octopus soft robots are expected to show greater potential in adapting to complex environments, human-computer interaction, and medical applications.

Revealing non-trivial information structures in aneural biological tissues via functional connectivity

A central challenge in the progression of a variety of open questions in biology, such as morphogenesis, wound healing, and development, is learning from empirical data how information is integrated to support tissue-level function and behavior. Information-theoretic approaches provide a quantitative framework for extracting patterns from data, but so far have been predominantly applied to neuronal systems at the tissue-level. Here, we demonstrate how time series of Ca2+ dynamics can be used to identify the structure and information dynamics of other biological tissues. To this end, we expressed the calcium reporter GCaMP6s in an organoid system of explanted amphibian epidermis derived from the African clawed frog Xenopus laevis, and imaged calcium activity pre- and post- a puncture injury, for six replicate organoids. We constructed functional connectivity networks by computing mutual information between cells from time series derived using medical imaging techniques to track intracellular Ca2+. We analyzed network properties including degree distribution, spatial embedding, and modular structure. We find organoid networks exhibit potential evidence for more connectivity than null models, with our models displaying high degree hubs and mesoscale community structure with spatial clustering. Utilizing functional connectivity networks, our model suggests the tissue retains non-random features after injury, displays long range correlations and structure, and non-trivial clustering that is not necessarily spatially dependent. In the context of this reconstruction method our results suggest increased integration after injury, possible cellular coordination in response to injury, and some type of generative structure of the anatomy. While we study Ca2+ in Xenopus epidermal cells, our computational approach and analyses highlight how methods developed to analyze functional connectivity in neuronal tissues can be generalized to any tissue and fluorescent signal type. We discuss expanded methods of analyses to improve models of non-neuronal information processing highlighting the potential of our framework to provide a bridge between neuroscience and more basal modes of information processing.

Multiagent Continual Coordination via Progressive Task Contextualization

Cooperative multiagent reinforcement learning (MARL) has attracted significant attention and has the potential for many real-world applications. Previous arts mainly focus on facilitating the coordination ability from different aspects (e.g., nonstationarity and credit assignment) in single-task or multitask scenarios, ignoring the stream of tasks that appear in a continual manner. This ignorance makes the continual coordination an unexplored territory, neither in problem formulation nor efficient algorithms designed. Toward tackling the mentioned issue, this article proposes an approach, multiagent continual coordination via progressive task contextualization (MACPro). The key point lies in obtaining a factorized policy, using shared feature extraction layers but separated independent task heads, each specializing in a specific class of tasks. The task heads can be progressively expanded based on the learned task contextualization. Moreover, to cater to the popular centralized training with decentralized execution (CTDE) paradigm in MARL, each agent learns to predict and adopt the most relevant policy head based on local information in a decentralized manner. We show in multiple multiagent benchmarks that existing continual learning methods fail, while MACPro is able to achieve close-to-optimal performance. More results also disclose the effectiveness of MACPro from multiple aspects, such as high generalization ability.

Few-Shot Class-Incremental Learning for Classification and Object Detection: A Survey

Few-shot Class-Incremental Learning (FSCIL) presents a unique challenge in Machine Learning (ML), as it necessitates the Incremental Learning (IL) of new classes from sparsely labeled training samples without forgetting previous knowledge. While this field has seen recent progress, it remains an active exploration area. This paper aims to provide a comprehensive and systematic review of FSCIL. In our in-depth examination, we delve into various facets of FSCIL, encompassing the problem definition, the discussion of the primary challenges of unreliable empirical risk minimization and the stability-plasticity dilemma, general schemes, and relevant problems of IL and Few-shot Learning (FSL). Besides, we offer an overview of benchmark datasets and evaluation metrics. Furthermore, we introduce the Few-shot Class-incremental Classification (FSCIC) methods from data-based, structure-based, and optimization-based approaches and the Few-shot Class-incremental Object Detection (FSCIOD) methods from anchor-free and anchor-based approaches. Beyond these, we present several promising research directions within FSCIL that merit further investigation.

Human Brain Inspired Artificial Intelligence Neural Networks

It is becoming increasingly evident that Artificial intelligence (AI) development draws inspiration from the architecture and functions of the human brain. This manuscript examines the alignment between key brain regions-such as the brainstem, sensory cortices, basal ganglia, thalamus, limbic system, and prefrontal cortex-and AI paradigms, including generic AI, machine learning, deep learning, and artificial general intelligence (AGI). By mapping these neural and computational architectures, I herein highlight how AI models progressively mimic the brain's complexity, from basic pattern recognition and association to advanced reasoning. Current challenges, such as overcoming learning limitations and achieving comparable neuroplasticity, are addressed alongside emerging innovations like neuromorphic computing. Given the rapid pace of AI advancements in recent years, this work underscores the importance of continuously reassessing our understanding as technology evolves exponentially.

The neurobench framework for benchmarking neuromorphic computing algorithms and systems

Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. This article presents NeuroBench, a benchmark framework for neuromorphic algorithms and systems, which is collaboratively designed from an open community of researchers across industry and academia. NeuroBench introduces a common set of tools and systematic methodology for inclusive benchmark measurement, delivering an objective reference framework for quantifying neuromorphic approaches in both hardware-independent and hardware-dependent settings. For latest project updates, visit the project website ( neurobench.ai ).

Two-Dimensional Materials for Brain-Inspired Computing Hardware

Recent breakthroughs in brain-inspired computing promise to address a wide range of problems from security to healthcare. However, the current strategy of implementing artificial intelligence algorithms using conventional silicon hardware is leading to unsustainable energy consumption. Neuromorphic hardware based on electronic devices mimicking biological systems is emerging as a low-energy alternative, although further progress requires materials that can mimic biological function while maintaining scalability and speed. As a result of their diverse unique properties, atomically thin two-dimensional (2D) materials are promising building blocks for next-generation electronics including nonvolatile memory, in-memory and neuromorphic computing, and flexible edge-computing systems. Furthermore, 2D materials achieve biorealistic synaptic and neuronal responses that extend beyond conventional logic and memory systems. Here, we provide a comprehensive review of the growth, fabrication, and integration of 2D materials and van der Waals heterojunctions for neuromorphic electronic and optoelectronic devices, circuits, and systems. For each case, the relationship between physical properties and device responses is emphasized followed by a critical comparison of technologies for different applications. We conclude with a forward-looking perspective on the key remaining challenges and opportunities for neuromorphic applications that leverage the fundamental properties of 2D materials and heterojunctions.

Asymmetrically connected reservoir networks learn better

We show that connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance. To this end, we systematically investigate the impact of network connectivity on its performance, i.e., we examine the symmetry and structure of the reservoir in relation to its computational power. Reservoirs with random and asymmetric connections are found to perform better for an exemplary Mackey-Glass time series than all structured reservoirs, including biologically inspired connectivities, such as small-world topologies. This result is quantified by the information processing capacity of the different network topologies which becomes highest for asymmetric and randomly connected networks.

Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines

Learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex information flow makes its implementation in biological and neuromorphic systems challenging. This has motivated the exploration of alternative learning mechanisms that can operate locally and do not rely on exact gradients. In this work, we introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. Using an Ornstein-Uhlenbeck process with adaptive dynamics, our method balances exploration and exploitation during learning, driven by deviations from error predictions, akin to reward prediction error. Operating in continuous time, Ornstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning in dynamic, time-evolving environments. We validate our approach across a range of different tasks, including supervised learning and reinforcement learning in feedforward and recurrent systems. Additionally, we demonstrate that it can perform meta-learning, adjusting hyper-parameters autonomously. Our results indicate that OUA provides a promising alternative to traditional gradient-based methods, with potential applications in neuromorphic computing. It also hints at a possible mechanism for noise-driven learning in the brain, where stochastic neurotransmitter release may guide synaptic adjustments.

Probabilistic metaplasticity for continual learning with memristors in spiking networks

Edge devices operating in dynamic environments critically need the ability to continually learn without catastrophic forgetting. The strict resource constraints in these devices pose a major challenge to achieve this, as continual learning entails memory and computational overhead. Crossbar architectures using memristor devices offer energy efficiency through compute-in-memory and hold promise to address this issue. However, memristors often exhibit low precision and high variability in conductance modulation, rendering them unsuitable for continual learning solutions that require precise modulation of weight magnitude for consolidation. Current approaches fall short to address this challenge directly and rely on auxiliary high-precision memory, leading to frequent memory access, high memory overhead, and energy dissipation. In this research, we propose probabilistic metaplasticity, which consolidates weights by modulating their update probability rather than magnitude. The proposed mechanism eliminates high-precision modification to weight magnitudes and, consequently, the need for auxiliary high-precision memory. We demonstrate the efficacy of the proposed mechanism by integrating probabilistic metaplasticity into a spiking network trained on an error threshold with low-precision memristor weights. Evaluations of continual learning benchmarks show that probabilistic metaplasticity achieves performance equivalent to state-of-the-art continual learning models with high-precision weights while consuming ~ 67% lower memory for additional parameters and up to ~ 60× lower energy during parameter updates compared to an auxiliary memory-based solution. The proposed model shows potential for energy-efficient continual learning with low-precision emerging devices.

Adult Neurogenesis Reconciles Flexibility and Stability of Olfactory Perceptual Memory

In brain regions featuring ongoing plasticity, the task of quickly encoding new information without overwriting old memories presents a significant challenge. In the rodent olfactory bulb, which is renowned for substantial structural plasticity driven by adult neurogenesis and persistent turnover of dendritic spines, we show that by synergistically combining both types of plasticity this flexibility-stability dilemma can be overcome. To do so, we develop a computational model for structural plasticity in the olfactory bulb and show that it is the maturation process of adult-born neurons that enables the bulb to learn quickly and forget slowly. Particularly important are the transient enhancement of the plasticity, excitability, and susceptibility to apoptosis that characterizes young neurons. The model captures many experimental observations and makes a number of testable predictions. Overall, it identifies memory consolidation as an important role of adult neurogenesis in olfaction and exemplifies how the brain can maintain stable memories despite ongoing extensive neurogenesis and synaptic plasticity.

Minds in movement: embodied cognition in the age of artificial intelligence

This theme issue brings together researchers from diverse fields to assess the current status and future prospects of embodied cognition in the age of generative artificial intelligence. In this introduction, we first clarify our view of embodiment as a potentially unifying concept in the study of cognition, characterizing this as a perspective that questions mind-body dualism and recognizes a profound continuity between sensorimotor action in the world and more abstract forms of cognition. We then consider how this unifying concept is developed and elaborated by the other contributions to this issue, identifying the following two key themes: (i) the role of language in cognition and its entanglement with the body and (ii) bodily mechanisms of interpersonal perception and alignment across the domains of social affiliation, teaching and learning. On balance, we consider that embodied approaches to the study of cognition, culture and evolution remain promising, but will require greater integration across disciplines to fully realize their potential. We conclude by suggesting that researchers will need to be ready and able to meet the various methodological, theoretical and practical challenges this will entail and remain open to encountering markedly different viewpoints about how and why embodiment matters. This article is the part of this theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Rapid context inference in a thalamocortical model using recurrent neural networks

Cognitive flexibility is a fundamental ability that enables humans and animals to exhibit appropriate behaviors in various contexts. The thalamocortical interactions between the prefrontal cortex (PFC) and the mediodorsal thalamus (MD) have been identified as crucial for inferring temporal context, a critical component of cognitive flexibility. However, the neural mechanism responsible for context inference remains unknown. To address this issue, we propose a PFC-MD neural circuit model that utilizes a Hebbian plasticity rule to support rapid, online context inference. Specifically, the model MD thalamus can infer temporal contexts from prefrontal inputs within a few trials. This is achieved through the use of PFC-to-MD synaptic plasticity with pre-synaptic traces and adaptive thresholding, along with winner-take-all normalization in the MD. Furthermore, our model thalamus gates context-irrelevant neurons in the PFC, thus facilitating continual learning. We evaluate our model performance by having it sequentially learn various cognitive tasks. Incorporating an MD-like component alleviates catastrophic forgetting of previously learned contexts and demonstrates the transfer of knowledge to future contexts. Our work provides insight into how biological properties of thalamocortical circuits can be leveraged to achieve rapid context inference and continual learning.

Physics-Informed Explainable Continual Learning on Graphs

Temporal graph learning has attracted great attention with its ability to deal with dynamic graphs. Although current methods are reasonably accurate, most of them are unexplainable due to their black-box nature. It remains a challenge to explain how temporal graph learning models adapt to information evolution. Furthermore, with the increasing application of artificial intelligence in various scientific domains, such as chemistry and biomedicine, the importance of delivering not only precise outcomes but also offering explanations regarding the learning models becomes paramount. This transparency aids users in comprehending the decision-making procedures and instills greater confidence in the generated models. To address this issue, this article proposes a novel physics-informed explainable continual learning (PiECL), focusing on temporal graphs. Our proposed method utilizes physical and mathematical algorithms to quantify the disturbance of new data to previous knowledge for obtaining changed information over time. As the proposed model is based on theories in physics, it can provide a transparent underlying mechanism for information evolution detection, thus enhancing explainability. The experimental results on three real-world datasets demonstrate that PiECL can explain the learning process, and the generated model outperforms other state-of-the-art methods. PiECL shows tremendous potential for explaining temporal graph learning in various scientific contexts.

A computational study of how an α- to γ-motoneurone collateral can mitigate velocity-dependent stretch reflexes during voluntary movement

The primary motor cortex does not uniquely or directly produce alpha motoneurone (α-MN) drive to muscles during voluntary movement. Rather, α-MN drive emerges from the synthesis and competition among excitatory and inhibitory inputs from multiple descending tracts, spinal interneurons, sensory inputs, and proprioceptive afferents. One such fundamental input is velocity-dependent stretch reflexes in lengthening muscles, which should be inhibited to enable voluntary movement. It remains an open question, however, the extent to which unmodulated stretch reflexes disrupt voluntary movement, and whether and how they are inhibited in limbs with numerous multiarticular muscles. We used a computational model of a Rhesus Macaque arm to simulate movements with feedforward α-MN commands only, and with added velocity-dependent stretch reflex feedback. We found that velocity-dependent stretch reflex caused movement-specific, typically large and variable disruptions to arm movements. These disruptions were greatly reduced when modulating velocity-dependent stretch reflex feedback (i) as per the commonly proposed (but yet to be clarified) idealized alpha-gamma (α-γ) coactivation or (ii) an alternative α-MN collateral projection to homonymous γ-MNs. We conclude that such α-MN collaterals are a physiologically tenable propriospinal circuit in the mammalian fusimotor system. These collaterals could still collaborate with α-γ coactivation, and the few skeletofusimotor fibers (β-MNs) in mammals, to create a flexible fusimotor ecosystem to enable voluntary movement. By locally and automatically regulating the highly nonlinear neuro-musculo-skeletal mechanics of the limb, these collaterals could be a critical low-level enabler of learning, adaptation, and performance via higher-level brainstem, cerebellar, and cortical mechanisms.

Concepts and applications of digital twins in healthcare and medicine

The digital twin (DT) is a concept widely used in industry to create digital replicas of physical objects or systems. The dynamic, bi-directional link between the physical entity and its digital counterpart enables a real-time update of the digital entity. It can predict perturbations related to the physical object's function. The obvious applications of DTs in healthcare and medicine are extremely attractive prospects that have the potential to revolutionize patient diagnosis and treatment. However, challenges including technical obstacles, biological heterogeneity, and ethical considerations make it difficult to achieve the desired goal. Advances in multi-modal deep learning methods, embodied AI agents, and the metaverse may mitigate some difficulties. Here, we discuss the basic concepts underlying DTs, the requirements for implementing DTs in medicine, and their current and potential healthcare uses. We also provide our perspective on five hallmarks for a healthcare DT system to advance research in this field.

Knowledge transfer in lifelong machine learning: a systematic literature review

Lifelong Machine Learning (LML) denotes a scenario involving multiple sequential tasks, each accompanied by its respective dataset, in order to solve specific learning problems. In this context, the focus of LML techniques is on utilizing already acquired knowledge to adapt to new tasks efficiently. Essentially, LML concerns about facing new tasks while exploiting the knowledge previously gathered from earlier tasks not only to help in adapting to new tasks but also to enrich the understanding of past ones. By understanding this concept, one can better grasp one of the major obstacles in LML, known as Knowledge Transfer (KT). This systematic literature review aims to explore state-of-the-art KT techniques within LML and assess the evaluation metrics and commonly utilized datasets in this field, thereby keeping the LML research community updated with the latest developments. From an initial pool of 417 articles from four distinguished databases, 30 were deemed highly pertinent for the information extraction phase. The analysis recognizes four primary KT techniques: Replay, Regularization, Parameter Isolation, and Hybrid. This study delves into the characteristics of these techniques across both neural network (NN) and non-neural network (non-NN) frameworks, highlighting their distinct advantages that have captured researchers' interest. It was found that the majority of the studies focused on supervised learning within an NN modelling framework, particularly employing Parameter Isolation and Hybrid for KT. The paper concludes by pinpointing research opportunities, including investigating non-NN models for Replay and exploring applications outside of computer vision (CV).

Reconciling shared versus context-specific information in a neural network model of latent causes

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could (1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, (2) capture human data on curriculum effects in schema learning, and (3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.

An alpha- to gamma-motoneurone collateral can mitigate velocity-dependent stretch reflexes during voluntary movement: A computational study

The primary motor cortex does not uniquely or directly produce alpha motoneurone (α-MN) drive to muscles during voluntary movement. Rather, α-MN drive emerges from the synthesis and competition among excitatory and inhibitory inputs from multiple descending tracts, spinal interneurons, sensory inputs, and proprioceptive afferents. One such fundamental input is velocity-dependent stretch reflexes in lengthening muscles, which should be inhibited to enable voluntary movement. It remains an open question, however, the extent to which unmodulated stretch reflexes disrupt voluntary movement, and whether and how they are inhibited in limbs with numerous multi-articular muscles. We used a computational model of a Rhesus Macaque arm to simulate movements with feedforward α-MN commands only, and with added velocity-dependent stretch reflex feedback. We found that velocity-dependent stretch reflex caused movement-specific, typically large and variable disruptions to arm movements. These disruptions were greatly reduced when modulating velocity-dependent stretch reflex feedback (i) as per the commonly proposed (but yet to be clarified) idealized alpha-gamma (α-γ) co-activation or (ii) an alternative α-MN collateral projection to homonymous γ-MNs. We conclude that such α-MN collaterals are a physiologically tenable, but previously unrecognized, propriospinal circuit in the mammalian fusimotor system. These collaterals could still collaborate with α-γ co-activation, and the few skeletofusimotor fibers (β-MNs) in mammals, to create a flexible fusimotor ecosystem to enable voluntary movement. By locally and automatically regulating the highly nonlinear neuro-musculo-skeletal mechanics of the limb, these collaterals could be a critical low-level enabler of learning, adaptation, and performance via higher-level brainstem, cerebellar and cortical mechanisms.

Never-Ending Learning for Explainable Brain Computing

Exploring the nature of human intelligence and behavior is a longstanding pursuit in cognitive neuroscience, driven by the accumulation of knowledge, information, and data across various studies. However, achieving a unified and transparent interpretation of findings presents formidable challenges. In response, an explainable brain computing framework is proposed that employs the never-ending learning paradigm, integrating evidence combination and fusion computing within a Knowledge-Information-Data (KID) architecture. The framework supports continuous brain cognition investigation, utilizing joint knowledge-driven forward inference and data-driven reverse inference, bolstered by the pre-trained language modeling techniques and the human-in-the-loop mechanisms. In particular, it incorporates internal evidence learning through multi-task functional neuroimaging analyses and external evidence learning via topic modeling of published neuroimaging studies, all of which involve human interactions at different stages. Based on two case studies, the intricate uncertainty surrounding brain localization in human reasoning is revealed. The present study also highlights the potential of systematization to advance explainable brain computing, offering a finer-grained understanding of brain activity patterns related to human intelligence.

NSF DARE-Transforming modeling in neurorehabilitation: Four threads for catalyzing progress

We present an overview of the Conference on Transformative Opportunities for Modeling in Neurorehabilitation held in March 2023. It was supported by the Disability and Rehabilitation Engineering (DARE) program from the National Science Foundation's Engineering Biology and Health Cluster. The conference brought together experts and trainees from around the world to discuss critical questions, challenges, and opportunities at the intersection of computational modeling and neurorehabilitation to understand, optimize, and improve clinical translation of neurorehabilitation. We organized the conference around four key, relevant, and promising Focus Areas for modeling: Adaptation & Plasticity, Personalization, Human-Device Interactions, and Modeling 'In-the-Wild'. We identified four common threads across the Focus Areas that, if addressed, can catalyze progress in the short, medium, and long terms. These were: (i) the need to capture and curate appropriate and useful data necessary to develop, validate, and deploy useful computational models (ii) the need to create multi-scale models that span the personalization spectrum from individuals to populations, and from cellular to behavioral levels (iii) the need for algorithms that extract as much information from available data, while requiring as little data as possible from each client (iv) the insistence on leveraging readily available sensors and data systems to push model-driven treatments from the lab, and into the clinic, home, workplace, and community. The conference archive can be found at (dare2023.usc.edu). These topics are also extended by three perspective papers prepared by trainees and junior faculty, clinician researchers, and federal funding agency representatives who attended the conference.

The maps of meaning consciousness theory

In simple terms, consciousness is constituted by multiple goals for action and the continuous adjudication of such goals to implement action, which is referred to as the maps of meaning (MoM) consciousness theory. The MoM theory triangulates through three parallel corollaries: action (behavior), mechanism (morphology/pathophysiology), and goals (teleology). (1) An organism's consciousness contains fluid, nested goals. These goals are not intentionality, but intersectionality, via the Darwinian byproduct of embodiment meeting the world, i.e., Darwinian inclusive fitness or randomization and then survival of the fittest. (2) These goals are formed via a gradual descent under inclusive fitness and are the abstraction of a "match" between the evolutionary environment and the organism. (3) Human consciousness implements the brain efficiency hypothesis, genetics, epigenetics, and experience-crystallized efficiencies, not necessitating best or objective but fitness, i.e., perceived efficiency based on one's adaptive environment. These efficiencies are objectively arbitrary but determine the operation and level of one's consciousness, termed as extreme thrownness. (4) Since inclusive fitness drives efficiencies in the physiologic mechanism, morphology, and behavior (action) and originates one's goals, embodiment is necessarily entangled to human consciousness as it is at the intersection of mechanism or action (both necessitating embodiment) occurring in the world that determines fitness. (5) Perception is the operant process of consciousness and is the de facto goal adjudication process of consciousness. Goal operationalization is fundamentally efficiency-based via one's unique neuronal mapping as a byproduct of genetics, epigenetics, and experience. (6) Perception involves information intake and information discrimination, equally underpinned by efficiencies of inclusive fitness via extreme thrownness. Perception is not a 'frame rate' but Bayesian priors of efficiency based on one's extreme thrownness. (7) Consciousness and human consciousness are modular (i.e., a scalar level of richness, which builds up like building blocks) and dimensionalized (i.e., cognitive abilities become possibilities as the emergent phenomena at various modularities such as the stratified factors in factor analysis). (8) The meta dimensions of human consciousness seemingly include intelligence quotient, personality (five-factor model), richness of perception intake, and richness of perception discrimination, among other potentialities. (9) Future consciousness research should utilize factor analysis to parse modularities and dimensions of human consciousness and animal models.

Analyzing Biomedical Datasets with Symbolic Tree Adaptive Resonance Theory

Biomedical datasets distill many mechanisms of human diseases, linking diseases to genes and phenotypes (signs and symptoms of disease), genetic mutations to altered protein structures, and altered proteins to changes in molecular functions and biological processes. It is desirable to gain new insights from these data, especially with regard to the uncovering of hierarchical structures relating disease variants. However, analysis to this end has proven difficult due to the complexity of the connections between multi-categorical symbolic data. This article proposes symbolic tree adaptive resonance theory (START), with additional supervised, dual-vigilance (DV-START), and distributed dual-vigilance (DDV-START) formulations, for the clustering of multi-categorical symbolic data from biomedical datasets by demonstrating its utility in clustering variants of Charcot–Marie–Tooth disease using genomic, phenotypic, and proteomic data.

Special Issue-Biosensors and Neuroscience: Is Biosensors Engineering Ready to Embrace Design Principles from Neuroscience?

In partnership with the Air Force Office of Scientific Research (AFOSR), the National Science Foundation's (NSF) Emerging Frontiers and Multidisciplinary Activities (EFMA) office of the Directorate for Engineering (ENG) launched an Emerging Frontiers in Research and Innovation (EFRI) topic for the fiscal years FY22 and FY23 entitled "Brain-inspired Dynamics for Engineering Energy-Efficient Circuits and Artificial Intelligence" (BRAID) [...].

Muscle redundancy is greatly reduced by the spatiotemporal nature of neuromuscular control

Animals must control numerous muscles to produce forces and movements with their limbs. Current theories of motor optimization and synergistic control are predicated on the assumption that there are multiple highly diverse feasible activations for any motor task ("muscle redundancy"). Here, we demonstrate that the dimensionality of the neuromuscular control problem is greatly reduced when adding the temporal constraints inherent to any sequence of motor commands: the physiological time constants for muscle activation-contraction dynamics. We used a seven-muscle model of a human finger to fully characterize the seven-dimensional polytope of all possible motor commands that can produce fingertip force vector in any direction in 3D, in alignment with the core models of Feasibility Theory. For a given sequence of seven force vectors lasting 300 ms, a novel single-step extended linear program finds the 49-dimensional polytope of all possible motor commands that can produce the sequence of forces. We find that muscle redundancy is severely reduced when the temporal limits on muscle activation-contraction dynamics are added. For example, allowing a generous ± 12% change in muscle activation within 50 ms allows visiting only ∼ 7% of the feasible activation space in the next time step. By considering that every motor command conditions future commands, we find that the motor-control landscape is much more highly structured and spatially constrained than previously recognized. We discuss how this challenges traditional computational and conceptual theories of motor control and neurorehabilitation for which muscle redundancy is a foundational assumption.

Online dynamical learning and sequence memory with neuromorphic nanowire networks

Nanowire Networks (NWNs) belong to an emerging class of neuromorphic systems that exploit the unique physical properties of nanostructured materials. In addition to their neural network-like physical structure, NWNs also exhibit resistive memory switching in response to electrical inputs due to synapse-like changes in conductance at nanowire-nanowire cross-point junctions. Previous studies have demonstrated how the neuromorphic dynamics generated by NWNs can be harnessed for temporal learning tasks. This study extends these findings further by demonstrating online learning from spatiotemporal dynamical features using image classification and sequence memory recall tasks implemented on an NWN device. Applied to the MNIST handwritten digit classification task, online dynamical learning with the NWN device achieves an overall accuracy of 93.4%. Additionally, we find a correlation between the classification accuracy of individual digit classes and mutual information. The sequence memory task reveals how memory patterns embedded in the dynamical features enable online learning and recall of a spatiotemporal sequence pattern. Overall, these results provide proof-of-concept of online learning from spatiotemporal dynamics using NWNs and further elucidate how memory can enhance learning.

Incremental Learning of Goal-Directed Actions in a Dynamic Environment by a Robot Using Active Inference

This study investigated how a physical robot can adapt goal-directed actions in dynamically changing environments, in real-time, using an active inference-based approach with incremental learning from human tutoring examples. Using our active inference-based model, while good generalization can be achieved with appropriate parameters, when faced with sudden, large changes in the environment, a human may have to intervene to correct actions of the robot in order to reach the goal, as a caregiver might guide the hands of a child performing an unfamiliar task. In order for the robot to learn from the human tutor, we propose a new scheme to accomplish incremental learning from these proprioceptive-exteroceptive experiences combined with mental rehearsal of past experiences. Our experimental results demonstrate that using only a few tutoring examples, the robot using our model was able to significantly improve its performance on new tasks without catastrophic forgetting of previously learned tasks.

Bio-inspired, task-free continual learning through activity regularization

The ability to sequentially learn multiple tasks without forgetting is a key skill of biological brains, whereas it represents a major challenge to the field of deep learning. To avoid catastrophic forgetting, various continual learning (CL) approaches have been devised. However, these usually require discrete task boundaries. This requirement seems biologically implausible and often limits the application of CL methods in the real world where tasks are not always well defined. Here, we take inspiration from neuroscience, where sparse, non-overlapping neuronal representations have been suggested to prevent catastrophic forgetting. As in the brain, we argue that these sparse representations should be chosen on the basis of feed forward (stimulus-specific) as well as top-down (context-specific) information. To implement such selective sparsity, we use a bio-plausible form of hierarchical credit assignment known as Deep Feedback Control (DFC) and combine it with a winner-take-all sparsity mechanism. In addition to sparsity, we introduce lateral recurrent connections within each layer to further protect previously learned representations. We evaluate the new sparse-recurrent version of DFC on the split-MNIST computer vision benchmark and show that only the combination of sparsity and intra-layer recurrent connections improves CL performance with respect to standard backpropagation. Our method achieves similar performance to well-known CL methods, such as Elastic Weight Consolidation and Synaptic Intelligence, without requiring information about task boundaries. Overall, we showcase the idea of adopting computational principles from the brain to derive new, task-free learning algorithms for CL.

High-resolution CMOS-based biosensor for assessing hippocampal circuit dynamics in experience-dependent plasticity

Experiential richness creates tissue-level changes and synaptic plasticity as patterns emerge from rhythmic spatiotemporal activity of large interconnected neuronal assemblies. Despite numerous experimental and computational approaches at different scales, the precise impact of experience on network-wide computational dynamics remains inaccessible due to the lack of applicable large-scale recording methodology. We here demonstrate a large-scale multi-site biohybrid brain circuity on-CMOS-based biosensor with an unprecedented spatiotemporal resolution of 4096 microelectrodes, which allows simultaneous electrophysiological assessment across the entire hippocampal-cortical subnetworks from mice living in an enriched environment (ENR) and standard-housed (SD) conditions. Our platform, empowered with various computational analyses, reveals environmental enrichment's impacts on local and global spatiotemporal neural dynamics, firing synchrony, topological network complexity, and large-scale connectome. Our results delineate the distinct role of prior experience in enhancing multiplexed dimensional coding formed by neuronal ensembles and error tolerance and resilience to random failures compared to standard conditions. The scope and depth of these effects highlight the critical role of high-density, large-scale biosensors to provide a new understanding of the computational dynamics and information processing in multimodal physiological and experience-dependent plasticity conditions and their role in higher brain functions. Knowledge of these large-scale dynamics can inspire the development of biologically plausible computational models and computational artificial intelligence networks and expand the reach of neuromorphic brain-inspired computing into new applications.

Dual memory model for experience-once task-incremental lifelong learning

Experience replay (ER) is a widely-adopted neuroscience-inspired method to perform lifelong learning. Nonetheless, existing ER-based approaches consider very coarse memory modules with simple memory and rehearsal mechanisms that cannot fully exploit the potential of memory replay. Evidence from neuroscience has provided fine-grained memory and rehearsal mechanisms, such as the dual-store memory system consisting of PFC-HC circuits. However, the computational abstraction of these processes is still very challenging. To address these problems, we introduce the Dual-Memory (Dual-MEM) model emulating the memorization, consolidation, and rehearsal process in the PFC-HC dual-store memory circuit. Dual-MEM maintains an incrementally updated short-term memory to benefit current-task learning. At the end of the current task, short-term memories will be consolidated into long-term ones for future rehearsal to alleviate forgetting. For the Dual-MEM optimization, we propose two learning policies that emulate different memory retrieval strategies: Direct Retrieval Learning and Mixup Retrieval Learning. Extensive evaluations on eight benchmarks demonstrate that Dual-MEM delivers compelling performance while maintaining high learning and memory utilization efficiencies under the challenging experience-once setting.

Continual Deep Learning for Time Series Modeling

The multi-layer structures of Deep Learning facilitate the processing of higher-level abstractions from data, thus leading to improved generalization and widespread applications in diverse domains with various types of data. Each domain and data type presents its own set of challenges. Real-world time series data may have a non-stationary data distribution that may lead to Deep Learning models facing the problem of catastrophic forgetting, with the abrupt loss of previously learned knowledge. Continual learning is a paradigm of machine learning to handle situations when the stationarity of the datasets may no longer be true or required. This paper presents a systematic review of the recent Deep Learning applications of sensor time series, the need for advanced preprocessing techniques for some sensor environments, as well as the summaries of how to deploy Deep Learning in time series modeling while alleviating catastrophic forgetting with continual learning methods. The selected case studies cover a wide collection of various sensor time series applications and can illustrate how to deploy tailor-made Deep Learning, advanced preprocessing techniques, and continual learning algorithms from practical, real-world application aspects.

Neuromuscular control: from a biomechanist's perspective

Understanding neural control of movement necessitates a collaborative approach between many disciplines, including biomechanics, neuroscience, and motor control. Biomechanics grounds us to the laws of physics that our musculoskeletal system must obey. Neuroscience reveals the inner workings of our nervous system that functions to control our body. Motor control investigates the coordinated motor behaviours we display when interacting with our environment. The combined efforts across the many disciplines aimed at understanding human movement has resulted in a rich and rapidly growing body of literature overflowing with theories, models, and experimental paradigms. As a result, gathering knowledge and drawing connections between the overlapping but seemingly disparate fields can be an overwhelming endeavour. This review paper evolved as a need for us to learn of the diverse perspectives underlying current understanding of neuromuscular control. The purpose of our review paper is to integrate ideas from biomechanics, neuroscience, and motor control to better understand how we voluntarily control our muscles. As biomechanists, we approach this paper starting from a biomechanical modelling framework. We first define the theoretical solutions (i.e., muscle activity patterns) that an individual could feasibly use to complete a motor task. The theoretical solutions will be compared to experimental findings and reveal that individuals display structured muscle activity patterns that do not span the entire theoretical solution space. Prevalent neuromuscular control theories will be discussed in length, highlighting optimality, probabilistic principles, and neuromechanical constraints, that may guide individuals to families of muscle activity solutions within what is theoretically possible. Our intention is for this paper to serve as a primer for the neuromuscular control scientific community by introducing and integrating many of the ideas common across disciplines today, as well as inspire future work to improve the representation of neural control in biomechanical models.

Edge Computing in Nature: Minimal pre-processing of multi-muscle ensembles of spindle signals improves discriminability of limb movements

Multiple proprioceptive signals, like those from muscle spindles, are thought to enable robust estimates of body configuration. Yet, it remains unknown whether spindle signals suffice to discriminate limb movements. Here, a simulated 4-musculotendon, 2-joint planar limb model produced repeated cycles of five end-point trajectories in forward and reverse directions, which generated spindle Ia and II afferent signals (proprioceptors for velocity and length, respectively) from each musculotendon. We find that cross-correlation of the 8D time series of raw firing rates (four Ia, four II) cannot discriminate among most movement pairs (∼ 29% accuracy). However, projecting these signals onto their 1st and 2nd principal components greatly improves discriminability of movement pairs (82% accuracy). We conclude that high-dimensional ensembles of muscle proprioceptors can discriminate among limb movements-but only after dimensionality reduction. This may explain the pre-processing of some afferent signals before arriving at the somatosensory cortex, such as processing of cutaneous signals at the cat's cuneate nucleus.

Genetic dissection of mutual interference between two consecutive learning tasks in Drosophila

Animals can continuously learn different tasks to adapt to changing environments and, therefore, have strategies to effectively cope with inter-task interference, including both proactive interference (Pro-I) and retroactive interference (Retro-I). Many biological mechanisms are known to contribute to learning, memory, and forgetting for a single task, however, mechanisms involved only when learning sequential different tasks are relatively poorly understood. Here, we dissect the respective molecular mechanisms of Pro-I and Retro-I between two consecutive associative learning tasks in Drosophila. Pro-I is more sensitive to an inter-task interval (ITI) than Retro-I. They occur together at short ITI (<20 min), while only Retro-I remains significant at ITI beyond 20 min. Acutely overexpressing Corkscrew (CSW), an evolutionarily conserved protein tyrosine phosphatase SHP2, in mushroom body (MB) neurons reduces Pro-I, whereas acute knockdown of CSW exacerbates Pro-I. Such function of CSW is further found to rely on the γ subset of MB neurons and the downstream Raf/MAPK pathway. In contrast, manipulating CSW does not affect Retro-I as well as a single learning task. Interestingly, manipulation of Rac1, a molecule that regulates Retro-I, does not affect Pro-I. Thus, our findings suggest that learning different tasks consecutively triggers distinct molecular mechanisms to tune proactive and retroactive interference.

Remembrance of things perceived: Adding thalamocortical function to artificial neural networks

Recent research has illuminated the complexity and importance of the thalamocortical system but it has been difficult to identify what computational functions it performs. Meanwhile, deep-learning artificial neural networks (ANNs) based on bio-inspired models of purely cortical circuits have achieved surprising success solving sophisticated cognitive problems associated historically with human intelligence. Nevertheless, the limitations and shortcomings of artificial intelligence (AI) based on such ANNs are becoming increasingly clear. This review considers how the addition of thalamocortical connectivity and its putative functions related to cortical attention might address some of those shortcomings. Such bio-inspired models are now providing both testable theories of biological cognition and improved AI technology, much of which is happening outside the usual academic venues.

Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks

Artificial neural networks are known to suffer from catastrophic forgetting: when learning multiple tasks sequentially, they perform well on the most recent task at the expense of previously learned tasks. In the brain, sleep is known to play an important role in incremental learning by replaying recent and old conflicting memory traces. Here we tested the hypothesis that implementing a sleep-like phase in artificial neural networks can protect old memories during new training and alleviate catastrophic forgetting. Sleep was implemented as off-line training with local unsupervised Hebbian plasticity rules and noisy input. In an incremental learning framework, sleep was able to recover old tasks that were otherwise forgotten. Previously learned memories were replayed spontaneously during sleep, forming unique representations for each class of inputs. Representational sparseness and neuronal activity corresponding to the old tasks increased while new task related activity decreased. The study suggests that spontaneous replay simulating sleep-like dynamics can alleviate catastrophic forgetting in artificial neural networks.

The challenges of lifelong learning in biological and artificial systems

How do biological systems learn continuously throughout their lifespans, adapting to change while retaining old knowledge, and how can these principles be applied to artificial learning systems? In this Forum article we outline challenges and strategies of 'lifelong learning' in biological and artificial systems, and argue that a collaborative study of each system's failure modes can benefit both.

Bayesian continual learning via spiking neural networks

Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps toward the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.

Recent Advances at the Interface of Neuroscience and Artificial Neural Networks

Biological neural networks adapt and learn in diverse behavioral contexts. Artificial neural networks (ANNs) have exploited biological properties to solve complex problems. However, despite their effectiveness for specific tasks, ANNs are yet to realize the flexibility and adaptability of biological cognition. This review highlights recent advances in computational and experimental research to advance our understanding of biological and artificial intelligence. In particular, we discuss critical mechanisms from the cellular, systems, and cognitive neuroscience fields that have contributed to refining the architecture and training algorithms of ANNs. Additionally, we discuss how recent work used ANNs to understand complex neuronal correlates of cognition and to process high throughput behavioral data.

Contributions by metaplasticity to solving the Catastrophic Forgetting Problem

Catastrophic forgetting (CF) refers to the sudden and severe loss of prior information in learning systems when acquiring new information. CF has been an Achilles heel of standard artificial neural networks (ANNs) when learning multiple tasks sequentially. The brain, by contrast, has solved this problem during evolution. Modellers now use a variety of strategies to overcome CF, many of which have parallels to cellular and circuit functions in the brain. One common strategy, based on metaplasticity phenomena, controls the future rate of change at key connections to help retain previously learned information. However, the metaplasticity properties so far used are only a subset of those existing in neurobiology. We propose that as models become more sophisticated, there could be value in drawing on a richer set of metaplasticity rules, especially when promoting continual learning in agents moving about the environment.

Developing Intelligent Robots that Grasp Affordance

Humans and robots operating in unstructured environments both need to classify objects through haptic exploration and use them in various tasks, but currently they differ greatly in their strategies for acquiring such capabilities. This review explores nascent technologies that promise more convergence. A novel form of artificial intelligence classifies objects according to sensory percepts during active exploration and decides on efficient sequences of exploratory actions to identify objects. Representing objects according to the collective experience of manipulating them provides a substrate for discovering causality and affordances. Such concepts that generalize beyond explicit training experiences are an important aspect of human intelligence that has eluded robots. For robots to acquire such knowledge, they will need an extended period of active exploration and manipulation similar to that employed by infants. The efficacy, efficiency and safety of such behaviors depends on achieving smooth transitions between movements that change quickly from exploratory to executive to reflexive. Animals achieve such smoothness by using a hierarchical control scheme that is fundamentally different from those of conventional robotics. The lowest level of that hierarchy, the spinal cord, starts to self-organize during spontaneous movements in the fetus. This allows its connectivity to reflect the mechanics of the musculoskeletal plant, a bio-inspired process that could be used to adapt spinal-like middleware for robots. Implementation of these extended and essential stages of fetal and infant development is impractical, however, for mechatronic hardware that does not heal and replace itself like biological tissues. Instead such development can now be accomplished in silico and then cloned into physical robots, a strategy that could transcend human performance.

Learning plastic matching of robot dynamics in closed-loop central pattern generators

Animals achieve agile locomotion performance with reduced control effort and energy efficiency by leveraging compliance in their muscles and tendons. However, it is not known how biological locomotion controllers learn to leverage the intelligence embodied in their leg mechanics. Here we present a framework to match control patterns and mechanics based on the concept of short-term elasticity and long-term plasticity. Inspired by animals, we design a robot, Morti, with passive elastic legs. The quadruped robot Morti is controlled by a bioinspired closed-loop central pattern generator that is designed to elastically mitigate short-term perturbations using sparse contact feedback. By minimizing the amount of corrective feedback on the long term, Morti learns to match the controller to its mechanics and learns to walk within 1 h. By leveraging the advantages of its mechanics, Morti improves its energy efficiency by 42% without explicit minimization in the cost function.

Three types of incremental learning

Incrementally learning new information from a non-stationary stream of data, referred to as 'continual learning', is a key feature of natural intelligence, but a challenging problem for deep neural networks. In recent years, numerous deep learning methods for continual learning have been proposed, but comparing their performances is difficult due to the lack of a common framework. To help address this, we describe three fundamental types, or 'scenarios', of continual learning: task-incremental, domain-incremental and class-incremental learning. Each of these scenarios has its own set of challenges. To illustrate this, we provide a comprehensive empirical comparison of currently used continual learning strategies, by performing the Split MNIST and Split CIFAR-100 protocols according to each scenario. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of the effectiveness of different strategies. The proposed categorization aims to structure the continual learning field, by forming a key foundation for clearly defining benchmark problems.