Elucidating multifinal and equifinal pathways to developmental disorders by constructing real-world neurorobotic models

Vigorous research has been conducted to accumulate biological and theoretical knowledge about neurodevelopmental disorders, including molecular, neural, computational, and behavioral characteristics; however, these findings remain fragmentary and do not elucidate integrated mechanisms. An obstacle is the heterogeneity of developmental pathways causing clinical phenotypes. Additionally, in symptom formations, the primary causes and consequences of developmental learning processes are often indistinguishable. Herein, we review developmental neurorobotic experiments tackling problems related to the dynamic and complex properties of neurodevelopmental disorders. Specifically, we focus on neurorobotic models under predictive processing lens for the study of developmental disorders. By constructing neurorobotic models with predictive processing mechanisms of learning, perception, and action, we can simulate formations of integrated causal relationships among neurodynamical, computational, and behavioral characteristics in the robot agents while considering developmental learning processes. This framework has the potential to bind neurobiological hypotheses (excitation-inhibition imbalance and functional disconnection), computational accounts (unusual encoding of uncertainty), and clinical symptoms. Developmental neurorobotic approaches may serve as a complementary research framework for integrating fragmented knowledge and overcoming the heterogeneity of neurodevelopmental disorders.

Saturated and asymmetric saturated control for projective synchronization of inertial neural networks with delays and discontinuous activations through matrix measure method

The projective synchronization work presented in this article is focused on a class of nonlinear discontinuous coupled inertial neural networks with mixed time-varying delays and a cluster topological structure. The synchronization problem for discontinuous coupled inertial neural networks with clustering topology is examined in consideration with the mismatched parameters and the mutual influence among various clusters. To determine the required conditions for network convergence under the influence of an extensive range of impulses, the matrix measure technique and the average impulsive intervals are employed. To illustrate the effectiveness of the theoretical findings, graphical representation of varied impulsive ranges for multiple cases are provided using numerical simulations.

Grounding Context in Embodied Cognitive Robotics

Biological agents are context-dependent systems that exhibit behavioral flexibility. The internal and external information agents process, their actions, and emotions are all grounded in the context within which they are situated. However, in the field of cognitive robotics, the concept of context is far from being clear with most studies making little to no reference to it. The aim of this paper is to provide an interpretation of the notion of context and its core elements based on different studies in natural agents, and how these core contextual elements have been modeled in cognitive robotics, to introduce a new hypothesis about the interactions between these contextual elements. Here, global context is categorized as agent-related, environmental, and task-related context. The interaction of their core elements, allows agents to first select self-relevant tasks depending on their current needs, or for learning and mastering their environment through exploration. Second, to perform a task and continuously monitor its performance. Third, to abandon a task in case its execution is not going as expected. Here, the monitoring of prediction error, the difference between sensorimotor predictions and incoming sensory information, is at the core of behavioral flexibility during situated action cycles. Additionally, monitoring prediction error dynamics and its comparison with the expected reduction rate should indicate the agent its overall performance on executing the task. Sensitivity to performance evokes emotions that function as the driving element for autonomous behavior which, at the same time, depends on the processing of the interacting core elements. Taking all these into account, an interactionist model of contexts and their core elements is proposed. The model is embodied, affective, and situated, by means of the processing of the agent-related and environmental core contextual elements. Additionally, it is grounded in the processing of the task-related context and the associated situated action cycles during task execution. Finally, the model proposed here aims to guide how artificial agents should process the core contextual elements of the agent-related and environmental context to give rise to the task-related context, allowing agents to autonomously select a task, its planning, execution, and monitoring for behavioral flexibility.

Goal-Directed Planning and Goal Understanding by Extended Active Inference: Evaluation through Simulated and Physical Robot Experiments

We show that goal-directed action planning and generation in a teleological framework can be formulated by extending the active inference framework. The proposed model, which is built on a variational recurrent neural network model, is characterized by three essential features. These are that (1) goals can be specified for both static sensory states, e.g., for goal images to be reached and dynamic processes, e.g., for moving around an object, (2) the model cannot only generate goal-directed action plans, but can also understand goals through sensory observation, and (3) the model generates future action plans for given goals based on the best estimate of the current state, inferred from past sensory observations. The proposed model is evaluated by conducting experiments on a simulated mobile agent as well as on a real humanoid robot performing object manipulation.

The brain and its time: intrinsic neural timescales are key for input processing

We process and integrate multiple timescales into one meaningful whole. Recent evidence suggests that the brain displays a complex multiscale temporal organization. Different regions exhibit different timescales as described by the concept of intrinsic neural timescales (INT); however, their function and neural mechanisms remains unclear. We review recent literature on INT and propose that they are key for input processing. Specifically, they are shared across different species, i.e., input sharing. This suggests a role of INT in encoding inputs through matching the inputs' stochastics with the ongoing temporal statistics of the brain's neural activity, i.e., input encoding. Following simulation and empirical data, we point out input integration versus segregation and input sampling as key temporal mechanisms of input processing. This deeply grounds the brain within its environmental and evolutionary context. It carries major implications in understanding mental features and psychiatric disorders, as well as going beyond the brain in integrating timescales into artificial intelligence.

Anti-Synchronization of Delayed State-Based Switched Inertial Neural Networks

In this article, global anti-synchronization control for a class of state-based switched inertial neural networks (SBSINNs) with time-varying delays is considered. Based on the hybrid control strategies and Lyapunov stability theory, several criteria are obtained to ensure global anti-synchronization of the underlying SBSINNs. Furthermore, we consider the global asymptotic anti-synchronization directly from the SBSINNs themselves with a nonreduced-order method. Finally, a numerical simulation is given to illustrate the effectiveness of the results.

Personogenesis Through Imitating Human Behavior in a Humanoid Robot "Alter3"

In this study, we report the investigations conducted on the mimetic behavior of a new humanoid robot called Alter3. Alter3 autonomously imitates the motions of a person in front of it and stores the motion sequences in its memory. Alter3 also uses a self-simulator to simulate its own motions before executing them and generates a self-image. If the visual perception (of a person's motion being imitated) and the imitating self-image differ significantly, Alter3 retrieves a motion sequence closer to the target motion from its memory and executes it. We investigate how this mimetic behavior develops interacting with human, by analyzing memory dynamics and information flow between Alter3 and a interacting person. One important observation from this study is that when Alter3 fails to imitate a person's motion, the person tend to imitate Alter3 instead. This tendency is quantified by the alternation of the direction of information flow. This spontaneous role-switching behavior between a human and Alter3 is a way to initiate personality formation (i.e., personogenesis) in Alter3.

Novel results on finite-time stabilization of state-based switched chaotic inertial neural networks with distributed delays

The p-norm finite-time stabilization (FTS) issue of a class of state-based switched inertial chaotic neural networks (SBSCINNs) with distributed time-varying delays is investigated. By using a suitable variable transformation, such second-order SBSCINNs are turned into the first-order differential equations. Then some novel criteria are obtained to stabilize SBSCINNs in a finite time based on the theory of finite-time control and non-smooth analysis together with designing two proper delay-dependent feedback controllers. Besides, the settling time of FTS is also estimated and discussed. Finally, the validity and practicability of the deduced theoretical results are verified by examples and applications.

New Criteria on Global Stabilization of Delayed Memristive Neural Networks With Inertial Item

In this paper, we are concerned with global stabilization for a kind of delayed memristive neural network with an inertial term. By building a new Lyapunov functional and designing a feedback controller, we obtain some new results on global stabilization of the addressed delayed memristive inertial neural networks (MINNs). An adaptive control strategy is also designed to realize the global stabilization. Compared with the reduced-order method used in the existing literature, we consider the stabilization directly from the MINNs themselves without a reduced-order method. In addition, the new results proposed here are shown as algebraic criteria, which are easy to test. At last, some simulations are given to show the validity of the derived criteria.

Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network

It is crucial to ask how agents can achieve goals by generating action plans using only partial models of the world acquired through habituated sensory-motor experiences. Although many existing robotics studies use a forward model framework, there are generalization issues with high degrees of freedom. The current study shows that the predictive coding (PC) and active inference (AIF) frameworks, which employ a generative model, can develop better generalization by learning a prior distribution in a low dimensional latent state space representing probabilistic structures extracted from well habituated sensory-motor trajectories. In our proposed model, learning is carried out by inferring optimal latent variables as well as synaptic weights for maximizing the evidence lower bound, while goal-directed planning is accomplished by inferring latent variables for maximizing the estimated lower bound. Our proposed model was evaluated with both simple and complex robotic tasks in simulation, which demonstrated sufficient generalization in learning with limited training data by setting an intermediate value for a regularization coefficient. Furthermore, comparative simulation results show that the proposed model outperforms a conventional forward model in goal-directed planning, due to the learned prior confining the search of motor plans within the range of habituated trajectories.

Further results on finite-time synchronization of delayed inertial memristive neural networks via a novel analysis method

In this paper, we propose a novel analysis method to investigate the finite-time synchronization (FTS) control problem of the drive-response inertial memristive neural networks (IMNNs) with mixed time-varying delays (MTVDs). Firstly, an improved control scheme is proposed under the delay-independent conditions, which can work even when the past state cannot be measured or the specific time delay function is unknown. Secondly, based on the assumption of bounded activation functions, we establish a new Lemma, which can effectively deal with the difficulties caused by memristive connection weights and MTVDs. Thirdly, by constructing a suitable Lyapunov functions and using a new inequality method, novel sufficient conditions to ensure the FTS for the discussed IMNNs are obtained. Compared with the existing results, our results obtained in a more general framework are more practical. Finally, some numerical simulations are given to substantiate the effectiveness of the theoretical results.

Global exponential stabilization and lag synchronization control of inertial neural networks with time delays

The global exponential stabilization and lag synchronization control of delayed inertial neural networks (INNs) are investigated. By constructing nonnegative function and employing inequality techniques, several new results about exponential stabilization and exponential lag synchronization are derived via adaptive control. And the theoretical outcomes are developed directly from the INNs themselves without variable substitution. In addition, the synchronization results are also applied to image encryption and decryption. Finally, an example is presented to illustrate the validity of the derived results.

Predictive learning: its key role in early cognitive development

What is a fundamental ability for cognitive development? Although many researchers have been addressing this question, no shared understanding has been acquired yet. We propose that predictive learning of sensorimotor signals plays a key role in early cognitive development. The human brain is known to represent sensorimotor signals in a predictive manner, i.e. it attempts to minimize prediction error between incoming sensory signals and top–down prediction. We extend this view and suggest that two mechanisms for minimizing prediction error lead to the development of cognitive abilities during early infancy. The first mechanism is to update an immature predictor. The predictor must be trained through sensorimotor experiences because it does not inherently have prediction ability. The second mechanism is to execute an action anticipated by the predictor. Interacting with other individuals often increases prediction error, which can be minimized by executing one's own action corresponding to others’ action. Our experiments using robotic systems replicated developmental dynamics observed in infants. The capabilities of self–other cognition and goal-directed action were acquired based on the first mechanism, whereas imitation and prosocial behaviours emerged based on the second mechanism. Our theory further provides a potential mechanism for autism spectrum condition. Atypical tolerance for prediction error is hypothesized to be a cause of perceptual and social difficulties.

This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’.

Stabilization of Second-Order Memristive Neural Networks With Mixed Time Delays via Nonreduced Order

In this brief, we investigate a class of second-order memristive neural networks (SMNNs) with mixed time-varying delays. Based on nonsmooth analysis, the Lyapunov stability theory, and adaptive control theory, several new results ensuring global stabilization of the SMNNs are obtained. In addition, compared with the reduced-order method used in the existing research studies, we consider the global stabilization directly from the SMNNs themselves without the reduced-order method. Finally, we give some numerical simulations to show the effectiveness of the results.

A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition

This study introduces PV-RNN, a novel variational RNN inspired by predictive-coding ideas. The model learns to extract the probabilistic structures hidden in fluctuating temporal patterns by dynamically changing the stochasticity of its latent states. Its architecture attempts to address two major concerns of variational Bayes RNNs: how latent variables can learn meaningful representations and how the inference model can transfer future observations to the latent variables. PV-RNN does both by introducing adaptive vectors mirroring the training data, whose values can then be adapted differently during evaluation. Moreover, prediction errors during backpropagation-rather than external inputs during the forward computation-are used to convey information to the network about the external data. For testing, we introduce error regression for predicting unseen sequences as inspired by predictive coding that leverages those mechanisms. As in other variational Bayes RNNs, our model learns by maximizing a lower bound on the marginal likelihood of the sequential data, which is composed of two terms: the negative of the expectation of prediction errors and the negative of the Kullback-Leibler divergence between the prior and the approximate posterior distributions. The model introduces a weighting parameter, the meta-prior, to balance the optimization pressure placed on those two terms. We test the model on two data sets with probabilistic structures and show that with high values of the meta-prior, the network develops deterministic chaos through which the randomness of the data is imitated. For low values, the model behaves as a random process. The network performs best on intermediate values and is able to capture the latent probabilistic structure with good generalization. Analyzing the meta-prior's impact on the network allows us to precisely study the theoretical value and practical benefits of incorporating stochastic dynamics in our model. We demonstrate better prediction performance on a robot imitation task with our model using error regression compared to a standard variational Bayes model lacking such a procedure.

The detour problem in a stochastic environment: Tolman revisited

We designed a grid world task to study human planning and re-planning behavior in an unknown stochastic environment. In our grid world, participants were asked to travel from a random starting point to a random goal position while maximizing their reward. Because they were not familiar with the environment, they needed to learn its characteristics from experience to plan optimally. Later in the task, we randomly blocked the optimal path to investigate whether and how people adjust their original plans to find a detour. To this end, we developed and compared 12 different models. These models were different on how they learned and represented the environment and how they planned to catch the goal. The majority of our participants were able to plan optimally. We also showed that people were capable of revising their plans when an unexpected event occurred. The result from the model comparison showed that the model-based reinforcement learning approach provided the best account for the data and outperformed heuristics in explaining the behavioral data in the re-planning trials.

Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatiotemporal Scales RNN Model

This letter proposes a novel predictive coding type neural network model, the predictive multiple spatiotemporal scales recurrent neural network (P-MSTRNN). The P-MSTRNN learns to predict visually perceived human whole-body cyclic movement patterns by exploiting multiscale spatiotemporal constraints imposed on network dynamics by using differently sized receptive fields as well as different time constant values for each layer. After learning, the network can imitate target movement patterns by inferring or recognizing corresponding intentions by means of the regression of prediction error. Results show that the network can develop a functional hierarchy by developing a different type of dynamic structure at each layer. The letter examines how model performance during pattern generation, as well as predictive imitation, varies depending on the stage of learning. The number of limit cycle attractors corresponding to target movement patterns increases as learning proceeds. Transient dynamics developing early in the learning process successfully perform pattern generation and predictive imitation tasks. The letter concludes that exploitation of transient dynamics facilitates successful task performance during early learning periods.

Autonomous exploration of mobile robots through deep neural networks

The exploration problem of mobile robots aims to allow mobile robots to explore an unknown environment. We describe an indoor exploration algorithm for mobile robots using a hierarchical structure that fuses several convolutional neural network layers with decision-making process. The whole system is trained end to end by taking only visual information (RGB-D information) as input and generates a sequence of main moving direction as output so that the robot achieves autonomous exploration ability. The robot is a TurtleBot with a Kinect mounted on it. The model is trained and tested in a real world environment. And the training data set is provided for download. The outputs of the test data are compared with the human decision. We use Gaussian process latent variable model to visualize the feature map of last convolutional layer, which proves the effectiveness of this deep convolution neural network mode. We also present a novel and lightweight deep-learning library libcnn especially for deep-learning processing of robotics tasks.

Motor-Skill Learning in an Insect Inspired Neuro-Computational Control System

In nature, insects show impressive adaptation and learning capabilities. The proposed computational model takes inspiration from specific structures of the insect brain: after proposing key hypotheses on the direct involvement of the mushroom bodies (MBs) and on their neural organization, we developed a new architecture for motor learning to be applied in insect-like walking robots. The proposed model is a nonlinear control system based on spiking neurons. MBs are modeled as a nonlinear recurrent spiking neural network (SNN) with novel characteristics, able to memorize time evolutions of key parameters of the neural motor controller, so that existing motor primitives can be improved. The adopted control scheme enables the structure to efficiently cope with goal-oriented behavioral motor tasks. Here, a six-legged structure, showing a steady-state exponentially stable locomotion pattern, is exposed to the need of learning new motor skills: moving through the environment, the structure is able to modulate motor commands and implements an obstacle climbing procedure. Experimental results on a simulated hexapod robot are reported; they are obtained in a dynamic simulation environment and the robot mimicks the structures of Drosophila melanogaster.

Robotic action acquisition with cognitive biases in coarse-grained state space

Some of the authors have previously proposed a cognitively inspired reinforcement learning architecture (LS-Q) that mimics cognitive biases in humans. LS-Q adaptively learns under uniform, coarse-grained state division and performs well without parameter tuning in a giant-swing robot task. However, these results were shown only in simulations. In this study, we test the validity of the LS-Q implemented in a robot in a real environment. In addition, we analyze the learning process to elucidate the mechanism by which the LS-Q adaptively learns under the partially observable environment. We argue that the LS-Q may be a versatile reinforcement learning architecture, which is, despite its simplicity, easily applicable and does not require well-prepared settings.

Development of compositional and contextual communicable congruence in robots by using dynamic neural network models

The current study presents neurorobotics experiments on acquisition of skills for "communicable congruence" with human via learning. A dynamic neural network model which is characterized by its multiple timescale dynamics property was utilized as a neuromorphic model for controlling a humanoid robot. In the experimental task, the humanoid robot was trained to generate specific sequential movement patterns as responding to various sequences of imperative gesture patterns demonstrated by the human subjects by following predefined compositional semantic rules. The experimental results showed that (1) the adopted MTRNN can achieve generalization by learning in the lower feature perception level by using a limited set of tutoring patterns, (2) the MTRNN can learn to extract compositional semantic rules with generalization in its higher level characterized by slow timescale dynamics, (3) the MTRNN can develop another type of cognitive capability for controlling the internal contextual processes as situated to on-going task sequences without being provided with cues for explicitly indicating task segmentation points. The analysis on the dynamic property developed in the MTRNN via learning indicated that the aforementioned cognitive mechanisms were achieved by self-organization of adequate functional hierarchy by utilizing the constraint of the multiple timescale property and the topological connectivity imposed on the network configuration. These results of the current research could contribute to developments of socially intelligent robots endowed with cognitive communicative competency similar to that of human.

Ideomotor feedback control in a recurrent neural network

The architecture of a neural network controlling an unknown environment is presented. It is based on a randomly connected recurrent neural network from which both perception and action are simultaneously read and fed back. There are two concurrent learning rules implementing a sort of ideomotor control: (i) perception is learned along the principle that the network should predict reliably its incoming stimuli; (ii) action is learned along the principle that the prediction of the network should match a target time series. The coherent behavior of the neural network in its environment is a consequence of the interaction between the two principles. Numerical simulations show a promising performance of the approach, which can be turned into a local and better "biologically plausible" algorithm.

On learning navigation behaviors for small mobile robots with reservoir computing architectures

This paper proposes a general reservoir computing (RC) learning framework that can be used to learn navigation behaviors for mobile robots in simple and complex unknown partially observable environments. RC provides an efficient way to train recurrent neural networks by letting the recurrent part of the network (called reservoir) be fixed while only a linear readout output layer is trained. The proposed RC framework builds upon the notion of navigation attractor or behavior that can be embedded in the high-dimensional space of the reservoir after learning. The learning of multiple behaviors is possible because the dynamic robot behavior, consisting of a sensory-motor sequence, can be linearly discriminated in the high-dimensional nonlinear space of the dynamic reservoir. Three learning approaches for navigation behaviors are shown in this paper. The first approach learns multiple behaviors based on the examples of navigation behaviors generated by a supervisor, while the second approach learns goal-directed navigation behaviors based only on rewards. The third approach learns complex goal-directed behaviors, in a supervised way, using a hierarchical architecture whose internal predictions of contextual switches guide the sequence of basic navigation behaviors toward the goal.

Autonomous learning in humanoid robotics through mental imagery

In this paper we focus on modeling autonomous learning to improve performance of a humanoid robot through a modular artificial neural networks architecture. A model of a neural controller is presented, which allows a humanoid robot iCub to autonomously improve its sensorimotor skills. This is achieved by endowing the neural controller with a secondary neural system that, by exploiting the sensorimotor skills already acquired by the robot, is able to generate additional imaginary examples that can be used by the controller itself to improve the performance through a simulated mental training. Results and analysis presented in the paper provide evidence of the viability of the approach proposed and help to clarify the rational behind the chosen model and its implementation.

Projective simulation for artificial intelligence

We propose a model of a learning agent whose interaction with the environment is governed by a simulation-based projection, which allows the agent to project itself into future situations before it takes real action. Projective simulation is based on a random walk through a network of clips, which are elementary patches of episodic memory. The network of clips changes dynamically, both due to new perceptual input and due to certain compositional principles of the simulation process. During simulation, the clips are screened for specific features which trigger factual action of the agent. The scheme is different from other, computational, notions of simulation, and it provides a new element in an embodied cognitive science approach to intelligent action and learning. Our model provides a natural route for generalization to quantum-mechanical operation and connects the fields of reinforcement learning and quantum computation.