Rapid learning and flexible memory in “habit” tasks in rats trained with brain stimulation reward

Profiles of animal consciousness: A species-sensitive, two-tier account to quality and distribution

The science of animal consciousness investigates (i) which animal species are conscious (the distribution question) and (ii) how conscious experience differs in detail between species (the quality question). We propose a framework which clearly distinguishes both questions and tackles both of them. This two-tier account distinguishes consciousness along ten dimensions and suggests cognitive capacities which serve as distinct operationalizations for each dimension. The two-tier account achieves three valuable aims: First, it separates strong and weak indicators of the presence of consciousness. Second, these indicators include not only different specific contents but also differences in the way particular contents are processed (by processes of learning, reasoning or abstraction). Third, evidence of consciousness from each dimension can be combined to derive the distinctive multi-dimensional consciousness profile of various species. Thus, the two-tier account shows how the kind of conscious experience of different species can be systematically compared.

Probabilistic generative modeling and reinforcement learning extract the intrinsic features of animal behavior

It is one of the ultimate goals of ethology to understand the generative process of animal behavior, and the ability to reproduce and control behavior is an important step in this field. However, it is not easy to achieve this goal in systems with complex and stochastic dynamics such as animal behavior. In this study, we have shown that MDN-RNN,a type of probabilistic deep generative model, is able to reproduce stochastic animal behavior with high accuracy by modeling the behavior of C. elegans. Furthermore, we found that the model learns different dynamics in a disentangled representation as a time-evolving Gaussian mixture. Finally, by combining the model and reinforcement learning, we were able to extract a behavioral policy of goal-directed behavior in silico, and showed that it can be used for regulating the behavior of real animals. This set of methods will be applicable not only to animal behavior but also to broader areas such as neuroscience and robotics.

Bilateral responses of rat ventral striatum tonically active neurons to unilateral medial forebrain bundle stimulation

Unilateral medial forebrain bundle (MFB) stimulation is an extremely effective promoter of reinforcement learning irrespective of the conditioned cue's laterality. The effectiveness of unilateral MFB stimulation, which activates the mesolimbic pathway connecting the ventral tegmental area to the ventral striatum (vStr), is surprising considering that these fibers rarely cross to the contralateral hemisphere. Specifically, this type of biased fiber distribution entails the activation of brain structures that are primarily ipsilateral to the stimulated MFB, along with weak to negligible activation of the contralateral structures, thus impeding the formation of a cue-outcome association. To better understand the spread of activation of MFB stimulation across hemispheres, we studied whether unilateral MFB stimulation primarily activates the ipsilateral vStr or the vStr of both hemispheres. We simultaneously recorded neuronal activity in the vStr of both hemispheres in response to several sets of unilateral MFB stimulation in anesthetized and freely moving rats. Unilateral MFB stimulation evoked strong stimulus-dependent activation of vStr tonically active neurons (TANs), presumably the cholinergic interneurons, in both hemispheres. However, the TANs' activation patterns and responsiveness depended on whether the stimulus was delivered ipsilaterally or contralaterally to the recorded neuron. These findings indicate that unilateral MFB stimulation effectively activates the vStr in both hemispheres in a stimulus-dependent manner which may serve as neuronal substrate for the formation of cue-outcome associations during reinforcement learning.

Brain-Machine Interface-Based Rat-Robot Behavior Control

Brain-machine interface (BMI) provides a bidirectional pathway between the brain and external facilities. The machine-to-brain pathway makes it possible to send artificial information back into the biological brain, interfering neural activities and generating sensations. The idea of the BMI-assisted bio-robotic animal system is accomplished by stimulations on specific sites of the nervous system. With the technology of BMI, animals' locomotion behavior can be precisely controlled as robots, which made the animal turning into bio-robot. In this chapter, we reviewed our lab works focused on rat-robot navigation. The principles of rat-robot system have been briefly described first, including the target brain sites chosen for locomotion control and the design of remote control system. Some methodological advances made by optogenetic technologies for better modulation control have then been introduced. Besides, we also introduced our implementation of "mind-controlled" rat navigation system. Moreover, we have presented our efforts made on combining biological intelligence with artificial intelligence, with developments of automatic control and training system assisted with images or voices inputs. We concluded this chapter by discussing further developments to acquire environmental information as well as promising applications with write-in BMIs.

Remote-Controlled Fully Implantable Neural Stimulator for Freely Moving Small Animal

The application of a neural stimulator to small animals is highly desired for the investigation of electrophysiological studies and development of neuroprosthetic devices. For this purpose, it is essential for the device to be implemented with the capabilities of full implantation and wireless control. Here, we present a fully implantable stimulator with remote controllability, compact size, and minimal power consumption. Our stimulator consists of modular units of (1) a surface-type cortical array for inducing directional change of a rat, (2) a depth-type array for providing rewards, and (3) a package for accommodating the stimulating electronics, a battery and ZigBee telemetry, all of which are assembled after independent fabrication and implantation using customized flat cables and connectors. All three modules were packaged using liquid crystal polymer (LCP) to avoid any chemical reaction after implantation. After bench-top evaluation of device functionality, the stimulator was implanted into rats to train the animals to turn to the left (or right) following a directional cue applied to the barrel cortex. Functionality of the device was also demonstrated in a three-dimensional (3D) maze structure, by guiding the rats to better navigate in the maze. The movement of the rat could be wirelessly controlled by a combination of artificial sensation evoked by the surface electrode array and reward stimulation. We could induce rats to turn left or right in free space and help their navigation through the maze. The polymeric packaging and modular design could encapsulate the devices with strict size limitations, which made it possible to fully implant the device into rats. Power consumption was minimized by a dual-mode power-saving scheme with duty cycling. The present study demonstrated feasibility of the proposed neural stimulator to be applied to neuroprosthesis research.

Spectral Modification by Operant Conditioning of Cortical Theta Suppression in Rats

Objective

Brain activity is known to be voluntarily controllable by neurofeedback, a kind of electroencephalographic (EEG) operant conditioning. Although its efficacy in clinical effects has been reported, it is yet to be uncovered whether or how a specific band activity is controllable. Here, we examined EEG spectral profiles along with conditioning training of a specific brain activity, theta band (4–8 Hz) amplitude, in rats.

Methods

During training, the experimental group received electrical stimulation to the medial forebrain bundle contingent to suppression of theta activity, while the control group received stimulation non-contingent to its own band activity.

Results

In the experimental group, theta activity gradually decreased within the training session, while there was an increase of theta activity in the control group. There was a significant difference in theta activity during the sessions between the two groups. The spectral theta peak, originally located at 7 Hz, shifted further towards higher frequencies in the experimental group.

Conclusion

Our results showed that an operant conditioning technique could train rats to control their specific EEG activity indirectly, and it may be used as an animal model for studying how neuronal systems work in human neurofeedback.

Human Mind Control of Rat Cyborg's Continuous Locomotion with Wireless Brain-to-Brain Interface

Brain-machine interfaces (BMIs) provide a promising information channel between the biological brain and external devices and are applied in building brain-to-device control. Prior studies have explored the feasibility of establishing a brain-brain interface (BBI) across various brains via the combination of BMIs. However, using BBI to realize the efficient multidegree control of a living creature, such as a rat, to complete a navigation task in a complex environment has yet to be shown. In this study, we developed a BBI from the human brain to a rat implanted with microelectrodes (i.e., rat cyborg), which integrated electroencephalogram-based motor imagery and brain stimulation to realize human mind control of the rat’s continuous locomotion. Control instructions were transferred from continuous motor imagery decoding results with the proposed control models and were wirelessly sent to the rat cyborg through brain micro-electrical stimulation. The results showed that rat cyborgs could be smoothly and successfully navigated by the human mind to complete a navigation task in a complex maze. Our experiments indicated that the cooperation through transmitting multidimensional information between two brains by computer-assisted BBI is promising.

Maze learning by a hybrid brain-computer system

The combination of biological and artificial intelligence is particularly driven by two major strands of research: one involves the control of mechanical, usually prosthetic, devices by conscious biological subjects, whereas the other involves the control of animal behaviour by stimulating nervous systems electrically or optically. However, to our knowledge, no study has demonstrated that spatial learning in a computer-based system can affect the learning and decision making behaviour of the biological component, namely a rat, when these two types of intelligence are wired together to form a new intelligent entity. Here, we show how rule operations conducted by computing components contribute to a novel hybrid brain-computer system, i.e., ratbots, exhibit superior learning abilities in a maze learning task, even when their vision and whisker sensation were blocked. We anticipate that our study will encourage other researchers to investigate combinations of various rule operations and other artificial intelligence algorithms with the learning and memory processes of organic brains to develop more powerful cyborg intelligence systems. Our results potentially have profound implications for a variety of applications in intelligent systems and neural rehabilitation.

Automatic Training of Rat Cyborgs for Navigation

A rat cyborg system refers to a biological rat implanted with microelectrodes in its brain, via which the outer electrical stimuli can be delivered into the brain in vivo to control its behaviors. Rat cyborgs have various applications in emergency, such as search and rescue in disasters. Prior to a rat cyborg becoming controllable, a lot of effort is required to train it to adapt to the electrical stimuli. In this paper, we build a vision-based automatic training system for rat cyborgs to replace the time-consuming manual training procedure. A hierarchical framework is proposed to facilitate the colearning between rats and machines. In the framework, the behavioral states of a rat cyborg are visually sensed by a camera, a parameterized state machine is employed to model the training action transitions triggered by rat's behavioral states, and an adaptive adjustment policy is developed to adaptively adjust the stimulation intensity. The experimental results of three rat cyborgs prove the effectiveness of our system. To the best of our knowledge, this study is the first to tackle automatic training of animal cyborgs.

Bio-robots automatic navigation with electrical reward stimulation

Bio-robots that controlled by outer stimulation through brain computer interface (BCI) suffer from the dependence on realtime guidance of human operators. Current automatic navigation methods for bio-robots focus on the controlling rules to force animals to obey man-made commands, with animals' intelligence ignored. This paper proposes a new method to realize the automatic navigation for bio-robots with electrical micro-stimulation as real-time rewards. Due to the reward-seeking instinct and trial-and-error capability, bio-robot can be steered to keep walking along the right route with rewards and correct its direction spontaneously when rewards are deprived. In navigation experiments, rat-robots learn the controlling methods in short time. The results show that our method simplifies the controlling logic and realizes the automatic navigation for rat-robots successfully. Our work might have significant implication for the further development of bio-robots with hybrid intelligence.

Operant conditioning of rat navigation using electrical stimulation for directional cues and rewards

Operant conditioning is often used to train a desired behavior in an animal. The contingency between a specific behavior and a reward is required for successful training. Here, we compared the effectiveness of two different mazes for training turning behaviors in response to directional cues in Sprague-Dawley rats. Forty-three rats were implanted with electrodes into the medial forebrain bundle and the left and right somatosensory cortices for reward and cues. Among them, thirteen rats discriminated between the left and right somatosensory stimulations to obtain rewards. They were trained to learn ipsilateral turning response to the stimulation of the left or right somatosensory cortex in either the T-maze (Group T) or the E| maze (Group W). Performance was measured by the navigation speed in the mazes. Performances of rats in Group T were enhanced faster than those in Group W. A significant correlation between performances during training and performance in final testing was observed in Group T starting with the fifth training session while such a correlation was not observed in Group W until the tenth training session. The training mazes did not however affect the performances in the final test. These results suggest that a simple maze is better than a complicated maze for training animals to learn directions and direct cortical stimulation can be used as a cue for direction training.