Can Machine Learning Personalize Cardiovascular Therapy in Sepsis?

Large randomized trials in sepsis have generally failed to find effective novel treatments. This is increasingly attributed to patient heterogeneity, including heterogeneous cardiovascular changes in septic shock. We discuss the potential for machine learning systems to personalize cardiovascular resuscitation in sepsis. While the literature is replete with proofs of concept, the technological readiness of current systems is low, with a paucity of clinical trials and proven patient benefit. Systems may be vulnerable to confounding and poor generalization to new patient populations or contemporary patterns of care. Typical electronic health records do not capture rich enough data, at sufficient temporal resolution, to produce systems that make actionable treatment suggestions. To resolve these issues, we recommend a simultaneous focus on technical challenges and removing barriers to translation. This will involve improving data quality, adopting causally grounded models, prioritizing safety assessment and integration into healthcare workflows, conducting randomized clinical trials and aligning with regulatory requirements.

EEG decoding for datasets with heterogenous electrode configurations using transfer learning graph neural networks

Objective. Brain-machine interfacing (BMI) has greatly benefited from adopting machine learning methods for feature learning that require extensive data for training, which are often unavailable from a single dataset. Yet, it is difficult to combine data across labs or even data within the same lab collected over the years due to the variation in recording equipment and electrode layouts resulting in shifts in data distribution, changes in data dimensionality, and altered identity of data dimensions. Our objective is to overcome this limitation and learn from many different and diverse datasets across labs with different experimental protocols.Approach. To tackle the domain adaptation problem, we developed a novel machine learning framework combining graph neural networks (GNNs) and transfer learning methodologies for non-invasive motor imagery (MI) EEG decoding, as an example of BMI. Empirically, we focus on the challenges of learning from EEG data with different electrode layouts and varying numbers of electrodes. We utilize three MI EEG databases collected using very different numbers of EEG sensors (from 22 channels to 64) and layouts (from custom layouts to 10-20).Main results. Our model achieved the highest accuracy with lower standard deviations on the testing datasets. This indicates that the GNN-based transfer learning framework can effectively aggregate knowledge from multiple datasets with different electrode layouts, leading to improved generalization in subject-independent MI EEG classification.Significance. The findings of this study have important implications for brain-computer-interface research, as they highlight a promising method for overcoming the limitations posed by non-unified experimental setups. By enabling the integration of diverse datasets with varying electrode layouts, our proposed approach can help advance the development and application of BMI technologies.

Bill-EVR: An Embodied Virtual Reality Framework for Reward-and-Error-Based Motor Rehab-Learning

VR rehabilitation is an established field by now, however, it often refers to computer screen-based interactive rehabilitation activities. In recent years, there was an increased use of VR-headsets, which can provide an immersive virtual environment for real-world tasks, but they are lacking any physical interaction with the task objects and any proprioceptive feedback. Here, we focus on Embodied Virtual Reality (EVR), an emerging field where not only the visual input via VR-headset but also the haptic feedback is physically correct. This happens because subjects interact with physical objects that are veridically aligned in Virtual Reality. This technology lets us manipulate motor performance and motor learning through visual feedback perturbations. Bill-EVR is a framework that allows interventions in the performance of real-world tasks, such as playing pool billiard, engaging end-users in motivating life-like situations to trigger motor (re)learning - subjects see in VR and handle the real-world cue stick, the pool table and shoot physical balls. Specifically, we developed our platform to isolate and evaluate different mechanisms of motor learning to investigate its two main components, error-based and reward-based motor adaptation. This understanding can provide insights for improvements in neurorehabilitation: indeed, reward-based mechanisms are putatively impaired by degradation of the dopaminergic system, such as in Parkinson's disease, while error-based mechanisms are essential for recovering from stroke-induced movement errors. Due to its fully customisable features, our EVR framework can be used to facilitate the improvement of several conditions, providing a valid extension of VR-based implementations and constituting a motor learning tool that can be completely tailored to the individual needs of patients.

Towards AI-Controlled Movement Restoration: Learning FES-Cycling Stimulation with Reinforcement Learning

Functional electrical stimulation (FES) has been increasingly integrated with other rehabilitation devices, including rehabilitation robots. FES cycling is one of the common FES applications in rehabilitation, which is performed by stimulating leg muscles in a certain pattern. The appropriate pattern varies across individuals and requires manual tuning which can be time-consuming and challenging for the individual user. Here, we present an AI-based method for finding the patterns, which requires no extra hardware or sensors. Our method starts with finding model-based patterns using reinforcement learning (RL) and customised cycling models. Next, our method fine-tunes the pattern using real cycling data and offline RL. We test our method both in simulation and experimentally on a stationary tricycle. Our method can robustly deliver model-based patterns for different cycling configurations. In the experimental evaluation, the model-based pattern can induce higher cycling speed than an EMG-based pattern. And by using just 100 seconds of cycling data, our method can deliver a fine-tuned pattern with better cycling performance. Beyond FES cycling, this work is a case study, displaying the feasibility and potential of human-in-the-loop AI in real-world rehabilitation.

Mindcraft, a Mobile Mental Health Monitoring Platform for Children and Young People: Development and Acceptability Pilot Study

BACKGROUND

Children and young people's mental health is a growing public health concern, which is further exacerbated by the COVID-19 pandemic. Mobile health apps, particularly those using passive smartphone sensor data, present an opportunity to address this issue and support mental well-being.

OBJECTIVE

This study aimed to develop and evaluate a mobile mental health platform for children and young people, Mindcraft, which integrates passive sensor data monitoring with active self-reported updates through an engaging user interface to monitor their well-being.

METHODS

A user-centered design approach was used to develop Mindcraft, incorporating feedback from potential users. User acceptance testing was conducted with a group of 8 young people aged 15-17 years, followed by a pilot test with 39 secondary school students aged 14-18 years, which was conducted for a 2-week period.

RESULTS

Mindcraft showed encouraging user engagement and retention. Users reported that they found the app to be a friendly tool helping them to increase their emotional awareness and gain a better understanding of themselves. Over 90% of users (36/39, 92.5%) answered all active data questions on the days they used the app. Passive data collection facilitated the gathering of a broader range of well-being metrics over time, with minimal user intervention.

CONCLUSIONS

The Mindcraft app has shown promising results in monitoring mental health symptoms and promoting user engagement among children and young people during its development and initial testing. The app's user-centered design, the focus on privacy and transparency, and a combination of active and passive data collection strategies have all contributed to its efficacy and receptiveness among the target demographic. By continuing to refine and expand the app, the Mindcraft platform has the potential to contribute meaningfully to the field of mental health care for young people.

A wearable motion capture suit and machine learning predict disease progression in Friedreich's ataxia

Friedreich's ataxia (FA) is caused by a variant of the Frataxin (FXN) gene, leading to its downregulation and progressively impaired cardiac and neurological function. Current gold-standard clinical scales use simplistic behavioral assessments, which require 18- to 24-month-long trials to determine if therapies are beneficial. Here we captured full-body movement kinematics from patients with wearable sensors, enabling us to define digital behavioral features based on the data from nine FA patients (six females and three males) and nine age- and sex-matched controls, who performed the 8-m walk (8-MW) test and 9-hole peg test (9 HPT). We used machine learning to combine these features to longitudinally predict the clinical scores of the FA patients, and compared these with two standard clinical assessments, Spinocerebellar Ataxia Functional Index (SCAFI) and Scale for the Assessment and Rating of Ataxia (SARA). The digital behavioral features enabled longitudinal predictions of personal SARA and SCAFI scores 9 months into the future and were 1.7 and 4 times more precise than longitudinal predictions using only SARA and SCAFI scores, respectively. Unlike the two clinical scales, the digital behavioral features accurately predicted FXN gene expression levels for each FA patient in a cross-sectional manner. Our work demonstrates how data-derived wearable biomarkers can track personal disease trajectories and indicates the potential of such biomarkers for substantially reducing the duration or size of clinical trials testing disease-modifying therapies and for enabling behavioral transcriptomics.

Wearable full-body motion tracking of activities of daily living predicts disease trajectory in Duchenne muscular dystrophy

Artificial intelligence has the potential to revolutionize healthcare, yet clinical trials in neurological diseases continue to rely on subjective, semiquantitative and motivation-dependent endpoints for drug development. To overcome this limitation, we collected a digital readout of whole-body movement behavior of patients with Duchenne muscular dystrophy (DMD) (n = 21) and age-matched controls (n = 17). Movement behavior was assessed while the participant engaged in everyday activities using a 17-sensor bodysuit during three clinical visits over the course of 12 months. We first defined new movement behavioral fingerprints capable of distinguishing DMD from controls. Then, we used machine learning algorithms that combined the behavioral fingerprints to make cross-sectional and longitudinal disease course predictions, which outperformed predictions derived from currently used clinical assessments. Finally, using Bayesian optimization, we constructed a behavioral biomarker, termed the KineDMD ethomic biomarker, which is derived from daily-life behavioral data and whose value progresses with age in an S-shaped sigmoid curve form. The biomarker developed in this study, derived from digital readouts of daily-life movement behavior, can predict disease progression in patients with muscular dystrophy and can potentially track the response to therapy.

Assuring the safety of AI-based clinical decision support systems: a case study of the AI Clinician for sepsis treatment

Objectives

Establishing confidence in the safety of Artificial Intelligence (AI)-based clinical decision support systems is important prior to clinical deployment and regulatory approval for systems with increasing autonomy. Here, we undertook safety assurance of the AI Clinician, a previously published reinforcement learning-based treatment recommendation system for sepsis.

Methods

As part of the safety assurance, we defined four clinical hazards in sepsis resuscitation based on clinical expert opinion and the existing literature. We then identified a set of unsafe scenarios, intended to limit the action space of the AI agent with the goal of reducing the likelihood of hazardous decisions.

Results

Using a subset of the Medical Information Mart for Intensive Care (MIMIC-III) database, we demonstrated that our previously published ‘AI clinician’ recommended fewer hazardous decisions than human clinicians in three out of our four predefined clinical scenarios, while the difference was not statistically significant in the fourth scenario. Then, we modified the reward function to satisfy our safety constraints and trained a new AI Clinician agent. The retrained model shows enhanced safety, without negatively impacting model performance.

Discussion

While some contextual patient information absent from the data may have pushed human clinicians to take hazardous actions, the data were curated to limit the impact of this confounder.

Conclusion

These advances provide a use case for the systematic safety assurance of AI-based clinical systems towards the generation of explicit safety evidence, which could be replicated for other AI applications or other clinical contexts, and inform medical device regulatory bodies.

Movement-specific signaling is differentially distributed across motor cortex layer 5 projection neuron classes

Motor cortex generates descending output necessary for executing a wide range of limb movements. Although movement-related activity has been described throughout motor cortex, the spatiotemporal organization of movement-specific signaling in deep layers remains largely unknown. Here we record layer 5B population dynamics in the caudal forelimb area of motor cortex while mice perform a forelimb push/pull task and find that most neurons show movement-invariant responses, with a minority displaying movement specificity. Using cell-type-specific imaging, we identify that invariant responses dominate pyramidal tract (PT) neuron activity, with a small subpopulation representing movement type, whereas a larger proportion of intratelencephalic (IT) neurons display movement-type-specific signaling. The proportion of IT neurons decoding movement-type peaks prior to movement initiation, whereas for PT neurons, this occurs during movement execution. Our data suggest that layer 5B population dynamics largely reflect movement-invariant signaling, with information related to movement-type being routed through relatively small, distributed subpopulations of projection neurons.

Variation in global COVID-19 symptoms by geography and by chronic disease: A global survey using the COVID-19 Symptom Mapper

BACKGROUND

COVID-19 is typically characterised by a triad of symptoms: cough, fever and loss of taste and smell, however, this varies globally. This study examines variations in COVID-19 symptom profiles based on underlying chronic disease and geographical location.

METHODS

Using a global online symptom survey of 78,299 responders in 190 countries between 09/04/2020 and 22/09/2020, we conducted an exploratory study to examine symptom profiles associated with a positive COVID-19 test result by country and underlying chronic disease (single, co- or multi-morbidities) using statistical and machine learning methods.

FINDINGS

From the results of 7980 COVID-19 tested positive responders, we find that symptom patterns differ by country. For example, India reported a lower proportion of headache (22.8% vs 47.8%, p<1e-13) and itchy eyes (7.3% vs. 16.5%, p=2e-8) than other countries. As with geographic location, we find people differed in their reported symptoms if they suffered from specific chronic diseases. For example, COVID-19 positive responders with asthma (25.3% vs. 13.7%, p=7e-6) were more likely to report shortness of breath compared to those with no underlying chronic disease.

INTERPRETATION

We have identified variation in COVID-19 symptom profiles depending on geographic location and underlying chronic disease. Failure to reflect this symptom variation in public health messaging may contribute to asymptomatic COVID-19 spread and put patients with chronic diseases at a greater risk of infection. Future work should focus on symptom profile variation in the emerging variants of the SARS-CoV-2 virus. This is crucial to speed up clinical diagnosis, predict prognostic outcomes and target treatment.

FUNDING

We acknowledge funding to AAF by a UKRI Turing AI Fellowship and to CEC by a personal NIHR Career Development Fellowship (grant number NIHR-2016-090-015). JKQ has received grants from The Health Foundation, MRC, GSK, Bayer, BI, Asthma UK-British Lung Foundation, IQVIA, Chiesi AZ, and Insmed. This work is supported by BREATHE - The Health Data Research Hub for Respiratory Health [MC_PC_19004]. BREATHE is funded through the UK Research and Innovation Industrial Strategy Challenge Fund and delivered through Health Data Research UK. Imperial College London is grateful for the support from the Northwest London NIHR Applied Research Collaboration. The views expressed in this publication are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Playing the piano with a robotic third thumb: assessing constraints of human augmentation

Contemporary robotics gives us mechatronic capabilities for augmenting human bodies with extra limbs. However, how our motor control capabilities pose limits on such augmentation is an open question. We developed a Supernumerary Robotic 3rd Thumbs (SR3T) with two degrees-of-freedom controlled by the user’s body to endow them with an extra contralateral thumb on the hand. We demonstrate that a pianist can learn to play the piano with 11 fingers within an hour. We then evaluate 6 naïve and 6 experienced piano players in their prior motor coordination and their capability in piano playing with the robotic augmentation. We show that individuals’ augmented performance with the SR3T could be explained by our new custom motor coordination assessment, the Human Augmentation Motor Coordination Assessment (HAMCA) performed pre-augmentation. Our work demonstrates how supernumerary robotics can augment humans in skilled tasks and that individual differences in their augmentation capability are explainable by their individual motor coordination abilities.

Early life experience sets hard limits on motor learning as evidenced from artificial arm use

The study of artificial arms provides a unique opportunity to address long-standing questions on sensorimotor plasticity and development. Learning to use an artificial arm arguably depends on fundamental building blocks of body representation and would therefore be impacted by early life experience. We tested artificial arm motor-control in two adult populations with upper-limb deficiencies: a congenital group—individuals who were born with a partial arm, and an acquired group—who lost their arm following amputation in adulthood. Brain plasticity research teaches us that the earlier we train to acquire new skills (or use a new technology) the better we benefit from this practice as adults. Instead, we found that although the congenital group started using an artificial arm as toddlers, they produced increased error noise and directional errors when reaching to visual targets, relative to the acquired group who performed similarly to controls. However, the earlier an individual with a congenital limb difference was fitted with an artificial arm, the better their motor control was. Since we found no group differences when reaching without visual feedback, we suggest that the ability to perform efficient visual-based corrective movements is highly dependent on either biological or artificial arm experience at a very young age. Subsequently, opportunities for sensorimotor plasticity become more limited.

Deep learning multimodal fNIRS and EEG signals for bimanual grip force decoding

Objective.Non-invasive brain-machine interfaces (BMIs) offer an alternative, safe and accessible way to interact with the environment. To enable meaningful and stable physical interactions, BMIs need to decode forces. Although previously addressed in the unimanual case, controlling forces from both hands would enable BMI-users to perform a greater range of interactions. We here investigate the decoding of hand-specific forces.Approach.We maximise cortical information by using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) and developing a deep-learning architecture with attention and residual layers (cnnatt) to improve their fusion. Our task required participants to generate hand-specific force profiles on which we trained and tested our deep-learning and linear decoders.Main results.The use of EEG and fNIRS improved the decoding of bimanual force and the deep-learning models outperformed the linear model. In both cases, the greatest gain in performance was due to the detection of force generation. In particular, the detection of forces was hand-specific and better for the right dominant hand andcnnattwas better at fusing EEG and fNIRS. Consequently, the study ofcnnattrevealed that forces from each hand were differently encoded at the cortical level.Cnnattalso revealed traces of the cortical activity being modulated by the level of force which was not previously found using linear models.Significance.Our results can be applied to avoid hand-cross talk during hand force decoding to improve the robustness of BMI robotic devices. In particular, we improve the fusion of EEG and fNIRS signals and offer hand-specific interpretability of the encoded forces which are valuable during motor rehabilitation assessment.

Dissociable effects of age and Parkinson's disease on instruction-based learning

The cognitive deficits associated with Parkinson's disease vary across individuals and change across time, with implications for prognosis and treatment. Key outstanding challenges are to define the distinct behavioural characteristics of this disorder and develop diagnostic paradigms that can assess these sensitively in individuals. In a previous study, we measured different aspects of attentional control in Parkinson's disease using an established fMRI switching paradigm. We observed no deficits for the aspects of attention the task was designed to examine; instead those with Parkinson's disease learnt the operational requirements of the task more slowly. We hypothesized that a subset of people with early-to-mid stage Parkinson's might be impaired when encoding rules for performing new tasks. Here, we directly test this hypothesis and investigate whether deficits in instruction-based learning represent a characteristic of Parkinson's Disease. Seventeen participants with Parkinson's disease (8 male; mean age: 61.2 years), 18 older adults (8 male; mean age: 61.3 years) and 20 younger adults (10 males; mean age: 26.7 years) undertook a simple instruction-based learning paradigm in the MRI scanner. They sorted sequences of coloured shapes according to binary discrimination rules that were updated at two-minute intervals. Unlike common reinforcement learning tasks, the rules were unambiguous, being explicitly presented; consequently, there was no requirement to monitor feedback or estimate contingencies. Despite its simplicity, a third of the Parkinson's group, but only one older adult, showed marked increases in errors, 4 SD greater than the worst performing young adult. The pattern of errors was consistent, reflecting a tendency to misbind discrimination rules. The misbinding behaviour was coupled with reduced frontal, parietal and anterior caudate activity when rules were being encoded, but not when attention was initially oriented to the instruction slides or when discrimination trials were performed. Concomitantly, Magnetic Resonance Spectroscopy showed reduced gamma-Aminobutyric acid levels within the mid-dorsolateral prefrontal cortices of individuals who made misbinding errors. These results demonstrate, for the first time, that a subset of early-to-mid stage people with Parkinson's show substantial deficits when binding new task rules in working memory. Given the ubiquity of instruction-based learning, these deficits are likely to impede daily living. They will also confound clinical assessment of other cognitive processes. Future work should determine the value of instruction-based learning as a sensitive early marker of cognitive decline and as a measure of responsiveness to therapy in Parkinson's disease.

The measurement, evolution, and neural representation of action grammars of human behavior

Human behaviors from toolmaking to language are thought to rely on a uniquely evolved capacity for hierarchical action sequencing. Testing this idea will require objective, generalizable methods for measuring the structural complexity of real-world behavior. Here we present a data-driven approach for extracting action grammars from basic ethograms, exemplified with respect to the evolutionarily relevant behavior of stone toolmaking. We analyzed sequences from the experimental replication of ~ 2.5 Mya Oldowan vs. ~ 0.5 Mya Acheulean tools, finding that, while using the same "alphabet" of elementary actions, Acheulean sequences are quantifiably more complex and Oldowan grammars are a subset of Acheulean grammars. We illustrate the utility of our complexity measures by re-analyzing data from an fMRI study of stone toolmaking to identify brain responses to structural complexity. Beyond specific implications regarding the co-evolution of language and technology, this exercise illustrates the general applicability of our method to investigate naturalistic human behavior and cognition.

Embodied virtual reality for the study of real-world motor learning

Motor-learning literature focuses on simple laboratory-tasks due to their controlled manner and the ease to apply manipulations to induce learning and adaptation. Recently, we introduced a billiards paradigm and demonstrated the feasibility of real-world-neuroscience using wearables for naturalistic full-body motion-tracking and mobile-brain-imaging. Here we developed an embodied virtual-reality (VR) environment to our real-world billiards paradigm, which allows to control the visual feedback for this complex real-world task, while maintaining sense of embodiment. The setup was validated by comparing real-world ball trajectories with the trajectories of the virtual balls, calculated by the physics engine. We then ran our short-term motor learning protocol in the embodied VR. Subjects played billiard shots when they held the physical cue and hit a physical ball on the table while seeing it all in VR. We found comparable short-term motor learning trends in the embodied VR to those we previously reported in the physical real-world task. Embodied VR can be used for learning real-world tasks in a highly controlled environment which enables applying visual manipulations, common in laboratory-tasks and rehabilitation, to a real-world full-body task. Embodied VR enables to manipulate feedback and apply perturbations to isolate and assess interactions between specific motor-learning components, thus enabling addressing the current questions of motor-learning in real-world tasks. Such a setup can potentially be used for rehabilitation, where VR is gaining popularity but the transfer to the real-world is currently limited, presumably, due to the lack of embodiment.

Accuracy and Acceptability of Wearable Motion Tracking for Inpatient Monitoring Using Smartwatches

Inertial Measurement Units (IMUs) within an everyday consumer smartwatch offer a convenient and low-cost method to monitor the natural behaviour of hospital patients. However, their accuracy at quantifying limb motion, and clinical acceptability, have not yet been demonstrated. To this end we conducted a two-stage study: First, we compared the inertial accuracy of wrist-worn IMUs, both research-grade (Xsens MTw Awinda, and Axivity AX3) and consumer-grade (Apple Watch Series 3 and 5), and optical motion tracking (OptiTrack). Given the moderate to strong performance of the consumer-grade sensors, we then evaluated this sensor and surveyed the experiences and attitudes of hospital patients (N = 44) and staff (N = 15) following a clinical test in which patients wore smartwatches for 1.5–24 h in the second study. Results indicate that for acceleration, Xsens is more accurate than the Apple Series 5 and 3 smartwatches and Axivity AX3 (RMSE 1.66 ± 0.12 m·s⁻²; R² 0.78 ± 0.02; RMSE 2.29 ± 0.09 m·s⁻²; R² 0.56 ± 0.01; RMSE 2.14 ± 0.09 m·s⁻²; R² 0.49 ± 0.02; RMSE 4.12 ± 0.18 m·s⁻²; R² 0.34 ± 0.01 respectively). For angular velocity, Series 5 and 3 smartwatches achieved similar performances against Xsens with RMSE 0.22 ± 0.02 rad·s⁻¹; R² 0.99 ± 0.00; and RMSE 0.18 ± 0.01 rad·s⁻¹; R² 1.00± SE 0.00, respectively. Surveys indicated that in-patients and healthcare professionals strongly agreed that wearable motion sensors are easy to use, comfortable, unobtrusive, suitable for long-term use, and do not cause anxiety or limit daily activities. Our results suggest that consumer smartwatches achieved moderate to strong levels of accuracy compared to laboratory gold-standard and are acceptable for pervasive monitoring of motion/behaviour within hospital settings.

Neuroinflammation in the normal-appearing white matter (NAWM) of the multiple sclerosis brain causes abnormalities at the nodes of Ranvier

Changes to the structure of nodes of Ranvier in the normal-appearing white matter (NAWM) of multiple sclerosis (MS) brains are associated with chronic inflammation. We show that the paranodal domains in MS NAWM are longer on average than control, with Kv1.2 channels dislocated into the paranode. These pathological features are reproduced in a model of chronic meningeal inflammation generated by the injection of lentiviral vectors for the lymphotoxin-α (LTα) and interferon-γ (IFNγ) genes. We show that tumour necrosis factor (TNF), IFNγ, and glutamate can provoke paranodal elongation in cerebellar slice cultures, which could be reversed by an N-methyl-D-aspartate (NMDA) receptor blocker. When these changes were inserted into a computational model to simulate axonal conduction, a rapid decrease in velocity was observed, reaching conduction failure in small diameter axons. We suggest that glial cells activated by pro-inflammatory cytokines can produce high levels of glutamate, which triggers paranodal pathology, contributing to axonal damage and conduction deficits.

Current thinking on the mechanisms by which multiple sclerosis gives rise to cumulative neurological disability revolves largely around focal lesions of inflammation and demyelination. However, some of the debilitating symptoms, such as severe fatigue, might be better explained by a more diffuse pathology. This study shows that paranodes in the white matter become abnormal as a result of neuroinflammation, which may be the result of the action of cytokines that cause glia to release glutamate.

Motor learning in real-world pool billiards

The neurobehavioral mechanisms of human motor-control and learning evolved in free behaving, real-life settings, yet this is studied mostly in reductionistic lab-based experiments. Here we take a step towards a more real-world motor neuroscience using wearables for naturalistic full-body motion-tracking and the sports of pool billiards to frame a real-world skill learning experiment. First, we asked if well-known features of motor learning in lab-based experiments generalize to a real-world task. We found similarities in many features such as multiple learning rates, and the relationship between task-related variability and motor learning. Our data-driven approach reveals the structure and complexity of movement, variability, and motor learning, enabling an in-depth understanding of the structure of motor learning in three ways: First, while expecting most of the movement learning is done by the cue-wielding arm, we find that motor learning affects the whole body, changing motor-control from head to toe. Second, during learning, all subjects decreased their movement variability and their variability in the outcome. Subjects who were initially more variable were also more variable after learning. Lastly, when screening the link across subjects between initial variability in individual joints and learning, we found that only the initial variability in the right forearm supination shows a significant correlation to the subjects' learning rates. This is in-line with the relationship between learning and variability: while learning leads to an overall reduction in movement variability, only initial variability in specific task-relevant dimensions can facilitate faster learning.

Brain Activity Reveals Multiple Motor-Learning Mechanisms in a Real-World Task

Many recent studies found signatures of motor learning in neural beta oscillations (13-30 Hz), and specifically in the post-movement beta rebound (PMBR). All these studies were in controlled laboratory-tasks in which the task designed to induce the studied learning mechanism. Interestingly, these studies reported opposing dynamics of the PMBR magnitude over learning for the error-based and reward-based tasks (increase vs. decrease, respectively). Here, we explored the PMBR dynamics during real-world motor-skill-learning in a billiards task using mobile-brain-imaging. Our EEG recordings highlight the opposing dynamics of PMBR magnitudes (increase vs. decrease) between different subjects performing the same task. The groups of subjects, defined by their neural dynamics, also showed behavioral differences expected for different learning mechanisms. Our results suggest that when faced with the complexity of the real-world different subjects might use different learning mechanisms for the same complex task. We speculate that all subjects combine multi-modal mechanisms of learning, but different subjects have different predominant learning mechanisms.

Neurobehavioural signatures in race car driving: a case study

Recent technological developments in mobile brain and body imaging are enabling new frontiers of real-world neuroscience. Simultaneous recordings of body movement and brain activity from highly skilled individuals as they demonstrate their exceptional skills in real-world settings, can shed new light on the neurobehavioural structure of human expertise. Driving is a real-world skill which many of us acquire to different levels of expertise. Here we ran a case-study on a subject with the highest level of driving expertise-a Formula E Champion. We studied the driver's neural and motor patterns while he drove a sports car on the "Top Gear" race track under extreme conditions (high speed, low visibility, low temperature, wet track). His brain activity, eye movements and hand/foot movements were recorded. Brain activity in the delta, alpha, and beta frequency bands showed causal relation to hand movements. We herein demonstrate the feasibility of using mobile brain and body imaging even in very extreme conditions (race car driving) to study the sensory inputs, motor outputs, and brain states which characterise complex human skills.

A.Eye Drive: Gaze-based semi-autonomous wheelchair interface

Existing wheelchair control interfaces, such as sip & puff or screen based gaze-controlled cursors, are challenging for the severely disabled to navigate safely and independently as users continuously need to interact with an interface during navigation. This puts a significant cognitive load on users and prevents them from interacting with the environment in other forms during navigation. We have combined eyetracking/gaze-contingent intention decoding with computer vision context-aware algorithms and autonomous navigation drawn from self-driving vehicles to allow paralysed users to drive by eye, simply by decoding natural gaze about where the user wants to go: A.Eye Drive. Our "Zero UI" driving platform allows users to look and interact visually with at an object or destination of interest in their visual scene, and the wheelchair autonomously takes the user to the intended destination, while continuously updating the computed path for static and dynamic obstacles. This intention decoding technology empowers the end-user by promising more independence through their own agency.

"Wink to grasp" - comparing eye, voice & EMG gesture control of grasp with soft-robotic gloves

The ability of robotic rehabilitation devices to support paralysed end-users is ultimately limited by the degree to which human-machine-interaction is designed to be effective and efficient in translating user intention into robotic action. Specifically, we evaluate the novel possibility of binocular eye-tracking technology to detect voluntary winks from involuntary blink commands, to establish winks as a novel low-latency control signal to trigger robotic action. By wearing binocular eye-tracking glasses we enable users to directly observe their environment or the actuator and trigger movement actions, without having to interact with a visual display unit or user interface. We compare our novel approach to two conventional approaches for controlling robotic devices based on electromyo-graphy (EMG) and speech-based human-computer interaction technology. We present an integrated software framework based on ROS that allows transparent integration of these multiple modalities with a robotic system. We use a soft-robotic SEM glove (Bioservo Technologies AB, Sweden) to evaluate how the 3 modalities support the performance and subjective experience of the end-user when movement assisted. All 3 modalities are evaluated in streaming, closed-loop control operation for grasping physical objects. We find that wink control shows the lowest error rate mean with lowest standard deviation of (0.23 ± 0.07, mean ± SEM) followed by speech control (0.35 ± 0. 13) and EMG gesture control (using the Myo armband by Thalamic Labs), with the highest mean and standard deviation (0.46 ± 0.16). We conclude that with our novel own developed eye-tracking based approach to control assistive technologies is a well suited alternative to conventional approaches, especially when combined with 3D eye-tracking based robotic end-point control.

Towards free 3D end-point control for robotic-assisted human reaching using binocular eye tracking

Eye-movements are the only directly observable behavioural signals that are highly correlated with actions at the task level, and proactive of body movements and thus reflect action intentions. Moreover, eye movements are preserved in many movement disorders leading to paralysis (or amputees) from stroke, spinal cord injury, Parkinson's disease, multiple sclerosis, and muscular dystrophy among others. Despite this benefit, eye tracking is not widely used as control interface for robotic interfaces in movement impaired patients due to poor human-robot interfaces. We demonstrate here how combining 3D gaze tracking using our GT3D binocular eye tracker with custom designed 3D head tracking system and calibration method enables continuous 3D end-point control of a robotic arm support system. The users can move their own hand to any location of the workspace by simple looking at the target and winking once. This purely eye tracking based system enables the end-user to retain free head movement and yet achieves high spatial end point accuracy in the order of 6 cm RMSE error in each dimension and standard deviation of 4 cm. 3D calibration is achieved by moving the robot along a 3 dimensional space filling Peano curve while the user is tracking it with their eyes. This results in a fully automated calibration procedure that yields several thousand calibration points versus standard approaches using a dozen points, resulting in beyond state-of-the-art 3D accuracy and precision.

Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control With Natural Hand Kinematics

Matching the dexterity, versatility, and robustness of the human hand is still an unachieved goal in bionics, robotics, and neural engineering. A major limitation for hand prosthetics lies in the challenges of reliably decoding user intention from muscle signals when controlling complex robotic hands. Most of the commercially available prosthetic hands use muscle-related signals to decode a finite number of predefined motions and some offer proportional control of open/close movements of the whole hand. Here, in contrast, we aim to offer users flexible control of individual joints of their artificial hand. We propose a novel framework for decoding neural information that enables a user to independently control 11 joints of the hand in a continuous manner-much like we control our natural hands. Toward this end, we instructed six able-bodied subjects to perform everyday object manipulation tasks combining both dynamic, free movements (e.g., grasping) and isometric force tasks (e.g., squeezing). We recorded the electromyographic and mechanomyographic activities of five extrinsic muscles of the hand in the forearm, while simultaneously monitoring 11 joints of hand and fingers using a sensorized data glove that tracked the joints of the hand. Instead of learning just a direct mapping from current muscle activity to intended hand movement, we formulated a novel autoregressive approach that combines the context of previous hand movements with instantaneous muscle activity to predict future hand movements. Specifically, we evaluated a linear vector autoregressive moving average model with exogenous inputs and a novel Gaussian process ( ) autoregressive framework to learn the continuous mapping from hand joint dynamics and muscle activity to decode intended hand movement. Our approach achieves high levels of performance (RMSE of 8°/s and ). Crucially, we use a small set of sensors that allows us to control a larger set of independently actuated degrees of freedom of a hand. This novel undersensored control is enabled through the combination of nonlinear autoregressive continuous mapping between muscle activity and joint angles. The system evaluates the muscle signals in the context of previous natural hand movements. This enables us to resolve ambiguities in situations, where muscle signals alone cannot determine the correct action as we evaluate the muscle signals in their context of natural hand movements. autoregression is a particularly powerful approach which makes not only a prediction based on the context but also represents the associated uncertainty of its predictions, thus enabling the novel notion of risk-based control in neuroprosthetics. Our results suggest that autoregressive approaches with exogenous inputs lend themselves for natural, intuitive, and continuous control in neurotechnology, with the particular focus on prosthetic restoration of natural limb function, where high dexterity is required for complex movements.

Robotic psychophysics system for assessment, diagnosis and rehabilitation of the neurological causes of falls in the elderly

Falls are the leading causes of unintentional injuries in the elderly and thus a pose a major hazard to our ageing society. We present the FOHEPO (FOot HEight POsitioning) system to measure, diagnose and eventually rehabilitate ageing-related neurological causes of falls. We hypothesise that both perceptual and motor variability is likely to increase with age and may lead to imprecise perception and movements causing trip overs, the major triggers of falls. Our robotic experimental system automatically measures and tracks different sources of noise in the nervous system: visual perception noise of obstacle height, proprioceptive noise of localising raising one's foot to a desired height, noise in the visual feedback of the foot movements. We developed age-appropriate psychophysical measurement protocols shorter than standard protocols for perceptual and motor accuracy. These quantify individual subjects perceptual and movement accuracy thresholds through their psychometric curves. Therefore, these platform measurements will enable us to estimate fall probabilities quantitatively, i.e. the chance that a foot will clip an obstacle because subjects did not add a sufficient safety factor when clearing it. Potentially, we can use our FOHEPO system in a game-ified setting to rehabilitate elderly users to move with larger safety factors so as to reduce their risks of trip-over.

Mind meets machine Malcolm Gay Farrar, Straus and Giroux, 2015. 275 pp

High-profile researchers take center stage in a quest to build a brain-machine interface.

Decoding of human hand actions to handle missing limbs in neuroprosthetics

The only way we can interact with the world is through movements, and our primary interactions are via the hands, thus any loss of hand function has immediate impact on our quality of life. However, to date it has not been systematically assessed how coordination in the hand's joints affects every day actions. This is important for two fundamental reasons. Firstly, to understand the representations and computations underlying motor control "in-the-wild" situations, and secondly to develop smarter controllers for prosthetic hands that have the same functionality as natural limbs. In this work we exploit the correlation structure of our hand and finger movements in daily-life. The novelty of our idea is that instead of averaging variability out, we take the view that the structure of variability may contain valuable information about the task being performed. We asked seven subjects to interact in 17 daily-life situations, and quantified behavior in a principled manner using CyberGlove body sensor networks that, after accurate calibration, track all major joints of the hand. Our key findings are: (1) We confirmed that hand control in daily-life tasks is very low-dimensional, with four to five dimensions being sufficient to explain 80-90% of the variability in the natural movement data. (2) We established a universally applicable measure of manipulative complexity that allowed us to measure and compare limb movements across tasks. We used Bayesian latent variable models to model the low-dimensional structure of finger joint angles in natural actions. (3) This allowed us to build a naïve classifier that within the first 1000 ms of action initiation (from a flat hand start configuration) predicted which of the 17 actions was going to be executed-enabling us to reliably predict the action intention from very short-time-scale initial data, further revealing the foreseeable nature of hand movements for control of neuroprosthetics and tele operation purposes. (4) Using the Expectation-Maximization algorithm on our latent variable model permitted us to reconstruct with high accuracy (<5-6° MAE) the movement trajectory of missing fingers by simply tracking the remaining fingers. Overall, our results suggest the hypothesis that specific hand actions are orchestrated by the brain in such a way that in the natural tasks of daily-life there is sufficient redundancy and predictability to be directly exploitable for neuroprosthetics.

Saltatory conduction in unmyelinated axons: clustering of Na(+) channels on lipid rafts enables micro-saltatory conduction in C-fibers

THE ACTION POTENTIAL (AP), THE FUNDAMENTAL SIGNAL OF THE NERVOUS SYSTEM, IS CARRIED BY TWO TYPES OF AXONS: unmyelinated and myelinated fibers. In the former the action potential propagates continuously along the axon as established in large-diameter fibers. In the latter axons the AP jumps along the nodes of Ranvier-discrete, anatomically specialized regions which contain very high densities of sodium ion (Na(+)) channels. Therefore, saltatory conduction is thought as the hallmark of myelinated axons, which enables faster and more reliable propagation of signals than in unmyelinated axons of same outer diameter. Recent molecular anatomy showed that in C-fibers, the very thin (0.1 μm diameter) axons of the peripheral nervous system, Nav1.8 channels are clustered together on lipid rafts that float in the cell membrane. This localized concentration of Na(+) channels resembles in structure the ion channel organization at the nodes of Ranvier, yet it is currently unknown whether this translates into an equivalent phenomenon of saltatory conduction or related-functional benefits and efficiencies. Therefore, we modeled biophysically realistic unmyelinated axons with both conventional and lipid-raft based organization of Na(+) channels. We find that APs are reliably conducted in a micro-saltatory fashion along lipid rafts. Comparing APs in unmyelinated fibers with and without lipid rafts did not reveal any significant difference in either the metabolic cost or AP propagation velocity. By investigating the efficiency of AP propagation over Nav1.8 channels, we find however that the specific inactivation properties of these channels significantly increase the metabolic cost of signaling in C-fibers.

Axonal noise as a source of synaptic variability

Post-synaptic potential (PSP) variability is typically attributed to mechanisms inside synapses, yet recent advances in experimental methods and biophysical understanding have led us to reconsider the role of axons as highly reliable transmission channels. We show that in many thin axons of our brain, the action potential (AP) waveform and thus the Ca++ signal controlling vesicle release at synapses will be significantly affected by the inherent variability of ion channel gating. We investigate how and to what extent fluctuations in the AP waveform explain observed PSP variability. Using both biophysical theory and stochastic simulations of central and peripheral nervous system axons from vertebrates and invertebrates, we show that channel noise in thin axons (<1 µm diameter) causes random fluctuations in AP waveforms. AP height and width, both experimentally characterised parameters of post-synaptic response amplitude, vary e.g. by up to 20 mV and 0.5 ms while a single AP propagates in C-fibre axons. We show how AP height and width variabilities increase with a ¾ power-law as diameter decreases and translate these fluctuations into post-synaptic response variability using biophysical data and models of synaptic transmission. We find for example that for mammalian unmyelinated axons with 0.2 µm diameter (matching cerebellar parallel fibres) axonal noise alone can explain half of the PSP variability in cerebellar synapses. We conclude that axonal variability may have considerable impact on synaptic response variability. Thus, in many experimental frameworks investigating synaptic transmission through paired-cell recordings or extracellular stimulation of presynaptic neurons, causes of variability may have been confounded. We thereby show how bottom-up aggregation of molecular noise sources contributes to our understanding of variability observed at higher levels of biological organisation.

Near optimal combination of sensory and motor uncertainty in time during a naturalistic perception-action task

Most behavioral tasks have time constraints for successful completion, such as catching a ball in flight. Many of these tasks require trading off the time allocated to perception and action, especially when only one of the two is possible at any time. In general, the longer we perceive, the smaller the uncertainty in perceptual estimates. However, a longer perception phase leaves less time for action, which results in less precise movements. Here we examine subjects catching a virtual ball. Critically, as soon as subjects began to move, the ball became invisible. We study how subjects trade-off sensory and movement uncertainty by deciding when to initiate their actions. We formulate this task in a probabilistic framework and show that subjects' decisions when to start moving are statistically near optimal given their individual sensory and motor uncertainties. Moreover, we accurately predict individual subject's task performance. Thus we show that subjects in a natural task are quantitatively aware of how sensory and motor variability depend on time and act so as to minimize overall task variability.

Stochastic simulations on the reliability of action potential propagation in thin axons

It is generally assumed that axons use action potentials (APs) to transmit information fast and reliably to synapses. Yet, the reliability of transmission along fibers below 0.5 microm diameter, such as cortical and cerebellar axons, is unknown. Using detailed models of rodent cortical and squid axons and stochastic simulations, we show how conduction along such thin axons is affected by the probabilistic nature of voltage-gated ion channels (channel noise). We identify four distinct effects that corrupt propagating spike trains in thin axons: spikes were added, deleted, jittered, or split into groups depending upon the temporal pattern of spikes. Additional APs may appear spontaneously; however, APs in general seldom fail (<1%). Spike timing is jittered on the order of milliseconds over distances of millimeters, as conduction velocity fluctuates in two ways. First, variability in the number of Na channels opening in the early rising phase of the AP cause propagation speed to fluctuate gradually. Second, a novel mode of AP propagation (stochastic microsaltatory conduction), where the AP leaps ahead toward spontaneously formed clusters of open Na channels, produces random discrete jumps in spike time reliability. The combined effect of these two mechanisms depends on the pattern of spikes. Our results show that axonal variability is a general problem and should be taken into account when considering both neural coding and the reliability of synaptic transmission in densely connected cortical networks, where small synapses are typically innervated by thin axons. In contrast we find that thicker axons above 0.5 microm diameter are reliable.

Ion-Channel Noise Places Limits on the Miniaturization of the Brain’s Wiring

The action potential (AP) is transmitted by the concerted action of voltage-gated ion channels. Thermodynamic fluctuations in channel proteins produce probabilistic gating behavior, causing channel noise. Miniaturizing signaling systems increases susceptibility to noise, and with many cortical, cerebellar, and peripheral axons <0.5 mum diameter [1, 2 and 3], channel noise could be significant [4 and 5]. Using biophysical theory and stochastic simulations, we investigated channel-noise limits in unmyelinated axons. Axons of diameter below 0.1 microm become inoperable because single, spontaneously opening Na channels generate spontaneous AP at rates that disrupt communication. This limiting diameter is relatively insensitive to variations in biophysical parameters (e.g., channel properties and density, membrane conductance and leak) and will apply to most spiking axons. We demonstrate that the essential molecular machinery can, in theory, fit into 0.06 microm diameter axons. However, a comprehensive survey of anatomical data shows a lower limit for AP-conducting axons of 0.08-0.1 microm diameter. Thus, molecular fluctuations constrain the wiring density of brains. Fluctuations have implications for epilepsy and neuropathic pain because changes in channel kinetics or axonal properties can change the rate at which channel noise generates spontaneous activity.

Coordinated righting behaviour in locusts

A locust placed upside down on a flat surface uses a predictable sequence of leg movements to right itself. To analyse this behaviour, we made use of a naturally occurring state of quiescence (thanatosis) to position locusts in a standardised upside-down position from which they spontaneously right themselves. Locusts grasped around the pronotum enter a state of thanatosis during which the limbs can be manipulated into particular postures, where they remain, and the animal can be placed upside down on the ground. When released, thanatosis lasts 4–456 s (mean 73 s) before the animal suddenly becomes active again and rights itself within a further 600 ms. Thanatosis is characterised by very low levels of leg motor activity. During righting, one hind leg provides most of the downward force against the ground that rolls the body around a longitudinal axis towards the other side. The driving force is produced by femoral levation (relative to the body) at the trochanter and by tibial extension. As the animal rolls over, the hind leg on the other side is also levated at the trochanter, so that it does not obstruct the movement. The forelegs and middle legs are not required for successful righting but they can help initially to tip the locust to one side, and at the end of the movement they help stop the roll as the animal turns upright. Individual locusts have a preferred righting direction but can, nevertheless, roll to either side. Locusts falling upside down through the air use both passive and active mechanisms to right themselves before they land. Without active movements, falling locusts tend to rotate into an upright position, but most locusts extend their hind leg tibiae and/or spread their wings, which increases the success of mid-air righting from 28 to 49 % when falling from 30 cm. The rapid and reliable righting behaviour of locusts reduces the time spent in a vulnerable upside-down position. Their narrow body geometry, large hind legs, which can generate substantial dorsally directed force, and the particular patterns of coordinated movements of the legs on both sides of the body are the key features that permit locusts to right themselves effectively. The reliability of autonomous multi-legged robots may be enhanced by incorporating these features into their design.