NeuroKinect: A Novel Low-Cost 3Dvideo-EEG System for Epileptic Seizure Motion Quantification

Deep learning approaches for seizure video analysis: A review

Seizure events can manifest as transient disruptions in the control of movements which may be organized in distinct behavioral sequences, accompanied or not by other observable features such as altered facial expressions. The analysis of these clinical signs, referred to as semiology, is subject to observer variations when specialists evaluate video-recorded events in the clinical setting. To enhance the accuracy and consistency of evaluations, computer-aided video analysis of seizures has emerged as a natural avenue. In the field of medical applications, deep learning and computer vision approaches have driven substantial advancements. Historically, these approaches have been used for disease detection, classification, and prediction using diagnostic data; however, there has been limited exploration of their application in evaluating video-based motion detection in the clinical epileptology setting. While vision-based technologies do not aim to replace clinical expertise, they can significantly contribute to medical decision-making and patient care by providing quantitative evidence and decision support. Behavior monitoring tools offer several advantages such as providing objective information, detecting challenging-to-observe events, reducing documentation efforts, and extending assessment capabilities to areas with limited expertise. The main applications of these could be (1) improved seizure detection methods; (2) refined semiology analysis for predicting seizure type and cerebral localization. In this paper, we detail the foundation technologies used in vision-based systems in the analysis of seizure videos, highlighting their success in semiology detection and analysis, focusing on work published in the last 7 years. We systematically present these methods and indicate how the adoption of deep learning for the analysis of video recordings of seizures could be approached. Additionally, we illustrate how existing technologies can be interconnected through an integrated system for video-based semiology analysis. Each module can be customized and improved by adapting more accurate and robust deep learning approaches as these evolve. Finally, we discuss challenges and research directions for future studies.

Computer vision for automated seizure detection and classification: A systematic review

Computer vision (CV) shows increasing promise as an efficient, low-cost tool for video seizure detection and classification. Here, we provide an overview of the fundamental concepts needed to understand CV and summarize the structure and performance of various model architectures used in video seizure analysis. We conduct a systematic literature review of the PubMed, Embase, and Web of Science databases from January 1, 2000 to September 15, 2023, to identify the strengths and limitations of CV seizure analysis methods and discuss the utility of these models when applied to different clinical seizure phenotypes. Reviews, nonhuman studies, and those with insufficient or poor quality data are excluded from the review. Of the 1942 records identified, 45 meet inclusion criteria and are analyzed. We conclude that the field has shown tremendous growth over the past 2 decades, leading to several model architectures with impressive accuracy and efficiency. The rapid and scalable detection offered by CV models holds the potential to reduce sudden unexpected death in epilepsy and help alleviate resource limitations in epilepsy monitoring units. However, a lack of standardized, thorough validation measures and concerns about patient privacy remain important obstacles for widespread acceptance and adoption. Investigation into the performance of models across varied datasets from clinical and nonclinical environments is an essential area for further research.

The history of motion photography to video electroencephalography in the study of functional seizures and related seizure disorders: The first 100 years

This historical note highlights pivotal events of technology progressing between the late 19th and the 20th century to capture functional seizures and other related seizure episodes. From Charcot's initial use of photography for his study of hysteria at the Salpêtrière to the development of cinematography by Muybridge and Marey to study motion to the initial use of video electroencephalography (vEEG) through a pairing of cinematography with EEG, and the advent of EEG telemetry to eventually the development of modern epilepsy monitoring unit through the adoption of cameras and an improved long-term monitoring vEEG system.

Anatomy-guided domain adaptation for 3D in-bed human pose estimation

3D human pose estimation is a key component of clinical monitoring systems. The clinical applicability of deep pose estimation models, however, is limited by their poor generalization under domain shifts along with their need for sufficient labeled training data. As a remedy, we present a novel domain adaptation method, adapting a model from a labeled source to a shifted unlabeled target domain. Our method comprises two complementary adaptation strategies based on prior knowledge about human anatomy. First, we guide the learning process in the target domain by constraining predictions to the space of anatomically plausible poses. To this end, we embed the prior knowledge into an anatomical loss function that penalizes asymmetric limb lengths, implausible bone lengths, and implausible joint angles. Second, we propose to filter pseudo labels for self-training according to their anatomical plausibility and incorporate the concept into the Mean Teacher paradigm. We unify both strategies in a point cloud-based framework applicable to unsupervised and source-free domain adaptation. Evaluation is performed for in-bed pose estimation under two adaptation scenarios, using the public SLP dataset and a newly created dataset. Our method consistently outperforms various state-of-the-art domain adaptation methods, surpasses the baseline model by 31%/66%, and reduces the domain gap by 65%/82%. Source code is available at https://github.com/multimodallearning/da-3dhpe-anatomy.

Novel 3D video action recognition deep learning approach for near real time epileptic seizure classification

Seizure semiology is a well-established method to classify epileptic seizure types, but requires a significant amount of resources as long-term Video-EEG monitoring needs to be visually analyzed. Therefore, computer vision based diagnosis support tools are a promising approach. In this article, we utilize infrared (IR) and depth (3D) videos to show the feasibility of a 24/7 novel object and action recognition based deep learning (DL) monitoring system to differentiate between epileptic seizures in frontal lobe epilepsy (FLE), temporal lobe epilepsy (TLE) and non-epileptic events. Based on the largest 3Dvideo-EEG database in the world (115 seizures/+680,000 video-frames/427GB), we achieved a promising cross-subject validation f1-score of 0.833±0.061 for the 2 class (FLE vs. TLE) and 0.763 ± 0.083 for the 3 class (FLE vs. TLE vs. non-epileptic) case, from 2 s samples, with an automated semi-specialized depth (Acc.95.65%) and Mask R-CNN (Acc.96.52%) based cropping pipeline to pre-process the videos, enabling a near-real-time seizure type detection and classification tool. Our results demonstrate the feasibility of our novel DL approach to support 24/7 epilepsy monitoring, outperforming all previously published methods.

Feasibility of video/audio monitoring in the analysis of motion and treatment effects on night-time seizures - Interventional study

THE AIM OF THE STUDY

This pilot study assessed the ability of a video/audio-based seizure monitoring system to evaluate (I) baseline frequency and severity of nocturnal seizures with motor features in patients with drug-resistant epilepsy (DRE) and (II) the individual effect of brivaracetam (BRV) treatment on number, duration and movement intensity of these seizure types. Algorithmic feature analysis was developed for assessment of qualitative changes in movement intensity measurements within seizure types before and after BRV intervention.

MATERIALS AND METHODS

Night-time motor seizures of recruited patients were recorded in two separate four-week monitoring periods. The first period defined a prescreening phase (n = 13 patients) to establish a baseline, and the second period defined the intervention phase (n = 9 patients), with BRV initiated during the second week of the second monitoring period. All recorded nights were analyzed by an expert video reviewer, and all unequivocal seizures were classified by an epileptologist. Seizure frequencies using both seizure diaries and video monitoring were compared. The effect of BRV on both seizure duration and movement intensity was assessed by numerical comparison of visual features calculated from motion characteristics of the video, as well as spectral features from the recorded audio. The statistical significance of changes in seizure duration and intensity before and after the intervention were investigated by Wilcoxon rank-sum test and visual inspection of Kernel density estimation.

RESULTS

8 patients marked seizures in their seizure diaries during the prescreening phase. During the three-week follow-up, three patients achieved > 50% seizure decrease, four patients did not respond to treatment, and two patients experienced worsening of seizures. Five patients were able to document 40-70% of their seizures compared to the video/audio monitoring system. According to the signal feature analysis the intervention decreased movement intensity with clear clinical significance in three patients, whereas statistically significant differences in features appeared in 8 out of 9 patients.

CONCLUSIONS

The novel video/audio monitoring system improved the evaluation of treatment effect compared to the seizure diaries and succeeded in providing a comparative intra-patient assessment of the movement intensity and duration of the recorded seizures.

Healthcare applications of single camera markerless motion capture: a scoping review

Background

Single camera markerless motion capture has the potential to facilitate at home movement assessment due to the ease of setup, portability, and affordable cost of the technology. However, it is not clear what the current healthcare applications of single camera markerless motion capture are and what information is being collected that may be used to inform clinical decision making. This review aims to map the available literature to highlight potential use cases and identify the limitations of the technology for clinicians and researchers interested in the collection of movement data.

Survey Methodology

Studies were collected up to 14 January 2022 using Pubmed, CINAHL and SPORTDiscus using a systematic search. Data recorded included the description of the markerless system, clinical outcome measures, and biomechanical data mapped to the International Classification of Functioning, Disability and Health Framework (ICF). Studies were grouped by patient population.

Results

A total of 50 studies were included for data collection. Use cases for single camera markerless motion capture technology were identified for Neurological Injury in Children and Adults; Hereditary/Genetic Neuromuscular Disorders; Frailty; and Orthopaedic or Musculoskeletal groups. Single camera markerless systems were found to perform well in studies involving single plane measurements, such as in the analysis of infant general movements or spatiotemporal parameters of gait, when evaluated against 3D marker-based systems and a variety of clinical outcome measures. However, they were less capable than marker-based systems in studies requiring the tracking of detailed 3D kinematics or fine movements such as finger tracking.

Conclusions

Single camera markerless motion capture offers great potential for extending the scope of movement analysis outside of laboratory settings in a practical way, but currently suffers from a lack of accuracy where detailed 3D kinematics are required for clinical decision making. Future work should therefore focus on improving tracking accuracy of movements that are out of plane relative to the camera orientation or affected by occlusion, such as supination and pronation of the forearm.

Optical Flow Estimation Improves Automated Seizure Detection in Neonatal EEG

PURPOSE

Existing automated seizure detection algorithms report sensitivities between 43% and 77% and specificities between 56% and 90%. The algorithms suffer from false alarms when applied to neonatal EEG because of the high degree of nurse handling and rhythmic patting used to soothe neonates. Computer vision technology that quantifies movement in real time could distinguish artifactual motion and improve automated neonatal seizure detection algorithms.

METHODS

The authors used video EEG recordings from 43 neonates undergoing monitoring for seizures as part of the NEOLEV2 clinical trial. The Persyst neonatal automated seizure detection algorithm ran in real time during study EEG acquisitions. Computer vision algorithms were applied to extract detailed accounts of artifactual movement of the neonate or people near the neonate though dense optical flow estimation.

RESULTS

Using the methods mentioned above, 197 periods of patting activity were identified and quantified, of which 45 generated false-positive automated seizure detection events. A binary patting detection algorithm was trained with a subset of 470 event videos. This supervised detection algorithm was applied to a testing subset of 187 event videos with 8 false-positive events, which resulted in a 24% reduction in false-positive automated seizure detections and a 50% reduction in false-positive events caused by neonatal care patting, while maintaining 11 of 12 true-positive seizure detection events.

CONCLUSIONS

This work presents a novel approach to improving automated seizure detection algorithms used during neonatal video EEG monitoring. This artifact detection mechanism can improve the ability of a seizure detector algorithm to distinguish between artifact and true seizure activity.

Guidelines for Conducting Ethical Artificial Intelligence Research in Neurology: A Systematic Approach for Clinicians and Researchers

Pre-emptive recognition of the ethical implications of study design and algorithm choices in artificial intelligence (AI) research is an important but challenging process. AI applications have begun to transition from a promising future to clinical reality in neurology. As the clinical management of neurology is often concerned with discrete, often unpredictable, and highly consequential events linked to multimodal data streams over long timescales, forthcoming advances in AI have great potential to transform care for patients. However, critical ethical questions have been raised with implementation of the first AI applications in clinical practice. Clearly, AI will have far-reaching potential to promote, but also to endanger, ethical clinical practice. This article employs an anticipatory ethics approach to scrutinize how researchers in neurology can methodically identify ethical ramifications of design choices early in the research and development process, with a goal of pre-empting unintended consequences that may violate principles of ethical clinical care. First, we discuss the use of a systematic framework for researchers to identify ethical ramifications of various study design and algorithm choices. Second, using epilepsy as a paradigmatic example, anticipatory clinical scenarios that illustrate unintended ethical consequences are discussed, and failure points in each scenario evaluated. Third, we provide practical recommendations for understanding and addressing ethical ramifications early in methods development stages. Awareness of the ethical implications of study design and algorithm choices that may unintentionally enter AI is crucial to ensuring that incorporation of AI into neurology care leads to patient benefit rather than harm.

Supporting the Assessment of Hereditary Transthyretin Amyloidosis Patients Based On 3-D Gait Analysis and Machine Learning

Hereditary Transthyretin Amyloidosis (vATTR-V30M) is a rare and highly incapacitating sensorimotor neuropathy caused by an inherited mutation (Val30Met), which typically affects gait, among other symptoms. In this context, we investigated the possibility of using machine learning (ML) techniques to build a model(s) that can be used to support the detection of the Val30Met mutation (possibility of developing the disease), as well as symptom onset detection for the disease, given the gait characteristics of a person. These characteristics correspond to 24 gait parameters computed from 3-D body data, provided by a Kinect v2 camera, acquired from a person while walking towards the camera. To build the model(s), different ML algorithms were explored: k-nearest neighbors, decision tree, random forest, support vector machines (SVM), and multilayer perceptron. For a dataset corresponding to 66 subjects (25 healthy controls, 14 asymptomatic mutation carriers, and 27 patients) and several gait cycles per subject, we were able to obtain a model that distinguishes between controls and vATTR-V30M mutation carriers (with or without symptoms) with a mean accuracy of 92% (SVM). We also obtained a model that distinguishes between asymptomatic and symptomatic carriers with a mean accuracy of 98% (SVM). These results are very relevant, since this is the first study that proposes a ML approach to support vATTR-V30M patient assessment based on gait, being a promising foundation for the development of a computer-aided diagnosis tool to help clinicians in the identification and follow-up of this disease. Furthermore, the proposed method may also be used for other neuropathies.

On seizure semiology

The clinical expression of seizures represents the main symptomatic burden of epilepsy. Neural mechanisms of semiologic production in epilepsy, especially for complex behaviors, remain poorly known. In a framework of epilepsy as a network rather than as a focal disorder, we can think of semiology as being dynamically produced by a set of interconnected structures, in which specific rhythmic interactions, and not just anatomical localization, are likely to play an important part in clinical expression. This requires a paradigm shift in how we think about seizure organization, including from a presurgical evaluation perspective. Semiology is a key data source, albeit with significant methodological challenges for its use in research, including observer bias and choice of semiologic categories. Better understanding of semiologic categorization and pathophysiological correlates is relevant to seizure classification systems. Advances in knowledge of neural mechanisms as well as anatomic correlates of different semiologic patterns could help improve knowledge of epilepsy networks and potentially contribute to therapeutic innovations.

Monitoring the frequency and duration of epileptic seizures: "A journey through time"

Seizure monitoring plays an undeniably important role in diagnosing and managing epileptic seizures. Establishing the frequency and duration of seizures is crucial for assessing the burden of this chronic neurological disease, selecting treatment methods, determining how frequently these methods are applied, and informing short and long-term therapeutic decisions. Over the years, seizure monitoring tools and methods have evolved and become increasingly sophisticated; from home seizure diaries to EEG monitoring to cutting-edge responsive neurostimulation systems. In this article, the various methods of seizure monitoring are reviewed.

Machine Learning Techniques for Personalized Detection of Epileptic Events in Clinical Video Recordings

Continuous patient monitoring is essential to achieve an effective and optimal patient treatment in the intensive care unit. In the specific case of epilepsy it is the only way to achieve a correct diagnosis and a subsequent optimal medication plan if possible. In addition to automatic vital sign monitoring, epilepsy patients need manual monitoring by trained personnel, a task that is very difficult to be performed continuously for each patient. Moreover, epileptic manifestations are highly personalized even within the same type of epilepsy. In this work we assess two machine learning methods, dictionary learning and an autoencoder based on long short-term memory (LSTM) cells, on the task of personalized epileptic event detection in videos, with a set of features that were specifically developed with an emphasis on high motion sensitivity. According to the strengths of each method we have selected different types of epilepsy, one with convulsive behaviour and one with very subtle motion. The results on five clinical patients show a highly promising ability of both methods to detect the epileptic events as anomalies deviating from the stable/normal patient status.

An integrative method to quantitatively detect nocturnal motor seizures

In this proof-of-concept investigation, we demonstrate a marker-free video-based method to detect nocturnal motor seizures across a spectrum of motor seizure types, in a nighttime setting with a single adult female with refractory epilepsy. In doing so, we further explore the intermediate biosignals, visually mapping seizure "fingerprints" to seizure types. The method is designed to be flexible enough to generalize to unseen data, and shows promising performance characteristics for low-cost seizure detection and classification. The dataset contained recordings from 27 recorded nights. Seizure events were observed in 22 of these nights, with 36 unequivocally confirmed seizures. Each seizure was classified by an expert epileptologist according to both the ILAE 2017 standard and the Lüders semiological classification guidelines, yielding 5 of the ILAE-recognized seizure types and 7 distinct seizure semiologies. Evaluation was based on inference of motion, oscillation, and sound signals extracted from the recordings. The model architecture consisted of two feature extraction and event determination layers and one thresholding layer, establishing a simple framework for multimodal seizure analysis. Training of the optimal parameters was done by randomly resampling the event hits for each signal, and choosing a threshold that kept an expected 90 % sensitivity for the sample distribution. With the cut-off values selected, statistical performance was calculated for two target seizure groups: those containing a clonic component, and those containing a tonic component. When tuned to 90 % sensitivity, the system achieved a very low false discovery rate of 0.038/hour when targeting seizures with a clonic component, and a clinically-relevant rate of 1.02/hour when targeting seizures with a tonic component. These results indicate a sensitive method for detecting various nocturnal motor seizure types, and a high potential to differentiate motor seizures based on their video and audio signal characteristics. Paired with the low cost of this technique, both cost savings and improved quality of care might be achieved through further development and commercialization of this method.

Motion Signatures for the Analysis of Seizure Evolution in Epilepsy

In epilepsy, semiology refers to the study of patient behavior and movement, and their temporal evolution during epileptic seizures. Understanding semiology provides clues to the cerebral networks underpinning the epileptic episode and is a vital resource in the pre-surgical evaluation. Recent advances in video analytics have been helpful in capturing and quantifying epileptic seizures. Nevertheless, the automated representation of the evolution of semiology, as examined by neurologists, has not been appropriately investigated. From initial seizure symptoms until seizure termination, motion patterns of isolated clinical manifestations vary over time. Furthermore, epileptic seizures frequently evolve from one clinical manifestation to another, and their understanding cannot be overlooked during a presurgery evaluation. Here, we propose a system capable of computing motion signatures from videos of face and hand semiology to provide quantitative information on the motion, and the correlation between motions. Each signature is derived from a sparse saliency representation established by the magnitude of the optical flow field. The developed computer-aided tool provides a novel approach for physicians to analyze semiology as a flow of signals without interfering in the healthcare environment. We detect and quantify semiology using detectors based on deep learning and via a novel signature scheme, which is independent of the amount of data and seizure differences. The system reinforces the benefits of computer vision for non-obstructive clinical applications to quantify epileptic seizures recorded in real-life healthcare conditions.

Validation of a Single RGB-D Camera for Gait Assessment of Polyneuropathy Patients

Motion analysis systems based on a single markerless RGB-D camera are more suitable for clinical practice than multi-camera marker-based reference systems. Nevertheless, the validity of RGB-D cameras for motor function assessment in some diseases affecting gait, such as Transthyretin Familial Amyloid Polyneuropathy (TTR-FAP), is yet to be investigated. In this study, the agreement between the Kinect v2 and a reference system for obtaining spatiotemporal and kinematic gait parameters was evaluated in the context of TTR-FAP. 3-D body joint data provided by both systems were acquired from ten TTR-FAP symptomatic patients, while performing ten gait trials. For each gait cycle, we computed several spatiotemporal and kinematic gait parameters. We then determined, for each parameter, the Bland Altman’s bias and 95% limits of agreement, as well as the Pearson’s and concordance correlation coefficients, between systems. The obtained results show that an affordable, portable and non-invasive system based on an RGB-D camera can accurately obtain most of the studied gait parameters (excellent or good agreement for eleven spatiotemporal and one kinematic). This system can bring more objectivity to motor function assessment of polyneuropathy patients, potentially contributing to an improvement of TTR-FAP treatment and understanding, with great benefits to the patients’ quality of life.

Deep Motion Analysis for Epileptic Seizure Classification

Visual motion clues such as facial expression and pose are natural semiology features which an epileptologist observes to identify epileptic seizures. However, these cues have not been effectively exploited for automatic detection due to the diverse variations in seizure appearance within and between patients. Here we present a multi-modal analysis approach to quantitatively classify patients with mesial temporal lobe (MTLE) and extra-temporal lobe (ETLE) epilepsy, relying on the fusion of facial expressions and pose dynamics. We propose a new deep learning approach that leverages recent advances in Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to automatically extract spatiotemporal features from facial and pose semiology using recorded videos. A video dataset from 12 patients with MTLE and 6 patients with ETLEin an Australian hospital has been collected for experiments. Our experiments show that facial semiology and body movements can be effectively recognized and tracked, and that they provide useful evidence to identify the type of epilepsy. A multi-fold cross-validation of the fusion model exhibited an average test accuracy of 92.10%, while a leave-one-subject-out cross-validation scheme, which is the first in the literature, achieves an accuracy of 58.49%. The proposed approach is capable of modelling semiology features which effectively discriminate between seizures arising from temporal and extra-temporal brain areas. Our approach can be used as a virtual assistant, which will save time, improve patient safety and provide objective clinical analysis to assist with clinical decision making.

Full-body motion assessment: Concurrent validation of two body tracking depth sensors versus a gold standard system during gait

RGB-D cameras provide 3-D body joint data in a low-cost, portable and non-intrusive way, when compared with reference motion capture systems used in laboratory settings. In this contribution, we evaluate the validity of both Microsoft Kinect versions (v1 and v2) for motion analysis against a Qualisys system in a simultaneous protocol. Two different walking directions in relation to the Kinect (towards - WT, and away - WA) were explored. For each gait trial, measures related with all body parts were computed: velocity of all joints, distance between symmetrical joints, and angle at some joints. For each measure, we compared each Kinect version and Qualisys by obtaining the mean true error and mean absolute error, Pearson's correlation coefficient, and optical-to-depth ratio. Although both Kinect v1 and v2 and/or WT and WA data present similar accuracy for some measures, better results were achieved, overall, when using WT data provided by the Kinect v2, especially for velocity measures. Moreover, the velocity and distance presented better results than angle measures. Our results show that both Kinect versions can be an alternative to more expensive systems such as Qualisys, for obtaining distance and velocity measures as well as some angles metrics (namely the knee angles). This conclusion is important towards the off-lab non-intrusive assessment of motor function in different areas, including sports and healthcare.

Understanding Patients' Behavior: Vision-Based Analysis of Seizure Disorders

A substantial proportion of patients with functional neurological disorders (FND) are being incorrectly diagnosed with epilepsy because their semiology resembles that of epileptic seizures (ES). Misdiagnosis may lead to unnecessary treatment and its associated complications. Diagnostic errors often result from an overreliance on specific clinical features. Furthermore, the lack of electrophysiological changes in patients with FND can also be seen in some forms of epilepsy, making diagnosis extremely challenging. Therefore, understanding semiology is an essential step for differentiating between ES and FND. Existing sensor-based and marker-based systems require physical contact with the body and are vulnerable to clinical situations such as patient positions, illumination changes, and motion discontinuities. Computer vision and deep learning are advancing to overcome these limitations encountered in the assessment of diseases and patient monitoring; however, they have not been investigated for seizure disorder scenarios. Here, we propose and compare two marker-free deep learning models, a landmark-based and a region-based model, both of which are capable of distinguishing between seizures from video recordings. We quantify semiology by using either a fusion of reference points and flow fields, or through the complete analysis of the body. Average leave-one-subject-out cross-validation accuracies for the landmark-based and region-based approaches of 68.1% and 79.6% in our dataset collected from 35 patients, reveal the benefit of video analytics to support automated identification of semiology in the challenging conditions of a hospital setting.

A hierarchical multimodal system for motion analysis in patients with epilepsy

During seizures, a myriad of clinical manifestations may occur. The analysis of these signs, known as seizure semiology, gives clues to the underlying cerebral networks involved. When patients with drug-resistant epilepsy are monitored to assess their suitability for epilepsy surgery, semiology is a vital component to the presurgical evaluation. Specific patterns of facial movements, head motions, limb posturing and articulations, and hand and finger automatisms may be useful in distinguishing between mesial temporal lobe epilepsy (MTLE) and extratemporal lobe epilepsy (ETLE). However, this analysis is time-consuming and dependent on clinical experience and training. Given this limitation, an automated analysis of semiological patterns, i.e., detection, quantification, and recognition of body movement patterns, has the potential to help increase the diagnostic precision of localization. While a few single modal quantitative approaches are available to assess seizure semiology, the automated quantification of patients' behavior across multiple modalities has seen limited advances in the literature. This is largely due to multiple complicated variables commonly encountered in the clinical setting, such as analyzing subtle physical movements when the patient is covered or room lighting is inadequate. Semiology encompasses the stepwise/temporal progression of signs that is reflective of the integration of connected neuronal networks. Thus, single signs in isolation are far less informative. Taking this into account, here, we describe a novel modular, hierarchical, multimodal system that aims to detect and quantify semiologic signs recorded in 2D monitoring videos. Our approach can jointly learn semiologic features from facial, body, and hand motions based on computer vision and deep learning architectures. A dataset collected from an Australian quaternary referral epilepsy unit analyzing 161 seizures arising from the temporal (n = 90) and extratemporal (n = 71) brain regions has been used in our system to quantitatively classify these types of epilepsy according to the semiology detected. A leave-one-subject-out (LOSO) cross-validation of semiological patterns from the face, body, and hands reached classification accuracies ranging between 12% and 83.4%, 41.2% and 80.1%, and 32.8% and 69.3%, respectively. The proposed hierarchical multimodal system is a potential stepping-stone towards developing a fully automated semiology analysis system to support the assessment of epilepsy.

MRI Markers and Functional Performance in Patients With CIS and MS: A Cross-Sectional Study

Introduction: Brain atrophy is a widely accepted marker of disease severity with association to clinical disability in multiple sclerosis (MS). It is unclear to which extent this association reflects common age effects on both atrophy and function. Objective: To explore how functional performance in gait, upper extremities and cognition is associated with brain atrophy in patients with Clinically Isolated Syndrome (CIS) and relapsing-remitting MS (RRMS), controlling for effects of age and sex. Methods: In 27 patients with CIS, 59 with RRMS (EDSS ≤3) and 63 healthy controls (HC), 3T MRI were analyzed for T2 lesion count (T2C), volume (T2V) and brain volumes [normalized brain volume (NBV), gray matter volume (NGMV), white matter volume (NWMV), thalamic volume (NThalV)]. Functional performance was measured with short maximum walking speed (SMSW speed), 9-hole peg test (9HPT) and symbol digit modalities test (SDMT). Linear regression models were created for functional variables with stepwise inclusion of age, sex and MR imaging markers. Results: CIS differed from HC only in T2C and T2V. RRMS differed from HC in NBV, NGMV and NThalV, T2C and T2V, but not in NWMV. A strong association with age was seen in HC, CIS and RRMS groups for NBV (r = -0.5 to -0.6) and NGMV (r = -0.6 to -0.8). Associations with age were seen in HC and RRMS but not CIS for NThalV (r = -0.3; r = -0.5), T2C (rs = 0.3; rs = 0.2) and T2V (rs = 0.3; rs = 0.3). No effect of age was seen on NWMV. Correlations of functional performance with age in RRMS were seen for SMSW speed, 9HPTand SDMT (r = -0.27 to -0.46). Regression analyses yielded significant models only in the RRMS group for 9HPT, SMSW speed and EDSS. These included NBV, NGMV, NThalV, NWMV, logT2V, age and sex as predictors. NThalV was the only MRI variable predicting a functional measure (9HPTr) with a higher standardized beta than age and sex (R2 = 0.36, p < 1e-04). Conclusion: Thalamic atrophy was a stronger predictor of hand function (9HPT) in RRMS, than age and sex. This underlines the clinical relevance of thalamic atrophy and the relevance of hand function as a clinical marker even in mildly disabled patients.

Guided Embodiment and Potential Applications of Tutor Systems in Language Instruction and Rehabilitation

Intelligent tutor systems (ITSs) in mobile devices take us through learning tasks and make learning ubiquitous, autonomous, and at low cost (Nye, 2015). In this paper, we describe guided embodiment as an ITS essential feature for second language learning (L2) and aphasia rehabilitation (ARe) that enhances efficiency in the learning process. In embodiment, cognitive processes, here specifically language (re)learning are grounded in actions and gestures (Pecher and Zwaan, 2005; Fischer and Zwaan, 2008; Dijkstra and Post, 2015). In order to guide users through embodiment, ITSs must track action and gesture, and give corrective feed-back to achieve the users' goals. Therefore, sensor systems are essential to guided embodiment. In the next sections, we describe sensor systems that can be implemented in ITS for guided embodiment.

The first Transthyretin Familial Amyloid Polyneuropathy gait quantification study - preliminary results

Transthyretin Familial Amyloid Polyneuropathy (TTR-FAP) is a rare neurological disease caused by a genetic mutation with a variable presentation and consequent challenging diagnosis, complex follow-up and treatment. At this moment, this condition has no cure and treatment options are under development. One of the disease's implications is a definite and progressive motor impairment that from the early stages compromises walking ability and daily life activities. The detection of this impairment is key for the disease onset diagnosis. With the goal of improving diagnosis of the symptoms and patients' quality of life, the authors have assessed the gait characteristics of subjects suffering from this condition. This contribution shows the results of a preliminary study, using a non-intrusive, markerless vision-based gait analysis tool. To the best of our knowledge, the reported results constitute the first gait analysis data of TTR-FAP mutation carriers.

A novel portable, low-cost kinect-based system for motion analysis in neurological diseases

Many neurological diseases, such as Parkinson's disease and epilepsy, can significantly impair the motor function of the patients, often leading to a dramatic loss of their quality of life. Human motion analysis is regarded as fundamental towards an early diagnosis and enhanced follow-up in this type of diseases. In this contribution, we present NeuroKinect, a novel system designed for motion analysis in neurological diseases. This system includes an RGB-D camera (Microsoft Kinect) and two integrated software applications, KiT (KinecTracker) and KiMA (Kinect Motion Analyzer). The applications enable the preview, acquisition, review and management of data provided by the sensor, which are then used for motion analysis of relevant events. NeuroKinect is a portable, low-cost and markerless solution that is suitable for use in the clinical environment. Furthermore, it is able to provide quantitative support to the clinical assessment of different neurological diseases with movement impairments, as demonstrated by its usage in two different clinical routine scenarios: gait analysis in Parkinson's disease and seizure semiology analysis in epilepsy.