Automated pose estimation captures key aspects of General Movements at eight to 17 weeks from conventional videos

Assessment of spontaneous movements of newborns on second or third day of life using computer-aided video analysis

According to current knowledge, impaired spontaneous movements of infants can be an early indicator of developmental difficulties. This study aimed to evaluate velocity, acceleration, and parameters describing the range, nature, and location of individual limb movements in infants with normal pregnancy and delivery histories on the second or third day of life. General Movement Assessment was used to qualitatively assess spontaneous activity, while computer-aided movement analysis provided a quantitative assessment based on video recordings. Statistical analysis revealed significant differences in limb movement parameters between the left and right sides. Additionally, the results indicated that limb movements in infants with writhing movements were dynamic, exhibiting greater range and a circular shape. In contrast, infants with poor repertoire movements showed less variation in mean velocity, acceleration, and range of motion. These findings confirm the feasibility of using computer-aided video analysis to support early neonatal diagnosis by objectifying movement descriptions through quantitative measures, contributing valuable insights to the current understanding of spontaneous movements in newborns, particularly during the second and third days of life.

Artificial intelligence-powered 3D analysis of video-based caregiver-child interactions

We introduce HARMONI, a three-dimensional (3D) computer vision and audio processing method for analyzing caregiver-child behavior and interaction from observational videos. HARMONI operates at subsecond resolution, estimating 3D mesh representations and spatial interactions of humans, and adapts to challenging natural environments using an environment-targeted synthetic data generation module. Deployed on 500 hours from the SEEDLingS dataset, HARMONI generates detailed quantitative measurements of 3D human behavior previously unattainable through manual efforts or 2D methods. HARMONI identifies longitudinal trends in child-caregiver interaction, including child movement, body pose, dyadic touch, visibility, and conversational turns. The integrated visual and audio analysis further reveals multimodal trends, including associations between child conversational turns and movement. Open-sourced for large-scale analysis, HARMONI facilitates advancements in human development research. HARMONI achieves 63 to 80% consistency on key attributes with human annotators on SEEDLingS and 84 to 93% consistency on videos taken from a laboratory setting while achieving >100 times savings in time.

A systematic review of portable technologies for the early assessment of motor development in infants

Early screening and evaluation of infant motor development are crucial for detecting motor deficits and enabling timely interventions. Traditional clinical assessments are often subjective, without fully capturing infants' "real-world" behavior. This has sparked interest in portable, low-cost technologies to objectively and precisely measure infant motion at home, with a goal of enhancing ecological validity. In this systematic review, we explored the current landscape of portable, technology-based solutions to assess early motor development (within the first year), outlining the prevailing challenges and future directions. We reviewed 66 publications, which utilized video, sensors, or a combination of technologies. There were three key applications of these technologies: (1) automating clinical assessments, (2) illuminating new measures of motor development, and (3) predicting developmental outcomes. There was a promising trend toward earlier and more accurate detection using portable technologies. Additional research and demographic diversity are needed to develop fully automated, robust, and user-friendly tools. Registration & Protocol OSF Registries https://doi.org/10.17605/OSF.IO/R6JAE .

Learning Developmental Age From 3D Infant Kinetics Using Adaptive Graph Neural Networks

Reliable methods for the neurodevelopmental assessment of infants are essential for early detection of problems that may need prompt interventions. Spontaneous motor activity, or 'kinetics', is shown to provide a powerful surrogate measure of upcoming neurodevelopment. However, its assessment is by and large qualitative and subjective, focusing on visually identified, age-specific gestures. In this work, we introduce Kinetic Age (KA), a novel data-driven metric that quantifies neurodevelopmental maturity by predicting an infant's age based on their movement patterns. KA offers an interpretable and generalizable proxy for motor development. Our method leverages 3D video recordings of infants, processed with pose estimation to extract spatio-temporal series of anatomical landmarks, which are released as a new openly available dataset. These data are modeled using adaptive graph convolutional networks (AAGCNs), able to capture the spatio-temporal dependencies in infant movements. We also show that our data-driven approach achieves improvement over traditional machine learning baselines based on manually engineered features.

Artificial intelligence detects awareness of functional relation with the environment in 3 month old babies

A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering an infant's foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism~world connection.

Correlation between pose estimation features regarding movements towards the midline in early infancy

In infants, spontaneous movement towards the midline (MTM) indicates the initiation of anti-gravity ability development. Markerless 2D pose estimation is a cost-effective, time-efficient, and quantifiable alternative to movement assessment. We aimed to establish correlations between pose estimation features and MTM in early-age infants. Ninety-four infant videos were analysed to calculate the percentage and rate of MTM occurrence. 2D Pose estimation processed the videos and determined the distances and areas using wrist and ankle landmark coordinates. We collected data using video recordings from 20 infants aged 8-16 weeks post-term age. Correlations between MTM observations and distance values were evaluated. Differences in areas between groups of videos showing MTM and no MTM in the total, lower-limb, and upper-limb categories were examined. MTM observations revealed common occurrences of hand-to-trunk and foot-to-foot movements. Weak correlations were noted between limb distances to the midbody imaginary line and MTM occurrence values. Lower MTM showed significant differences in the lower part (p = 0.003) and whole area (p = 0.001). Video recording by parents or guardians could extract features using 2D pose estimation, assisting in the early identification of MTM in infants. Further research is required to assess a larger sample size with the diversity of MTM motor behaviour, and later developmental skills, and collect data from at-risk infants.

TwinEDA: a sustainable deep-learning approach for limb-position estimation in preterm infants' depth images

Early diagnosis of neurodevelopmental impairments in preterm infants is currently based on the visual analysis of newborns' motion patterns by trained operators. To help automatize this time-consuming and qualitative procedure, we propose a sustainable deep-learning algorithm for accurate limb-pose estimation from depth images. The algorithm consists of a convolutional neural network (TwinEDA) relying on architectural blocks that require limited computation while ensuring high performance in prediction. To ascertain its low computational costs and assess its application in on-the-edge computing, TwinEDA was additionally deployed on a cost-effective single-board computer. The network was validated on a dataset of 27,000 depth video frames collected during the actual clinical practice from 27 preterm infants. When compared to the main state-of-the-art competitor, TwinEDA is twice as fast to predict a single depth frame and four times as light in terms of memory, while performing similarly in terms of Dice similarity coefficient (0.88). This result suggests that the pursuit of efficiency does not imply the detriment of performance. This work is among the first to propose an automatic and sustainable limb-position estimation approach for preterm infants. This represents a significant step towards the development of broadly accessible clinical monitoring applications.

The developing brain: Challenges and opportunities to promote school readiness in young children at risk of neurodevelopmental disorders in low- and middle-income countries

This paper discusses possibilities for early detection and early intervention in infants with or at increased risk of neurodevelopmental disorders in low- and middle-income countries (LMICs). The brain's high rate of developmental activity in the early years post-term challenges early detection. It also offers opportunities for early intervention and facilitation of school readiness. The paper proposes that in the first year post-term two early detection options are feasible for LMICs: (a) caregiver screening questionnaires that carry little costs but predict neurodevelopmental disorders only moderately well; (b) the Hammersmith Infant Neurological Examination and Standardized Infant NeuroDevelopmental Assessment (SINDA) which are easy tools that predict neurodisability well but require assessment by health professionals. The young brain's neuroplasticity offers great opportunities for early intervention. Ample evidence indicates that families play a critical role in early intervention of infants at increased risk of neurodevelopmental disorders. Other interventional key elements are responsive parenting and stimulation of infant development. The intervention's composition and delivery mode depend on the infant's risk profile. For instance, in infants with moderately increased risk (e.g., preterm infants) lay community health workers may provide major parts of intervention, whereas in children with neurodisability (e.g., cerebral palsy) health professionals play a larger role.

Generating depth images of preterm infants in given poses using GANs

BACKGROUND AND OBJECTIVES

The use of deep learning for preterm infant's movement monitoring has the potential to support clinicians in early recognizing motor and behavioural disorders. The development of deep learning algorithms is, however, hampered by the lack of publicly available annotated datasets.

METHODS

To mitigate the issue, this paper presents a Generative Adversarial Network-based framework to generate images of preterm infants in a given pose. The framework consists of a bibranch encoder and a conditional Generative Adversarial Network, to generate a rough image and a refined version of it, respectively.

RESULTS

Evaluation was performed on the Moving INfants In RGB-D dataset which has 12.000 depth frames from 12 preterm infants. A low Fréchet inception distance (142.9) and an inception score (2.8) close to that of real-image distribution (2.6) are obtained. The results achieved show the potentiality of the framework in generating realistic depth images of preterm infants in a given pose.

CONCLUSIONS

Pursuing research on the generation of new data may enable researchers to propose increasingly advanced and effective deep learning-based monitoring systems.

Automated Movement Analysis to Predict Cerebral Palsy in Very Preterm Infants: An Ambispective Cohort Study

The General Movements Assessment requires extensive training. As an alternative, a novel automated movement analysis was developed and validated in preterm infants. Infants < 31 weeks’ gestational age or birthweight ≤ 1500 g evaluated at 3−5 months using the general movements assessment were included in this ambispective cohort study. The C-statistic, sensitivity, specificity, positive predictive value, and negative predictive value were calculated for a predictive model. A total of 252 participants were included. The median gestational age and birthweight were 274/7 weeks (range 256/7−292/7 weeks) and 960 g (range 769−1215 g), respectively. There were 29 cases of cerebral palsy (11.5%) at 18−24 months, the majority of which (n = 22) were from the retrospective cohort. Mean velocity in the vertical direction, median, standard deviation, and minimum quantity of motion constituted the multivariable model used to predict cerebral palsy. Sensitivity, specificity, positive, and negative predictive values were 55%, 80%, 26%, and 93%, respectively. C-statistic indicated good fit (C = 0.74). A cluster of four variables describing quantity of motion and variability of motion was able to predict cerebral palsy with high specificity and negative predictive value. This technology may be useful for screening purposes in very preterm infants; although, the technology likely requires further validation in preterm and high-risk term populations.

Early brain activity: Translations between bedside and laboratory

Neural activity is both a driver of brain development and a readout of developmental processes. Changes in neuronal activity are therefore both the cause and consequence of neurodevelopmental compromises. Here, we review the assessment of neuronal activities in both preclinical models and clinical situations. We focus on issues that require urgent translational research, the challenges and bottlenecks preventing translation of biomedical research into new clinical diagnostics or treatments, and possibilities to overcome these barriers. The key questions are (i) what can be measured in clinical settings versus animal experiments, (ii) how do measurements relate to particular stages of development, and (iii) how can we balance practical and ethical realities with methodological compromises in measurements and treatments.

Healthcare applications of single camera markerless motion capture: a scoping review

Background

Single camera markerless motion capture has the potential to facilitate at home movement assessment due to the ease of setup, portability, and affordable cost of the technology. However, it is not clear what the current healthcare applications of single camera markerless motion capture are and what information is being collected that may be used to inform clinical decision making. This review aims to map the available literature to highlight potential use cases and identify the limitations of the technology for clinicians and researchers interested in the collection of movement data.

Survey Methodology

Studies were collected up to 14 January 2022 using Pubmed, CINAHL and SPORTDiscus using a systematic search. Data recorded included the description of the markerless system, clinical outcome measures, and biomechanical data mapped to the International Classification of Functioning, Disability and Health Framework (ICF). Studies were grouped by patient population.

Results

A total of 50 studies were included for data collection. Use cases for single camera markerless motion capture technology were identified for Neurological Injury in Children and Adults; Hereditary/Genetic Neuromuscular Disorders; Frailty; and Orthopaedic or Musculoskeletal groups. Single camera markerless systems were found to perform well in studies involving single plane measurements, such as in the analysis of infant general movements or spatiotemporal parameters of gait, when evaluated against 3D marker-based systems and a variety of clinical outcome measures. However, they were less capable than marker-based systems in studies requiring the tracking of detailed 3D kinematics or fine movements such as finger tracking.

Conclusions

Single camera markerless motion capture offers great potential for extending the scope of movement analysis outside of laboratory settings in a practical way, but currently suffers from a lack of accuracy where detailed 3D kinematics are required for clinical decision making. Future work should therefore focus on improving tracking accuracy of movements that are out of plane relative to the camera orientation or affected by occlusion, such as supination and pronation of the forearm.

Deep learning-based quantitative analyses of spontaneous movements and their association with early neurological development in preterm infants

This study aimed to develop quantitative assessments of spontaneous movements in high-risk preterm infants based on a deep learning algorithm. Video images of spontaneous movements were recorded in very preterm infants at the term-equivalent age. The Hammersmith Infant Neurological Examination (HINE) was performed in infants at 4 months of corrected age. Joint positional data were extracted using a pretrained pose-estimation model. Complexity and similarity indices of joint angle and angular velocity in terms of sample entropy and Pearson correlation coefficient were compared between the infants with HINE < 60 and ≥ 60. Video images of spontaneous movements were recorded in 65 preterm infants at term-equivalent age. Complexity indices of joint angles and angular velocities differed between the infants with HINE < 60 and ≥ 60 and correlated positively with HINE scores in most of the joints at the upper and lower extremities (p < 0.05). Similarity indices between each joint angle or joint angular velocity did not differ between the two groups in most of the joints at the upper and lower extremities. Quantitative assessments of spontaneous movements in preterm infants are feasible using a deep learning algorithm and sample entropy. The results indicated that complexity indices of joint movements at both the upper and lower extremities can be potential candidates for detecting developmental outcomes in preterm infants.

A Pose-based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants

The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results. However, the prospect of automating these processes may improve accessibility of the assessment and also enhance the understanding of movement development of infants. Previous works have established the viability of using pose-based features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a series of new and improved features, and a feature fusion pipeline for this classification task. We also introduce the RVI-38 dataset, a series of videos captured as part of routine clinical care. By utilising this challenging dataset we establish the robustness of several motion features for classification, subsequently informing the design of our proposed feature fusion framework based upon the GMA. We evaluate our proposed framework's classification performance using both the RVI-38 dataset and the publicly available MINI-RGBD dataset. We also implement several other methods from the literature for direct comparison using these two independent datasets. Our experimental results and feature analysis show that our proposed pose-based method performs well across both datasets. The proposed features afford us the opportunity to include finer detail than previous methods, and further model GMA specific body movements. These new features also allow us to take advantage of additional body-part specific information as a means of improving the overall classification performance, whilst retaining GMA relevant, interpretable, and shareable features.

Technology-assisted quantification of movement to predict infants at high risk of motor disability: A systematic review

AIM

To systematically review the scientific literature to determine the predictive validity of technology-assisted measures of observable infant movement in infants less than six months of corrected age (CA) to identify high-risk of motor disability.

METHOD

A comprehensive search for randomised and non-randomised controlled trials, cohort studies and cross-comparison trials was performed on five electronic databases up to Feb 2021. Studies were included if they quantified infant movement before 6 months CA using some method of technology-assistance and compared the instrumented measure to a diagnostic clinical measure of neurodevelopment. Studies were excluded if they did not report a technology-assisted measure of infant movement. Methodological quality of the included studies was assessed using the Downs and Black scale.

RESULTS

23 studies met the full inclusion and exclusion criteria. Methodological quality of the included papers ranged from 9 to 24 (out of 26) on the Downs and Black scale. Infant movement assessments included the General Movements Assessment (GMA) and domains of the Hammersmith Infant Neurological Assessment (HINE). Studies used 2D video recordings, RGB-Depth recordings, accelerometry, and electromagnetic motion tracking technologies to quantify movement. Analytical approaches and movement features of interest were individual and varied. Technology assisted quantitative assessments identified cases of later diagnosed CP with sensitivity 44-100 %, specificity 59-95 %, Area under the ROC Curve 82-93 %; and typical development with sensitivity range 30-46 %, specificity 88-95 %, Area under the ROC Curve 68 %.

INTERPRETATION

Technology-assisted assessments of movement in infants less than 6 months CA using current technologies are feasible. Validation of measurement tools are limited. Although methods and results appear promising clinical uptake of technology-assisted assessments remains limited.

Age of Diagnosis, Fidelity and Acceptability of an Early Diagnosis Clinic for Cerebral Palsy: A Single Site Implementation Study

Cerebral palsy (CP) diagnosis is historically late, at between 12 and 24 months. We aimed to determine diagnosis age, fidelity to recommended tests and acceptability to parents and referrers of an early diagnosis clinic to implement a recent evidence-based clinical guideline for the early diagnosis of CP. A prospective observational case series of infants <12 months with detectable risks for CP attending our clinic was completed with data analysed cross-sectionally. Infants had a high risk of CP diagnosis at a mean age of 4.4 (standard deviation [SD] 2.3) months and CP diagnosis at 8.5 [4.1] months. Of the 109 infants seen, 57% had a diagnosis of CP or high risk of CP, showing high specificity to our inclusion criteria. Parent and referrer acceptability of the clinic was high. Paediatricians had the highest rate of referral (39%) followed by allied health (31%), primary carer (14%) and other health workers (16%). Fidelity to the guideline was also high. All infants referred <5 mths had the General Movements Assessment (GMA) and all except one had the Hammersmith Infant Neurological Examination (HINE) administered. N = 92 (84%) of infants seen had neuroimaging, including n = 53 (49%) who had magnetic resonance imaging (MRI), showing recommended tests are feasible. Referral to CP-specific interventions was at 4.7 [3.0] months, sometimes before referral to clinic. Clinicians can be confident CP can be diagnosed well under 12 months using recommended tools. This clinic model is acceptable to parents and referrers and supports access to CP-specific early interventions when they are likely to be most effective.

Video-based neonatal state assessment method for timing of procedures

BACKGROUND

Procedures should be performed when an infant is most receptive to disruptions in order to reduce the stress on the infant. However, frequent direct observations place a heavy burden on medical staff. There is therefore a need for a method for quantitatively and automatically evaluating the neonatal state.

METHODS

Ten infants in our hospital were enrolled in this study. The states of the infants were assessed by medical staff using the Brazelton Neonatal Behavioral Assessment Scale and were recorded on video at the same time. The recorded states were reclassified as activity levels, a new state classification method that includes middle activity, which is the appropriate time for a procedure. Using image analysis, motions of the infant were quantified as two indices: activity and pause time. Activity and pause time were compared for each activity level. The cutoff values of the indices were calculated, and the sensitivity and specificity of the middle activity were calculated.

RESULTS

There was a significant difference between all groups of activity level (P < 0.01). The maximum sensitivity and specificity of middle activity were 71.7% and 51.2%, respectively.

CONCLUSIONS

The neonatal state of infants can be quantitatively and automatically evaluated using video cameras, and the activity level can be used to determine an appropriate time for procedures in infants. This will reduce the burden on medical staff and lead to less stressful procedures for infants.

Automated movement recognition to predict motor impairment in high-risk infants: a systematic review of diagnostic test accuracy and meta-analysis

AIM

To assess the sensitivity and specificity of automated movement recognition in predicting motor impairment in high-risk infants.

METHOD

We searched MEDLINE, Embase, PsycINFO, CINAHL, Web of Science, and Scopus databases and identified additional studies from the references of relevant studies. We included studies that evaluated automated movement recognition in high-risk infants to predict motor impairment, including cerebral palsy (CP) and non-CP motor impairments. Two authors independently assessed studies for inclusion, extracted data, and assessed methodological quality using the Quality Assessment of Diagnostic Accuracy Studies-2. Meta-analyses were performed using hierarchical summary receiver operating characteristic models.

RESULTS

Of 6536 articles, 13 articles assessing 59 movement variables in 1248 infants under 5 months corrected age were included. Of these, 143 infants had CP. The overall sensitivity and specificity for motor impairment were 0.73 (95% confidence interval [CI] 0.68-0.77) and 0.70 (95% CI 0.65-0.75) respectively. Comparatively, clinical General Movements Assessment (GMA) was found to have sensitivity and specificity of 98% (95% CI 74-100) and 91% (95% CI 83-93) respectively. Sensor-based technologies had higher specificity (0.88, 95% CI 0.80-0.93).

INTERPRETATION

Automated movement recognition technology remains inferior to clinical GMA. The strength of this study is its meta-analysis to summarize performance, although generalizability of these results is limited by study heterogeneity.

Precision Medicine in Neonates: A Tailored Approach to Neonatal Brain Injury

Despite advances in neonatal care to prevent neonatal brain injury and neurodevelopmental impairment, predicting long-term outcome in neonates at risk for brain injury remains difficult. Early prognosis is currently based on cranial ultrasound (CUS), MRI, EEG, NIRS, and/or general movements assessed at specific ages, and predicting outcome in an individual (precision medicine) is not yet possible. New algorithms based on large databases and machine learning applied to clinical, neuromonitoring, and neuroimaging data and genetic analysis and assays measuring multiple biomarkers (omics) can fulfill the needs of modern neonatology. A synergy of all these techniques and the use of automatic quantitative analysis might give clinicians the possibility to provide patient-targeted decision-making for individualized diagnosis, therapy, and outcome prediction. This review will first focus on common neonatal neurological diseases, associated risk factors, and most common treatments. After that, we will discuss how precision medicine and machine learning (ML) approaches could change the future of prediction and prognosis in this field.

Early Diagnostics and Early Intervention in Neurodevelopmental Disorders-Age-Dependent Challenges and Opportunities

This review discusses early diagnostics and early intervention in developmental disorders in the light of brain development. The best instruments for early detection of cerebral palsy (CP) with or without intellectual disability are neonatal magnetic resonance imaging, general movements assessment at 2-4 months and from 2-4 months onwards, the Hammersmith Infant Neurological Examination and Standardized Infant NeuroDevelopmental Assessment. Early detection of autism spectrum disorders (ASD) is difficult; its first signs emerge at the end of the first year. Prediction with the Modified Checklist for Autism in Toddlers and Infant Toddler Checklist is possible to some extent and improves during the second year, especially in children at familial risk of ASD. Thus, prediction improves substantially when transient brain structures have been replaced by permanent circuitries. At around 3 months the cortical subplate has dissolved in primary motor and sensory cortices; around 12 months the cortical subplate in prefrontal and parieto-temporal cortices and cerebellar external granular layer have disappeared. This review stresses that families are pivotal in early intervention. It summarizes evidence on the effectiveness of early intervention in medically fragile neonates, infants at low to moderate risk, infants with or at high risk of CP and with or at high risk of ASD.

Writhing Movement Detection in Newborns on the Second and Third Day of Life Using Pose-Based Feature Machine Learning Classification

Observation of neuromotor development at an early stage of an infant’s life allows for early diagnosis of deficits and the beginning of the therapeutic process. General movement assessment is a method of spontaneous movement observation, which is the foundation for contemporary attempts at objectification and computer-aided diagnosis based on video recordings’ analysis. The present study attempts to automatically detect writhing movements, one of the normal general movement categories presented by newborns in the first weeks of life. A set of 31 recordings of newborns on the second and third day of life was divided by five experts into videos containing writhing movements (with occurrence time) and poor repertoire, characterized by a lower quality of movement in relation to the norm. Novel, objective pose-based features describing the scope, nature, and location of each limb’s movement are proposed. Three machine learning algorithms are evaluated in writhing movements’ detection in leave-one-out cross-validation for different feature extraction time windows and overlapping time. The experimental results make it possible to indicate the optimal parameters for which 80% accuracy was achieved. Based on automatically detected writhing movement percent in the video, infant movements are classified as writhing movements or poor repertoire with an area under the ROC (receiver operating characteristics) curve of 0.83.

AI Approaches Towards Prechtl's Assessment of General Movements: A Systematic Literature Review

General movements (GMs) are spontaneous movements of infants up to five months post-term involving the whole body varying in sequence, speed, and amplitude. The assessment of GMs has shown its importance for identifying infants at risk for neuromotor deficits, especially for the detection of cerebral palsy. As the assessment is based on videos of the infant that are rated by trained professionals, the method is time-consuming and expensive. Therefore, approaches based on Artificial Intelligence have gained significantly increased attention in the last years. In this article, we systematically analyze and discuss the main design features of all existing technological approaches seeking to transfer the Prechtl's assessment of general movements from an individual visual perception to computer-based analysis. After identifying their shared shortcomings, we explain the methodological reasons for their limited practical performance and classification rates. As a conclusion of our literature study, we conceptually propose a methodological solution to the defined problem based on the groundbreaking innovation in the area of Deep Learning.

Early Motor Development Predicts Clinical Outcomes of Siblings at High-Risk for Autism: Insight from an Innovative Motion-Tracking Technology

Atypical motor patterns are potential early markers and predictors of later diagnosis of Autism Spectrum Disorder (ASD). This study aimed to investigate the early motor trajectories of infants at high-risk (HR) of ASD through MOVIDEA, a semi-automatic software developed to analyze 2D and 3D videos and provide objective kinematic features of their movements. MOVIDEA was developed within the Italian Network for early detection of Autism Spectrum Disorder (NIDA Network), which is currently coordinating the most extensive surveillance program for infants at risk for neurodevelopmental disorders (NDDs). MOVIDEA was applied to video recordings of 53 low-risk (LR; siblings of typically developing children) and 50 HR infants’ spontaneous movements collected at 10 days and 6, 12, 18, and 24 weeks. Participants were grouped based on their clinical outcome (18 HR received an NDD diagnosis, 32 HR and 53 LR were typically developing). Results revealed that early developmental trajectories of specific motor parameters were different in HR infants later diagnosed with NDDs from those of infants developing typically. Since MOVIDEA was useful in the association of quantitative measures with specific early motor patterns, it should be applied to the early detection of ASD/NDD markers.

General Movement Assessment from videos of computed 3D infant body models is equally effective compared to conventional RGB video rating

BACKGROUND

General Movement Assessment (GMA) is a powerful tool to predict Cerebral Palsy (CP). Yet, GMA requires substantial training challenging its broad implementation in clinical routine. This inspired a world-wide quest for automated GMA.

AIMS

To test whether a low-cost, marker-less system for three-dimensional motion capture from RGB depth sequences using a whole body infant model may serve as the basis for automated GMA.

STUDY DESIGN

Clinical case study at an academic neurodevelopmental outpatient clinic.

SUBJECTS

Twenty-nine high risk infants were assessed at their clinical follow-up at 2-4 month corrected age (CA). Their neurodevelopmental outcome was assessed regularly up to 12-31 months CA.

OUTCOME MEASURES

GMA according to Hadders-Algra by a masked GMA-expert of conventional and computed 3D body model ("SMIL motion") videos of the same GMs. Agreement between both GMAs was tested using dichotomous and graded scaling with Kappa and intraclass correlations, respectively. Sensitivity and specificity to predict CP at ≥12 months CA were assessed.

RESULTS

Agreement of the two GMA ratings was moderate-good for GM-complexity (κ = 0.58; ICC = 0.874 [95%CI 0.730; 0.941]) and substantial-good for fidgety movements (FMs; Kappa = 0.78, ICC = 0.926 [95%CI 0.843; 0.965]). Five children were diagnosed with CP (four bilateral, one unilateral CP). The GMs of the child with unilateral CP were twice rated as mildly abnormal with FMs. GM-complexity and somewhat less FMs, of both conventional and SMIL motion videos predicted bilateral CP comparably to published literature.

CONCLUSIONS

Our computed infant 3D full body model is an attractive starting point for automated GMA in infants at risk of CP.

Automated Measures of Force and Motion Can Improve Our Understanding of Infants’ Motor Persistence

Every day, young learners are confronted with challenges. The degree to which they persist in overcoming those challenges, and the different ways they persist, provides critical insights into the various cognitive, motoric, and affective processes that drive behavior. Here, we present a systematic overview of the methodologies that have been traditionally used to study persistence, and offer suggestions for new approaches to the study of persistence that will make strides in moving the field forward. We argue that automated measures of force and motion, which have long been used in the study of infants’ motoric behavior, can provide a means to unravel the psychological processes that guide infants’ trying behavior. To illustrate this, we present a case study that highlights the novel lessons to be learned by the use of automated measures of force and motion regarding infants’ persistence, along with an analysis of the benefits and drawbacks of this approach, as well as detailed instructions for application. In sum, we conclude that these measures, when used in conjunction with more traditional approaches, will provide creative new insights into the nature and development of early persistence.

Movidea: A Software Package for Automatic Video Analysis of Movements in Infants at Risk for Neurodevelopmental Disorders

Early detecting the presence of neurodevelopmental disorders plays an important role in the effectiveness of the treatment. In this paper, we present a novel tool to extract motion features using single camera video recordings of infants. The Movidea software was developed to allow the operator to track the movement of end-effectors of infants in free moving conditions and extract movement features automatically. Movidea was used by different operators to analyze a set of video recordings and its performance was evaluated. The results showed that Movidea performance did not vary with the operator, and the tracking was also stable in home-video recordings. Even if the setup allowed for a two-dimensional analysis, most of the informative content of the movement was maintained. The reliability of the measures and features extracted, as well as the easiness of use, may boost the uptake of the proposed solution in clinical settings. Movidea overcomes the current limitation in the clinical practice in early detection of neurodevelopmental disorders by providing objective measures based on reliable data, and adds a new tool for the motor analysis of infants through unobtrusive technology.

Automatic assessment of the myoclonus severity from videos recorded according to standardized Unified Myoclonus Rating Scale protocol and using human pose and body movement analysis

PURPOSE

Myoclonus in progressive myoclonus epilepsy type 1 (EPM1) patients shows marked variability, which presents a substantial challenge in devising treatment and conducting clinical trials. Consequently, fast and objective myoclonus quantification methods are needed.

METHODS

Ten video-recorded unified myoclonus rating scale (UMRS) myoclonus with action tests were performed on EPM1 patients who were selected for the development and testing of the automatic myoclonus quantification method. Human pose and body movement analyses of the videos were used to identify body keypoints and further analyze movement smoothness and speed. The automatic myoclonus rating scale (ARMS) was developed. It included the jerk count during movement score and the log dimensionless jerk (LDLJ) score to evaluate changes in the smoothness of movement.

RESULTS

The scores obtained with the automatic analyses showed moderate to strong significant correlation with the UMRS myoclonus with action scores. The jerk count of the primary keypoints and the LDLJ scores were effective in the evaluation of the myoclonic jerks during hand movements. They also correlated moderately to strongly with the total UMRS test panel scores (r2 = 0,77, P = 0,009 for the jerk count score and r2 = 0,88, P = 0,001 for the LDLJ score). The automatic analyses was weaker in quantification of the neck, trunk, and leg myoclonus.

CONCLUSION

Automatic quantification of myoclonic jerks using human pose and body movement analysis of patients' videos is feasible and was found to be quite consistent with the accepted clinical gold standard quantification method. Based on the results of this study, the automatic analytical method should be further developed and validated to improve myoclonus severity follow-up for EPM1 patients.

Validity of General Movement Assessment Based on Clinical and Home Videos

INTRODUCTION

The purposes of this study were to develop an instructional leaflet on home video recording for the General Movement Assessment (GMA) and to examine the concurrent and predictive validity of the GMA completed by physical therapists (PTs) and completed by parents.

METHODS

The GMA and the Alberta Infant Motor Scale (AIMS) were completed by PTs in the clinic. Parents completed the GMA following the instructional leaflet.

RESULTS

The content validity of the leaflet was 0.83. The consistency of the GMA results between sources was κ = 0.869. The concurrent validity of the GMA at a corrected age of 3 months was κ = 0.266 (PT) versus 0.525 (parent) using the 10th-percentile cutoffs of the AIMS. The positive likelihood ratio was 26 (PT) versus 25 (parents) at a corrected age of 12 months based on 5th-percentile cutoffs of the AIMS.

CONCLUSIONS

Home GMA videos are valid for clinical assessment following the instructional leaflet.

Machine Learning of Infant Spontaneous Movements for the Early Prediction of Cerebral Palsy: A Multi-Site Cohort Study

BACKGROUND

Early identification of cerebral palsy (CP) during infancy will provide opportunities for early therapies and treatments. The aim of the present study was to present a novel machine-learning model, the Computer-based Infant Movement Assessment (CIMA) model, for clinically feasible early CP prediction based on infant video recordings.

METHODS

The CIMA model was designed to assess the proportion (%) of CP risk-related movements using a time-frequency decomposition of the movement trajectories of the infant's body parts. The CIMA model was developed and tested on video recordings from a cohort of 377 high-risk infants at 9-15 weeks corrected age to predict CP status and motor function (ambulatory vs. non-ambulatory) at mean 3.7 years age. The performance of the model was compared with results of the general movement assessment (GMA) and neonatal imaging.

RESULTS

The CIMA model had sensitivity (92.7%) and specificity (81.6%), which was comparable to observational GMA or neonatal cerebral imaging for the prediction of CP. Infants later found to have non-ambulatory CP had significantly more CP risk-related movements (median: 92.8%, p = 0.02) compared with those with ambulatory CP (median: 72.7%).

CONCLUSION

The CIMA model may be a clinically feasible alternative to observational GMA.