A Contactless Method for Measuring Full-Day, Naturalistic Motor Behavior Using Wearable Inertial Sensors

Assessing Infant Gross Motor Performance With an At-Home Wearable

BACKGROUND

Early development of gross motor skills is foundational for the upcoming neurocognitive performance. Here, we studied whether at-home wearable measurements performed by the parents could be used to quantify and track infants' developing motor abilities.

METHODS

Unsupervised at-home measurements of the infants' spontaneous activity were made repeatedly by the parents using a multisensor wearable suit (altogether 620 measurements from 134 infants at age 4-22 months). Machine learning-based algorithms were developed to detect the reaching of gross motor milestones (GMM), to measure times spent in key postures, and to track the overall motor development longitudinally. Parental questionnaires regarding GMMs were used for developing the algorithms, and the results were benchmarked with the interrater agreement levels established by World Health Organization (WHO). A total of 97 infants were used for the algorithm development and cross-validation, whereas an external validation was done using 37 infants from an independent recruitment in the same hospital.

RESULTS

The algorithms detected the reaching of GMMs very accurately (cross-validation: accuracy, 90.9%-95.5%; external validation, 92.4%-96.8%), which compares well with the human experts in the WHO reference study. The wearable-derived postural times showed strong correlation to parental assessments (ρ = .48-.81). Individual trajectories of motor maturation showed strong correlation to infants' age (ρ = .93).

CONCLUSIONS

These findings suggest that infants' gross motor skills can be quantified reliably and automatically from unsupervised at-home wearable recordings. Such methodology could be used in health care practice and in all developmental studies for gaining real-world quantitation and tracking of infants' motor abilities.

Physical Activity in Pre-Ambulatory Children with Cerebral Palsy: An Exploratory Validation Study to Distinguish Active vs. Sedentary Time Using Wearable Sensors

Wearable inertial sensor technology affords opportunities to record the physical activity of young children in their natural environments. The interpretation of these data, however, requires validation. The purpose of this study was to develop and establish the criterion validity of a method of quantifying active and sedentary physical activity using an inertial sensor for pre-ambulatory children with cerebral palsy. Ten participants were video recorded during 30 min physical therapy sessions that encouraged gross motor play activities, and the video recording was behaviorally coded to identify active and sedentary time. A receiver operating characteristic curve identified the optimal threshold to maximize true positive and minimize false positive active time for eight participants in the development dataset. The threshold was 0.417 m/s2 and was then validated with the remaining two participants; the percent of true positives and true negatives was 92.2 and 89.7%, respectively. We conclude that there is potential for raw sensor data to be used to quantify active and sedentary time in pre-ambulatory children with physical disability, and raw acceleration data may be more generalizable than the sensor-specific activity counts commonly reported in the literature.

Identifying Infant Body Position from Inertial Sensors with Machine Learning: Which Parameters Matter?

The efficient classification of body position is crucial for monitoring infants' motor development. It may fast-track the early detection of developmental issues related not only to the acquisition of motor milestones but also to postural stability and movement patterns. In turn, this may facilitate and enhance opportunities for early intervention that are crucial for promoting healthy growth and development. The manual classification of human body position based on video recordings is labour-intensive, leading to the adoption of Inertial Motion Unit (IMU) sensors. IMUs measure acceleration, angular velocity, and magnetic field intensity, enabling the automated classification of body position. Many research teams are currently employing supervised machine learning classifiers that utilise hand-crafted features for data segment classification. In this study, we used a longitudinal dataset of IMU recordings made in the lab in three different play activities of infants aged 4-12 months. The classification was conducted based on manually annotated video recordings. We found superior performance of the CatBoost Classifier over the Random Forest Classifier in the task of classifying five positions based on IMU sensor data from infants, yielding excellent classification accuracy of the Supine (97.7%), Sitting (93.5%), and Prone (89.9%) positions. Moreover, using data ablation experiments and analysing the SHAP (SHapley Additive exPlanations) values, the study assessed the importance of various groups of features from both the time and frequency domains. The results highlight that both accelerometer and magnetometer data, especially their statistical characteristics, are critical contributors to improving the accuracy of body position classification.

Long-form recording of infant body position in the home using wearable inertial sensors

Long-form audio recordings have had a transformational effect on the study of infant language acquisition by using mobile, unobtrusive devices to gather full-day, real-time data that can be automatically scored. How can we produce similar data in service of measuring infants' everyday motor behaviors, such as body position? The aim of the current study was to validate long-form recordings of infant position (supine, prone, sitting, upright, held by caregiver) based on machine learning classification of data from inertial sensors worn on infants' ankles and thighs. Using over 100 h of video recordings synchronized with inertial sensor data from infants in their homes, we demonstrate that body position classifications are sufficiently accurate to measure infant behavior. Moreover, classification remained accurate when predicting behavior later in the session when infants and caregivers were unsupervised and went about their normal activities, showing that the method can handle the challenge of measuring unconstrained, natural activity. Next, we show that the inertial sensing method has convergent validity by replicating age differences in body position found using other methods with full-day data captured from inertial sensors. We end the paper with a discussion of the novel opportunities that long-form motor recordings afford for understanding infant learning and development.

Automatic assessment of infant carrying and holding using at-home wearable recordings

Assessing infant carrying and holding (C/H), or physical infant-caregiver interaction, is important for a wide range of contexts in development research. An automated detection and quantification of infant C/H is particularly needed in long term at-home studies where development of infants' neurobehavior is measured using wearable devices. Here, we first developed a phenomenological categorization for physical infant-caregiver interactions to support five different definitions of C/H behaviors. Then, we trained and assessed deep learning-based classifiers for their automatic detection from multi-sensor wearable recordings that were originally used for mobile assessment of infants' motor development. Our results show that an automated C/H detection is feasible at few-second temporal accuracy. With the best C/H definition, the automated detector shows 96% accuracy and 0.56 kappa, which is slightly less than the video-based inter-rater agreement between trained human experts (98% accuracy, 0.77 kappa). The classifier performance varies with C/H definition reflecting the extent to which infants' movements are present in each C/H variant. A systematic benchmarking experiment shows that the widely used actigraphy-based method ignores the normally occurring C/H behaviors. Finally, we show proof-of-concept for the utility of the novel classifier in studying C/H behavior across infant development. Particularly, we show that matching the C/H detections to individuals' gross motor ability discloses novel insights to infant-parent interaction.

Preliminary Technical Validation of LittleBeats™: A Multimodal Sensing Platform to Capture Cardiac Physiology, Motion, and Vocalizations

Across five studies, we present the preliminary technical validation of an infant-wearable platform, LittleBeats™, that integrates electrocardiogram (ECG), inertial measurement unit (IMU), and audio sensors. Each sensor modality is validated against data from gold-standard equipment using established algorithms and laboratory tasks. Interbeat interval (IBI) data obtained from the LittleBeats™ ECG sensor indicate acceptable mean absolute percent error rates for both adults (Study 1, N = 16) and infants (Study 2, N = 5) across low- and high-challenge sessions and expected patterns of change in respiratory sinus arrythmia (RSA). For automated activity recognition (upright vs. walk vs. glide vs. squat) using accelerometer data from the LittleBeats™ IMU (Study 3, N = 12 adults), performance was good to excellent, with smartphone (industry standard) data outperforming LittleBeats™ by less than 4 percentage points. Speech emotion recognition (Study 4, N = 8 adults) applied to LittleBeats™ versus smartphone audio data indicated a comparable performance, with no significant difference in error rates. On an automatic speech recognition task (Study 5, N = 12 adults), the best performing algorithm yielded relatively low word error rates, although LittleBeats™ (4.16%) versus smartphone (2.73%) error rates were somewhat higher. Together, these validation studies indicate that LittleBeats™ sensors yield a data quality that is largely comparable to those obtained from gold-standard devices and established protocols used in prior research.

Objective Measurement of Posture and Movement in Young Children Using Wearable Sensors and Customised Mathematical Approaches: A Systematic Review

Given the importance of young children's postures and movements to health and development, robust objective measures are required to provide high-quality evidence. This study aimed to systematically review the available evidence for objective measurement of young (0-5 years) children's posture and movement using machine learning and other algorithm methods on accelerometer data. From 1663 papers, a total of 20 papers reporting on 18 studies met the inclusion criteria. Papers were quality-assessed and data extracted and synthesised on sample, postures and movements identified, sensors used, model development, and accuracy. A common limitation of studies was a poor description of their sample data, yet over half scored adequate/good on their overall study design quality assessment. There was great diversity in all aspects examined, with evidence of increasing sophistication in approaches used over time. Model accuracy varied greatly, but for a range of postures and movements, models developed on a reasonable-sized (n > 25) sample were able to achieve an accuracy of >80%. Issues related to model development are discussed and implications for future research outlined. The current evidence suggests the rapidly developing field of machine learning has clear potential to enable the collection of high-quality evidence on the postures and movements of young children.

Sitting Capacity and Performance in Infants with Typical Development and Infants with Motor Delay

AIMS

Infants with neuromotor disorders demonstrate delays in sitting skills (decreased capacity) and are less likely to maintain independent sitting during play than their peers with typical development (decreased performance). This study aimed to quantify developmental trajectories of sitting capacity and sitting performance in infants with typical development and infants with significant motor delay and to assess whether the relationship between capacity and performance differs between the groups.

METHODS

Typically developing infants (n = 35) and infants with significant motor delay (n = 31) were assessed longitudinally over a year following early sitting readiness. The Gross Motor Function Measure (GMFM) Sitting Dimension was used to assess sitting capacity, and a 5-min free play observation was used to assess sitting performance.

RESULTS

Both capacity and performance increased at a faster rate initially, with more deceleration across time, in infants with typical development compared to infants with motor delay. At lower GMFM scores, changes in GMFM sitting were associated with larger changes in independent sitting for infants with typical development, and the association between GMFM sitting and independent sitting varied more across GMFM scores for typically developing infants.

CONCLUSIONS

Intervention and assessment for infants with motor delay should target both sitting capacity and sitting performance.

Charting infants' motor development at home using a wearable system: validation and comparison to physical growth charts

BACKGROUND

Early neurodevelopmental care and research are in urgent need of practical methods for quantitative assessment of early motor development. Here, performance of a wearable system in early motor assessment was validated and compared to developmental tracking of physical growth charts.

METHODS

Altogether 1358 h of spontaneous movement during 226 recording sessions in 116 infants (age 4-19 months) were analysed using a multisensor wearable system. A deep learning-based automatic pipeline quantified categories of infants' postures and movements at a time scale of seconds. Results from an archived cohort (dataset 1, N = 55 infants) recorded under partial supervision were compared to a validation cohort (dataset 2, N = 61) recorded at infants' homes by the parents. Aggregated recording-level measures including developmental age prediction (DAP) were used for comparison between cohorts. The motor growth was also compared with respective DAP estimates based on physical growth data (length, weight, and head circumference) obtained from a large cohort (N = 17,838 infants; age 4-18 months).

FINDINGS

Age-specific distributions of posture and movement categories were highly similar between infant cohorts. The DAP scores correlated tightly with age, explaining 97-99% (94-99% CI 95) of the variance at the group average level, and 80-82% (72-88%) of the variance in the individual recordings. Both the average motor and the physical growth measures showed a very strong fit to their respective developmental models (R² = 0.99). However, single measurements showed more modality-dependent variation that was lowest for motor (σ = 1.4 [1.3-1.5 CI 95] months), length (σ = 1.5 months), and combined physical (σ = 1.5 months) measurements, and it was clearly higher for the weight (σ = 1.9 months) and head circumference (σ = 1.9 months) measurements. Longitudinal tracking showed clear individual trajectories, and its accuracy was comparable between motor and physical measures with longer measurement intervals.

INTERPRETATION

A quantified, transparent and explainable assessment of infants' motor performance is possible with a fully automated analysis pipeline, and the results replicate across independent cohorts from out-of-hospital recordings. A holistic assessment of motor development provides an accuracy that is comparable with the conventional physical growth measures. A quantitative measure of infants' motor development may directly support individual diagnostics and care, as well as facilitate clinical research as an outcome measure in early intervention trials.

FUNDING

This work was supported by the Finnish Academy (314602, 335788, 335872, 332017, 343498), Finnish Pediatric Foundation (Lastentautiensäätiö), Aivosäätiö, Sigrid Jusélius Foundation, and HUS Children's Hospital/HUS diagnostic center research funds.

Comparison of End-to-End Neural Network Architectures and Data Augmentation Methods for Automatic Infant Motility Assessment Using Wearable Sensors

Infant motility assessment using intelligent wearables is a promising new approach for assessment of infant neurophysiological development, and where efficient signal analysis plays a central role. This study investigates the use of different end-to-end neural network architectures for processing infant motility data from wearable sensors. We focus on the performance and computational burden of alternative sensor encoder and time series modeling modules and their combinations. In addition, we explore the benefits of data augmentation methods in ideal and nonideal recording conditions. The experiments are conducted using a dataset of multisensor movement recordings from 7-month-old infants, as captured by a recently proposed smart jumpsuit for infant motility assessment. Our results indicate that the choice of the encoder module has a major impact on classifier performance. For sensor encoders, the best performance was obtained with parallel two-dimensional convolutions for intrasensor channel fusion with shared weights for all sensors. The results also indicate that a relatively compact feature representation is obtainable for within-sensor feature extraction without a drastic loss to classifier performance. Comparison of time series models revealed that feedforward dilated convolutions with residual and skip connections outperformed all recurrent neural network (RNN)-based models in performance, training time, and training stability. The experiments also indicate that data augmentation improves model robustness in simulated packet loss or sensor dropout scenarios. In particular, signal- and sensor-dropout-based augmentation strategies provided considerable boosts to performance without negatively affecting the baseline performance. Overall, the results provide tangible suggestions on how to optimize end-to-end neural network training for multichannel movement sensor data.

Quantifying States and Transitions of Emerging Postural Control for Children Not Yet Able to Sit Independently

Objective, quantitative postural data is limited for individuals who are non-ambulatory, especially for those who have not yet developed trunk control for sitting. There are no gold standard measurements to monitor the emergence of upright trunk control. Quantification of intermediate levels of postural control is critically needed to improve research and intervention for these individuals. Accelerometers and video were used to record postural alignment and stability for eight children with severe cerebral palsy aged 2 to 13 years, under two conditions, seated on a bench with only pelvic support and with additional thoracic support. This study developed an algorithm to classify vertical alignment and states of upright control; Stable, Wobble, Collapse, Rise and Fall from accelerometer data. Next, a Markov chain model was created to calculate a normative score for postural state and transition for each participant with each level of support. This tool allowed quantification of behaviors previously not captured in adult-based postural sway measures. Histogram and video recordings were used to confirm the output of the algorithm. Together, this tool revealed that providing external support allowed all participants: (1) to increase their time spent in the Stable state, and (2) to reduce the frequency of transitions between states. Furthermore, all participants except one showed improved state and transition scores when given external support.

Motor Overflow during Reaching in Infancy: Quantification of Limb Movement Using Inertial Motion Units

Early in life, infants exhibit motor overflow, which can be defined as the generation of involuntary movements accompanying purposeful actions. We present the results of a quantitative study exploring motor overflow in 4-month-old infants. This is the first study quantifying motor overflow with high accuracy and precision provided by Inertial Motion Units. The study aimed to investigate the motor activity across the non-acting limbs during goal-directed action. To this end, we used wearable motion trackers to measure infant motor activity during a baby-gym task designed to capture overflow during reaching movements. The analysis was conducted on the subsample of participants (n = 20), who performed at least four reaches during the task. A series of Granger causality tests revealed that the activity differed depending on the non-acting limb and the type of the reaching movement. Importantly, on average, the non-acting arm preceded the activation of the acting arm. In contrast, the activity of the acting arm was followed by the activation of the legs. This may be caused by their distinct purposes in supporting postural stability and efficiency of movement execution. Finally, our findings demonstrate the utility of wearable motion trackers for precise measurement of infant movement dynamics.

Intelligent wearable allows out-of-the-lab tracking of developing motor abilities in infants

Background

Early neurodevelopmental care needs better, effective and objective solutions for assessing infants' motor abilities. Novel wearable technology opens possibilities for characterizing spontaneous movement behavior. This work seeks to construct and validate a generalizable, scalable, and effective method to measure infants' spontaneous motor abilities across all motor milestones from lying supine to fluent walking.

Methods

A multi-sensor infant wearable was constructed, and 59 infants (age 5-19 months) were recorded during their spontaneous play. A novel gross motor description scheme was used for human visual classification of postures and movements at a second-level time resolution. A deep learning -based classifier was then trained to mimic human annotations, and aggregated recording-level outputs were used to provide posture- and movement-specific developmental trajectories, which enabled more holistic assessments of motor maturity.

Results

Recordings were technically successful in all infants, and the algorithmic analysis showed human-equivalent-level accuracy in quantifying the observed postures and movements. The aggregated recordings were used to train an algorithm for predicting a novel neurodevelopmental measure, Baba Infant Motor Score (BIMS). This index estimates maturity of infants' motor abilities, and it correlates very strongly (Pearson's r = 0.89, p < 1e-20) to the chronological age of the infant.

Conclusions

The results show that out-of-hospital assessment of infants' motor ability is possible using a multi-sensor wearable. The algorithmic analysis provides metrics of motility that are transparent, objective, intuitively interpretable, and they link strongly to infants' age. Such a solution could be automated and scaled to a global extent, holding promise for functional benchmarking in individualized patient care or early intervention trials.

Changes in the Complexity of Limb Movements during the First Year of Life across Different Tasks

Infants’ limb movements evolve from disorganized to more selectively coordinated during the first year of life as they learn to navigate and interact with an ever-changing environment more efficiently. However, how these coordination patterns change during the first year of life and across different contexts is unknown. Here, we used wearable motion trackers to study the developmental changes in the complexity of limb movements (arms and legs) at 4, 6, 9 and 12 months of age in two different tasks: rhythmic rattle-shaking and free play. We applied Multidimensional Recurrence Quantification Analysis (MdRQA) to capture the nonlinear changes in infants’ limb complexity. We show that the MdRQA parameters (entropy, recurrence rate and mean line) are task-dependent only at 9 and 12 months of age, with higher values in rattle-shaking than free play. Since rattle-shaking elicits more stable and repetitive limb movements than the free exploration of multiple objects, we interpret our data as reflecting an increase in infants’ motor control that allows for stable body positioning and easier execution of limb movements. Infants’ motor system becomes more stable and flexible with age, allowing for flexible adaptation of behaviors to task demands.

Online testing in developmental science: A guide to design and implementation

At present, most developmental psychology experiments use participants from a mere subsection of the world's population. Moreover, like other fields of psychology, many studies in developmental psychology suffer from low statistical power due to small samples and limited observations. Online testing holds promise as a way to achieve more representative and robust, better powered experiments. As participants do not have to visit in person, it is easier to access populations living further away from a developmental lab, enabling testing of more diverse populations (e.g., urban vs rural areas, various different nationalities or geographies), both within and beyond the researcher's home country. Furthermore, due to the codified nature of browser-based online testing, it is possible for multiple labs to carry out the exact same study, allowing for better replications. Because of these advantages, developmental researchers have started to move experiments online so that caregivers and their children can participate from their home environments. However, the transition from traditional lab testing to remote online testing brings many challenges. Laboratory studies of infant and child development are typically conducted under highly standardized conditions to control factors, such as distractors, distance to the screen, movement, and lighting, and often rely on specialized equipment for measuring behavior. In this chapter, we provide a guide for researchers considering online testing of a developmental population. The different sections comprise an overview of the decision-making processes and the state-of-the-art advances associated with, as well as tangible recommendations for, online data collection.