Integrating Behavior of Children with Profound Intellectual, Multiple, or Severe Motor Disabilities With Location and Environment Data Sensors for Independent Communication and Mobility: App Development and Pilot Testing

Multilevel and general linear modeling of weather and time effects on the emotional and behavioral states of children with profound intellectual and multiple disabilities

Introduction

Eliciting the emotional and behavioral states of children with severe or profound intellectual disabilities (IDs) and profound intellectual and multiple disabilities (PIMD) due to their complex and atypical developmental trajectories has become increasingly elusive. It is evident that the environment, influenced by weather conditions and time of the day, plays a pivotal role in molding children's behaviors, emotions, and interactions. This underscores the significance of the environment as a critical factor in exploring the communication dynamics of children with PIMD/IDs.

Methods

Over five months during fall and winter seasons, we conducted 105 video-recorded sessions with 20 children aged 8 to 16 with PIMD/IDs. These sessions aimed to capture the emotional and behavioral states interpreted by caregivers while simultaneously collecting indoor and outdoor weather indices, location, and time data. Using cross-classified multilevel and general linear models adjusted for individual characteristics and location variability with subsequent simple slope analyses, we examined the main and seasonal interaction effects of indoor and outdoor weather indices and time of the day on the emotional and behavioral states of children with PIMD/IDs.

Results

The models revealed that higher atmospheric pressure (atm), indicative of pleasant and favorable weather conditions, was associated with increased engagement (indoor: p < 0.01; outdoor: p < 0.01) and interest (outdoor: p < 0.01) behaviors. In contrast, engagement levels decreased before lunchtime (p < 0.01; p < 0.001), and inclement or unstable weather conditions characterized by low-pressure systems (p < 0.05) and stronger wind speed (p < 0.05) led to more refusal or disagreement. During winter, children displayed significantly more agreement with their caregivers (p < 0.001). Interestingly, they also engaged more on cloudy days (p < 0.05). Furthermore, simple slope analyses revealed that high atm conditions in fall were linked to more engagement (p < 0.05) while humid conditions predicted more assent behaviors (p < 0.001). However, cloudy weather predicted less attentional focusing (p < 0.05) and interest (p < 0.01) behaviors in winter.

Conclusion

This study confirms that fluctuations in weather indices, including seasonal changes and time of the day, can provide potential pathway indicators and supplement behavioral observations to elicit the behavioral states of children with PIMD/IDs. These findings highlight the importance of considering these factors when designing meaningful interactions and communication interventions for this population.

Passive Mobile Self-tracking of Mental Health by Veterans With Serious Mental Illness: Protocol for a User-Centered Design and Prospective Cohort Study

BACKGROUND

Serious mental illnesses (SMI) are common, disabling, and challenging to treat, requiring years of monitoring and treatment adjustments. Stress or reduced medication adherence can lead to rapid worsening of symptoms and behaviors. Illness exacerbations and relapses generally occur with little or no clinician awareness in real time, leaving limited opportunity to modify treatments. Previous research suggests that passive mobile sensing may be beneficial for individuals with SMI by helping them monitor mental health status and behaviors, and quickly detect worsening mental health for prompt assessment and intervention. However, there is too little research on its feasibility and acceptability and the extent to which passive data can predict changes in behaviors or symptoms.

OBJECTIVE

The aim of this research is to study the feasibility, acceptability, and safety of passive mobile sensing for tracking behaviors and symptoms of patients in treatment for SMI, as well as developing analytics that use passive data to predict changes in behaviors and symptoms.

METHODS

A mobile app monitors and transmits passive mobile sensor and phone utilization data, which is used to track activity, sociability, and sleep in patients with SMI. The study consists of a user-centered design phase and a mobile sensing phase. In the design phase, focus groups, interviews, and usability testing inform further app development. In the mobile sensing phase, passive mobile sensing occurs with participants engaging in weekly assessments for 9 months. Three- and nine-month interviews study the perceptions of passive mobile sensing and ease of app use. Clinician interviews before and after the mobile sensing phase study the usefulness and feasibility of app utilization in clinical care. Predictive analytic models are built, trained, and selected, and make use of machine learning methods. Models use sensor and phone utilization data to predict behavioral changes and symptoms.

RESULTS

The study started in October 2020. It has received institutional review board approval. The user-centered design phase, consisting of focus groups, usability testing, and preintervention clinician interviews, was completed in June 2021. Recruitment and enrollment for the mobile sensing phase began in October 2021.

CONCLUSIONS

Findings may inform the development of passive sensing apps and self-tracking in patients with SMI, and integration into care to improve assessment, treatment, and patient outcomes.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05023252; https://clinicaltrials.gov/ct2/show/NCT05023252.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/39010.

Machine learning-based classification of the movements of children with profound or severe intellectual or multiple disabilities using environment data features

Communication interventions have broadened from dialogical meaning-making, assessment approaches, to remote-controlled interactive objects. Yet, interpretation of the mostly pre-or protosymbolic, distinctive, and idiosyncratic movements of children with intellectual disabilities (IDs) or profound intellectual and multiple disabilities (PIMD) using computer-based assistive technology (AT), machine learning (ML), and environment data (ED: location, weather indices and time) remain insufficiently unexplored. We introduce a novel behavior inference computer-based communication-aid AT system structured on machine learning (ML) framework to interpret the movements of children with PIMD/IDs using ED. To establish a stable system, our study aimed to train, cross-validate (10-fold), test and compare the classification accuracy performance of ML classifiers (eXtreme gradient boosting [XGB], support vector machine [SVM], random forest [RF], and neural network [NN]) on classifying the 676 movements to 2, 3, or 7 behavior outcome classes using our proposed dataset recalibration (adding ED to movement datasets) with or without Boruta feature selection (53 child characteristics and movements, and ED-related features). Natural-child-caregiver-dyadic interactions observed in 105 single-dyad video-recorded (30-hour) sessions targeted caregiver-interpreted facial, body, and limb movements of 20 8-to 16-year-old children with PIMD/IDs and simultaneously app-and-sensor-collected ED. Classification accuracy variances and the influences of and the interaction among recalibrated dataset, feature selection, classifiers, and classes on the pooled classification accuracy rates were evaluated using three-way ANOVA. Results revealed that Boruta and NN-trained dataset in class 2 and the non-Boruta SVM-trained dataset in class 3 had >76% accuracy rates. Statistically significant effects indicating high classification rates (>60%) were found among movement datasets: with ED, non-Boruta, class 3, SVM, RF, and NN. Similar trends (>69%) were found in class 2, NN, Boruta-trained movement dataset with ED, and SVM and RF, and non-Boruta-trained movement dataset with ED in class 3. These results support our hypotheses that adding environment data to movement datasets, selecting important features using Boruta, using NN, SVM and RF classifiers, and classifying movements to 2 and 3 behavior outcomes can provide >73.3% accuracy rates, a promising performance for a stable ML-based behavior inference communication-aid AT system for children with PIMD/IDs.