Can consumer grade activity devices replace research grade actiwatches in youth mental health settings?

Accuracy and Precision of Consumer-Grade Wearable Activity Monitors for Assessing Time Spent in Sedentary Behavior in Children and Adolescents: Systematic Review

Background

A large number of wearable activity monitor models are released and used each year by consumers and researchers. As more studies are being carried out on children and adolescents in terms of sedentary behavior (SB) assessment, knowledge about accurate and precise monitoring devices becomes increasingly important.

Objective

The main aim of this systematic review was to investigate and communicate findings on the accuracy and precision of consumer-grade physical activity monitors in assessing the time spent in SB in children and adolescents.

Methods

Searches of PubMed (MEDLINE), Scopus, SPORTDiscus (full text), ProQuest, Open Access Theses and Dissertations, DART Europe E-theses Portal, and Networked Digital Library of Theses and Dissertations electronic databases were performed. All relevant studies that compared different types of consumer-grade monitors using a comparison method in the assessment of SB, published in European languages from 2015 onward were considered for inclusion. The risk of bias was estimated using Consensus-Based Standards for the Selection of Health Status Measurement Instruments. For enabling comparisons of accuracy measures within the studied outcome domain, measurement accuracy interpretation was based on group mean or percentage error values and 90% CI. Acceptable limits were predefined as –10% to +10% error in controlled and free-living settings. For determining the number of studies with group error percentages that fall within or outside one of the sides from previously defined acceptable limits, two 1-sided tests of equivalence were carried out, and the direction of measurement error was examined.

Results

A total of 8 studies complied with the predefined inclusion criteria, and 3 studies provided acceptable data for quantitative analyses. In terms of the presented accuracy comparisons, 14 were subsequently identified, with 6 of these comparisons being acceptable in terms of quantitative analysis. The results of the Cochran Q test indicated that the included studies did not share a common effect size (Q₅=82.86; P<.001). I², which represents the percentage of total variation across studies due to heterogeneity, amounted to 94%. The summary effect size based on the random effects model was not statistically significant (effect size=14.36, SE 12.04, 90% CI −5.45 to 34.17; P=.23). According to the equivalence test results, consumer-grade physical activity monitors did not generate equivalent estimates of SB in relation to the comparison methods. Majority of the studies (3/7, 43%) that reported the mean absolute percentage errors have reported values of <30%.

Conclusions

This is the first study that has attempted to synthesize available evidence on the accuracy and precision of consumer-grade physical activity monitors in measuring SB in children and adolescents. We found very few studies on the accuracy and almost no evidence on the precision of wearable activity monitors. The presented results highlight the large heterogeneity in this area of research.

Trial Registration

PROSPERO CRD42021251922; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=251922

Quality Evaluation of Free-living Validation Studies for the Assessment of 24-Hour Physical Behavior in Adults via Wearables: Systematic Review

Background

Wearable technology is a leading fitness trend in the growing commercial industry and an established method for collecting 24-hour physical behavior data in research studies. High-quality free-living validation studies are required to enable both researchers and consumers to make guided decisions on which study to rely on and which device to use. However, reviews focusing on the quality of free-living validation studies in adults are lacking.

Objective

This study aimed to raise researchers’ and consumers’ attention to the quality of published validation protocols while aiming to identify and compare specific consistencies or inconsistencies between protocols. We aimed to provide a comprehensive and historical overview of which wearable devices have been validated for which purpose and whether they show promise for use in further studies.

Methods

Peer-reviewed validation studies from electronic databases, as well as backward and forward citation searches (1970 to July 2021), with the following, required indicators were included: protocol must include real-life conditions, outcome must belong to one dimension of the 24-hour physical behavior construct (intensity, posture or activity type, and biological state), the protocol must include a criterion measure, and study results must be published in English-language journals. The risk of bias was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 tool with 9 questions separated into 4 domains (patient selection or study design, index measure, criterion measure, and flow and time).

Results

Of the 13,285 unique search results, 222 (1.67%) articles were included. Most studies (153/237, 64.6%) validated an intensity measure outcome such as energy expenditure. However, only 19.8% (47/237) validated biological state and 15.6% (37/237) validated posture or activity-type outcomes. Across all studies, 163 different wearables were identified. Of these, 58.9% (96/163) were validated only once. ActiGraph GT3X/GT3X+ (36/163, 22.1%), Fitbit Flex (20/163, 12.3%), and ActivPAL (12/163, 7.4%) were used most often in the included studies. The percentage of participants meeting the quality criteria ranged from 38.8% (92/237) to 92.4% (219/237). On the basis of our classification tree to evaluate the overall study quality, 4.6% (11/237) of studies were classified as low risk. Furthermore, 16% (38/237) of studies were classified as having some concerns, and 72.9% (173/237) of studies were classified as high risk.

Conclusions

Overall, free-living validation studies of wearables are characterized by low methodological quality, large variability in design, and focus on intensity. Future research should strongly aim at biological state and posture or activity outcomes and strive for standardized protocols embedded in a validation framework. Standardized protocols for free-living validation embedded in a framework are urgently needed to inform and guide stakeholders (eg, manufacturers, scientists, and consumers) in selecting wearables for self-tracking purposes, applying wearables in health studies, and fostering innovation to achieve improved validity.

Actigraphically measured psychomotor slowing in depression: systematic review and meta-analysis

Psychomotor slowing is a key feature of depressive disorders. Despite its great clinical importance, the pathophysiology and prevalence across different diagnoses and mood states are still poorly understood. Actigraphy allows unbiased, objective, and naturalistic assessment of physical activity as a marker of psychomotor slowing. Yet, the true effect-sizes remain unclear as recent, large systematic reviews are missing. We conducted a novel meta-analysis on actigraphically measured slowing in depression with strict inclusion and exclusion criteria for diagnosis ascertainment and sample duplications. Medline/PubMed and Web-of-Science were searched with terms combining mood-keywords and actigraphy-keywords until September 2021. Original research measuring actigraphy for ⩾24 h in at least two groups of depressed, remitted, or healthy participants and applying operationalized diagnosis was included. Studies in somatically ill patients, N < 10 participants/group, and studies using consumer-devices were excluded. Activity-levels between groups were compared using random-effects models with standardized-mean-differences and several moderators were examined. In total, 34 studies (n = 1804 patients) were included. Patients had lower activity than controls [standardized mean difference (s.m.d.) = -0.78, 95% confidence interval (CI) -0.99 to -0.57]. Compared to controls, patients with unipolar and bipolar disorder had lower activity than controls whether in depressed (unipolar: s.m.d. = -0.82, 95% CI -1.07 to -0.56; bipolar: s.m.d. = -0.94, 95% CI -1.41 to -0.46), or remitted/euthymic mood (unipolar: s.m.d. = -0.28, 95% CI -0.56 to 0.0; bipolar: s.m.d. = -0.92, 95% CI -1.36 to -0.47). None of the examined moderators had any significant effect. To date, this is the largest meta-analysis on actigraphically measured slowing in mood disorders. They are associated with lower activity, even in the remitted/euthymic mood-state. Studying objective motor behavior via actigraphy holds promise for informing screening and staging of affective disorders.

Diagnosis of Depressive Disorder Model on Facial Expression Based on Fast R-CNN

This study examines related literature to propose a model based on artificial intelligence (AI), that can assist in the diagnosis of depressive disorder. Depressive disorder can be diagnosed through a self-report questionnaire, but it is necessary to check the mood and confirm the consistency of subjective and objective descriptions. Smartphone-based assistance in diagnosing depressive disorders can quickly lead to their identification and provide data for intervention provision. Through fast region-based convolutional neural networks (R-CNN), a deep learning method that recognizes vector-based information, a model to assist in the diagnosis of depressive disorder can be devised by checking the position change of the eyes and lips, and guessing emotions based on accumulated photos of the participants who will repeatedly participate in the diagnosis of depressive disorder.

Distinguishing sleep from wake with a radar sensor: a contact-free real-time sleep monitor

This work aimed to evaluate whether a radar sensor can distinguish sleep from wakefulness in real time. The sensor detects body movements without direct physical contact with the subject and can be embedded in the roof of a hospital room for completely unobtrusive monitoring. We conducted simultaneous recordings with polysomnography, actigraphy, and radar on two groups: healthy young adults (n = 12, four nights per participant) and patients referred to a sleep examination (n = 28, one night per participant). We developed models for sleep/wake classification based on principles commonly used by actigraphy, including real-time models, and tested them on both datasets. We estimated a set of commonly reported sleep parameters from these data, including total-sleep-time, sleep-onset-latency, sleep-efficiency, and wake-after-sleep-onset, and evaluated the inter-method reliability of these estimates. Classification results were on-par with, or exceeding, those often seen for actigraphy. For real-time models in healthy young adults, accuracies were above 92%, sensitivities above 95%, specificities above 83%, and all Cohen's kappa values were above 0.81 compared to polysomnography. For patients referred to a sleep examination, accuracies were above 81%, sensitivities about 89%, specificities above 53%, and Cohen's kappa values above 0.44. Sleep variable estimates showed no significant intermethod bias, but the limits of agreement were quite wide for the group of patients referred to a sleep examination. Our results indicate that the radar has the potential to offer the benefits of contact-free real-time monitoring of sleep, both for in-patients and for ambulatory home monitoring.

Brief Research Report: A Pilot Study of Cognitive Behavioral Regulation Therapy (CBT-REG) for Young People at High Risk of Early Transition to Bipolar Disorders

Attempts to increase early identification of individuals in the early stages of bipolar disorders (i.e., individuals at high risk of bipolar disorders and/or experiencing a subthreshold syndrome with bipolar symptoms) have highlighted the need to develop high benefit-low risk interventions. We suggest that any new psychological therapy should (i) be acceptable to young people seeking help for the first time, (ii) be applicable to "at risk" conditions and sub-syndromal states and (iii) consider pluripotent factors that may be linked to illness progression not only for bipolar disorders specifically but also for other potential disease trajectories. However, evidence indicates that current interventions for youth with emerging mood disorders mainly represent approaches abbreviated from "disorder-specific" therapies used with older adults and are primarily offered to first episode cases of bipolar disorders who are also receiving psychotropic medication. This brief report discusses empirical findings used to construct core targets for therapeutic interventions that might reduce or delay transition to full-threshold bipolar disorders. We describe an intervention that includes strategies for problem-solving, reducing sleep-wake cycle disturbances, self-management of rumination and that addresses the needs of individuals with "sub-threshold" presentations who are probably at risk of developing a bipolar or other major mental disorders. Outcome data from a case series of 14 youth indicates that the intervention appears to demonstrate a relatively high benefit-to-risk ratio, promising levels of engagement with the therapy modules, and the therapy appears to be acceptable to a wide range of help-seeking youth with early expressions of bipolar psychopathology.

Evidence Base Update on Assessing Sleep in Youth

BACKGROUND

Sleep is vital to youth well-being and when it becomes disturbed - whether due to environmental or individual factors - mental and physical health suffer. Sleep problems can also be a symptom of underlying mental health disorders. Assessing different components of sleep, including quality and hygiene, can be useful both for identifying mental health problems and for measuring changes in well-being over time. However, there are dozens of sleep-related measures for youth and it can be difficult to determine which to select for a specific research or clinical purpose. The goal of this review was to identify sleep-related measures for clinical and/or research use in youth mental health settings, and to update the evidence base on this topic.

METHOD

We generated a list of candidate measures based on other reviews and searched in PubMed and PsycINFO using the terms "sleep" AND (measure OR assessment OR questionnaire) AND (psychometric OR reliability OR validity). Search results were limited to studies about children and adolescents (aged 2-17) published in English. Additional criteria for inclusion were that there had to be at least three publications reporting on the measure psychometrics in community or mental health populations. Sleep measures meeting these criteria were evaluated using the criteria set by De Los Reyes and Langer (2018).

RESULTS

Twenty-six measures, across four domains of sleep - insomnia, sleep hygiene, sleepiness, sleep quality - met inclusion criteria. Each measure had at least adequate clinical utility. No measure(s) emerged as superior across psychometric domains.

CONCLUSION

Clinicians and researchers must evaluate sleep measures for each use case, as the intended purpose will dictate which measure is best. Future research is necessary to evaluate measure performance in transdiagnostic mental health populations, including youth with serious mental illness.

Measuring circadian function in bipolar disorders: Empirical and conceptual review of physiological, actigraphic, and self-report approaches

BACKGROUND

Interest in biological clock pathways in bipolar disorders (BD) continues to grow, but there has yet to be an audit of circadian measurement tools for use in BD research and practice.

PROCEDURE

The International Society for Bipolar Disorders Chronobiology Task Force conducted a critical integrative review of circadian methods that have real-world applicability. Consensus discussion led to the selection of three domains to review-melatonin assessment, actigraphy, and self-report.

RESULTS

Measurement approaches used to quantify circadian function in BD are described in sufficient detail for researchers and clinicians to make pragmatic decisions about their use. A novel integration of the measurement literature is offered in the form of a provisional taxonomy distinguishing between circadian measures (the instruments and methods used to quantify circadian function, such as dim light melatonin onset) and circadian constructs (the biobehavioral processes to be measured, such as circadian phase).

CONCLUSIONS

Circadian variables are an important target of measurement in clinical practice and biomarker research. To improve reproducibility and clinical application of circadian constructs, an informed systematic approach to measurement is required. We trust that this review will decrease ambiguity in the literature and support theory-based consideration of measurement options.

Comparative validity of energy expenditure prediction algorithms using wearable devices for people with spinal cord injury

STUDY DESIGN

Cross-sectional validation study.

OBJECTIVES

To conduct a literature search for existing energy expenditure (EE) predictive algorithms using ActiGraph activity monitors for manual wheelchairs users (MWUs) with spinal cord injury (SCI), and evaluate their validity using an out-of-sample dataset.

SETTING

Research institution in Pittsburgh, USA.

METHODS

A literature search resulted in five articles containing five sets of predictive equations using an ActiGraph activity monitor for MWUs with SCI. Out-of-sample data were collected from 29 MWUs with chronic SCI who were asked to follow an activity protocol while wearing an ActiGraph GT9X Link on the dominant wrist. They also wore a portable metabolic cart which provided the criterion measure for EE. The out-of-sample dataset was used to evaluate the validity of the five sets of EE predictive equations.

RESULTS

None of the five sets of predictive equations demonstrated equivalence within 20% of the criterion measure based on an equivalence test. The mean absolute error for the five sets of predictive equations ranged from 0.87 to 6.41 kilocalories per minute (kcal min-1) when compared with the criterion measure, and the intraclass correlation estimates ranged from 0.06 to 0.59. The range between the Bland-Altman upper and lower limits of agreement was from 4.70 kcal min-1 to 25.09 kcal min-1.

CONCLUSIONS

The existing EE predictive equations based on ActiGraph monitors for MWUs with SCI showed varied performance when compared with the criterion measure. Their accuracies may not be sufficient to support future clinical and research use. More work is needed to develop more accurate EE predictive equations for this population.

AI in mental health

With the advent of digital approaches to mental health, modern artificial intelligence (AI), and machine learning in particular, is being used in the development of prediction, detection and treatment solutions for mental health care. In terms of treatment, AI is being incorporated into digital interventions, particularly web and smartphone apps, to enhance user experience and optimise personalised mental health care. In terms of prediction and detection, modern streams of abundant data mean that data-driven AI methods can be employed to develop prediction/detection models for mental health conditions. In particular, an individual's 'digital exhaust', the data gathered from their numerous personal digital device and social media interactions, can be mined for behavioural or mental health insights. Language, long considered a window into the human mind, can now be quantitatively harnessed as data with powerful computer-based natural language processing to also provide a method of inferring mental health. Furthermore, natural language processing can also be used to develop conversational agents used for therapeutic intervention.

Field-based Measurement of Sleep: Agreement between Six Commercial Activity Monitors and a Validated Accelerometer

OBJECTIVE

To examine agreement between multiple commercial activity monitors (CAMs) and a validated actigraph to measure sleep.

METHODS

Thirty adults without sleep disorders wore an Actiwatch Spectrum (AW) and alternated wearing 6 CAMs for one 24-h period each (Fitbit Alta, Jawbone Up3, Misfit Shine 2, Polar A360, Samsung Gear Fit2, Xiaomi Mi Band 2). Total sleep time (TST) and wake after sleep onset (WASO) were compared between edited AW and unedited CAM outputs. Comparisons between AW and CAM data were made via paired t-tests, mean absolute percent error (MAPE) calculations, and intra-class correlations (ICC). Intra-model reliability was performed in 10 participants who wore a pair of each AW and CAM model.

RESULTS

Fitbit, Jawbone, Misfit, and Xiaomi overestimated TST relative to AW (53.7-80.4 min, P ≤ .001). WASO was underestimated by Fitbit, Misfit, Samsung and Xiaomi devices (15.0-27.9 min; P ≤ .004) and overestimated by Polar (27.7 min, P ≤ .001). MAPEs ranged from 5.1% (Samsung) to 25.4% (Misfit) for TST and from 36.6% (Fitbit) to 165.1% (Polar) for WASO. TST ICCs ranged from .00 (Polar) to .92 (Samsung), while WASO ICCs ranged from .38 (Misfit) to .69 (Samsung). Differences were similar between poor sleepers (Pittsburgh Sleep Quality Index global score >5; n = 10) and good sleepers. Intra-model reliability analyses revealed minimal between-pair differences and high ICCs.

CONCLUSIONS

Agreement between CAMs and AW varied by device, with greater agreement observed for TST than WASO. While reliable, variability in agreement across CAMs with traditional actigraphy may complicate the interpretation of CAM data obtained for clinical or research purposes.

Sleep, circadian rhythm, and physical activity patterns in depressive and anxiety disorders: A 2-week ambulatory assessment study

BACKGROUND

Actigraphy may provide a more valid assessment of sleep, circadian rhythm (CR), and physical activity (PA) than self-reported questionnaires, but has not been used widely to study the association with depression/anxiety and their clinical characteristics.

METHODS

Fourteen-day actigraphy data of 359 participants with current (n = 93), remitted (n = 176), or no (n = 90) composite international diagnostic interview depression/anxiety diagnoses were obtained from the Netherlands Study of Depression and Anxiety. Objective estimates included sleep duration (SD), sleep efficiency, relative amplitude (RA) between day-time and night-time activity, mid sleep on free days (MSF), gross motor activity (GMA), and moderate-to-vigorous PA (MVPA). Self-reported measures included insomnia rating scale, SD, MSF, metabolic equivalent total, and MVPA.

RESULTS

Compared to controls, individuals with current depression/anxiety had a significantly different objective, but not self-reported, PA and CR: lower GMA (23.83 vs. 27.4 milli-gravity/day, p = .022), lower MVPA (35.32 vs. 47.64 min/day, p = .023), lower RA (0.82 vs. 0.83, p = .033). In contrast, self-reported, but not objective, sleep differed between people with current depression/anxiety compared to those without current disorders; people with current depression/anxiety reported both shorter and longer SD and more insomnia. More depressive/anxiety symptoms and number of depressive/anxiety diagnoses were associated with larger disturbances of the actigraphy measures.

CONCLUSION

Actigraphy provides ecologically valid information on sleep, CR, and PA that enhances data from self-reported questionnaires. As those with more severe or comorbid forms showed the lowest PA and most CR disruptions, the potential for adjunctive behavioral and chronotherapy interventions should be explored, as well as the potential of actigraphy to monitor treatment response to such interventions.