Rigorous performance evaluation (previously, "validation") for informed use of new technologies for sleep health measurement

Evolving trends in novel sleep tracking and sleep testing technology publications between 2020 and 2022

STUDY OBJECTIVES

To update sleep medicine providers regarding (1) published research on the uses and performance of novel sleep tracking and testing technologies, (2) the use of artificial intelligence to acquire and process sleep data, and (3) research trends and gaps regarding the development and/or evaluation of these technologies.

METHODS

Medline and Embase electronic databases were searched for studies utilizing screening and diagnostic sleep technologies, published between 2020 and 2022 in journals focusing on human sleep. Studies' quality was determined based on the Study Design criteria of The Oxford Center for Evidence-Based Medicine Levels of Evidence.

RESULTS

Ninety-six of 3,849 articles were included. Most studies were adult performance evaluation (validation) studies, often comparing a novel technology to polysomnography. Sleep tracker publications tended to be Unites States-based, nonindustry-funded, performance studies on healthy adults using non-Food and Drug Administration-cleared technologies. Sleep apnea testing technologies were more frequently industry-funded and Food and Drug Administration-cleared. All studied technologies utilized software with an algorithm and/or artificial intelligence. Few studies used randomized control designs, or accounted for recruitment/attrition biases associated with participants' age, race/ethnicity, or comorbid health conditions.

CONCLUSIONS

Evidence-based publications have not kept pace with the proliferation and landscape of consumer and clinical sleep technologies. Due to the variance in technologies used within sleep research, careful review of the software used within studies is recommended. Future publications may fill identified gaps by including underrepresented populations, maintaining independence from industry, and through rigorous study design.

CITATION

Holfinger S, Schutte-Rodin S, Ratnasoma D, et al. Evolving trends in novel sleep tracking and sleep testing technology publications between 2020 and 2022. J Clin Sleep Med. 2025;21(5):891-905.

Evaluating the performance of open-source and proprietary processing of actigraphy sleep estimation in children with suspected sleep disorders: a comparison with polysomnography

STUDY OBJECTIVES

Evaluate the performance of actigraphy-based open-source and proprietary sleep algorithms compared to polysomnography in children with suspected sleep disorders.

METHODS

In a sleep clinic, 110 children (5-12 years, 54% female, 50% black, 82% with sleep disorders) wore wrist-placed ActiGraph GT9X during overnight polysomnography. Actigraphy data were scored as sleep or wake using open-source GGIR and proprietary ActiLife software. Discrepancy and epoch-by-epoch analyses were conducted to assess agreement between algorithms and polysomnography, along with equivalence testing.

RESULTS

The open-source vanHees2015 algorithm showed good accuracy (79.5% ± 12.0%), sensitivity (81.1% ± 13.5%), and specificity (66.0% ± 23.8%) for sleep detection but was outperformed by the proprietary ActiLife algorithms. The magnitude and trend of bias for total sleep time (TST), sleep efficiency (SE), sleep onset latency, and wake after sleep onset were similar between algorithms. TST and SE were statistically equivalent for the Cole-Kripke (Actilife) and vanHees2015 algorithms compared to the Sadeh (Actilife) algorithm. The Cole-Kripke (ActiLife) demonstrated higher sensitivity (90.5%) to detect sleep but lower specificity (61.2%) than Cole-Kripke (GGIR) (sensitivity: 62.7%, specificity: 79.9%). Sadeh and Cole-Kripke estimated sleep outcomes were not statistically equivalent between implementations in ActiLife and GGIR.

CONCLUSIONS

The open-source vanHees2015 algorithm performed well but slightly worse than the proprietary ActiLife algorithms in children. The open-source nature vanHees2015 makes it ideal for clinical pediatric use. Implementation of the Sadeh and Cole-Kripke algorithms in the proprietary ActiLife and open-source GGIR software yield different sleep estimates, so comparisons between studies using these different implementations should be avoided.

Enhancing artificial intelligence-driven sleep apnea diagnosis: The critical importance of input signal proficiency with a focus on mandibular jaw movements

PURPOSE

This review aims to highlight the pivotal role of the mandibular jaw movement (MJM) signal in advancing artificial intelligence (AI)-powered technologies for diagnosing obstructive sleep apnea (OSA).

METHODS

A scoping review was conducted to evaluate various aspects of the MJM signal and their contribution to improving signal proficiency for users.

RESULTS

The comprehensive literature analysis is structured into four key sections, each addressing factors essential to signal proficiency. These factors include (1) the comprehensiveness of research, development, and application of MJM-based technology; (2) the physiological significance of the MJM signal for various clinical tasks; (3) the technical transparency; and (4) the interpretability of the MJM signal. Comparisons with the photoplethysmography (PPG) signal are made where applicable.

CONCLUSIONS

Proficiency in biosignal interpretation is essential for the success of AI-driven diagnostic tools and for maximizing the clinical benefits through enhanced physiological insight. Through rigorous research ensuring an enhanced understanding of the signal and its extensive validation, the MJM signal sets a new benchmark for the development of AI-driven diagnostic solutions in OSA diagnosis.

Performance evaluation of a machine learning-based methodology using dynamical features to detect nonwear intervals in actigraphy data in a free-living setting

GOAL AND AIMS

One challenge using wearable sensors is nonwear time. Without a nonwear (e.g., capacitive) sensor, actigraphy data quality can be biased by subjective determinations confounding sleep/wake classification. We developed and evaluated a machine learning algorithm supplemented by dynamic features to discern wear/nonwear episodes.

FOCUS TECHNOLOGY

Actigraphy data from wrist actigraph (Spectrum, Philips-Respironics).

REFERENCE TECHNOLOGY

The built-in nonwear sensor as "ground truth" to classify nonwear periods using other data, mimicking features of Actiwatch 2.

SAMPLE

Data were collected over 1week from employed adults (n = 853).

DESIGN

Extreme gradient boosting (XGBoost), a tree-based classifier algorithm, was used to classify wear/nonwear, supplemented by dynamic features calculated over various time windows.

CORE ANALYTICS

The performance of the proposed algorithm was tested over 30-second epochs. Additional analytics and exploratory analyses: Evaluation of the SHapley Additive exPlanations (SHAP) values to find the effectiveness of the dynamic features.

CORE OUTCOMES

The XGBoost classifier yielded substantial improvements in balanced accuracy, sensitivity, and specificity, including dynamic features and comparison to default actiwatch classification algorithms.

IMPORTANT SUPPLEMENTAL OUTCOMES

The proposed classifier effectively distinguished between valid and invalid days, and the duration of contiguous periods of nonwear correctly identified.

CORE CONCLUSION

Our findings highlight the potential of XGBoost using dynamic features of varying activity levels across the time series to provide insights on wear/nonwear classification using a large dataset. The methodology provides an alternative to laborious manual benchmarking of the data for similar devices that do not have a nonwear sensor.

Optimizing wearable single-channel electroencephalography sleep staging in a heterogeneous sleep-disordered population using transfer learning

STUDY OBJECTIVES

Although various wearable electroencephalography devices have been developed, performance evaluation of the devices and their associated automated sleep stage classification models is mostly limited to healthy participants. A major barrier for applying automated wearable electroencephalography sleep staging in clinical populations is the need for large-scale data for model training. We therefore investigated transfer learning as a strategy to overcome limited data availability and optimize automated single-channel electroencephalography sleep staging in people with sleep disorders.

METHODS

We acquired 52 single-channel frontopolar headband electroencephalography recordings from a heterogeneous sleep-disordered population with concurrent polysomnography (PSG). We compared 3 model training strategies: "pretraining" (ie, training on a larger dataset of 901 conventional PSGs), "training-from-scratch" (ie, training on wearable headband recordings), and "fine-tuning" (ie, training on conventional PSGs, followed by training on headband recordings). Performance was evaluated on all headband recordings using 10-fold cross-validation.

RESULTS

Highest performance for 5-stage classification was achieved with fine-tuning (κ = .778), significantly higher than with pretraining (κ = .769) and with training-from-scratch (κ = .733). No significant differences or systematic biases were observed with clinically relevant sleep parameters derived from PSG. All sleep disorder categories showed comparable performance.

CONCLUSIONS

This study emphasizes the importance of leveraging larger available datasets through deep transfer learning to optimize performance with limited data availability. Findings indicate strong similarity in data characteristics between conventional PSG and headband recordings. Altogether, results suggest the combination of the headband, classification model, and training methodology can be viable for sleep monitoring in the heterogeneous clinical population.

CITATION

van der Aar JF, van Gilst MM, van den Ende DA, et al. Optimizing wearable single-channel electroencephalography sleep staging in a heterogeneous sleep-disordered population using transfer learning. J Clin Sleep Med. 2025;21(2):315-323.

2024 Position Statement on the Use of Different Diagnostic Methods for Sleep Disorders in Adults - Brazilian Sleep Association

Introduction  The current document represents the official position of Associação Brasileira do Sono (ABS; Brazilian Sleep Association) on the application of different sleep studies and provides specific recommendations for the use of different types of polysomnography (PSG) and respiratory polygraphy. Materials and Methods  The present document was based on existing guidelines. The steering committee discussed its findings and developed recommendations and contraindications, which were refined in discussions with the advisory committee. Adaptations were made based on professional experience, pathophysiological knowledge, and theoretical reasoning, especially to cover topics not discussed in previous guidelines or to adapt recommendations to the context and current practices in Brazil. Results  A total of 55 recommendations were made, covering the following domains: professional requirements for the requisition and interpretation of sleep studies ( n  = 7); eligibility for different sleep studies ( n  = 9); diagnosis of sleep-disordered breathing (SDB; n  = 5); diagnosis of SDB in special conditions ( n  = 3); diagnosis of SDB in association with other sleep disorders and comorbidities ( n  = 3); sleep studies on the follow-up of patients with SDB ( n  = 9); sleep studies for positive air pressure titration ( n  = 3); diagnosis of other sleep disorders ( n  = 10); and sleep studies on other conditions ( n  = 6). Conclusion  The selection of the type of sleep study should be made carefully, considering resource constraints, clinical suspicion of moderate or severe obstructive sleep apnea (OSA), and individual patient needs, among other factors. It is crucial that health professionals receive appropriate training and board certification in sleep science, thus being able to determine the most suitable diagnostic method, understand their indications and limitations, and assure an accurate diagnosis for each patient.

Automatic sleep staging based on 24/7 EEG SubQ (UNEEG medical) data displays strong agreement with polysomnography in healthy adults

GOAL AND AIMS

Performance evaluation of automatic sleep staging on two-channel subcutaneous electroencephalography.

FOCUS TECHNOLOGY

UNEEG medical's 24/7 electroencephalography SubQ (the SubQ device) with deep learning model U-SleepSQ.

REFERENCE METHOD/TECHNOLOGY

Manually scored hypnograms from polysomnographic recordings.

SAMPLE

Twenty-two healthy adults with 1-6 recordings per participant. The clinical study was registered at ClinicalTrials.gov with the identifier NCT04513743.

DESIGN

Fine-tuning of U-Sleep in 11-fold cross-participant validation on 22 healthy adults. The resultant model was called U-SleepSQ.

CORE ANALYTICS

Bland-Altman analysis of sleep parameters. Advanced multiclass model performance metrics: stage-specific accuracy, specificity, sensitivity, kappa (κ), and F1 score. Additionally, Cohen's κ coefficient and macro F1 score. Longitudinal and participant-level performance evaluation.

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Exploration of model confidence quantification. Performance vs. age, sex, body mass index, SubQ implantation hemisphere, normalized entropy, transition index, and scores from the following three questionnaires: Morningness-Eveningness Questionnaire, World Health Organization's 5-item Well-being Index, and Major Depression Inventory.

CORE OUTCOMES

There was a strong agreement between the focus and reference method/technology.

IMPORTANT SUPPLEMENTAL OUTCOMES

The confidence score was a promising metric for estimating the reliability of each hypnogram classified by the system.

CORE CONCLUSION

The U-SleepSQ model classified hypnograms for healthy participants soon after implantation and longitudinally with a strong agreement with the gold standard of manually scored polysomnographics, exhibiting negligible temporal variation.

Breathless Nights and Cardiac Frights-How Snoring Is Breaking Hearts

While your nightly symphony may be testing your loved one's patience, it could also be giving your own heart reasons to complain [...].

OSA diagnosis goes wearable: are the latest devices ready to shine?

STUDY OBJECTIVES

From 2019-2023, the United States Food and Drug Administration has cleared 9 novel obstructive sleep apnea-detecting wearables for home sleep apnea testing, with many now commercially available for sleep clinicians to integrate into their clinical practices. To help clinicians comprehend these devices and their functionalities, we meticulously reviewed their operating mechanisms, sensors, algorithms, data output, and related performance evaluation literature.

METHODS

We collected information from PubMed, United States Food and Drug Administration clearance documents, ClinicalTrials.gov, and web sources, with direct industry input whenever feasible.

RESULTS

In this "device-centered" review, we broadly categorized these wearables into 2 main groups: those that primarily harness photoplethysmography data and those that do not. The former include the peripheral arterial tonometry-based devices. The latter was further broken down into 2 key subgroups: acoustic-based and respiratory effort-based devices. We provided a performance evaluation literature review and objectively compared device-derived metrics and specifications pertinent to sleep clinicians. Detailed demographics of study populations, exclusion criteria, and pivotal statistical analyses of the key validation studies are summarized.

CONCLUSIONS

In the foreseeable future, these novel obstructive sleep apnea-detecting wearables may emerge as primary diagnostic tools for patients at risk for moderate-to-severe obstructive sleep apnea without significant comorbidities. While more devices are anticipated to join this category, there remains a critical need for cross-device comparison studies as well as independent performance evaluation and outcome research in diverse populations. Now is the moment for sleep clinicians to immerse themselves in understanding these emerging tools to ensure our patient-centered care is improved through the appropriate implementation and utilization of these novel sleep technologies.

CITATION

Chiang AA, Jerkins E, Holfinger S, et al. OSA diagnosis goes wearable: are the latest devices ready to shine? J Clin Sleep Med. 2024;20(11):1823-1838.

A systematic review of the performance of actigraphy in measuring sleep stages

The accuracy of actigraphy for sleep staging is assumed to be poor, but examination is limited. This systematic review aimed to assess the performance of actigraphy in sleep stage classification of adults. A systematic search was performed using MEDLINE, Web of Science, Google Scholar, and Embase databases. We identified eight studies that compared sleep architecture estimates between wrist-worn actigraphy and polysomnography. Large heterogeneity was found with respect to how sleep stages were grouped, and the choice of metrics used to evaluate performance. Quantitative synthesis was not possible, so we performed a narrative synthesis of the literature. From the limited number of studies, we found that actigraphy-based sleep staging had some ability to classify different sleep stages compared with polysomnography.

Guidelines for the development, performance evaluation and validation of new sleep technologies (DEVSleepTech guidelines) - a protocol for a Delphi consensus study

New sleep technologies are being developed, refined and delivered at a fast pace. However, there are serious concerns about the validation and accuracy of new sleep-related technologies being made available, as many of them, especially consumer-sleep technologies, have not been tested in comparison with gold-standard methods or have been approved by health regulatory agencies. The importance of proper validation and performance evaluation of new sleep technologies has already been discussed in previous studies and some recommendations have already been published, but most of them do not employ standardized methodology and are not able to cover all aspects of new sleep technologies. The current protocol describes the methods of a Delphi consensus study to create guidelines for the development, performance evaluation and validation of new sleep devices and technologies. The resulting recommendations are not intended to be used as a quality assessment tool to evaluate individual articles, but rather to evaluate the overall procedures, studies and experiments performed to develop, evaluate performance and validate new technologies. We hope these guidelines can be helpful for researchers who work with new sleep technologies on the appraisal of their reliability and validation, for companies who are working on the development and refinement of new sleep technologies, and by regulatory agencies to evaluate new technologies that are looking for registration, approval or inclusion on health systems.

Performance evaluation of portable respiratory polygraphy for assessing sleep bruxism in adults

Background

Polysomnography (PSG) is the gold standard for sleep bruxism (SB) assessment, it is expensive, not widely accessible, and time‐consuming.

Objective

Given the increasing prevalence of SB, there is a growing need for an alternative, readily available, reliable and cost‐effective diagnostic method. This study aimed to evaluate the diagnostic validity of portable respiratory polygraphy (PRPG) compared with PSG for SB diagnosis.

Methods

One hundred and three subjects underwent simultaneous examinations using PRPG (NOX T3, NOX Medical) and PSG (NOX A1, NOX Medical) in a sleep laboratory.

Results

The mean Bruxism Episodes Index (BEI) measured by PRPG was 4.70 ± 3.98, whereas PSG yielded a mean BEI of 3.79 ± 3.08. The sensitivity for detecting sleep bruxism (BEI >2) by PRPG was 48.3%, with a specificity of 81.2%. The positive predictive value was estimated at 51.9%, and the negative predictive value at 78.9%. However, when distinguishing between mild bruxism (BEI >2 < 4) and severe bruxism (BEI >4), PRPG demonstrated a sensitivity of 77.8% and 68.3% and a specificity of 48.6% and 71.4%, respectively.

Conclusion

Polysomnography continues to be the SB diagnostic gold standard tool, as the sensitivity and specificity of PRPG are significantly lower when compared with PSG. Nevertheless, PRPG could serve as an alternative tool for SB screening or diagnosis, despite its limitations. Furthermore, our data indicate that comorbidities such as sleep apnea and sleep quality do not influence the diagnostic accuracy of PSG, suggesting its potential as a screening instrument in individuals with other sleep disorders.

Validation of Downloadable Mobile Snore Applications by Polysomnography (PSG)

Purpose

Obstructive sleep apnea (OSA) is a common breathing disorder during sleep that is associated with symptoms such as snoring, excessive daytime sleepiness, and breathing interruptions. Polysomnography (PSG) is the most reliable diagnostic test for OSA; however, its high cost and lengthy testing duration make it difficult to access for many patients. With the availability of free snore applications for home-monitoring, this study aimed to validate the top three ranked snore applications, namely SnoreLab (SL), Anti Snore Solution (ASS), and Sleep Cycle Alarm (SCA), using PSG.

Patients and Methods

Sixty participants underwent an overnight PSG while simultaneously using three identical smartphones with the tested apps to gather sleep and snoring data.

Results

The study discovered that all three applications were significantly correlated with the total recording time and snore counts of PSG, with ASS showing good agreement with snore counts. Furthermore, the Snore Score, Time Snoring of SL, and Sleep Quality of SCA had a significant correlation with the natural logarithm of apnea hypopnea index (lnAHI) of PSG. The Snore Score of SL and the Sleep Quality of SCA were shown to be useful for evaluating snore severity and for pre-diagnosing or predicting OSA above moderate levels.

Conclusion

These findings suggest that some parameters of free snore applications can be employed to monitor OSA progress, and future research could involve adjusted algorithms and larger-scale studies to further authenticate these downloadable snore and sleep applications.

State of the science and recommendations for using wearable technology in sleep and circadian research

Wearable sleep-tracking technology is of growing use in the sleep and circadian fields, including for applications across other disciplines, inclusive of a variety of disease states. Patients increasingly present sleep data derived from their wearable devices to their providers and the ever-increasing availability of commercial devices and new-generation research/clinical tools has led to the wide adoption of wearables in research, which has become even more relevant given the discontinuation of the Philips Respironics Actiwatch. Standards for evaluating the performance of wearable sleep-tracking devices have been introduced and the available evidence suggests that consumer-grade devices exceed the performance of traditional actigraphy in assessing sleep as defined by polysomnogram. However, clear limitations exist, for example, the misclassification of wakefulness during the sleep period, problems with sleep tracking outside of the main sleep bout or nighttime period, artifacts, and unclear translation of performance to individuals with certain characteristics or comorbidities. This is of particular relevance when person-specific factors (like skin color or obesity) negatively impact sensor performance with the potential downstream impact of augmenting already existing healthcare disparities. However, wearable sleep-tracking technology holds great promise for our field, given features distinct from traditional actigraphy such as measurement of autonomic parameters, estimation of circadian features, and the potential to integrate other self-reported, objective, and passively recorded health indicators. Scientists face numerous decision points and barriers when incorporating traditional actigraphy, consumer-grade multi-sensor devices, or contemporary research/clinical-grade sleep trackers into their research. Considerations include wearable device capabilities and performance, target population and goals of the study, wearable device outputs and availability of raw and aggregate data, and data extraction, processing, and analysis. Given the difficulties in the implementation and utilization of wearable sleep-tracking technology in real-world research and clinical settings, the following State of the Science review requested by the Sleep Research Society aims to address the following questions. What data can wearable sleep-tracking devices provide? How accurate are these data? What should be taken into account when incorporating wearable sleep-tracking devices into research? These outstanding questions and surrounding considerations motivated this work, outlining practical recommendations for using wearable technology in sleep and circadian research.

A Protocol for Evaluating Digital Technology for Monitoring Sleep and Circadian Rhythms in Older People and People Living with Dementia in the Community

Sleep and circadian rhythm disturbance are predictors of poor physical and mental health, including dementia. Long-term digital technology-enabled monitoring of sleep and circadian rhythms in the community has great potential for early diagnosis, monitoring of disease progression, and assessing the effectiveness of interventions. Before novel digital technology-based monitoring can be implemented at scale, its performance and acceptability need to be evaluated and compared to gold-standard methodology in relevant populations. Here, we describe our protocol for the evaluation of novel sleep and circadian technology which we have applied in cognitively intact older adults and are currently using in people living with dementia (PLWD). In this protocol, we test a range of technologies simultaneously at home (7-14 days) and subsequently in a clinical research facility in which gold standard methodology for assessing sleep and circadian physiology is implemented. We emphasize the importance of assessing both nocturnal and diurnal sleep (naps), valid markers of circadian physiology, and that evaluation of technology is best achieved in protocols in which sleep is mildly disturbed and in populations that are relevant to the intended use-case. We provide details on the design, implementation, challenges, and advantages of this protocol, along with examples of datasets.

Is it Time to Include Wearable Sleep Trackers in the Applied Psychologists' Toolbox?

Wearable sleep trackers are increasingly used in applied psychology. Particularly, the recent boom in the fitness tracking industry has resulted in a number of relatively inexpensive consumer-oriented devices that further enlarge the potential applications of ambulatory sleep monitoring. While being largely positioned as wellness tools, wearable sleep trackers could be considered useful health devices supported by a growing number of independent peer-reviewed studies evaluating their accuracy. The inclusion of sensors that monitor cardiorespiratory physiology, diurnal activity data, and other environmental signals allows for a comprehensive and multidimensional approach to sleep health and its impact on psychological well-being. Moreover, the increasingly common combination of wearable trackers and experience sampling methods has the potential to uncover within-individual processes linking sleep to daily experiences, behaviors, and other psychosocial factors. Here, we provide a concise overview of the state-of-the-art, challenges, and opportunities of using wearable sleep-tracking technology in applied psychology. Specifically, we review key device profiles, capabilities, and limitations. By providing representative examples, we highlight how scholars and practitioners can fully exploit the potential of wearable sleep trackers while being aware of the most critical pitfalls characterizing these devices. Overall, consumer wearable sleep trackers are increasingly recognized as a valuable method to investigate, assess, and improve sleep health. Incorporating such devices in research and professional practice might significantly improve the quantity and quality of the collected information while opening the possibility of involving large samples over representative time periods. However, a rigorous and informed approach to their use is necessary.

Performance of the Dreem 2 EEG headband, relative to polysomnography, for assessing sleep in Parkinson's disease

GOAL AND AIMS

To pilot the feasibility and evaluate the performance of an EEG wearable for measuring sleep in individuals with Parkinson's disease.

FOCUS TECHNOLOGY

Dreem Headband, Version 2.

REFERENCE TECHNOLOGY

Polysomnography.

SAMPLE

Ten individuals with Parkinson's disease.

DESIGN

Individuals wore Dreem Headband during a single night of polysomnography.

CORE ANALYTICS

Comparison of summary metrics, bias, and epoch-by-epoch analysis.

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Correlation of summary metrics with demographic and Parkinson's disease characteristics.

CORE OUTCOMES

Summary statistics showed Dreem Headband overestimated several sleep metrics, including total sleep, efficiency, deep sleep, and rapid eye movement sleep, with an exception in light sleep. Epoch-by-epoch analysis showed greater specificity than sensitivity, with adequate accuracy across sleep stages (0.55-0.82).

IMPORTANT SUPPLEMENTAL OUTCOMES

Greater Parkinson's disease duration and rapid eye movement behavior were associated with more wakefulness, and worse Parkinson's disease motor symptoms were associated with less deep sleep.

CORE CONCLUSION

The Dreem Headband performs similarly in Parkinson's disease as it did in non-Parkinson's disease samples and shows promise for improving access to sleep assessment in people with Parkinson's disease.

Evaluating Accuracy in Five Commercial Sleep-Tracking Devices Compared to Research-Grade Actigraphy and Polysomnography

The development of consumer sleep-tracking technologies has outpaced the scientific evaluation of their accuracy. In this study, five consumer sleep-tracking devices, research-grade actigraphy, and polysomnography were used simultaneously to monitor the overnight sleep of fifty-three young adults in the lab for one night. Biases and limits of agreement were assessed to determine how sleep stage estimates for each device and research-grade actigraphy differed from polysomnography-derived measures. Every device, except the Garmin Vivosmart, was able to estimate total sleep time comparably to research-grade actigraphy. All devices overestimated nights with shorter wake times and underestimated nights with longer wake times. For light sleep, absolute bias was low for the Fitbit Inspire and Fitbit Versa. The Withings Mat and Garmin Vivosmart overestimated shorter light sleep and underestimated longer light sleep. The Oura Ring underestimated light sleep of any duration. For deep sleep, bias was low for the Withings Mat and Garmin Vivosmart while other devices overestimated shorter and underestimated longer times. For REM sleep, bias was low for all devices. Taken together, these results suggest that proportional bias patterns in consumer sleep-tracking technologies are prevalent and could have important implications for their overall accuracy.

Consumer preferences for sleep-tracking wearables: The role of scientific evaluation and endorsement

OBJECTIVES

Accuracy and relevance to health outcomes are important to researchers and clinicians who use consumer sleep technologies, but economic demand motivates consumer sleep technology design. This report quantifies the value of scientific relevance to the general consumer in a dollar amount to convey the importance of device accuracy in terms that consumer sleep technology manufacturers can appreciate.

METHODS

Survey data were collected from 368 participants on Amazon mTurk. Participants ranked sleep metrics, evaluation methods, and scientific endorsement by perceived level of importance. Participants indicated their likelihood of purchasing a hypothetical consumer sleep technology that had either (1) not been evaluated or endorsed; (2) had been evaluated but not endorsed, and; (3) had been evaluated and endorsed by a sleep science authority. Demand curves determined the relative value of each consumer sleep technology.

RESULTS

Devices that were evaluated and endorsed had the most value, followed by those only evaluated, and then those with no evaluation. The unit price at which there was 50% probability of purchase increased by $30 or $48 for evaluation or endorsement, respectively, relative to a nonvalidated device. Respondents indicated the most valuable sleep metric was sleep duration, the most important evaluation method was against laboratory/hospital standards for sleep, and that the highest value of endorsement came from a medical institution.

CONCLUSIONS

Consumer demand is greatest for a device that has been evaluated by an independent laboratory and is endorsed by a medical institution. Consumer sleep technology manufacturers may be able to increase sales by partnering with sleep science authorities to produce a scientifically superior device.

Performance evaluation of the open-source Yet Another Spindle Algorithm sleep staging algorithm against gold standard manual evaluation of polysomnographic records in adolescence

GOAL AND AIMS

To evaluate an automatic sleep scoring algorithm against manual polysomnography sleep scoring.

FOCUS METHOD/TECHNOLOGY

Yet Another Spindle Algorithm automatic sleep staging algorithm.

REFERENCE METHOD/TECHNOLOGY

Manual sleep scoring.

SAMPLE

327 nights (151 healthy adolescents), from the NCANDA study.

DESIGN

Participants underwent one-to-three overnight polysomnography recordings, one consisting of an event-related-potential paradigm.

CORE ANALYTICS

Epoch by Epoch and discrepancy analyses (Bland Altman plots) were conducted on the overall sample.

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Epoch by Epoch and discrepancy analysis were repeated separately on standard polysomnography nights and event-related potential nights. Regression models were estimated on age, sex, scorer, and site of recording, separately on standard polysomnography nights and event-related potential nights.

CORE OUTCOMES

The Yet Another Spindle Algorithm sleep scoring algorithm's average sensitivity of 93.04% for Wake, 87.67% for N2, 84.46% for N3, 86.02% for rapid-eye-movement, and 40.39% for N1. Specificity was 96.75% for Wake, 97.31% for N1, 88.87% for N2, 97.99% for N3, and 97.70% for rapid-eye-movement. The Matthews Correlation Coefficient was highest in rapid-eye-movement sleep (0.85) while lowest in N1 (0.39). Cohen's Kappa mirrored Matthews Correlation Coefficient results. In Bland-Altman plots, the bias between Yet Another Spindle Algorithm and human scoring showed proportionality to the manual scoring measurement size.

IMPORTANT ADDITIONAL OUTCOMES

Yet Another Spindle Algorithm performance was reduced in event-related-potential/polysomnography nights for N3 and rapid-eye-movement. According to the Matthews Correlation Coefficient, the Yet Another Spindle Algorithm performance was affected by younger age, male sex, recording sites, and scorers.

CORE CONCLUSION

Results support the use of Yet Another Spindle Algorithm to score adolescents' polysomnography sleep records, possibly with classification outcomes supervised by an expert scorer.

Three Contactless Sleep Technologies Compared With Actigraphy and Polysomnography in a Heterogeneous Group of Older Men and Women in a Model of Mild Sleep Disturbance: Sleep Laboratory Study

BACKGROUND

Contactless sleep technologies (CSTs) hold promise for longitudinal, unobtrusive sleep monitoring in the community and at scale. They may be particularly useful in older populations wherein sleep disturbance, which may be indicative of the deterioration of physical and mental health, is highly prevalent. However, few CSTs have been evaluated in older people.

OBJECTIVE

This study evaluated the performance of 3 CSTs compared to polysomnography (PSG) and actigraphy in an older population.

METHODS

Overall, 35 older men and women (age: mean 70.8, SD 4.9 y; women: n=14, 40%), several of whom had comorbidities, including sleep apnea, participated in the study. Sleep was recorded simultaneously using a bedside radar (Somnofy [Vital Things]: n=17), 2 undermattress devices (Withings sleep analyzer [WSA; Withings Inc]: n=35; Emfit-QS [Emfit; Emfit Ltd]: n=17), PSG (n=35), and actigraphy (Actiwatch Spectrum [Philips Respironics]: n=18) during the first night in a 10-hour time-in-bed protocol conducted in a sleep laboratory. The devices were evaluated through performance metrics for summary measures and epoch-by-epoch classification. PSG served as the gold standard.

RESULTS

The protocol induced mild sleep disturbance with a mean sleep efficiency (SEFF) of 70.9% (SD 10.4%; range 52.27%-92.60%). All 3 CSTs overestimated the total sleep time (TST; bias: >90 min) and SEFF (bias: >13%) and underestimated wake after sleep onset (bias: >50 min). Sleep onset latency was accurately detected by the bedside radar (bias: <6 min) but overestimated by the undermattress devices (bias: >16 min). CSTs did not perform as well as actigraphy in estimating the all-night sleep summary measures. In an epoch-by-epoch concordance analysis, the bedside radar performed better in discriminating sleep versus wake (Matthew correlation coefficient [MCC]: mean 0.63, SD 0.12, 95% CI 0.57-0.69) than the undermattress devices (MCC of WSA: mean 0.41, SD 0.15, 95% CI 0.36-0.46; MCC of Emfit: mean 0.35, SD 0.16, 95% CI 0.26-0.43). The accuracy of identifying rapid eye movement and light sleep was poor across all CSTs, whereas deep sleep (ie, slow wave sleep) was predicted with moderate accuracy (MCC: >0.45) by both Somnofy and WSA. The deep sleep duration estimates of Somnofy correlated (r2=0.60; P<.01) with electroencephalography slow wave activity (0.75-4.5 Hz) derived from PSG, whereas for the undermattress devices, this correlation was not significant (WSA: r2=0.0096, P=.58; Emfit: r2=0.11, P=.21).

CONCLUSIONS

These CSTs overestimated the TST, and sleep stage prediction was unsatisfactory in this group of older people in whom SEFF was relatively low. Although it was previously shown that CSTs provide useful information on bed occupancy, which may be useful for particular use cases, the performance of these CSTs with respect to the TST and sleep stage estimation requires improvement before they can serve as an alternative to PSG in estimating most sleep variables in older individuals.

Performance of an open machine learning model to classify sleep/wake from actigraphy across ∼24-hour intervals without knowledge of rest timing

GOAL AND AIMS

Commonly used actigraphy algorithms are designed to operate within a known in-bed interval. However, in free-living scenarios this interval is often unknown. We trained and evaluated a sleep/wake classifier that operates on actigraphy over ∼24-hour intervals, without knowledge of in-bed timing.

FOCUS TECHNOLOGY

Actigraphy counts from ActiWatch Spectrum devices.

REFERENCE TECHNOLOGY

Sleep staging derived from polysomnography, supplemented by observation of wakefulness outside of the staged interval. Classifications from the Oakley actigraphy algorithm were additionally used as performance reference.

SAMPLE

Adults, sleeping in either a home or laboratory environment.

DESIGN

Machine learning was used to train and evaluate a sleep/wake classifier in a supervised learning paradigm. The classifier is a temporal convolutional network, a form of deep neural network.

CORE ANALYTICS

Performance was evaluated across ∼24 hours, and additionally restricted to only in-bed intervals, both in terms of epoch-by-epoch performance, and the discrepancy of summary statistics within the intervals.

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Performance of the trained model applied to the Multi-Ethnic Study of Atherosclerosis dataset.

CORE OUTCOMES

Over ∼24 hours, the temporal convolutional network classifier produced the same or better performance as the Oakley classifier on all measures tested. When restricting analysis to the in-bed interval, the temporal convolutional network remained favorable on several metrics.

IMPORTANT SUPPLEMENTAL OUTCOMES

Performance decreased on the Multi-Ethnic Study of Atherosclerosis dataset, especially when restricting analysis to the in-bed interval.

CORE CONCLUSION

A classifier using data labeled over ∼24-hour intervals allows for the continuous classification of sleep/wake without knowledge of in-bed intervals. Further development should focus on improving generalization performance.

Smart sleep: what to consider when adopting AI-enabled solutions in clinical practice of sleep medicine

Since the publication of its 2020 position statement on artificial intelligence (AI) in sleep medicine by the American Academy of Sleep Medicine, there has been a tremendous expansion of AI-related software and hardware options for sleep clinicians. To help clinicians understand the current state of AI and sleep medicine, and to further enable these solutions to be adopted into clinical practice, a discussion panel was conducted on June 7, 2022, at the Associated Professional Sleep Societies Sleep Conference in Charlotte, North Carolina. The article is a summary of key discussion points from this session, including aspects of considerations for the clinician in evaluating AI-enabled solutions including but not limited to what steps might be taken both by the Food and Drug Administration and clinicians to protect patients, logistical issues, technical challenges, billing and compliance considerations, education and training considerations, and other unique challenges specific to AI-enabled solutions. Our summary of this session is meant to support clinicians in efforts to assist in the clinical care of patients with sleep disorders utilizing AI-enabled solutions.

CITATION

Bandyopadhyay A, Bae C, Cheng H, et al. Smart sleep: what to consider when adopting AI-enabled solutions in clinical practice of sleep medicine. J Clin Sleep Med. 2023;19(10):1823-1833.

Sleep and Breathing Conference highlights 2023: a summary by ERS Assembly 4

This paper presents some of the highlights of the Sleep and Breathing Conference 2023 https://bit.ly/46MxJml.

Challenges of Applying Automated Polysomnography Scoring at Scale

Automatic polysomnography analysis can be leveraged to shorten scoring times, reduce associated costs, and ultimately improve the overall diagnosis of sleep disorders. Multiple and diverse strategies have been attempted for implementation of this technology at scale in the routine workflow of sleep centers. The field, however, is complex and presents unsolved challenges in a number of areas. Recent developments in computer science and artificial intelligence are nevertheless closing the gap. Technological advances are also opening new pathways for expanding our current understanding of the domain and its analysis.

Consumer Wearable Sleep Trackers: Are They Ready for Clinical Use?

As the importance of good sleep continues to gain public recognition, the market for sleep-monitoring devices continues to grow. Modern technology has shifted from simple sleep tracking to a more granular sleep health assessment. We examine the available functionalities of consumer wearable sleep trackers (CWSTs) and how they perform in healthy individuals and disease states. Additionally, the continuum of sleep technology from consumer-grade to medical-grade is detailed. As this trend invariably grows, we urge professional societies to develop guidelines encompassing the practical clinical use of CWSTs and how best to incorporate them into patient care plans.

Performance evaluation of Fitbit Charge 3 and actigraphy vs. polysomnography: Sensitivity, specificity, and reliability across participants and nights

GOAL AND AIMS

Compare the accuracy and reliability of sleep/wake classification between the Fitbit Charge 3 and the Micro Motionlogger actigraph when applying either the Cole-Kripke or Sadeh scoring algorithms. Accuracy was established relative to simultaneous Polysomnography recording. Focus technology: Fitbit Charge 3 and actigraphy. Reference technology: Polysomnography.

SAMPLE

Twenty-one university students (10 females).

DESIGN

Simultaneous Fitbit Charge 3, actigraphy, and polysomnography were recorded over 3 nights at the participants' homes.

CORE ANALYTICS

Total sleep time, wake after sleep onset, sensitivity, specificity, positive predictive value, and negative predictive value.

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Variability of specificity and negative predictive value across subjects and across nights.

CORE OUTCOMES

Fitbit Charge 3 and actigraphy using the Cole-Kripke or Sadeh algorithms exhibited similar sensitivity in classifying sleep segments relative to polysomnography (sensitivity of 0.95, 0.96, and 0.95, respectively). Fitbit Charge 3 was significantly more accurate in classifying wake segments (specificity of 0.69, 0.33, and 0.29, respectively). Fitbit Charge 3 also exhibited significantly higher positive predictive value than actigraphy (0.99 vs. 0.97 and 0.97, respectively) and a negative predictive value that was significantly higher only relative to the Sadeh algorithm (0.41 vs. 0.25, respectively).

IMPORTANT ADDITIONAL OUTCOMES

Fitbit Charge 3 exhibited significantly lower standard deviation in specificity values across subjects and negative predictive value across nights.

CORE CONCLUSION

This study demonstrates that Fitbit Charge 3 is more accurate and reliable in identifying wake segments than the examined FDA-approved Micro Motionlogger actigraphy device. The results also highlight the need to create devices that record and save raw multi-sensor data, which are necessary for developing open-source sleep or wake classification algorithms.

Sleep medicine provider perceptions and attitudes regarding consumer sleep technology

STUDY OBJECTIVES

This study assessed perceptions and attitudes of sleep medicine providers regarding consumer sleep technology (CST).

METHODS

A convenience sample of n = 176 practicing sleep medicine and behavioral sleep medicine experts was obtained using social media and the American Academy of Sleep Medicine directory. Providers completed a questionnaire that assessed perceptions and attitudes about patient use of CST in the clinical setting.

RESULTS

The sample included both adult and pediatric psychologists, physicians, and advanced practice providers from a variety of health settings. Providers reported 36% (3%-95%) of patients used CST, and the most common devices seen by providers were wrist-worn devices followed by smartphone apps. The most common perceived patient motivations for frequent use were to measure sleep and self-discovery. Across sleep disorders, clinicians did not endorse frequent CST use; the highest reported use was for assisting patients in the completion of sleep diaries. Overall devices were rated as somewhat accurate and neutral regarding helpfulness. In qualitative responses, providers associated CST use with increased patient engagement but increased orthosomnia and misperceptions about sleep.

CONCLUSIONS

CST is frequently encountered in the sleep medicine clinic, and providers view CST as somewhat accurate but neither helpful nor unhelpful in clinical practice. Although providers viewed these devices as useful to drive patient engagement/awareness and track sleep patterns, providers also viewed them as a contributor to orthosomnia and misperceptions about sleep.

CITATION

Addison C, Grandner MA, Baron KG. Sleep medicine provider perceptions and attitudes regarding consumer sleep technology. J Clin Sleep Med. 2023;19(8):1457-1463.

Belun Ring (Belun Sleep System BLS-100): Deep learning-facilitated wearable enables obstructive sleep apnea detection, apnea severity categorization, and sleep stage classification in patients suspected of obstructive sleep apnea

GOAL AND AIMS

Our objective was to evaluate the performance of Belun Ring with second-generation deep learning algorithms in obstructive sleep apnea (OSA) detection, OSA severity categorization, and sleep stage classification.

FOCUS TECHNOLOGY

Belun Ring with second-generation deep learning algorithms REFERENCE TECHNOLOGY: In-lab polysomnography (PSG) SAMPLE: Eighty-four subjects (M: F = 1:1) referred for an overnight sleep study were eligible. Of these, 26% had PSG-AHI<5; 24% had PSG-AHI 5-15; 23% had PSG-AHI 15-30; 27% had PSG-AHI ≥ 30.

DESIGN

Rigorous performance evaluation by comparing Belun Ring to concurrent in-lab PSG using the 4% rule.

CORE ANALYTICS

Pearson's correlation coefficient, Student's paired t-test, diagnostic accuracy, sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, Cohen's kappa coefficient (kappa), Bland-Altman plots with bias and limits of agreement, receiver operating characteristics curves with area under the curve, and confusion matrix.

CORE OUTCOMES

The accuracy, sensitivity, specificity, and kappa in categorizing AHI ≥ 5 were 0.85, 0.92, 0.64, and 0.58, respectively. The accuracy, sensitivity, specificity, and Kappa in categorizing AHI ≥ 15 were 0.89, 0.91, 0.88, and 0.79, respectively. The accuracy, sensitivity, specificity, and Kappa in categorizing AHI ≥ 30 were 0.91, 0.83, 0.93, and 0.76, respectively. BSP2 also achieved an accuracy of 0.88 in detecting wake, 0.82 in detecting NREM, and 0.90 in detecting REM sleep.

CORE CONCLUSION

Belun Ring with second-generation algorithms detected OSA with good accuracy and demonstrated a moderate-to-substantial agreement in categorizing OSA severity and classifying sleep stages.

Evaluation of a device-agnostic approach to predict sleep from raw accelerometry data collected by Apple Watch Series 7, Garmin Vivoactive 4, and ActiGraph GT9X Link in children with sleep disruptions

GOAL AND AIMS

Evaluate the performance of a sleep scoring algorithm applied to raw accelerometry data collected from research-grade and consumer wearable actigraphy devices against polysomnography.

FOCUS METHOD/TECHNOLOGY

Automatic sleep/wake classification using the Sadeh algorithm applied to raw accelerometry data from ActiGraph GT9X Link, Apple Watch Series 7, and Garmin Vivoactive 4.

REFERENCE METHOD/TECHNOLOGY

Standard manual PSG sleep scoring.

SAMPLE

Fifty children with disrupted sleep (M = 8.5 years, range = 5-12 years, 42% Black, 64% male).

DESIGN

Participants underwent to single night lab polysomnography while wearing ActiGraph, Apple, and Garmin devices.

CORE ANALYTICS

Discrepancy and epoch-by-epoch analyses for sleep/wake classification (devices vs. polysomnography).

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Equivalence testing for sleep/wake classification (research-grade actigraphy vs. commercial devices).

CORE OUTCOMES

Compared to polysomnography, accuracy, sensitivity, and specificity were 85.5, 87.4, and 76.8, respectively, for Actigraph; 83.7, 85.2, and 75.8, respectively, for Garmin; and 84.6, 86.2, and 77.2, respectively, for Apple. The magnitude and trend of bias for total sleep time, sleep efficiency, sleep onset latency, and wake after sleep were similar between the research and consumer wearable devices.

IMPORTANT ADDITIONAL OUTCOMES

Equivalence testing indicated that total sleep time and sleep efficiency estimates from the research and consumer wearable devices were statistically significantly equivalent.

CORE CONCLUSION

This study demonstrates that raw acceleration data from consumer wearable devices has the potential to be harnessed to predict sleep in children. While further work is needed, this strategy could overcome current limitations related to proprietary algorithms for predicting sleep in consumer wearable devices.

Electrocardiogram Monitoring Wearable Devices and Artificial-Intelligence-Enabled Diagnostic Capabilities: A Review

Worldwide, population aging and unhealthy lifestyles have increased the incidence of high-risk health conditions such as cardiovascular diseases, sleep apnea, and other conditions. Recently, to facilitate early identification and diagnosis, efforts have been made in the research and development of new wearable devices to make them smaller, more comfortable, more accurate, and increasingly compatible with artificial intelligence technologies. These efforts can pave the way to the longer and continuous health monitoring of different biosignals, including the real-time detection of diseases, thus providing more timely and accurate predictions of health events that can drastically improve the healthcare management of patients. Most recent reviews focus on a specific category of disease, the use of artificial intelligence in 12-lead electrocardiograms, or on wearable technology. However, we present recent advances in the use of electrocardiogram signals acquired with wearable devices or from publicly available databases and the analysis of such signals with artificial intelligence methods to detect and predict diseases. As expected, most of the available research focuses on heart diseases, sleep apnea, and other emerging areas, such as mental stress. From a methodological point of view, although traditional statistical methods and machine learning are still widely used, we observe an increasing use of more advanced deep learning methods, specifically architectures that can handle the complexity of biosignal data. These deep learning methods typically include convolutional and recurrent neural networks. Moreover, when proposing new artificial intelligence methods, we observe that the prevalent choice is to use publicly available databases rather than collecting new data.

Consumer sleep technology for the screening of obstructive sleep apnea and snoring: current status and a protocol for a systematic review and meta-analysis of diagnostic test accuracy

There are concerns about the validation and accuracy of currently available consumer sleep technology for sleep-disordered breathing. The present report provides a background review of existing consumer sleep technologies and discloses the methods and procedures for a systematic review and meta-analysis of diagnostic test accuracy of these devices and apps for the detection of obstructive sleep apnea and snoring in comparison with polysomnography. The search will be performed in four databases (PubMed, Scopus, Web of Science, and the Cochrane Library). Studies will be selected in two steps, first by an analysis of abstracts followed by full-text analysis, and two independent reviewers will perform both phases. Primary outcomes include apnea-hypopnea index, respiratory disturbance index, respiratory event index, oxygen desaturation index, and snoring duration for both index and reference tests, as well as the number of true positives, false positives, true negatives, and false negatives for each threshold, as well as for epoch-by-epoch and event-by-event results, which will be considered for the calculation of surrogate measures (including sensitivity, specificity, and accuracy). Diagnostic test accuracy meta-analyses will be performed using the Chu and Cole bivariate binomial model. Mean difference meta-analysis will be performed for continuous outcomes using the DerSimonian and Laird random-effects model. Analyses will be performed independently for each outcome. Subgroup and sensitivity analyses will evaluate the effects of the types (wearables, nearables, bed sensors, smartphone applications), technologies (e.g., oximeter, microphone, arterial tonometry, accelerometer), the role of manufacturers, and the representativeness of the samples.

Daytime Sleep-Tracking Performance of Four Commercial Wearable Devices During Unrestricted Home Sleep

Purpose

Previous studies have found that many commercial wearable devices can accurately track sleep-wake patterns in laboratory or home settings. However, nearly all previous studies tested devices under conditions with fixed time in bed (TIB) and during nighttime sleep episodes only. Despite its relevance to shift workers and others with irregular sleep schedules, it is largely unknown how devices track daytime sleep. Therefore, we tested the sleep-tracking performance of four commercial wearable devices during unrestricted home daytime sleep.

Participants and Methods

Participants were 16 healthy young adults (6 men, 10 women; 26.6 ± 4.6 years, mean ± SD) with habitual daytime sleep schedules. Participants slept at home for 1 week under unrestricted conditions (ie, self-selecting TIB) using a set of four commercial wearable devices and completed reference sleep logs. Wearables included the Fatigue Science ReadiBand, Fitbit Inspire HR, Oura Ring, and Polar Vantage V Titan. Daytime sleep episode TIB biases and frequencies of missed and false-positive daytime sleep episodes were examined.

Results

TIB bias was low in general for all devices on most daytime sleep episodes, but some exhibited large biases (eg, >1 h). Total missed daytime sleep episodes were as follows: Fatigue Science: 3.6%; Fitbit: 4.8%; Oura: 6.0%; Polar: 37.3%. Missed episodes occurred most often when TIB was short (eg, naps <4 h).

Conclusion

When daytime sleep episodes were recorded, the devices generally exhibited similar performance for tracking TIB (ie, most episodes had low bias). However, the devices failed to detect some daytime episodes, which occurred most often when TIB was short, but varied across devices (especially Polar, which missed over one-third of episodes). Findings suggest that accurate daytime sleep tracking is largely achievable with commercial wearable devices. However, performance differences for missed recordings suggest that some devices vary in reliability (especially for naps), but improvements could likely be made with changes to algorithm sensitivities.

Sleep characterization with smart wearable devices: a call for standardization and consensus recommendations

The general public increasingly adopts smart wearable devices to quantify sleep characteristics and dedicated devices for sleep assessment. The rapid evolution of technology has outpaced the ability to implement validation approaches and demonstrate relevant clinical applicability. There are untapped opportunities to validate and refine consumer devices in partnership with scientists in academic institutions, patients, and the private sector to allow effective integration into clinical management pathways and facilitate trust in adoption once reliability and validity have been demonstrated. We call for the formation of a working group involving stakeholders from academia, clinical care and industry to develop clear professional recommendations to facilitate appropriate and optimized clinical utilization of such technologies.