The Development and Accuracy of the THIM Wearable Device for Estimating Sleep and Wakefulness

Performance of consumer wrist-worn sleep tracking devices compared to polysomnography: a meta-analysis

STUDY OBJECTIVES

The use of sleep tracking devices is increasing as people become more aware of the importance of sleep and interested in monitoring their patterns. With many devices on the market, we conducted a meta-analysis comparing sleep scoring data from consumer wrist-worn sleep tracking devices with polysomnography to validate the accuracy of these devices.

METHODS

We retrieved studies from the databases of SCOPUS, EMBASE, Cochrane Library, PubMed, Web of Science, and KoreaMed and OVID Medline up to March 2024. We compared personal data about participants and information on objective sleep parameters.

RESULTS

From 24 studies, data of 798 patient using Fitbit, Jawbone, myCadian watch, WHOOP strap, Garmin, Basis B1, Zulu Watch, Huami Arc, E4 wristband, Fatigue Science Readiband, Apple Watch, or Xiaomi Mi Band 5 were analyzed. There were significant differences in total sleep time (mean difference, -16.854; 95% confidence interval, [-26.332; -7.375]), sleep efficiency (mean difference, -4.691; 95% confidence interval, [-7.079; -2.302]), sleep latency (mean difference, 2.574; 95% confidence interval, [0.606; 4.542]), and wake after sleep onset (mean difference, 13.255; 95% confidence interval, [4.522; 21.988]) between all consumer sleep tracking devices and polysomnography. In subgroup analysis, there was no significant difference in wake after sleep onset between Fitbit and polysomnography. There was also no significant difference in sleep latency between other devices and polysomnography. Fitbit measured sleep latency longer than other devices, and other devices measured wake after sleep onset longer. Based on Begg and Egger's test, there was no publication bias in total sleep time and sleep efficiency.

CONCLUSIONS

Wrist-worn sleep tracking devices, although popular, are not as reliable as polysomnography in measuring key sleep parameters such as total sleep time, sleep efficiency, and sleep latency. Physicians and consumers should be aware of their limitations and interpret results carefully, though they can still be useful for tracking general sleep patterns. Further improvements and clinical studies are needed to enhance their accuracy.

CITATION

Lee YJ, Lee JY, Cho JH, Kang YJ, Choi JH. Performance of consumer wrist-worn sleep tracking devices compared to polysomnography: a meta-analysis. J Clin Sleep Med. 2025;21(3):573-582.

Comparing the efficacy of technology-enabled treatments for insomnia: study protocol for a randomized controlled trial

Chronic insomnia is a prevalent sleep disorder where <1% of patients receive the recommended first-line treatment; Cognitive Behavioural Therapy for Insomnia. Digital technologies and self-managed therapies are scalable solutions to address this critical gap in patient care, but it is presently difficult to know which therapies are best. This study will test the comparative efficacy and cost-benefits of Intensive Sleep Retraining administered by the THIM sleep tracker, Sleep Healthy Using the Internet (SHUTi) treatment program, and their combination (THIM then SHUTi) versus a waitlist control group. This study is a 4 (treatment: +/- THIM and +/- SHUTi) × 3 (time: pretreatment, posttreatment, and 2-month follow-up) randomized controlled trial. Participants who meet the diagnostic criteria for Chronic Insomnia Disorder will be randomized to one of four groups. Sleep and daytime functioning symptoms will be assessed via self-report daily and weekly questionnaires, and objective sleep trackers during treatment and for 2 weeks at pre-treatment, post-treatment, and 2-month follow-up. The primary outcome is total wake time, with a reduction of ≥30 minutes considered a clinically meaningful difference. For the primary analysis, the interaction between the treatment group and time on total wake time will be analyzed using repeated measures analyses of variance (ANOVA). This project was approved by the Southern Adelaide Clinical Human Research Ethics Committee (2021/HRE00414) and registered in the Australian and New Zealand Clinical Trials Registry (ACTRN12622000778785). As the first study to investigate the comparative efficacy of two different technology-enabled treatments for insomnia, this study will help inform clinicians and public health policy regarding the use cases for public and private health-funded technology-enabled options for insomnia.

SLUMBR: SLeep statUs estiMation from aBdominal Respiratory effort

Accurately monitoring sleep for extended periods remains a challenge due to the cumbersome nature of conventional gold-standard techniques. We propose a novel deep learning method to estimate sleep status from an easily acquired abdominal respiratory effort signal. Our end-to-end convolutional neural network, developed on 476 hours of manually annotated polysomnography recordings from 53 participants, achieves an area under the curve of 0.90, and a more balanced performance across sensitivity and specificity than previous studies: 0.85 and 0.82, respectively. This method eliminates the need for obtrusive equipment and manual processing, paving the way for more accessible sleep monitoring solutions.

Validity and reliability of the Oura Ring Generation 3 (Gen3) with Oura sleep staging algorithm 2.0 (OSSA 2.0) when compared to multi-night ambulatory polysomnography: A validation study of 96 participants and 421,045 epochs

PURPOSE

To evaluate the validity and the reliability of the Oura Ring Generation 3 (Gen3) with Oura Sleep Staging Algorithm 2.0 (OSSA 2.0) through multi-night polysomnography (PSG).

PARTICIPANTS AND METHODS

Participants were 96 generally healthy Japanese men and women aged between 20 and 70 years contributing with 421,045 30-s epochs. Sleep scoring was performed according to American Academy of Sleep Medicine criteria. Each participant could contribute with a maximum of three polysomnography (PSG) nights. Within-participant means were created for each sleep measure and paired t-tests were used to compare equivalent measures obtained from the PSG and Oura Rings (non-dominant and dominant hand). Agreement between sleep measures were assessed using Bland-Altman plots. Interrater reliability for epoch accuracy was determined by prevalence-adjusted and bias-adjusted kappa (PABAK).

RESULTS

The Oura Ring did not significantly differ from PSG for the measures time in bed, total sleep time, sleep onset latency, sleep period time, wake after sleep onset, time spent in light sleep, and time spent in deep sleep. Oura Rings worn on the non-dominant- and dominant-hand underestimated sleep efficiency by 1.1 %-1.5 % and time spent in REM sleep by 4.1-5.6 min. The Oura Ring had a sensitivity of 94.4 %-94.5 %, specificity of 73.0 %-74.6 %, a predictive value for sleep of 95.9 %-96.1 %, a predictive value for wake of 66.6 %-67.0 %, and accuracy of 91.7 %-91.8 %. PABAK was 0.83-0.84 and reliability was 94.8 %. Sleep staging accuracy ranged between 75.5 % (light sleep) and 90.6 % (REM sleep).

CONCLUSIONS

The Oura Ring Gen3 with OSSA 2.0 shows good agreement with PSG for global sleep measures and time spent in light and deep sleep.

The efficacy of intensive sleep retraining for insomnia: A systematic review and research agenda

Intensive sleep retraining (ISR) is a brief behavioural treatment for sleep onset insomnia, administered in just a single overnight treatment session. This systematic review evaluates existing trials about the efficacy of intensive sleep retraining for treating insomnia, to inform whether there is enough evidence to recommend its use for clinical practice. A systematic literature search was conducted across three databases, yielding 108 results. Of these studies, three were deemed suitable for inclusion in this review. The included studies consistently reported significant reductions in insomnia symptoms following intensive sleep retraining, particularly decreases in sleep diary-derived sleep latency and increases in total sleep time. Based on these inconclusive but promising findings, a research agenda is proffered to test intensive sleep retraining as a treatment for insomnia. Large randomised controlled trials are needed to elucidate the potential benefits of intensive sleep retraining for different populations with insomnia, as are mechanistic trials to test which components underlie its seemingly therapeutic effects. Since more practical modalities of intensive sleep retraining administration have been developed, such trials are more feasible to conduct now than ever before.

The effectiveness of exercise on global cognitive function, balance, depression symptoms, and sleep quality in patients with mild cognitive impairment: A systematic review and meta-analysis

This review aimed to examine the effectiveness of exercise on global cognitive function, balance, depression symptoms, and sleep quality in patients with mild cognitive impairment. And systematically retrieved five electronic databases, including the Cochrane library, PubMed, Embase, Web of Science, and PsycINFO, from inception to May 2022. Of 1102 studies, twenty-one studies were included in this meta-analysis. The polled results revealed that exercise could significantly improve global cognitive function (SMD = 0.64, 95%CI: 0.36 to 0.91, Z = 4.56, P < 0.00001), balance (SMD = 0.62, 95%CI: 0.30 to 0.95, Z = 4.56, P = 0.0001) and depression symptoms (SMD = -0.37, 95%CI: -0.64 to -0.10, Z = 2.70, P = 0.007). The exercise was a promising intervention with the potential to be applied in people with mild cognitive impairment.

Wearable Device-Delivered Intensive Sleep Retraining as an Adjunctive Treatment to Kickstart Cognitive-Behavioral Therapy for Insomnia

Intensive Sleep Retraining is a behavioral treatment for sleep onset insomnia that produces substantial benefits in symptoms after a single treatment session. This technique involves falling asleep and waking up shortly afterward repeatedly: a process that is thought to retrain people to fall asleep quickly when attempting sleep. Although originally confined to the sleep laboratory, recent technological developments mean that this technique is feasible to self-administer at home. With multiple randomised controlled trials required to confirm its efficacy, Intensive Sleep Retraining may serve as an adjunctive treatment to cognitive-behavioral therapy for insomnia, improving short-term efficacy by kick-starting treatment gains.

Pyjamas, Polysomnography and Professional Athletes: The Role of Sleep Tracking Technology in Sport

Technological advances in sleep monitoring have seen an explosion of devices used to gather important sleep metrics. These devices range from instrumented 'smart pyjamas' through to at-home polysomnography devices. Alongside these developments in sleep technologies, there have been concomitant increases in sleep monitoring in athletic populations, both in the research and in practical settings. The increase in sleep monitoring in sport is likely due to the increased knowledge of the importance of sleep in the recovery process and performance of an athlete, as well as the well-reported challenges that athletes can face with their sleep. This narrative review will discuss: (1) the importance of sleep to athletes; (2) the various wearable tools and technologies being used to monitor sleep in the sport setting; (3) the role that sleep tracking devices may play in gathering information about sleep; (4) the reliability and validity of sleep tracking devices; (5) the limitations and cautions associated with sleep trackers; and, (6) the use of sleep trackers to guide behaviour change in athletes. We also provide some practical recommendations for practitioners working with athletes to ensure that the selection of such devices and technology will meet the goals and requirements of the athlete.

Emerging applications of objective sleep assessments towards the improved management of insomnia

Self-reported sleep difficulties are the primary concern associated with diagnosis and treatment of chronic insomnia. This said, in-home sleep monitoring technology in combination with self-reported sleep outcomes may usefully assist with the management of insomnia. The rapid acceleration in consumer sleep technology capabilities together with their growing use by consumers means that the implementation of clinically useful techniques to more precisely diagnose and better treat insomnia are now possible. This review describes emerging techniques which may facilitate better identification and management of insomnia through objective sleep monitoring. Diagnostic techniques covered include insomnia phenotyping, better detection of comorbid sleep disorders, and identification of patients potentially at greatest risk of adverse outcomes. Treatment techniques reviewed include the administration of therapies (e.g., Intensive Sleep Retraining, digital treatment programs), methods to assess and improve treatment adherence, and sleep feedback to address concerns about sleep and sleep loss. Gaps in sleep device capabilities are also discussed, such as the practical assessment of circadian rhythms. Proof-of-concept studies remain needed to test these sleep monitoring-supported techniques in insomnia patient populations, with the goal to progress towards more precise diagnoses and efficacious treatments for individuals with insomnia.

Validation of Fitbit Inspire 2TM Against Polysomnography in Adults Considering Adaptation for Use

PURPOSE

The commercialization of sleep activity tracking devices has made it possible to manage sleep quality at home. However, it is necessary to verify the reliability and accuracy of wearable devices through comparison with polysomnography (PSG), which is the standard for tracking sleep activity. This study aimed to monitor overall sleep activity using Fitbit Inspire 2™ (FBI2) and to evaluate its performance and effectiveness through PSG under the same conditions.

PATIENTS AND METHODS

We compared the FBI2 and PSG data of nine participants (four male and five female participants; average age, 39 years) without severe sleeping problems. The participants wore FBI2 continuously for 14 days, considering the period of adaptation to the device. FBI2 and PSG sleep data were compared using paired t-tests, Bland-Altman plots, and epoch-by-epoch analysis for 18 samples by pooling data from two replicates.

RESULTS

The average values for each sleep stage obtained from FBI2 and PSG showed significant differences in the total sleep time (TST), deep sleep, and rapid eye motion (REM). In the Bland-Altman analysis, TST (P = 0.02), deep sleep (P = 0.05), and REM (P = 0.03) were significantly overstated in FBI2 compared to PSG. In addition, time in bed, sleep efficiency, and wake after sleep onset were overestimated, while light sleep was underestimated. However, these differences were not statistically significant. FBI2 showed a high sensitivity (93.9%) and low specificity (13.1%), with an accuracy of 76%. The sensitivity and specificity of each sleep stage was 54.3% and 62.3%, respectively, for light sleep, 84.8% and 50.1%, respectively, for deep sleep, and 86.4% and 59.1%, respectively for REM sleep.

CONCLUSION

The use of FBI2 as an objective tool for measuring sleep in daily life can be considered appropriate. However, further research is warranted on its application in participants with sleep-wake problems.

New and Emerging Approaches to Better Define Sleep Disruption and Its Consequences

Current approaches to quantify and diagnose sleep disorders and circadian rhythm disruption are imprecise, laborious, and often do not relate well to key clinical and health outcomes. Newer emerging approaches that aim to overcome the practical and technical constraints of current sleep metrics have considerable potential to better explain sleep disorder pathophysiology and thus to more precisely align diagnostic, treatment and management approaches to underlying pathology. These include more fine-grained and continuous EEG signal feature detection and novel oxygenation metrics to better encapsulate hypoxia duration, frequency, and magnitude readily possible via more advanced data acquisition and scoring algorithm approaches. Recent technological advances may also soon facilitate simple assessment of circadian rhythm physiology at home to enable sleep disorder diagnostics even for “non-circadian rhythm” sleep disorders, such as chronic insomnia and sleep apnea, which in many cases also include a circadian disruption component. Bringing these novel approaches into the clinic and the home settings should be a priority for the field. Modern sleep tracking technology can also further facilitate the transition of sleep diagnostics from the laboratory to the home, where environmental factors such as noise and light could usefully inform clinical decision-making. The “endpoint” of these new and emerging assessments will be better targeted therapies that directly address underlying sleep disorder pathophysiology via an individualized, precision medicine approach. This review outlines the current state-of-the-art in sleep and circadian monitoring and diagnostics and covers several new and emerging approaches to better define sleep disruption and its consequences.

Impact of COVID-19 social-distancing on sleep timing and duration during a university semester

Social-distancing directives to contain community transmission of the COVID-19 virus can be expected to affect sleep timing, duration or quality. Remote work or school may increase time available for sleep, with benefits for immune function and mental health, particularly in those individuals who obtain less sleep than age-adjusted recommendations. Young adults are thought to regularly carry significant sleep debt related in part to misalignment between endogenous circadian clock time and social time. We examined the impact of social-distancing measures on sleep in young adults by comparing sleep self-studies submitted by students enrolled in a university course during the 2020 summer session (entirely remote instruction, N = 80) with self-studies submitted by students enrolled in the same course during previous summer semesters (on-campus instruction, N = 452; cross-sectional study design). Self-studies included 2–8 week sleep diaries, two chronotype questionnaires, written reports, and sleep tracker (Fitbit) data from a subsample. Students in the 2020 remote instruction semester slept later, less efficiently, less at night and more in the day, but did not sleep more overall despite online, asynchronous classes and ~44% fewer work days compared to students in previous summers. Subjectively, the net impact on sleep was judged as positive or negative in equal numbers of students, with students identifying as evening types significantly more likely to report a positive impact, and morning types a negative impact. Several features of the data suggest that the average amount of sleep reported by students in this summer course, historically and during the 2020 remote school semester, represents a homeostatic balance, rather than a chronic deficit. Regardless of the interpretation, the results provide additional evidence that social-distancing measures affect sleep in heterogeneous ways.

A Systematic Review of Sensing Technologies for Wearable Sleep Staging

Designing wearable systems for sleep detection and staging is extremely challenging due to the numerous constraints associated with sensing, usability, accuracy, and regulatory requirements. Several researchers have explored the use of signals from a subset of sensors that are used in polysomnography (PSG), whereas others have demonstrated the feasibility of using alternative sensing modalities. In this paper, a systematic review of the different sensing modalities that have been used for wearable sleep staging is presented. Based on a review of 90 papers, 13 different sensing modalities are identified. Each sensing modality is explored to identify signals that can be obtained from it, the sleep stages that can be reliably identified, the classification accuracy of systems and methods using the sensing modality, as well as the usability constraints of the sensor in a wearable system. It concludes that the two most common sensing modalities in use are those based on electroencephalography (EEG) and photoplethysmography (PPG). EEG-based systems are the most accurate, with EEG being the only sensing modality capable of identifying all the stages of sleep. PPG-based systems are much simpler to use and better suited for wearable monitoring but are unable to identify all the sleep stages.

Automatic Sleep Stage Classification of Children with Sleep-Disordered Breathing Using the Modularized Network

PURPOSE

To develop an automatic sleep stage analysis model for children and evaluate the effect of the model on the diagnosis of sleep-disordered breathing (SDB).

PATIENTS AND METHODS

Three hundred and forty-four SDB patients aged between 2 to 18 years who completed polysomnography (PSG) to assess the severity of the disease were enrolled in this study. We developed deep neural networks to stage sleep from electroencephalography (EEG), electrooculography (EOG) and electromyogram (EMG). The model performance was estimated by accuracy, precision, recall, F1-score, and Cohen's Kappa coefficient (ĸ). And we compared the difference in calculation of sleep parameters among the technicians, the model ensemble, and the single-channel EEG model.

RESULTS

The numbers of raw data divided into training, validation, and testing were 240, 36, and 68, respectively. The best performance appeared in the model ensemble of which the accuracy was 83.36% (ĸ=0.7817) in 5-stages, and the accuracy was 96.76% (ĸ=0.8236) in 2-stages. The single-channel EEG model showed the classification satisfyingly as well. There was no significant difference in TST, SE, SOL, time in W, time in N1+N2, time in N3, and OAHI between technician and the model (P>0.05). On the datasets from sleep-EDF-13 and sleep-EDF-18, the average classification accuracies achieved were 92.76% and 91.94% in 5-stages by using the proposed method, respectively.

CONCLUSION

This research established the model for pediatric automatic sleep stage classification with satisfying reliability and generalizability. In addition, it could be applied for calculating quantitative sleep parameters and evaluating the severity of SDB.