Detection of obstructive sleep apnea using Belun Sleep Platform wearable with neural network-based algorithm and its combined use with STOP-Bang questionnaire

Correction: Detection of obstructive sleep apnea using Belun Sleep Platform wearable with neural network-based algorithm and its combined use with STOP-Bang questionnaire

[This corrects the article DOI: 10.1371/journal.pone.0258040.].

State of the science and recommendations for using wearable technology in sleep and circadian research

Wearable sleep-tracking technology is of growing use in the sleep and circadian fields, including for applications across other disciplines, inclusive of a variety of disease states. Patients increasingly present sleep data derived from their wearable devices to their providers and the ever-increasing availability of commercial devices and new-generation research/clinical tools has led to the wide adoption of wearables in research, which has become even more relevant given the discontinuation of the Philips Respironics Actiwatch. Standards for evaluating the performance of wearable sleep-tracking devices have been introduced and the available evidence suggests that consumer-grade devices exceed the performance of traditional actigraphy in assessing sleep as defined by polysomnogram. However, clear limitations exist, for example, the misclassification of wakefulness during the sleep period, problems with sleep tracking outside of the main sleep bout or nighttime period, artifacts, and unclear translation of performance to individuals with certain characteristics or comorbidities. This is of particular relevance when person-specific factors (like skin color or obesity) negatively impact sensor performance with the potential downstream impact of augmenting already existing healthcare disparities. However, wearable sleep-tracking technology holds great promise for our field, given features distinct from traditional actigraphy such as measurement of autonomic parameters, estimation of circadian features, and the potential to integrate other self-reported, objective, and passively recorded health indicators. Scientists face numerous decision points and barriers when incorporating traditional actigraphy, consumer-grade multi-sensor devices, or contemporary research/clinical-grade sleep trackers into their research. Considerations include wearable device capabilities and performance, target population and goals of the study, wearable device outputs and availability of raw and aggregate data, and data extraction, processing, and analysis. Given the difficulties in the implementation and utilization of wearable sleep-tracking technology in real-world research and clinical settings, the following State of the Science review requested by the Sleep Research Society aims to address the following questions. What data can wearable sleep-tracking devices provide? How accurate are these data? What should be taken into account when incorporating wearable sleep-tracking devices into research? These outstanding questions and surrounding considerations motivated this work, outlining practical recommendations for using wearable technology in sleep and circadian research.

Obstructive sleep apnea diagnosis and beyond using portable monitors

Obstructive sleep apnea (OSA) is a chronic sleep and breathing disorder with significant health complications, including cardiovascular disease and neurocognitive impairments. To ensure timely treatment, there is a need for a portable, accurate and rapid method of diagnosing OSA. This review examines the use of various physiological signals used in the detection of respiratory events and evaluates their effectiveness in portable monitors (PM) relative to gold standard polysomnography. The primary objective is to explore the relationship between these physiological parameters and OSA, their application in calculating the apnea hypopnea index (AHI), the standard metric for OSA diagnosis, and the derivation of non-AHI metrics that offer additional diagnostic value. It is found that increasing the number of parameters in PMs does not necessarily improve OSA detection. Several factors can cause performance variations among different PMs, even if they extract similar signals. The review also highlights the potential of PMs to be used beyond OSA diagnosis. These devices possess parameters that can be utilized to obtain endotypic and other non-AHI metrics, enabling improved characterization of the disorder and personalized treatment strategies. Advancements in PM technology, coupled with thorough evaluation and validation of these devices, have the potential to revolutionize OSA diagnosis, personalized treatment, and ultimately improve health outcomes for patients with OSA. By identifying the key factors influencing performance and exploring the application of PMs beyond OSA diagnosis, this review aims to contribute to the ongoing development and utilization of portable, efficient, and effective diagnostic tools for OSA.

Belun Ring (Belun Sleep System BLS-100): Deep learning-facilitated wearable enables obstructive sleep apnea detection, apnea severity categorization, and sleep stage classification in patients suspected of obstructive sleep apnea

GOAL AND AIMS

Our objective was to evaluate the performance of Belun Ring with second-generation deep learning algorithms in obstructive sleep apnea (OSA) detection, OSA severity categorization, and sleep stage classification.

FOCUS TECHNOLOGY

Belun Ring with second-generation deep learning algorithms REFERENCE TECHNOLOGY: In-lab polysomnography (PSG) SAMPLE: Eighty-four subjects (M: F = 1:1) referred for an overnight sleep study were eligible. Of these, 26% had PSG-AHI<5; 24% had PSG-AHI 5-15; 23% had PSG-AHI 15-30; 27% had PSG-AHI ≥ 30.

DESIGN

Rigorous performance evaluation by comparing Belun Ring to concurrent in-lab PSG using the 4% rule.

CORE ANALYTICS

Pearson's correlation coefficient, Student's paired t-test, diagnostic accuracy, sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, Cohen's kappa coefficient (kappa), Bland-Altman plots with bias and limits of agreement, receiver operating characteristics curves with area under the curve, and confusion matrix.

CORE OUTCOMES

The accuracy, sensitivity, specificity, and kappa in categorizing AHI ≥ 5 were 0.85, 0.92, 0.64, and 0.58, respectively. The accuracy, sensitivity, specificity, and Kappa in categorizing AHI ≥ 15 were 0.89, 0.91, 0.88, and 0.79, respectively. The accuracy, sensitivity, specificity, and Kappa in categorizing AHI ≥ 30 were 0.91, 0.83, 0.93, and 0.76, respectively. BSP2 also achieved an accuracy of 0.88 in detecting wake, 0.82 in detecting NREM, and 0.90 in detecting REM sleep.

CORE CONCLUSION

Belun Ring with second-generation algorithms detected OSA with good accuracy and demonstrated a moderate-to-substantial agreement in categorizing OSA severity and classifying sleep stages.

The Role of Novel Digital Clinical Tools in the Screening or Diagnosis of Obstructive Sleep Apnea: Systematic Review

BACKGROUND

Digital clinical tools are a new technology that can be used in the screening or diagnosis of obstructive sleep apnea (OSA), notwithstanding the crucial role of polysomnography, the gold standard.

OBJECTIVE

This study aimed to identify, gather, and analyze the most accurate digital tools and smartphone-based health platforms used for OSA screening or diagnosis in the adult population.

METHODS

We performed a comprehensive literature search of PubMed, Scopus, and Web of Science databases for studies evaluating the validity of digital tools in OSA screening or diagnosis until November 2022. The risk of bias was assessed using the Joanna Briggs Institute critical appraisal tool for diagnostic test accuracy studies. The sensitivity, specificity, and area under the curve (AUC) were used as discrimination measures.

RESULTS

We retrieved 1714 articles, 41 (2.39%) of which were included in the study. From these 41 articles, we found 7 (17%) smartphone-based tools, 10 (24%) wearables, 11 (27%) bed or mattress sensors, 5 (12%) nasal airflow devices, and 8 (20%) other sensors that did not fit the previous categories. Only 8 (20%) of the 41 studies performed external validation of the developed tool. Of these, the highest reported values for AUC, sensitivity, and specificity were 0.99, 96%, and 92%, respectively, for a clinical cutoff of apnea-hypopnea index (AHI)≥30. These values correspond to a noncontact audio recorder that records sleep sounds, which are then analyzed by a deep learning technique that automatically detects sleep apnea events, calculates the AHI, and identifies OSA. Looking at the studies that only internally validated their models, the work that reported the highest accuracy measures showed AUC, sensitivity, and specificity values of 1.00, 100%, and 96%, respectively, for a clinical cutoff AHI≥30. It uses the Sonomat-a foam mattress that, aside from recording breath sounds, has pressure sensors that generate voltage when deformed, thus detecting respiratory movements, and uses it to classify OSA events.

CONCLUSIONS

These clinical tools presented promising results with high discrimination measures (best results reached AUC>0.99). However, there is still a need for quality studies comparing the developed tools with the gold standard and validating them in external populations and other environments before they can be used in clinical settings.

TRIAL REGISTRATION

PROSPERO CRD42023387748; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=387748.

Are current clinical studies on artificial intelligence-based medical devices comprehensive enough to support a full health technology assessment? A systematic review

INTRODUCTION

Artificial Intelligence-based Medical Devices (AI-based MDs) are experiencing exponential growth in healthcare. This study aimed to investigate whether current studies assessing AI contain the information required for health technology assessment (HTA) by HTA bodies.

METHODS

We conducted a systematic literature review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology to extract articles published between 2016 and 2021 related to the assessment of AI-based MDs. Data extraction focused on study characteristics, technology, algorithms, comparators, and results. AI quality assessment and HTA scores were calculated to evaluate whether the items present in the included studies were concordant with the HTA requirements. We performed a linear regression for the HTA and AI scores with the explanatory variables of the impact factor, publication date, and medical specialty. We conducted a univariate analysis of the HTA score and a multivariate analysis of the AI score with an alpha risk of 5 %.

RESULTS

Of 5578 retrieved records, 56 were included. The mean AI quality assessment score was 67 %; 32 % of articles had an AI quality score ≥ 70 %, 50 % had a score between 50 % and 70 %, and 18 % had a score under 50 %. The highest quality scores were observed for the study design (82 %) and optimisation (69 %) categories, whereas the scores were lowest in the clinical practice category (23 %). The mean HTA score was 52 % for all seven domains. 100 % of the studies assessed clinical effectiveness, whereas only 9 % evaluated safety, and 20 % evaluated economic issues. There was a statistically significant relationship between the impact factor and the HTA and AI scores (both p = 0.046).

DISCUSSION

Clinical studies on AI-based MDs have limitations and often lack adapted, robust, and complete evidence. High-quality datasets are also required because the output data can only be trusted if the inputs are reliable. The existing assessment frameworks are not specifically designed to assess AI-based MDs. From the perspective of regulatory authorities, we suggest that these frameworks should be adapted to assess the interpretability, explainability, cybersecurity, and safety of ongoing updates. From the perspective of HTA agencies, we highlight that transparency, professional and patient acceptance, ethical issues, and organizational changes are required for the implementation of these devices. Economic assessments of AI should rely on a robust methodology (business impact or health economic models) to provide decision-makers with more reliable evidence.

CONCLUSION

Currently, AI studies are insufficient to cover HTA prerequisites. HTA processes also need to be adapted because they do not consider the important specificities of AI-based MDs. Specific HTA workflows and accurate assessment tools should be designed to standardise evaluations, generate reliable evidence, and create confidence.

The Belun sleep platform to diagnose obstructive sleep apnea in patients with hypertension and high cardiovascular risk

STUDY OBJECTIVE

Current hypertension guidelines recommend that at-risk individuals be screened for obstructive sleep apnea (OSA). The Belun Ring is a wearable OSA diagnostic device worn on the palmar side of the proximal phalanx of the index finger.

METHODS

We recruited 129 participants (age: 60 ± 8 years, male sex: 88%, BMI: 27 ± 4 kg/m 2 ) with hypertension and high cardiovascular risk for a simultaneous polysomnography and Belun Ring monitoring for one night. Epworth Sleepiness Scale score more than 10 was detected in 27 (21.0%) participants.

RESULTS

In the 127 participants who completed the study, the apnea-hypopnea index (AHI) derived from polysomnography was 18.1 (interquartile range: 33.0) events/h and that derived from the Belun Ring was 19.5 (interquartile range: 23.3) events/h [intraclass correlation coefficient: 0.882, 95% confidence interval (95% CI): 0.837-0.916]. A Bland-Altman plot showed the difference between the Belun Ring and polysomnography AHIs to be -1.3 ± 10.4 events/h. Area under the receiver operating characteristic for the Belun Ring AHI was 0.961 (95% CI: 0.932-0.990, P  < 0.001). When the Belun Ring AHI of at least 15 events/h was used to diagnose OSA, the sensitivity, specificity, positive predictive value, and negative predictive value were 95.7, 77.6, 85.3, and 93.8%, respectively. The overall accuracy was 87.4%. The Cohen's kappa agreement was 0.74 ± 0.09 ( P  < 0.001). Similar results were obtained when the oxygen desaturation index was used to diagnose OSA.

CONCLUSION

A high prevalence of OSA was detected in patients with hypertension and high cardiovascular risk. The Belun Ring is a reliable device for OSA diagnosis similar to polysomnography.

STOP-BANG QUESTIONNAIRE - AN EASY TOOL FOR IDENTIFYING OBSTRUCTIVE SLEEP APNEA SYNDROME IN PATIENTS WITH TYPE 2 DIABETES MELLITUS

BACKGROUND

Patients with type 2 diabetes mellitus (T2DM) have a higher risk of developing obstructive sleep apnea (OSA) compared to the general population. Our study aims to analyze the usefulness of the STOP-BANG score, tool which was not yet validated in patients with diabetets, as a tool that estimates the severity of OSA, in patients with T2DM.

METHODS

120 patients, who answered the STOP-BANG questionnaire and underwent polysomnography, were included in the study. The patients were divided into 3 groups, depending on the severity of OSA, defined by the apnea/hypopnea index (AHI).

RESULTS

A significant percentage of participants (42.1%) had a severe form of OSA (AHI ≥30) and a high percentage of subjects had a STOP-BANG score ≥5 (58.7%), equivalent to a severe form of the disease. The STOP-BANG score increased proportionally with AHI (p<0.001). The area under the ROC curve for the STOP-Bang score indicated an optimal cut-off value of 4.5, with a sensitivity of 88.2% and a specificity of 62.9% (p <0.001), STOP-BANG score ≥5 being an independent predictor for severe OSA in patients with T2DM.

CONCLUSIONS

The STOP-BANG score can be used in patients with diabetes to detect severe OSA in order to establish appropriate therapeutic measures.