Unique Clinical Language Patterns Among Expert Vestibular Providers Can Predict Vestibular Diagnoses

Applications of Natural Language Processing in Otolaryngology: A Scoping Review

OBJECTIVE

To review the current literature on the applications of natural language processing (NLP) within the field of otolaryngology.

DATA SOURCES

MEDLINE, EMBASE, SCOPUS, Cochrane Library, Web of Science, and CINAHL.

METHODS

The preferred reporting Items for systematic reviews and meta-analyzes extension for scoping reviews checklist was followed. Databases were searched from the date of inception up to Dec 26, 2023. Original articles on the application of language-based models to otolaryngology patient care and research, regardless of publication date, were included. The studies were classified under the 2011 Oxford CEBM levels of evidence.

RESULTS

One-hundred sixty-six papers with a median publication year of 2024 (range 1982, 2024) were included. Sixty-one percent (102/166) of studies used ChatGPT and were published in 2023 or 2024. Sixty studies used NLP for clinical education and decision support, 42 for patient education, 14 for electronic medical record improvement, 5 for triaging, 4 for trainee education, 4 for patient monitoring, 3 for telemedicine, and 1 for medical translation. For research, 37 studies used NLP for extraction, classification, or analysis of data, 17 for thematic analysis, 5 for evaluating scientific reporting, and 4 for manuscript preparation.

CONCLUSION

The role of NLP in otolaryngology is evolving, with ChatGPT passing OHNS board simulations, though its clinical application requires improvement. NLP shows potential in patient education and post-treatment monitoring. NLP is effective at extracting data from unstructured or large data sets. There is limited research on NLP in trainee education and administrative tasks. Guidelines for NLP use in research are critical.

Development and Validation of a Machine Learning Model for Detection and Classification of Vertigo

PURPOSE

This study aims to investigate whether artificial intelligence can improve the diagnostic accuracy of vertigo related diseases.

EXPERIMENTAL DESIGN

Based on the clinical guidelines, clinical symptoms and laboratory test results were extracted from electronic medical records as variables. These variables were then input into a machine learning diagnostic model for classification and diagnosis. This study encompasses two primary objectives: Task 1 to distinguish between patients with Benign Paroxysmal Positional Vertigo (BPPV) and non-BPPV. In Task 2, further classifying non-BPPV patients into Ménière's Disease (MD), Vestibular Migraine (VM), and Sudden Sensorineural Hearing Loss accompanied by Vertigo (SSNHLV). The sensitivity, precision, and area under the curve (AUC) metric is primarily used to assess the performance of the machine learning model development phase in a prospective validation cohort.

RESULTS

In our study, 1789 patients were recruited as the training cohort and 1148 patients as the prospective validation cohort. The comprehensive diagnostic performance of the XGBoost model surpasses that of traditional models. The sensitivity, accuracy, and AUC in task 1 were 98.32%, 87.03%, and 0.947, respectively. In task 2, the sensitivity values for MD, SSNHLV, and VM were 89.00%, 100.0%, and 79.40%, respectively. The precision values were 88.80%, 100.0%, and 80.00%, respectively. The AUC values were 0.933, 1.000, and 0.931, respectively. The model can significantly improve the accuracy of diagnosing vertigo diseases.

CONCLUSIONS

This system may enhance the accuracy of classification and diagnosis of vertigo diseases. It offers initial therapy or referrals to clinical doctors, particularly in resource-limited settings.

LEVEL OF EVIDENCE

NA Laryngoscope, 135:1652-1660, 2025.

Machine learning models and classification algorithms in the diagnosis of vestibular migraine: A systematic review and meta-analysis

OBJECTIVES

To perform a systematic review and meta-analysis to evaluate the effectiveness of machine learning (ML) algorithms in the diagnosis of vestibular migraine.

BACKGROUND

Due to the absence of defined biomarkers for diagnosing vestibular migraine (VM), it is valuable to determine which clinical, physical, and exploratory information is most crucial to diagnosing this disease. The use of artificial intelligence tools could streamline this process.

METHODS

This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and searched for records from PubMed, Scopus, and Web of Science. Observational (case-control and cohort) studies were included to assess the ability of artificial intelligence (AI) to distinguish VM from other vestibular disorders. Risk of bias and applicability concerns were assessed using the Quality Assessment of Diagnostic Accuracy Studies-AI tool.

RESULTS

A total of 14 articles were included in the systematic review, and 10 were eligible for meta-analysis. The main inputs included for the ML algorithms were anamnesis (medical history), physical examination, results from audiological and vestibular tests, and imaging. The global sensitivity was 0.85 (95% confidence interval [CI] 0.73-0.92, I2 = 96%), while the global specificity was 0.89 (95% CI 0.84-0.93, I2 = 95%). The pooled diagnostic odds ratio was 48.15 (95% CI 17.64-131.43, I2 = 97%). Using the bivariate model, the area under the curve and for the summary receiver operating characteristic curve, using the 10 available studies, was 0.94 (95% CI 0.86-0.96).

CONCLUSION

Machine learning algorithms could be used as effective tools for the diagnosis process in VM. The use of models trained with three to four inputs yield the highest accuracy, compared to other strategies. However, the design and validation of these studies could be improved to ensure the reproducibility and generalizability of results.

Gender Differences in Letters of Recommendations and Personal Statements for Neurotology Fellowship over 10 Years: A Deep Learning Linguistic Analysis

OBJECTIVE

Personal statements (PSs) and letters of recommendation (LORs) are critical components of the neurotology fellowship application process but can be subject to implicit biases. This study evaluated general and deep learning linguistic differences between the applicant genders over a 10-year span.

STUDY DESIGN

Retrospective cohort.

SETTING

Two institutions.

MAIN OUTCOME MEASURES

PSs and LORs were collected from 2014 to 2023 from two institutions. The Valence Aware Dictionary and Sentiment Reasoner (VADER) natural language processing (NLP) package was used to compare the positive or negative sentiment in LORs and PSs. Next, the deep learning tool, Empath, categorized the text into scores, and Wilcoxon rank sum tests were performed for comparisons between applicant gender.

RESULTS

Among 177 applicants over 10 years, 120 were males and 57 were females. There were no differences in word count or VADER sentiment scores between genders for both LORs and PSs. However, among Empath sentiment categories, male applicants had more words of trust ( p = 0.03) and leadership ( p = 0.002) in LORs. Temporally, the trends show a consistently higher VADER sentiment and Empath "trust" and "leader" in male LORs from 2014 to 2019, after which there was no statistical significance in sentiment scores between genders, and females even have higher scores of trust and leadership in 2023.

CONCLUSIONS

Linguistic content overall favored male applicants because they were more frequently described as trustworthy and leaders. However, the temporal analysis of linguistic differences between male and female applicants found an encouraging trend suggesting a reduction of gender bias in recent years, mirroring an increased composition of women in neurotology over time.

Artificial intelligence and headache

BACKGROUND AND METHODS

In this narrative review, we introduce key artificial intelligence (AI) and machine learning (ML) concepts, aimed at headache clinicians and researchers. Thereafter, we thoroughly review the use of AI in headache, based on a comprehensive literature search across PubMed, Embase and IEEExplore. Finally, we discuss limitations, as well as ethical and political perspectives.

RESULTS

We identified six main research topics. First, natural language processing can be used to effectively extract and systematize unstructured headache research data, such as from electronic health records. Second, the most common application of ML is for classification of headache disorders, typically based on clinical record data, or neuroimaging data, with accuracies ranging from around 60% to well over 90%. Third, ML is used for prediction of headache disease trajectories. Fourth, ML shows promise in forecasting of headaches using self-reported data such as triggers and premonitory symptoms, data from wearable sensors and external data. Fifth and sixth, ML can be used for prediction of treatment responses and inference of treatment effects, respectively, aiming to optimize and individualize headache management.

CONCLUSIONS

The potential uses of AI and ML in headache are broad, but, at present, many studies suffer from poor reporting and lack out-of-sample evaluation, and most models are not validated in a clinical setting.

Diagnostic accuracy and usability of the EMBalance decision support system for vestibular disorders in primary care: proof of concept randomised controlled study results

BACKGROUND

Dizziness and imbalance are common symptoms that are often inadequately diagnosed or managed, due to a lack of dedicated specialists. Decision Support Systems (DSS) may support first-line physicians to diagnose and manage these patients based on personalised data.

AIM

To examine the diagnostic accuracy and application of the EMBalance DSS for diagnosis and management of common vestibular disorders in primary care.

METHODS

Patients with persistent dizziness were recruited from primary care in Germany, Greece, Belgium and the UK and randomised to primary care clinicians assessing the patients with (+ DSS) versus assessment without (- DSS) the EMBalance DSS. Subsequently, specialists in neuro-otology/audiovestibular medicine performed clinical evaluation of each patient in a blinded way to provide the "gold standard" against which the + DSS, - DSS and the DSS as a standalone tool (i.e. without the final decision made by the clinician) were validated.

RESULTS

One hundred ninety-four participants (age range 25-85, mean = 57.7, SD = 16.7 years) were assigned to the + DSS (N = 100) and to the - DSS group (N = 94). The diagnosis suggested by the + DSS primary care physician agreed with the expert diagnosis in 54%, compared to 41.5% of cases in the - DSS group (odds ratio 1.35). Similar positive trends were observed for management and further referral in the + DSS vs. the - DSS group. The standalone DSS had better diagnostic and management accuracy than the + DSS group.

CONCLUSION

There were trends for improved vestibular diagnosis and management when using the EMBalance DSS. The tool requires further development to improve its diagnostic accuracy, but holds promise for timely and effective diagnosis and management of dizzy patients in primary care.

TRIAL REGISTRATION NUMBER

NCT02704819 (clinicaltrials.gov).

Recent Advances in the Application of Artificial Intelligence in Otorhinolaryngology-Head and Neck Surgery

This study presents an up-to-date survey of the use of artificial intelligence (AI) in the field of otorhinolaryngology, considering opportunities, research challenges, and research directions. We searched PubMed, the Cochrane Central Register of Controlled Trials, Embase, and the Web of Science. We initially retrieved 458 articles. The exclusion of non-English publications and duplicates yielded a total of 90 remaining studies. These 90 studies were divided into those analyzing medical images, voice, medical devices, and clinical diagnoses and treatments. Most studies (42.2%, 38/90) used AI for image-based analysis, followed by clinical diagnoses and treatments (24 studies). Each of the remaining two subcategories included 14 studies. Machine learning and deep learning have been extensively applied in the field of otorhinolaryngology. However, the performance of AI models varies and research challenges remain.

A Novel Diagnostic Prediction Model for Vestibular Migraine

BACKGROUND

Increasing morbidity and misdiagnosis of vestibular migraine (VM) gravely affect the treatment of the disease as well as the patients' quality of life. A powerful diagnostic prediction model is of great importance for management of the disease in the clinical setting.

MATERIALS AND METHODS

Patients with a main complaint of dizziness were invited to join this prospective study. The diagnosis of VM was made according to the International Classification of Headache Disorders. Study variables were collected from a rigorous questionnaire survey, clinical evaluation, and laboratory tests for the development of a novel predictive diagnosis model for VM.

RESULTS

A total of 235 patients were included in this study: 73 were diagnosed with VM and 162 were diagnosed with non-VM vertigo. Compared with non-VM vertigo patients, serum magnesium levels in VM patients were lower. Following the logistic regression analysis of risk factors, a predictive model was developed based on 6 variables: age, sex, autonomic symptoms, hypertension, cognitive impairment, and serum Mg²⁺ concentration. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve was 0.856, which was better than some of the reported predictive models.

CONCLUSION

With high sensitivity and specificity, the proposed logistic model has a very good predictive capability for the diagnosis of VM. It can be used as a screening tool as well as a complementary diagnostic tool for primary care providers and other clinicians who are non-experts of VM.