1
|
Ostrowska M, Kacała P, Onolememen D, Vaughan-Lane K, Sisily Joseph A, Ostrowski A, Pietruszewska W, Banaszewski J, Wróbel MJ. To trust or not to trust: evaluating the reliability and safety of AI responses to laryngeal cancer queries. Eur Arch Otorhinolaryngol 2024; 281:6069-6081. [PMID: 38652298 PMCID: PMC11512842 DOI: 10.1007/s00405-024-08643-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2024] [Accepted: 03/26/2024] [Indexed: 04/25/2024]
Abstract
PURPOSE As online health information-seeking surges, concerns mount over the quality and safety of accessible content, potentially leading to patient harm through misinformation. On one hand, the emergence of Artificial Intelligence (AI) in healthcare could prevent it; on the other hand, questions raise regarding the quality and safety of the medical information provided. As laryngeal cancer is a prevalent head and neck malignancy, this study aims to evaluate the utility and safety of three large language models (LLMs) as sources of patient information about laryngeal cancer. METHODS A cross-sectional study was conducted using three LLMs (ChatGPT 3.5, ChatGPT 4.0, and Bard). A questionnaire comprising 36 inquiries about laryngeal cancer was categorised into diagnosis (11 questions), treatment (9 questions), novelties and upcoming treatments (4 questions), controversies (8 questions), and sources of information (4 questions). The population of reviewers consisted of 3 groups, including ENT specialists, junior physicians, and non-medicals, who graded the responses. Each physician evaluated each question twice for each model, while non-medicals only once. Everyone was blinded to the model type, and the question order was shuffled. Outcome evaluations were based on a safety score (1-3) and a Global Quality Score (GQS, 1-5). Results were compared between LLMs. The study included iterative assessments and statistical validations. RESULTS Analysis revealed that ChatGPT 3.5 scored highest in both safety (mean: 2.70) and GQS (mean: 3.95). ChatGPT 4.0 and Bard had lower safety scores of 2.56 and 2.42, respectively, with corresponding quality scores of 3.65 and 3.38. Inter-rater reliability was consistent, with less than 3% discrepancy. About 4.2% of responses fell into the lowest safety category (1), particularly in the novelty category. Non-medical reviewers' quality assessments correlated moderately (r = 0.67) with response length. CONCLUSIONS LLMs can be valuable resources for patients seeking information on laryngeal cancer. ChatGPT 3.5 provided the most reliable and safe responses among the models evaluated.
Collapse
Affiliation(s)
- Magdalena Ostrowska
- Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland
| | - Paulina Kacała
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland
| | - Deborah Onolememen
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland
| | - Katie Vaughan-Lane
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland.
| | - Anitta Sisily Joseph
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland
| | - Adam Ostrowski
- Department of Urology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland
| | - Wioletta Pietruszewska
- Department of Otolaryngology, Laryngological Oncology, Audiology and Phoniatrics, Medical University of Lodz, ul Żeromskiego 113, 90-549, Lodz, Poland
| | - Jacek Banaszewski
- Department of Otolaryngology, Head and Neck Oncology, Poznan University of Medical Science, ul Przybyszewskiego 49, 60-355, Poznań, Poland
| | - Maciej J Wróbel
- Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Torun, ul.Marie Sklodowskiej-Curie 9, 85-094, Bydgoszcz, Poland
| |
Collapse
|
2
|
Afify AY, Ashry MH. Exploring the risk of second primary malignancies in laryngeal cancer survivors: insights from the SEER database. Eur Arch Otorhinolaryngol 2024; 281:4409-4417. [PMID: 38775833 PMCID: PMC11266365 DOI: 10.1007/s00405-024-08731-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2024] [Accepted: 05/09/2024] [Indexed: 07/24/2024]
Abstract
PURPOSE We intended to investigate the risk for second primary malignancy (SPM) development in Laryngeal Cancer (LC) survivors. We conducted a population-based analysis of SPM risk using the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) database. METHODS Data of selected LC survivors from the SEER database between 2000 and 2020 were examined. Standardized Incidence ratios (SIRs) for SPM development were calculated, followed by detailed stratification according to anatomical site and different latency periods. RESULTS A total of 8413 SPMs were observed in our extracted cohort. The collective standardized incidence of SPMs was 2.12 (95% CI 2.07-2.17) compared to the US population, with an absolute excess risk (AER) of 201.73 per 10,000 individuals. The highest SPM risks were observed in patients with young age at diagnosis, females, and American Indians/Alaska natives. Increased SPM risks were reported in patients receiving all modalities of treatment including surgery, chemotherapy, and radiotherapy. Most SPMs were detected in solid organs such as the lungs and bronchus, oral cavity and pharynx, and prostate. The highest increased risks of developing SPMs were observed in Trachea, larynx, oral cavity and pharynx, lung and bronchus, and esophagus. CONCLUSIONS The risk of SPMs in LC survivors was significantly increased compared to the general US population. Accordingly, a more impactful cancer surveillance strategy for LC patients should be implemented.
Collapse
Affiliation(s)
| | - Mohamed Hady Ashry
- School of Medicine, New Giza University (NGU), Giza, Egypt
- Medical Research Platform, Cairo, Egypt
| |
Collapse
|
3
|
Arora A, Tsigelny IF, Kouznetsova VL. Laryngeal cancer diagnosis via miRNA-based decision tree model. Eur Arch Otorhinolaryngol 2024; 281:1391-1399. [PMID: 38147113 DOI: 10.1007/s00405-023-08383-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 11/27/2023] [Indexed: 12/27/2023]
Abstract
PURPOSE Laryngeal cancer (LC) is the most common head and neck cancer, which often goes undiagnosed due to the inaccessible nature of current diagnosis methods in some parts of the world. Many recent studies have shown that microRNAs (miRNAs) are crucial biomarkers for a variety of cancers. METHODS In this study, we create a decision tree model for the diagnosis of laryngeal cancer using a created series of miRNA attributes, such as sequence-based characteristics, predicted miRNA target genes, and gene pathways. This series of attributes is extracted from both differentially expressed blood-based miRNAs in laryngeal cancer and random, non-associated with cancer miRNAs. RESULTS Several machine-learning (ML) algorithms were tested in the ML model, and the Hoeffding Tree classifier yields the highest accuracy (86.8%) in miRNAs-based recognition of laryngeal cancer. Furthermore, our model is validated with the independent laryngeal cancer datasets and can accurately diagnose laryngeal cancer with 86% accuracy. We also explored the biological relationships of the attributes used in our model to understand their relationship with cancer proliferation or suppression pathways. CONCLUSION Our study demonstrates that the proposed model and an inexpensive miRNA testing strategy have the potential to serve as an additional method for diagnosing laryngeal cancer.
Collapse
Affiliation(s)
- Aarav Arora
- REHS Program, San Diego Supercomputer Center, UC San Diego, La Jolla, CA, USA
| | - Igor F Tsigelny
- San Diego Supercomputer Center, UC San Diego, La Jolla, CA, USA.
- BiAna, La Jolla, CA, USA.
- Department of Neurosciences, UC San Diego, La Jolla, CA, USA.
- CureScience, San Diego, CA, USA.
| | - Valentina L Kouznetsova
- San Diego Supercomputer Center, UC San Diego, La Jolla, CA, USA
- BiAna, La Jolla, CA, USA
- CureScience, San Diego, CA, USA
| |
Collapse
|
4
|
Xu J, Huang S, Yan Q, Jiang J, Hu N, Zhang W, Shi L, Deng M, Tang X, Liao J. Causes of death and nomogram for patients with oncologic hepato-biliary-pancreatic disorders: A large-cohort study. Medicine (Baltimore) 2024; 103:e37187. [PMID: 38394524 PMCID: PMC11309702 DOI: 10.1097/md.0000000000037187] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Accepted: 01/17/2024] [Indexed: 02/25/2024] Open
Abstract
The improvement of digestive cancer survival results in increased morbidity of noncancerous comorbidities. This study aimed at clarifying causes of death (COD) and predicting overall survival (OS) in patients diagnosed with liver cancer, gallbladder cancer, cholangiocarcinoma, and pancreatic cancer. We used the Surveillance, Epidemic, and End Results database to extract information. Nomograms of multivariate Cox regression was used to predict OS of cancer patients. The models were evaluated using the concordance indexes (C-indexes), the receiver operating characteristic curves and calibration curves. Respectively 58,895, 15,324, 30,708, and 109,995 cases with cancer of liver, gallbladder, bile duct or pancreas were retrieved between 2000 and 2020. Approximately 80% deaths occurred within 1 years after cancer diagnosis. Sequence in noncancerous COD proportion was diverse, while diseases of heart always accounted for a great part. Risks of death from most noncancerous COD were significantly higher than that of the cancer-free population. Nomograms were developed by predictors of interest such as age, therapy and TNM stage. The concordance indexes of nomograms were 0.756, 0.729, 0.763, and 0.760 respectively, well-calibrating to the reality. The 0.5-, 1-, and 2-year areas under the receiver operating characteristic curve were about 0.800, indicating good reliability and accuracy. Noncancerous COD accounted for larger part in gallbladder cancer and cholangiocarcinoma. Noncancerous COD showed an upward trend as follow-up time extended and the majorities were diseases of heart, cerebrovascular disease, chronic liver disease and cirrhosis. The novel OS-nomograms can provide personalized prognosis information with satisfactory accuracy.
Collapse
Affiliation(s)
- Jia Xu
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Shu Huang
- Department of Gastroenterology, Lianshui County People’ Hospital, Huaian, China
- Department of Gastroenterology, Lianshui People’ Hospital of Kangda College Affiliated to Nanjing Medical University, Huaian, China
| | - Qiong Yan
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Jiao Jiang
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Nan Hu
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Wei Zhang
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Lei Shi
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Mingming Deng
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Xiaowei Tang
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Jingyuan Liao
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| |
Collapse
|
5
|
Liao J, Xu J, Huang S, Jiang J, Hu N, Zhang W, Shi L, Deng M, Tang X, Yan Q. Cause of death among gastric cancer survivors in the United States from 2000 to 2020. Medicine (Baltimore) 2024; 103:e37219. [PMID: 38394549 PMCID: PMC11309694 DOI: 10.1097/md.0000000000037219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 01/18/2024] [Indexed: 02/25/2024] Open
Abstract
A number of studies have been conducted to explore the survival of gastric cancer (GC) patients, while studies about non-cancer causes of death in patients with GC are not well-conducted. The aim of this study was to deeply investigate the causes of death (COD) in GC patients, especially non-cancer ones. The Surveillance, Epidemiology and End Results (SEER) database was used to extract information including demographics, tumor characteristics and causes of death of GC patients meeting the inclusion criteria. The patients were stratified by demographic and clinical parameters. Standardized mortality ratios (SMRs) were calculated for all causes of death at different follow-up periods. A total of 116,437 patients with GC diagnosed between 2000 and 2020 were retrieved from the SEER database. Of these, 85,827 deaths occurred during the follow-up period, most of which occurred within 1 year after GC diagnosis. GC (n = 49,746; 58%) was the leading COD, followed by other cancer (n = 21,135; 25%) and non-cancer causes (n = 14,946; 17%). Diseases of heart were the most common non-cancer cause of death, accounting for 30%, followed by cerebrovascular diseases (n = 917; 6%) and chronic obstructive pulmonary disease (n = 900; 6%). Although gastric cancer remains the most common cause of death in gastric cancer patients, it should not be ignored that the risk of non-cancer causes tends to increase with the length of the latency period. These findings may provide important insights into the healthcare management of gastric cancer patients at various follow-up intervals.
Collapse
Affiliation(s)
- Jingyuan Liao
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Jia Xu
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Shu Huang
- Department of Gastroenterology, Lianshui County People’ Hospital, Huaian, China
- Department of Gastroenterology, Lianshui People’ Hospital of Kangda College Affiliated to Nanjing Medical University, Huaian, China
| | - Jiao Jiang
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Nan Hu
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Wei Zhang
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Lei Shi
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Mingming Deng
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Xiaowei Tang
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| | - Qiong Yan
- Department of Gastroenterology, the Affiliated Hospital of Southwest Medical University, Luzhou, China
- Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province, Luzhou, China
| |
Collapse
|