1
|
Tenda ED, Yunus RE, Zulkarnaen B, Yugo MR, Pitoyo CW, Asaf MM, Islamiyati TN, Pujitresnani A, Setiadharma A, Henrina J, Rumende CM, Wulani V, Harimurti K, Lydia A, Shatri H, Soewondo P, Yusuf PA. Comparison of the Discrimination Performance of AI Scoring and the Brixia Score in Predicting COVID-19 Severity on Chest X-Ray Imaging: Diagnostic Accuracy Study. JMIR Form Res 2024; 8:e46817. [PMID: 38451633 PMCID: PMC10958333 DOI: 10.2196/46817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 09/28/2023] [Accepted: 12/29/2023] [Indexed: 03/08/2024] Open
Abstract
BACKGROUND The artificial intelligence (AI) analysis of chest x-rays can increase the precision of binary COVID-19 diagnosis. However, it is unknown if AI-based chest x-rays can predict who will develop severe COVID-19, especially in low- and middle-income countries. OBJECTIVE The study aims to compare the performance of human radiologist Brixia scores versus 2 AI scoring systems in predicting the severity of COVID-19 pneumonia. METHODS We performed a cross-sectional study of 300 patients suspected with and with confirmed COVID-19 infection in Jakarta, Indonesia. A total of 2 AI scores were generated using CAD4COVID x-ray software. RESULTS The AI probability score had slightly lower discrimination (area under the curve [AUC] 0.787, 95% CI 0.722-0.852). The AI score for the affected lung area (AUC 0.857, 95% CI 0.809-0.905) was almost as good as the human Brixia score (AUC 0.863, 95% CI 0.818-0.908). CONCLUSIONS The AI score for the affected lung area and the human radiologist Brixia score had similar and good discrimination performance in predicting COVID-19 severity. Our study demonstrated that using AI-based diagnostic tools is possible, even in low-resource settings. However, before it is widely adopted in daily practice, more studies with a larger scale and that are prospective in nature are needed to confirm our findings.
Collapse
Affiliation(s)
- Eric Daniel Tenda
- Department of Internal Medicine, Pulmonology and Critical Care Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Reyhan Eddy Yunus
- Department of Radiology, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Benny Zulkarnaen
- Department of Radiology, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Muhammad Reynalzi Yugo
- Department of Radiology, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Ceva Wicaksono Pitoyo
- Department of Internal Medicine, Pulmonology and Critical Care Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Moses Mazmur Asaf
- Department of Radiology, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Tiara Nur Islamiyati
- Department of Internal Medicine, Pulmonology and Critical Care Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Arierta Pujitresnani
- Department of Medical Physiology and Biophysics/ Medical Technology Cluster IMERI, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
| | - Andry Setiadharma
- Department of Internal Medicine, Pulmonology and Critical Care Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Joshua Henrina
- Department of Internal Medicine, Pulmonology and Critical Care Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Cleopas Martin Rumende
- Department of Internal Medicine, Pulmonology and Critical Care Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Vally Wulani
- Department of Radiology, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Kuntjoro Harimurti
- Department of Internal Medicine, Geriatric Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Aida Lydia
- Department of Internal Medicine, Nephrology and Hypertension Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Hamzah Shatri
- Department of Internal Medicine, Psychosomatic Division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Pradana Soewondo
- Department of Internal Medicine, Endocrinology - Metabolism - Diabetes division, Faculty of Medicine Universitas Indonesia, RSUPN Dr. Cipto Mangunkusumo, Universitas Indonesia, Jakarta, Indonesia
| | - Prasandhya Astagiri Yusuf
- Department of Medical Physiology and Biophysics/ Medical Technology Cluster IMERI, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
| |
Collapse
|
2
|
Grenier PA, Brun AL, Mellot F. [The contribution of artificial intelligence (AI) subsequent to the processing of thoracic imaging]. Rev Mal Respir 2024; 41:110-126. [PMID: 38129269 DOI: 10.1016/j.rmr.2023.12.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 11/27/2023] [Indexed: 12/23/2023]
Abstract
The contribution of artificial intelligence (AI) to medical imaging is currently the object of widespread experimentation. The development of deep learning (DL) methods, particularly convolution neural networks (CNNs), has led to performance gains often superior to those achieved by conventional methods such as machine learning. Radiomics is an approach aimed at extracting quantitative data not accessible to the human eye from images expressing a disease. The data subsequently feed machine learning models and produce diagnostic or prognostic probabilities. As for the multiple applications of AI methods in thoracic imaging, they are undergoing evaluation. Chest radiography is a practically ideal field for the development of DL algorithms able to automatically interpret X-rays. Current algorithms can detect up to 14 different abnormalities present either in isolation or in combination. Chest CT is another area offering numerous AI applications. Various algorithms have been specifically formed and validated for the detection and characterization of pulmonary nodules and pulmonary embolism, as well as segmentation and quantitative analysis of the extent of diffuse lung diseases (emphysema, infectious pneumonias, interstitial lung disease). In addition, the analysis of medical images can be associated with clinical, biological, and functional data (multi-omics analysis), the objective being to construct predictive approaches regarding disease prognosis and response to treatment.
Collapse
Affiliation(s)
- P A Grenier
- Délégation à la recherche clinique et l'innovation, hôpital Foch, Suresnes, France.
| | - A L Brun
- Service de radiologie, hôpital Foch, Suresnes, France
| | - F Mellot
- Service de radiologie, hôpital Foch, Suresnes, France
| |
Collapse
|
3
|
Aleksandra S, Robert K, Klaudia K, Dawid L, Mariusz S. Artificial Intelligence in Optimizing the Functioning of Emergency Departments; a Systematic Review of Current Solutions. ARCHIVES OF ACADEMIC EMERGENCY MEDICINE 2024; 12:e22. [PMID: 38572221 PMCID: PMC10988184 DOI: 10.22037/aaem.v12i1.2110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/05/2024]
Abstract
Introduction The burgeoning burden on emergency departments is a global challenge that we have been confronting for many years. Emerging artificial intelligence (AI)-based solutions may constitute a critical component in the optimization of these units. This systematic review was conducted to thoroughly examine and summarize the currently available AI solutions, assess potential benefits from their implementation, and identify anticipated directions of further development in this fascinating and rapidly evolving field. Methods This systematic review utilized data compiled from three key scientific databases: PubMed (2045 publications), Scopus (877 publications), and Web of Science (2495 publications). After meticulous removal of duplicates, we conducted a detailed analysis of 2052 articles, including 147 full-text papers. From these, we selected 51 of the most pertinent and representative publications for the review. Results Overall the present research indicates that due to high accuracy and sensitivity of machine learning (ML) models it's reasonable to use AI in support of doctors as it can show them the potential diagnosis, which could save time and resources. However, AI-generated diagnoses should be verified by a doctor as AI is not infallible. Conclusions Currently available AI algorithms are capable of analysing complex medical data with unprecedented precision and speed. Despite AI's vast potential, it is still a nascent technology that is often perceived as complicated and challenging to implement. We propose that a pivotal point in effectively harnessing this technology is the close collaboration between medical professionals and AI experts. Future research should focus on further refining AI algorithms, performing comprehensive validation, and introducing suitable legal regulations and standard procedures, thereby fully leveraging the potential of AI to enhance the quality and efficiency of healthcare delivery.
Collapse
Affiliation(s)
- Szymczyk Aleksandra
- Department of Emergency Medicine, Medical University of Gdansk, Smoluchowskiego 17, 80-214 Gdansk, Poland
| | | | | | | | | |
Collapse
|
4
|
Gefter WB, Prokop M, Seo JB, Raoof S, Langlotz CP, Hatabu H. Human-AI Symbiosis: A Path Forward to Improve Chest Radiography and the Role of Radiologists in Patient Care. Radiology 2024; 310:e232778. [PMID: 38259206 PMCID: PMC10831473 DOI: 10.1148/radiol.232778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 12/08/2023] [Accepted: 12/18/2023] [Indexed: 01/24/2024]
Affiliation(s)
- Warren B. Gefter
- From the Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, Pa (W.B.G.); Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands (M.P.); Department of Radiology, Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea (J.B.S.); Department of Medicine and Radiology, Zucker School of Medicine, Hofstra/Northwell and Lung Institute, Lenox Hill Hospital, New York, NY (S.R.); Department of Radiology and Biomedical Informatics and Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, Calif (C.P.L.); and Center for Pulmonary Functional Imaging, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis St, Boston, MA 02215 (H.H.)
| | - Mathias Prokop
- From the Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, Pa (W.B.G.); Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands (M.P.); Department of Radiology, Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea (J.B.S.); Department of Medicine and Radiology, Zucker School of Medicine, Hofstra/Northwell and Lung Institute, Lenox Hill Hospital, New York, NY (S.R.); Department of Radiology and Biomedical Informatics and Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, Calif (C.P.L.); and Center for Pulmonary Functional Imaging, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis St, Boston, MA 02215 (H.H.)
| | - Joon Beom Seo
- From the Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, Pa (W.B.G.); Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands (M.P.); Department of Radiology, Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea (J.B.S.); Department of Medicine and Radiology, Zucker School of Medicine, Hofstra/Northwell and Lung Institute, Lenox Hill Hospital, New York, NY (S.R.); Department of Radiology and Biomedical Informatics and Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, Calif (C.P.L.); and Center for Pulmonary Functional Imaging, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis St, Boston, MA 02215 (H.H.)
| | - Suhail Raoof
- From the Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, Pa (W.B.G.); Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands (M.P.); Department of Radiology, Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea (J.B.S.); Department of Medicine and Radiology, Zucker School of Medicine, Hofstra/Northwell and Lung Institute, Lenox Hill Hospital, New York, NY (S.R.); Department of Radiology and Biomedical Informatics and Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, Calif (C.P.L.); and Center for Pulmonary Functional Imaging, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis St, Boston, MA 02215 (H.H.)
| | - Curtis P. Langlotz
- From the Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, Pa (W.B.G.); Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands (M.P.); Department of Radiology, Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea (J.B.S.); Department of Medicine and Radiology, Zucker School of Medicine, Hofstra/Northwell and Lung Institute, Lenox Hill Hospital, New York, NY (S.R.); Department of Radiology and Biomedical Informatics and Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, Calif (C.P.L.); and Center for Pulmonary Functional Imaging, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis St, Boston, MA 02215 (H.H.)
| | - Hiroto Hatabu
- From the Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, Pa (W.B.G.); Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands (M.P.); Department of Radiology, Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea (J.B.S.); Department of Medicine and Radiology, Zucker School of Medicine, Hofstra/Northwell and Lung Institute, Lenox Hill Hospital, New York, NY (S.R.); Department of Radiology and Biomedical Informatics and Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, Calif (C.P.L.); and Center for Pulmonary Functional Imaging, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis St, Boston, MA 02215 (H.H.)
| |
Collapse
|
5
|
Morís DI, de Moura J, Marcos PJ, Rey EM, Novo J, Ortega M. Comprehensive analysis of clinical data for COVID-19 outcome estimation with machine learning models. Biomed Signal Process Control 2023; 84:104818. [PMID: 36915863 PMCID: PMC9995330 DOI: 10.1016/j.bspc.2023.104818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2022] [Revised: 11/22/2022] [Accepted: 03/05/2023] [Indexed: 03/11/2023]
Abstract
COVID-19 is a global threat for the healthcare systems due to the rapid spread of the pathogen that causes it. In such situation, the clinicians must take important decisions, in an environment where medical resources can be insufficient. In this task, the computer-aided diagnosis systems can be very useful not only in the task of supporting the clinical decisions but also to perform relevant analyses, allowing them to understand better the disease and the factors that can identify the high risk patients. For those purposes, in this work, we use several machine learning algorithms to estimate the outcome of COVID-19 patients given their clinical information. Particularly, we perform 2 different studies: the first one estimates whether the patient is at low or at high risk of death whereas the second estimates if the patient needs hospitalization or not. The results of the analyses of this work show the most relevant features for each studied scenario, as well as the classification performance of the considered machine learning models. In particular, the XGBoost algorithm is able to estimate the need for hospitalization of a patient with an AUC-ROC of 0 . 8415 ± 0 . 0217 while it can also estimate the risk of death with an AUC-ROC of 0 . 7992 ± 0 . 0104 . Results have demonstrated the great potential of the proposal to determine those patients that need a greater amount of medical resources for being at a higher risk. This provides the healthcare services with a tool to better manage their resources.
Collapse
Affiliation(s)
- Daniel I Morís
- Centro de Investigación CITIC, Universidade da Coruña, Campus de Elviña, s/n, 15071 A Coruña, Spain.,Grupo VARPA, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, Xubias de Arriba, 84, 15006 A Coruña, Spain
| | - Joaquim de Moura
- Centro de Investigación CITIC, Universidade da Coruña, Campus de Elviña, s/n, 15071 A Coruña, Spain.,Grupo VARPA, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, Xubias de Arriba, 84, 15006 A Coruña, Spain
| | - Pedro J Marcos
- Dirección Asistencial y Servicio de Neumología, Complejo Hospitalario Universitario de A Coruña (CHUAC), Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, Sergas, 15006 A Coruña, Spain
| | - Enrique Míguez Rey
- Grupo de Investigación en Virología Clínica, Sección de Enfermedades Infecciosas, Servicio de Medicina Interna, Instituto de Investigación Biomédica de A Coruña (INIBIC), Área Sanitaria A Coruña y CEE (ASCC), SERGAS, 15006 A Coruña, Spain
| | - Jorge Novo
- Centro de Investigación CITIC, Universidade da Coruña, Campus de Elviña, s/n, 15071 A Coruña, Spain.,Grupo VARPA, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, Xubias de Arriba, 84, 15006 A Coruña, Spain
| | - Marcos Ortega
- Centro de Investigación CITIC, Universidade da Coruña, Campus de Elviña, s/n, 15071 A Coruña, Spain.,Grupo VARPA, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, Xubias de Arriba, 84, 15006 A Coruña, Spain
| |
Collapse
|
6
|
Banoei MM, Rafiepoor H, Zendehdel K, Seyyedsalehi MS, Nahvijou A, Allameh F, Amanpour S. Unraveling complex relationships between COVID-19 risk factors using machine learning based models for predicting mortality of hospitalized patients and identification of high-risk group: a large retrospective study. Front Med (Lausanne) 2023; 10:1170331. [PMID: 37215714 PMCID: PMC10192907 DOI: 10.3389/fmed.2023.1170331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 04/11/2023] [Indexed: 05/24/2023] Open
Abstract
Background At the end of 2019, the coronavirus disease 2019 (COVID-19) pandemic increased the hospital burden of COVID-19 caused by the SARS-Cov-2 and became the most significant health challenge for nations worldwide. The severity and high mortality of COVID-19 have been correlated with various demographic characteristics and clinical manifestations. Prediction of mortality rate, identification of risk factors, and classification of patients played a crucial role in managing COVID-19 patients. Our purpose was to develop machine learning (ML)-based models for the prediction of mortality and severity among patients with COVID-19. Identifying the most important predictors and unraveling their relationships by classification of patients to the low-, moderate- and high-risk groups might guide prioritizing treatment decisions and a better understanding of interactions between factors. A detailed evaluation of patient data is believed to be important since COVID-19 resurgence is underway in many countries. Results The findings of this study revealed that the ML-based statistically inspired modification of the partial least square (SIMPLS) method could predict the in-hospital mortality among COVID-19 patients. The prediction model was developed using 19 predictors including clinical variables, comorbidities, and blood markers with moderate predictability (Q2 = 0.24) to separate survivors and non-survivors. Oxygen saturation level, loss of consciousness, and chronic kidney disease (CKD) were the top mortality predictors. Correlation analysis showed different correlation patterns among predictors for each non-survivor and survivor cohort separately. The main prediction model was verified using other ML-based analyses with a high area under the curve (AUC) (0.81-0.93) and specificity (0.94-0.99). The obtained data revealed that the mortality prediction model can be different for males and females with diverse predictors. Patients were classified into four clusters of mortality risk and identified the patients at the highest risk of mortality, which accentuated the most significant predictors correlating with mortality. Conclusion An ML model for predicting mortality among hospitalized COVID-19 patients was developed considering the interactions between factors that may reduce the complexity of clinical decision-making processes. The most predictive factors related to patient mortality were identified by assessing and classifying patients into different groups based on their sex and mortality risk (low-, moderate-, and high-risk groups).
Collapse
Affiliation(s)
| | - Haniyeh Rafiepoor
- Cancer Biology Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Kazem Zendehdel
- Cancer Biology Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
- Cancer Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Monireh Sadat Seyyedsalehi
- Cancer Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Azin Nahvijou
- Cancer Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Farshad Allameh
- Gastroenterology Ward, Imam Khomeini Hospital Complex (IKHC), Tehran University of Medical Sciences, Tehran, Iran
| | - Saeid Amanpour
- Cancer Biology Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
7
|
Lee HW, Yang HJ, Kim H, Kim UH, Kim DH, Yoon SH, Ham SY, Nam BD, Chae KJ, Lee D, Yoo JY, Bak SH, Kim JY, Kim JH, Kim KB, Jung JI, Lim JK, Lee JE, Chung MJ, Lee YK, Kim YS, Lee SM, Kwon W, Park CM, Kim YH, Jeong YJ, Jin KN, Goo JM. Deep Learning With Chest Radiographs for Making Prognoses in Patients With COVID-19: Retrospective Cohort Study. J Med Internet Res 2023; 25:e42717. [PMID: 36795468 PMCID: PMC9937110 DOI: 10.2196/42717] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Revised: 11/12/2022] [Accepted: 01/11/2023] [Indexed: 02/17/2023] Open
Abstract
BACKGROUND An artificial intelligence (AI) model using chest radiography (CXR) may provide good performance in making prognoses for COVID-19. OBJECTIVE We aimed to develop and validate a prediction model using CXR based on an AI model and clinical variables to predict clinical outcomes in patients with COVID-19. METHODS This retrospective longitudinal study included patients hospitalized for COVID-19 at multiple COVID-19 medical centers between February 2020 and October 2020. Patients at Boramae Medical Center were randomly classified into training, validation, and internal testing sets (at a ratio of 8:1:1, respectively). An AI model using initial CXR images as input, a logistic regression model using clinical information, and a combined model using the output of the AI model (as CXR score) and clinical information were developed and trained to predict hospital length of stay (LOS) ≤2 weeks, need for oxygen supplementation, and acute respiratory distress syndrome (ARDS). The models were externally validated in the Korean Imaging Cohort of COVID-19 data set for discrimination and calibration. RESULTS The AI model using CXR and the logistic regression model using clinical variables were suboptimal to predict hospital LOS ≤2 weeks or the need for oxygen supplementation but performed acceptably in the prediction of ARDS (AI model area under the curve [AUC] 0.782, 95% CI 0.720-0.845; logistic regression model AUC 0.878, 95% CI 0.838-0.919). The combined model performed better in predicting the need for oxygen supplementation (AUC 0.704, 95% CI 0.646-0.762) and ARDS (AUC 0.890, 95% CI 0.853-0.928) compared to the CXR score alone. Both the AI and combined models showed good calibration for predicting ARDS (P=.079 and P=.859). CONCLUSIONS The combined prediction model, comprising the CXR score and clinical information, was externally validated as having acceptable performance in predicting severe illness and excellent performance in predicting ARDS in patients with COVID-19.
Collapse
Affiliation(s)
- Hyun Woo Lee
- Division of Respiratory and Critical Care, Department of Internal Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Republic of Korea
- College of Medicine, Seoul National University, Seoul, Republic of Korea
| | - Hyun Jun Yang
- College of Medicine, Seoul National University, Seoul, Republic of Korea
| | - Hyungjin Kim
- College of Medicine, Seoul National University, Seoul, Republic of Korea
- Department of Radiology, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - Ue-Hwan Kim
- AI Graduate School, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| | - Dong Hyun Kim
- College of Medicine, Seoul National University, Seoul, Republic of Korea
- Department of Radiology, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Republic of Korea
| | - Soon Ho Yoon
- College of Medicine, Seoul National University, Seoul, Republic of Korea
- Department of Radiology, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - Soo-Youn Ham
- Department of Radiology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Bo Da Nam
- Department of Radiology, Soonchunhyang University Seoul Hospital, Soonchunhyang University College of Medicine, Seoul, Republic of Korea
| | - Kum Ju Chae
- Department of Radiology, Research Institute of Clinical Medicine, Jeonbuk National University-Biomedical Research Institute of Jeonbuk National University Hospital, Jeonju, Republic of Korea
| | - Dabee Lee
- Department of Radiology, Dankook University Hospital, Cheonan, Republic of Korea
| | - Jin Young Yoo
- Department of Radiology, Chungbuk National University Hospital, Cheongju, Republic of Korea
| | - So Hyeon Bak
- Department of Radiology, Kangwon National University Hospital, Kangwon National University School of Medicine, Chuncheon, Republic of Korea
| | - Jin Young Kim
- Department of Radiology, Keimyung University Dongsan Hospital, Keimyung University School of Medicine, Daegu, Republic of Korea
| | - Jin Hwan Kim
- Department of Radiology, Chungnam National University Hospital, College of Medicine, Daejeon, Republic of Korea
| | - Ki Beom Kim
- Department of Radiology, Daegu Fatima Hospital, Daegu, Republic of Korea
| | - Jung Im Jung
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea
| | - Jae-Kwang Lim
- Department of Radiology, Kyungpook National University Hospital, School of Medicine, Kyungpook National University, Daegu, Republic of Korea
| | - Jong Eun Lee
- Department of Radiology, Chonnam National University Hospital, Gwangju, Republic of Korea
| | - Myung Jin Chung
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Young Kyung Lee
- Department of Radiology, Seoul Medical Center, Seoul, Republic of Korea
| | - Young Seon Kim
- Department of Radiology, Yeungnam University Hospital, Yeungnam University College of Medicine, Daegu, Republic of Korea
| | - Sang Min Lee
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Woocheol Kwon
- Department of Radiology, Ewha Womans University Seoul Hospital, Seoul, Republic of Korea
| | - Chang Min Park
- College of Medicine, Seoul National University, Seoul, Republic of Korea
- Department of Radiology, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - Yun-Hyeon Kim
- Department of Radiology, Chonnam National University Hospital, Gwangju, Republic of Korea
| | - Yeon Joo Jeong
- Department of Radiology, Pusan National University Hospital, Pusan National University School of Medicine and Biomedical Research Institute, Busan, Republic of Korea
| | - Kwang Nam Jin
- College of Medicine, Seoul National University, Seoul, Republic of Korea
- Department of Radiology, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Republic of Korea
| | - Jin Mo Goo
- College of Medicine, Seoul National University, Seoul, Republic of Korea
- Department of Radiology, Seoul National University Medical Research Center, Seoul, Republic of Korea
| |
Collapse
|
8
|
Walston SL, Matsumoto T, Miki Y, Ueda D. Artificial intelligence-based model for COVID-19 prognosis incorporating chest radiographs and clinical data; a retrospective model development and validation study. Br J Radiol 2022; 95:20220058. [PMID: 36193755 PMCID: PMC9733620 DOI: 10.1259/bjr.20220058] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 08/19/2022] [Accepted: 08/23/2022] [Indexed: 11/19/2022] Open
Abstract
OBJECTIVES The purpose of this study was to develop an artificial intelligence-based model to prognosticate COVID-19 patients at admission by combining clinical data and chest radiographs. METHODS This retrospective study used the Stony Brook University COVID-19 dataset of 1384 inpatients. After exclusions, 1356 patients were randomly divided into training (1083) and test datasets (273). We implemented three artificial intelligence models, which classified mortality, ICU admission, or ventilation risk. Each model had three submodels with different inputs: clinical data, chest radiographs, and both. We showed the importance of the variables using SHapley Additive exPlanations (SHAP) values. RESULTS The mortality prediction model was best overall with area under the curve, sensitivity, specificity, and accuracy of 0.79 (0.72-0.86), 0.74 (0.68-0.79), 0.77 (0.61-0.88), and 0.74 (0.69-0.79) for the clinical data-based model; 0.77 (0.69-0.85), 0.67 (0.61-0.73), 0.81 (0.67-0.92), 0.70 (0.64-0.75) for the image-based model, and 0.86 (0.81-0.91), 0.76 (0.70-0.81), 0.77 (0.61-0.88), 0.76 (0.70-0.81) for the mixed model. The mixed model had the best performance (p value < 0.05). The radiographs ranked fourth for prognostication overall, and first of the inpatient tests assessed. CONCLUSIONS These results suggest that prognosis models become more accurate if AI-derived chest radiograph features and clinical data are used together. ADVANCES IN KNOWLEDGE This AI model evaluates chest radiographs together with clinical data in order to classify patients as having high or low mortality risk. This work shows that chest radiographs taken at admission have significant COVID-19 prognostic information compared to clinical data other than age and sex.
Collapse
Affiliation(s)
| | | | - Yukio Miki
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University,1-4-3 Asahi-machi, Abeno-ku, Osaka, Japan
| | | |
Collapse
|