1
|
Geric C, Tavaziva G, Breuninger M, Dheda K, Esmail A, Scott A, Kagujje M, Muyoyeta M, Reither K, Khan AJ, Benedetti A, Ahmad Khan F. Breaking the threshold: Developing multivariable models using computer-aided chest X-ray analysis for tuberculosis triage. Int J Infect Dis 2024; 147:107221. [PMID: 39233047 DOI: 10.1016/j.ijid.2024.107221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2024] [Revised: 08/01/2024] [Accepted: 08/22/2024] [Indexed: 09/06/2024] Open
Abstract
OBJECTIVES Computer-aided detection (CAD) software packages quantify tuberculosis (TB)-compatible chest X-ray (CXR) abnormality as continuous scores. In practice, a threshold value is selected for binary CXR classification. We assessed the diagnostic accuracy of an alternative approach to applying CAD for TB triage: incorporating CAD scores in multivariable modeling. METHODS We pooled individual patient data from four studies. Separately, for two commercial CAD, we used logistic regression to model microbiologically confirmed TB. Models included CAD score, study site, age, sex, human immunodeficiency virus status, and prior TB. We compared specificity at target sensitivities ≥90% between the multivariable model and the current threshold-based approach for CAD use. RESULTS We included 4,733/5,640 (84%) participants with complete covariate data (median age 36 years; 45% female; 22% with prior TB; 22% people living with human immunodeficiency virus). A total of 805 (17%) had TB. Multivariable models demonstrated excellent performance (areas under the receiver operating characteristic curve [95% confidence interval]: software A, 0.91 [0.90-0.93]; software B, 0.92 [0.91-0.93]). Compared with threshold scores, multivariable models increased specificity (e.g., at 90% sensitivity, threshold vs model specificity [95% confidence interval]: software A, 71% [68-74%] vs 75% [74-77%]; software B, 69% [63-75%] vs 75% [74-77%]). CONCLUSION Using CAD scores in multivariable models outperformed the current practice of CAD-threshold-based CXR classification for TB diagnosis.
Collapse
Affiliation(s)
- Coralie Geric
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montreal, Canada; Department of Medicine, McGill University, Montreal, Canada; Respiratory Epidemiology and Clinical Research Unit, Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, Canada
| | - Gamuchirai Tavaziva
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montreal, Canada; Respiratory Epidemiology and Clinical Research Unit, Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, Canada
| | - Marianne Breuninger
- Division of Infectious Diseases, Department I of Internal Medicine, University of Cologne, Cologne, Germany
| | - Keertan Dheda
- Centre for Lung Infection and Immunity Unit, Division of Pulmonology and UCT Lung Institute, University of Cape Town, Cape Town, South Africa; Faculty of Infectious and Tropical Diseases, Department of Infection Biology, London School of Hygiene and Tropical Medicine, London, United Kingdom
| | - Ali Esmail
- Centre for Lung Infection and Immunity Unit, Division of Pulmonology and UCT Lung Institute, University of Cape Town, Cape Town, South Africa
| | - Alex Scott
- Centre for Lung Infection and Immunity Unit, Division of Pulmonology and UCT Lung Institute, University of Cape Town, Cape Town, South Africa
| | - Mary Kagujje
- Tuberculosis Department, Centre for Infectious Disease Research in Zambia, Lusaka, Zambia
| | - Monde Muyoyeta
- Tuberculosis Department, Centre for Infectious Disease Research in Zambia, Lusaka, Zambia; Zambart, Lusaka, Zambia
| | - Klaus Reither
- Swiss Tropical and Public Health Institute, Allschwill, Switzerland; University of Basel, Basel, Switzerland
| | | | - Andrea Benedetti
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montreal, Canada; Department of Medicine, McGill University, Montreal, Canada; Department of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, Canada
| | - Faiz Ahmad Khan
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montreal, Canada; Department of Medicine, McGill University, Montreal, Canada; Respiratory Epidemiology and Clinical Research Unit, Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, Canada.
| |
Collapse
|
2
|
Bosman S, Ayakaka I, Muhairwe J, Kamele M, van Heerden A, Madonsela T, Labhardt ND, Sommer G, Bremerich J, Zoller T, Murphy K, van Ginneken B, Keter AK, Jacobs BKM, Bresser M, Signorell A, Glass TR, Lynen L, Reither K. Evaluation of C-Reactive Protein and Computer-Aided Analysis of Chest X-rays as Tuberculosis Triage Tests at Health Facilities in Lesotho and South Africa. Clin Infect Dis 2024:ciae378. [PMID: 39190813 DOI: 10.1093/cid/ciae378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Indexed: 08/29/2024] Open
Abstract
BACKGROUND To improve tuberculosis case-finding, rapid, non-sputum triage tests need to be developed according to the World Health Organization target product profile (TPP) (>90% sensitivity, >70% specificity). We prospectively evaluated and compared artificial intelligence-based, computer-aided detection software, CAD4TBv7, and C-reactive protein assay (CRP) as triage tests at health facilities in Lesotho and South Africa. METHODS Adults (≥18 years) presenting with ≥1 of the 4 cardinal tuberculosis symptoms were consecutively recruited between February 2021 and April 2022. After informed consent, each participant underwent a digital chest X-ray for CAD4TBv7 and a CRP test. Participants provided 1 sputum sample for Xpert MTB/RIF Ultra and Xpert MTB/RIF and 1 for liquid culture. Additionally, an expert radiologist read the chest X-rays via teleradiology. For primary analysis, a composite microbiological reference standard (ie, positive culture or Xpert Ultra) was used. RESULTS We enrolled 1392 participants, 48% were people with HIV and 24% had previously tuberculosis. The receiver operating characteristic curve for CAD4TBv7 and CRP showed an area under the curve of .87 (95% CI: .84-.91) and .80 (95% CI: .76-.84), respectively. At thresholds corresponding to 90% sensitivity, specificity was 68.2% (95% CI: 65.4-71.0%) and 38.2% (95% CI: 35.3-41.1%) for CAD4TBv7 and CRP, respectively. CAD4TBv7 detected tuberculosis as well as an expert radiologist. CAD4TBv7 almost met the TPP criteria for tuberculosis triage. CONCLUSIONS CAD4TBv7 is accurate as a triage test for patients with tuberculosis symptoms from areas with a high tuberculosis and HIV burden. The role of CRP in tuberculosis triage requires further research. CLINICAL TRIALS REGISTRATION Clinicaltrials.gov identifier: NCT04666311.
Collapse
Affiliation(s)
- Shannon Bosman
- Centre for Community Based Research, Human Sciences Research Council, Sweetwaters, South Africa
| | - Irene Ayakaka
- SolidarMed, Partnerships for Health, Maseru, Lesotho
| | | | | | - Alastair van Heerden
- Centre for Community Based Research, Human Sciences Research Council, Sweetwaters, South Africa
- SAMRC/WITS Developmental Pathways for Health Research Unit, University of the Witwatersrand, Johannesburg, South Africa
| | - Thandanani Madonsela
- Centre for Community Based Research, Human Sciences Research Council, Sweetwaters, South Africa
| | - Niklaus D Labhardt
- Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, Basel, Switzerland
- University of Basel, Basel, Switzerland
| | - Gregor Sommer
- University of Basel, Basel, Switzerland
- Department of Radiology, Clinic of Radiology and Nuclear Medicine, University Hospital Basel, Basel, Switzerland
- Institute of Radiology and Nuclear Medicine, Hirslanden Klinik St. Anna, Lucerne, Switzerland
| | - Jens Bremerich
- University of Basel, Basel, Switzerland
- Department of Radiology, Clinic of Radiology and Nuclear Medicine, University Hospital Basel, Basel, Switzerland
| | - Thomas Zoller
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Infectious Diseases, Respiratory and Critical Care Medicine, Berlin, Germany
| | - Keelin Murphy
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Bram van Ginneken
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Alfred K Keter
- Department of Clinical Sciences, Institute of Tropical Medicine, Antwerp, Belgium
| | - Bart K M Jacobs
- Department of Clinical Sciences, Institute of Tropical Medicine, Antwerp, Belgium
| | - Moniek Bresser
- University of Basel, Basel, Switzerland
- Department of Medicine, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
| | - Aita Signorell
- University of Basel, Basel, Switzerland
- Department of Medicine, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
| | - Tracy R Glass
- University of Basel, Basel, Switzerland
- Department of Medicine, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
| | - Lutgarde Lynen
- Department of Clinical Sciences, Institute of Tropical Medicine, Antwerp, Belgium
| | - Klaus Reither
- University of Basel, Basel, Switzerland
- Department of Medicine, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
| |
Collapse
|
3
|
Codlin AJ, Vo LNQ, Garg T, Banu S, Ahmed S, John S, Abdulkarim S, Muyoyeta M, Sanjase N, Wingfield T, Iem V, Squire B, Creswell J. Expanding molecular diagnostic coverage for tuberculosis by combining computer-aided chest radiography and sputum specimen pooling: a modeling study from four high-burden countries. BMC GLOBAL AND PUBLIC HEALTH 2024; 2:52. [PMID: 39100507 PMCID: PMC11291606 DOI: 10.1186/s44263-024-00081-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 07/05/2024] [Indexed: 08/06/2024]
Abstract
Background In 2022, fewer than half of persons with tuberculosis (TB) had access to molecular diagnostic tests for TB due to their high costs. Studies have found that the use of artificial intelligence (AI) software for chest X-ray (CXR) interpretation and sputum specimen pooling can each reduce the cost of testing. We modeled the combination of both strategies to estimate potential savings in consumables that could be used to expand access to molecular diagnostics. Methods We obtained Xpert testing and positivity data segmented into deciles by AI probability scores for TB from the community- and healthcare facility-based active case finding conducted in Bangladesh, Nigeria, Viet Nam, and Zambia. AI scores in the model were based on CAD4TB version 7 (Zambia) and qXR (all other countries). We modeled four ordinal screening and testing approaches involving AI-aided CXR interpretation to indicate individual and pooled testing. Setting a false negative rate of 5%, for each approach we calculated additional and cumulative savings over the baseline of universal Xpert testing, as well as the theoretical expansion in diagnostic coverage. Results In each country, the optimal screening and testing approach was to use AI to rule out testing in deciles with low AI scores and to guide pooled vs individual testing in persons with moderate and high AI scores, respectively. This approach yielded cumulative savings in Xpert tests over baseline ranging from 50.8% in Zambia to 57.5% in Nigeria and 61.5% in Bangladesh and Viet Nam. Using these savings, diagnostic coverage theoretically could be expanded by 34% to 160% across the different approaches and countries. Conclusions Using AI software data generated during CXR interpretation to inform a differentiated pooled testing strategy may optimize TB diagnostic test use, and could extend molecular tests to more people who need them. The optimal AI thresholds and pooled testing strategy varied across countries, which suggests that bespoke screening and testing approaches may be needed for differing populations and settings. Supplementary Information The online version contains supplementary material available at 10.1186/s44263-024-00081-2.
Collapse
Affiliation(s)
- Andrew James Codlin
- Friends for International TB Relief, Hanoi, Viet Nam
- Karolinska Institutet, Stockholm, Sweden
| | - Luan Nguyen Quang Vo
- Friends for International TB Relief, Hanoi, Viet Nam
- Karolinska Institutet, Stockholm, Sweden
| | | | | | | | | | | | - Monde Muyoyeta
- Centre for Infectious Disease Research in Zambia, Lusaka, Zambia
| | - Nsala Sanjase
- Centre for Infectious Disease Research in Zambia, Lusaka, Zambia
| | - Tom Wingfield
- Karolinska Institutet, Stockholm, Sweden
- Liverpool School of Tropical Medicine, Liverpool, United Kingdom
- Liverpool University Hospitals NHS Foundation Trust, Liverpool, United Kingdom
| | - Vibol Iem
- Liverpool School of Tropical Medicine, Liverpool, United Kingdom
| | - Bertie Squire
- Liverpool School of Tropical Medicine, Liverpool, United Kingdom
| | | |
Collapse
|
4
|
Chang JY, Makary MS. Evolving and Novel Applications of Artificial Intelligence in Thoracic Imaging. Diagnostics (Basel) 2024; 14:1456. [PMID: 39001346 PMCID: PMC11240935 DOI: 10.3390/diagnostics14131456] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2024] [Revised: 07/01/2024] [Accepted: 07/06/2024] [Indexed: 07/16/2024] Open
Abstract
The advent of artificial intelligence (AI) is revolutionizing medicine, particularly radiology. With the development of newer models, AI applications are demonstrating improved performance and versatile utility in the clinical setting. Thoracic imaging is an area of profound interest, given the prevalence of chest imaging and the significant health implications of thoracic diseases. This review aims to highlight the promising applications of AI within thoracic imaging. It examines the role of AI, including its contributions to improving diagnostic evaluation and interpretation, enhancing workflow, and aiding in invasive procedures. Next, it further highlights the current challenges and limitations faced by AI, such as the necessity of 'big data', ethical and legal considerations, and bias in representation. Lastly, it explores the potential directions for the application of AI in thoracic radiology.
Collapse
Affiliation(s)
- Jin Y Chang
- Department of Radiology, The Ohio State University College of Medicine, Columbus, OH 43210, USA
| | - Mina S Makary
- Department of Radiology, The Ohio State University College of Medicine, Columbus, OH 43210, USA
- Division of Vascular and Interventional Radiology, Department of Radiology, The Ohio State University Wexner Medical Center, Columbus, OH 43210, USA
| |
Collapse
|
5
|
Chen CF, Hsu CH, Jiang YC, Lin WR, Hong WC, Chen IY, Lin MH, Chu KA, Lee CH, Lee DL, Chen PF. A deep learning-based algorithm for pulmonary tuberculosis detection in chest radiography. Sci Rep 2024; 14:14917. [PMID: 38942819 PMCID: PMC11213931 DOI: 10.1038/s41598-024-65703-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Accepted: 06/24/2024] [Indexed: 06/30/2024] Open
Abstract
In tuberculosis (TB), chest radiography (CXR) patterns are highly variable, mimicking pneumonia and many other diseases. This study aims to evaluate the efficacy of Google teachable machine, a deep neural network-based image classification tool, to develop algorithm for predicting TB probability of CXRs. The training dataset included 348 TB CXRs and 3806 normal CXRs for training TB detection. We also collected 1150 abnormal CXRs and 627 normal CXRs for training abnormality detection. For external validation, we collected 250 CXRs from our hospital. We also compared the accuracy of the algorithm to five pulmonologists and radiological reports. In external validation, the AI algorithm showed areas under the curve (AUC) of 0.951 and 0.975 in validation dataset 1 and 2. The accuracy of the pulmonologists on validation dataset 2 showed AUC range of 0.936-0.995. When abnormal CXRs other than TB were added, AUC decreased in both human readers (0.843-0.888) and AI algorithm (0.828). When combine human readers with AI algorithm, the AUC further increased to 0.862-0.885. The TB CXR AI algorithm developed by using Google teachable machine in this study is effective, with the accuracy close to experienced clinical physicians, and may be helpful for detecting tuberculosis by CXR.
Collapse
Affiliation(s)
- Chiu-Fan Chen
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
- Shu-Zen Junior College of Medicine and Management, Kaohsiung, Taiwan, R.O.C
- Department of Nursing, Mei-Ho University, Pingtung, Taiwan, R.O.C
| | - Chun-Hsiang Hsu
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - You-Cheng Jiang
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - Wen-Ren Lin
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - Wei-Cheng Hong
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - I-Yuan Chen
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - Min-Hsi Lin
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - Kuo-An Chu
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - Chao-Hsien Lee
- Department of Nursing, Mei-Ho University, Pingtung, Taiwan, R.O.C
| | - David Lin Lee
- Division of Chest Medicine, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, R.O.C
| | - Po-Fan Chen
- Department of Obstetrics and Gynecology, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan, R.O.C..
- Quality Center, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan, R.O.C..
| |
Collapse
|
6
|
Torres-Fernandez D, Dalsuco J, Bramugy J, Bassat Q, Varo R. Innovative strategies for the surveillance, prevention, and management of pediatric infections applied to low-income settings. Expert Rev Anti Infect Ther 2024; 22:413-422. [PMID: 38739471 DOI: 10.1080/14787210.2024.2354839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 05/09/2024] [Indexed: 05/16/2024]
Abstract
INTRODUCTION Infectious diseases still cause a significant burden of morbidity and mortality among children in low- and middle-income countries (LMICs). There are ample opportunities for innovation in surveillance, prevention, and management, with the ultimate goal of improving survival. AREAS COVERED This review discusses the current status in the use and development of innovative strategies for pediatric infectious diseases in LMICs by focusing on surveillance, diagnosis, prevention, and management. Topics covered are: Minimally Invasive Tissue Sampling as a technique to accurately ascertain the cause of death; Genetic Surveillance to trace the pathogen genomic diversity and emergence of resistance; Artificial Intelligence as a multidisciplinary tool; Portable noninvasive imaging methods; and Prognostic Biomarkers to triage and risk stratify pediatric patients. EXPERT OPINION To overcome the specific hurdles in child health for LMICs, some innovative strategies appear at the forefront of research. If the development of these next-generation tools remains focused on accessibility, sustainability and capacity building, reshaping epidemiological surveillance, diagnosis, and treatment in LMICs, can become a reality and result in a significant public health impact. Their integration with existing healthcare infrastructures may revolutionize disease detection and surveillance, and improve child health and survival.
Collapse
Affiliation(s)
- David Torres-Fernandez
- ISGlobal, Hospital Clínic - Universitat de Barcelona, Barcelona, Spain
- Centro de Investigação em Saúde de Manhiça (CISM), Maputo, Mozambique
| | - Jessica Dalsuco
- Centro de Investigação em Saúde de Manhiça (CISM), Maputo, Mozambique
| | - Justina Bramugy
- Centro de Investigação em Saúde de Manhiça (CISM), Maputo, Mozambique
| | - Quique Bassat
- ISGlobal, Hospital Clínic - Universitat de Barcelona, Barcelona, Spain
- Centro de Investigação em Saúde de Manhiça (CISM), Maputo, Mozambique
- ICREA, Pg. Lluís Companys, Barcelona, Spain
- Pediatrics Department, Hospital Sant Joan de Déu, Universitat de Barcelona, Barcelona, Spain
- CIBER de Epidemiología y Salud Pública, Instituto de Salud Carlos III, Madrid, Spain
| | - Rosauro Varo
- ISGlobal, Hospital Clínic - Universitat de Barcelona, Barcelona, Spain
- Centro de Investigação em Saúde de Manhiça (CISM), Maputo, Mozambique
| |
Collapse
|
7
|
Hwang EJ, Jeong WG, David PM, Arentz M, Ruhwald M, Yoon SH. AI for Detection of Tuberculosis: Implications for Global Health. Radiol Artif Intell 2024; 6:e230327. [PMID: 38197795 PMCID: PMC10982823 DOI: 10.1148/ryai.230327] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Revised: 12/03/2023] [Accepted: 12/18/2023] [Indexed: 01/11/2024]
Abstract
Tuberculosis, which primarily affects developing countries, remains a significant global health concern. Since the 2010s, the role of chest radiography has expanded in tuberculosis triage and screening beyond its traditional complementary role in the diagnosis of tuberculosis. Computer-aided diagnosis (CAD) systems for tuberculosis detection on chest radiographs have recently made substantial progress in diagnostic performance, thanks to deep learning technologies. The current performance of CAD systems for tuberculosis has approximated that of human experts, presenting a potential solution to the shortage of human readers to interpret chest radiographs in low- or middle-income, high-tuberculosis-burden countries. This article provides a critical appraisal of developmental process reporting in extant CAD software for tuberculosis, based on the Checklist for Artificial Intelligence in Medical Imaging. It also explores several considerations to scale up CAD solutions, encompassing manufacturer-independent CAD validation, economic and political aspects, and ethical concerns, as well as the potential for broadening radiography-based diagnosis to other nontuberculosis diseases. Collectively, CAD for tuberculosis will emerge as a representative deep learning application, catalyzing advances in global health and health equity. Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning Supplemental material is available for this article. © RSNA, 2024.
Collapse
Affiliation(s)
- Eui Jin Hwang
- From the Department of Radiology, Seoul National University Hospital
and Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu,
Seoul 03080, Korea (E.J.H., S.H.Y.); Department of Radiology, Chonnam National
University Hwasun Hospital, Hwasun, Korea (W.G.J.); Faculty of Pharmacy,
University of Montréal, Montréal, Canada (P.M.D.);
OBVIA–Observatoire sur les Impacts Sociétaux de l'IA et du
Numérique, Québec, Canada (P.M.D.); and FIND–The Global
Alliance for Diagnostics, Geneva, Switzerland (M.A., M.R.)
| | - Won Gi Jeong
- From the Department of Radiology, Seoul National University Hospital
and Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu,
Seoul 03080, Korea (E.J.H., S.H.Y.); Department of Radiology, Chonnam National
University Hwasun Hospital, Hwasun, Korea (W.G.J.); Faculty of Pharmacy,
University of Montréal, Montréal, Canada (P.M.D.);
OBVIA–Observatoire sur les Impacts Sociétaux de l'IA et du
Numérique, Québec, Canada (P.M.D.); and FIND–The Global
Alliance for Diagnostics, Geneva, Switzerland (M.A., M.R.)
| | - Pierre-Marie David
- From the Department of Radiology, Seoul National University Hospital
and Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu,
Seoul 03080, Korea (E.J.H., S.H.Y.); Department of Radiology, Chonnam National
University Hwasun Hospital, Hwasun, Korea (W.G.J.); Faculty of Pharmacy,
University of Montréal, Montréal, Canada (P.M.D.);
OBVIA–Observatoire sur les Impacts Sociétaux de l'IA et du
Numérique, Québec, Canada (P.M.D.); and FIND–The Global
Alliance for Diagnostics, Geneva, Switzerland (M.A., M.R.)
| | - Matthew Arentz
- From the Department of Radiology, Seoul National University Hospital
and Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu,
Seoul 03080, Korea (E.J.H., S.H.Y.); Department of Radiology, Chonnam National
University Hwasun Hospital, Hwasun, Korea (W.G.J.); Faculty of Pharmacy,
University of Montréal, Montréal, Canada (P.M.D.);
OBVIA–Observatoire sur les Impacts Sociétaux de l'IA et du
Numérique, Québec, Canada (P.M.D.); and FIND–The Global
Alliance for Diagnostics, Geneva, Switzerland (M.A., M.R.)
| | - Morten Ruhwald
- From the Department of Radiology, Seoul National University Hospital
and Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu,
Seoul 03080, Korea (E.J.H., S.H.Y.); Department of Radiology, Chonnam National
University Hwasun Hospital, Hwasun, Korea (W.G.J.); Faculty of Pharmacy,
University of Montréal, Montréal, Canada (P.M.D.);
OBVIA–Observatoire sur les Impacts Sociétaux de l'IA et du
Numérique, Québec, Canada (P.M.D.); and FIND–The Global
Alliance for Diagnostics, Geneva, Switzerland (M.A., M.R.)
| | - Soon Ho Yoon
- From the Department of Radiology, Seoul National University Hospital
and Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu,
Seoul 03080, Korea (E.J.H., S.H.Y.); Department of Radiology, Chonnam National
University Hwasun Hospital, Hwasun, Korea (W.G.J.); Faculty of Pharmacy,
University of Montréal, Montréal, Canada (P.M.D.);
OBVIA–Observatoire sur les Impacts Sociétaux de l'IA et du
Numérique, Québec, Canada (P.M.D.); and FIND–The Global
Alliance for Diagnostics, Geneva, Switzerland (M.A., M.R.)
| |
Collapse
|
8
|
Scott AJ, Perumal T, Hohlfeld A, Oelofse S, Kühn L, Swanepoel J, Geric C, Ahmad Khan F, Esmail A, Ochodo E, Engel M, Dheda K. Diagnostic Accuracy of Computer-Aided Detection During Active Case Finding for Pulmonary Tuberculosis in Africa: A Systematic Review and Meta-analysis. Open Forum Infect Dis 2024; 11:ofae020. [PMID: 38328498 PMCID: PMC10849117 DOI: 10.1093/ofid/ofae020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 01/15/2024] [Indexed: 02/09/2024] Open
Abstract
Background Computer-aided detection (CAD) may be a useful screening tool for tuberculosis (TB). However, there are limited data about its utility in active case finding (ACF) in a community-based setting, and particularly in an HIV-endemic setting where performance may be compromised. Methods We performed a systematic review and evaluated articles published between January 2012 and February 2023 that included CAD as a screening tool to detect pulmonary TB against a microbiological reference standard (sputum culture and/or nucleic acid amplification test [NAAT]). We collected and summarized data on study characteristics and diagnostic accuracy measures. Two reviewers independently extracted data and assessed methodological quality against Quality Assessment of Diagnostic Accuracy Studies-2 criteria. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines were followed. Results Of 1748 articles reviewed, 5 met with the eligibility criteria and were included in this review. A meta-analysis revealed pooled sensitivity of 0.87 (95% CI, 0.78-0.96) and specificity of 0.74 (95% CI, 0.55-0.93), just below the World Health Organization (WHO)-recommended target product profile (TPP) for a screening test (sensitivity ≥0.90 and specificity ≥0.70). We found a high risk of bias and applicability concerns across all studies. Subgroup analyses, including the impact of HIV and previous TB, were not possible due to the nature of the reporting within the included studies. Conclusions This review provides evidence, specifically in the context of ACF, for CAD as a potentially useful and cost-effective screening tool for TB in a resource-poor HIV-endemic African setting. However, given methodological concerns, caution is required with regards to applicability and generalizability.
Collapse
Affiliation(s)
- Alex J Scott
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
| | - Tahlia Perumal
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
| | - Ameer Hohlfeld
- Health Systems Research Unit, South African Medical Research Council, Cape Town, South Africa
| | - Suzette Oelofse
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
| | - Louié Kühn
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
| | - Jeremi Swanepoel
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
| | - Coralie Geric
- McGill International TB Centre, McGill University, Montreal, Quebec, Canada
| | - Faiz Ahmad Khan
- McGill International TB Centre, McGill University, Montreal, Quebec, Canada
| | - Aliasgar Esmail
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
| | - Eleanor Ochodo
- Kenya Medical Research Institute, Nairobi, Kenya
- Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University, Cape Town, South Africa
| | - Mark Engel
- Department of Medicine, University of Cape Town, Cape Town, South Africa
- Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa
| | - Keertan Dheda
- Division of Pulmonology, Department of Medicine, Centre for Lung Infection and Immunity, University of Cape Town Lung Institute, Cape Town, South Africa
- Centre for the Study of Antimicrobial Resistance, South African Medical Research Council and University of Cape Town, Cape Town, South Africa
- Department of Infection Biology, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, UK
| |
Collapse
|
9
|
Ghermi M, Messedi M, Adida C, Belarbi K, Djazouli MEA, Berrazeg ZI, Kallel Sellami M, Ghezini Y, Louati M. TubIAgnosis: A machine learning-based web application for active tuberculosis diagnosis using complete blood count data. Digit Health 2024; 10:20552076241278211. [PMID: 39224791 PMCID: PMC11367613 DOI: 10.1177/20552076241278211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 08/08/2024] [Indexed: 09/04/2024] Open
Abstract
Objective Tuberculosis remains a major global health challenge, with delayed diagnosis contributing to increased transmission and disease burden. While microbiological tests are the gold standard for confirming active tuberculosis, many cases lack microbiological evidence, necessitating additional clinical and laboratory data for diagnosis. The complete blood count (CBC), an inexpensive and widely available test, could provide a valuable tool for tuberculosis diagnosis by analyzing disturbances in blood parameters. This study aimed to develop and evaluate a machine learning (ML)-based web application, TubIAgnosis, for diagnosing active tuberculosis using CBC data. Methods We conducted a retrospective case-control study using data from 449 tuberculosis patients and 1200 healthy controls in Oran, Algeria, from January 2016 to April 2023. Eight ML algorithms were trained on 18 CBC parameters and demographic data. Model performance was evaluated using balanced accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC). Results The best-performing model, Extreme Gradient Boosting (XGB), achieved a balanced accuracy of 83.3%, AUC of 89.4%, sensitivity of 83.3%, and specificity of 83.3% on the testing dataset. Platelet-to-lymphocyte ratio was the most influential parameter in this ML predictive model. The best performing model (XGB) was made available online as a web application called TubIAgnosis, which is available free of charge at https://yh5f0z-ghermi-mohamed.shinyapps.io/TubIAgnosis/. Conclusions TubIAgnosis, a ML-based web application utilizing CBC data, demonstrated promising performance for diagnosing active tuberculosis. This accessible and cost-effective tool could complement existing diagnostic methods, particularly in resource-limited settings. Prospective studies are warranted to further validate and refine this approach.
Collapse
Affiliation(s)
- Mohamed Ghermi
- Biology of Microorganisms and Biotechnology Laboratory, University of Oran1 Ahmed Ben Bella, Oran, Algeria
- Biotechnology Department, University of Oran1 Ahmed Ben Bella, Oran, Algeria
| | - Meriam Messedi
- Molecular Bases of Human Diseases (LR19ES13), Faculty of Medicine, University of Sfax, Sfax, Tunisia
| | - Chahira Adida
- Biotechnology Department, University of Oran1 Ahmed Ben Bella, Oran, Algeria
| | - Kada Belarbi
- Biotechnology Department, University of Oran1 Ahmed Ben Bella, Oran, Algeria
| | - Mohamed El Amine Djazouli
- Occupational Medicine Service, Oran University Hospital Center, Faculty of Medicine, University of Oran1 Ahmed Ben Bella, Oran, Algeria
| | - Zahia Ibtissem Berrazeg
- Occupational Medicine Service, Oran University Hospital Center, Faculty of Medicine, University of Oran1 Ahmed Ben Bella, Oran, Algeria
| | | | - Younes Ghezini
- Occupational Medicine Service, Oran University Hospital Center, Faculty of Medicine, University of Oran1 Ahmed Ben Bella, Oran, Algeria
| | - Mahdi Louati
- National School of Electronics and Telecommunications of Sfax, University of Sfax, Sfax, Tunisia
| |
Collapse
|
10
|
Vanobberghen F, Keter AK, Jacobs BK, Glass TR, Lynen L, Law I, Murphy K, van Ginneken B, Ayakaka I, van Heerden A, Maama L, Reither K. Computer-aided detection thresholds for digital chest radiography interpretation in tuberculosis diagnostic algorithms. ERJ Open Res 2024; 10:00508-2023. [PMID: 38196890 PMCID: PMC10772898 DOI: 10.1183/23120541.00508-2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Accepted: 10/25/2023] [Indexed: 01/11/2024] Open
Abstract
Objectives Use of computer-aided detection (CAD) software is recommended to improve tuberculosis screening and triage, but threshold determination is challenging if reference testing has not been performed in all individuals. We aimed to determine such thresholds through secondary analysis of the 2019 Lesotho national tuberculosis prevalence survey. Methods Symptom screening and chest radiographs were performed in participants aged ≥15 years; those symptomatic or with abnormal chest radiographs provided samples for Xpert MTB/RIF and culture testing. Chest radiographs were processed using CAD4TB version 7. We used six methodological approaches to deal with participants who did not have bacteriological test results to estimate pulmonary tuberculosis prevalence and assess diagnostic accuracy. Results Among 17 070 participants, 5214 (31%) had their tuberculosis status determined; 142 had tuberculosis. Prevalence estimates varied between methodological approaches (0.83-2.72%). Using multiple imputation to estimate tuberculosis status for those eligible but not tested, and assuming those not eligible for testing were negative, a CAD4TBv7 threshold of 13 had a sensitivity of 89.7% (95% CI 84.6-94.8) and a specificity of 74.2% (73.6-74.9), close to World Health Organization (WHO) target product profile criteria. Assuming all those not tested were negative produced similar results. Conclusions This is the first study to evaluate CAD4TB in a community screening context employing a range of approaches to account for unknown tuberculosis status. The assumption that those not tested are negative - regardless of testing eligibility status - was robust. As threshold determination must be context specific, our analytically straightforward approach should be adopted to leverage prevalence surveys for CAD threshold determination in other settings with a comparable proportion of eligible but not tested participants.
Collapse
Affiliation(s)
- Fiona Vanobberghen
- Swiss Tropical and Public Health Institute, Allschwil, Switzerland
- University of Basel, Basel, Switzerland
| | - Alfred Kipyegon Keter
- Institute of Tropical Medicine, Antwerp, Belgium
- Centre for Community Based Research, Human Sciences Research Council, Pietermaritzburg, South Africa
- Ghent University, Ghent, Belgium
| | | | - Tracy R. Glass
- Swiss Tropical and Public Health Institute, Allschwil, Switzerland
- University of Basel, Basel, Switzerland
| | | | - Irwin Law
- Global Tuberculosis Programme, World Health Organization, Geneva, Switzerland
| | - Keelin Murphy
- Radboud University Medical Center, Nijmegen, Netherlands
| | | | - Irene Ayakaka
- SolidarMed, Partnerships for Health, Maseru, Lesotho
| | - Alastair van Heerden
- Centre for Community Based Research, Human Sciences Research Council, Pietermaritzburg, South Africa
- SAMRC/Wits Developmental Pathways for Health Research Unit (DPHRU), Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| | - Llang Maama
- Disease Control Directorate, National Tuberculosis Program, Ministry of Health, Maseru, Lesotho
| | - Klaus Reither
- Swiss Tropical and Public Health Institute, Allschwil, Switzerland
- University of Basel, Basel, Switzerland
| |
Collapse
|
11
|
Innes AL, Martinez A, Gao X, Dinh N, Hoang GL, Nguyen TBP, Vu VH, Luu THT, Le TTT, Lebrun V, Trieu VC, Tran NDB, Qin ZZ, Pham HM, Dinh VL, Nguyen BH, Truong TTH, Nguyen VC, Nguyen VN, Mai TH. Computer-Aided Detection for Chest Radiography to Improve the Quality of Tuberculosis Diagnosis in Vietnam's District Health Facilities: An Implementation Study. Trop Med Infect Dis 2023; 8:488. [PMID: 37999607 PMCID: PMC10675130 DOI: 10.3390/tropicalmed8110488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 10/20/2023] [Accepted: 10/25/2023] [Indexed: 11/25/2023] Open
Abstract
In Vietnam, chest radiography (CXR) is used to refer people for GeneXpert (Xpert) testing to diagnose tuberculosis (TB), demonstrating high yield for TB but a wide range of CXR abnormality rates. In a multi-center implementation study, computer-aided detection (CAD) was integrated into facility-based TB case finding to standardize CXR interpretation. CAD integration was guided by a programmatic framework developed for routine implementation. From April through December 2022, 24,945 CXRs from TB-vulnerable populations presenting to district health facilities were evaluated. Physicians interpreted all CXRs in parallel with CAD (qXR 3.0) software, for which the selected TB threshold score was ≥0.60. At three months, there was 47.3% concordance between physician and CAD TB-presumptive CXR results, 7.8% of individuals who received CXRs were referred for Xpert testing, and 858 people diagnosed with Xpert-confirmed TB per 100,000 CXRs. This increased at nine months to 76.1% concordant physician and CAD TB-presumptive CXRs, 9.6% referred for Xpert testing, and 2112 people with Xpert-confirmed TB per 100,000 CXRs. Our programmatic CAD-CXR framework effectively supported physicians in district facilities to improve the quality of referral for diagnostic testing and increase TB detection yield. Concordance between physician and CAD CXR results improved with training and was important to optimize Xpert testing.
Collapse
Affiliation(s)
- Anh L. Innes
- FHI 360 Asia Pacific Regional Office, Bangkok 10330, Thailand
| | | | - Xiaoming Gao
- FHI 360, Durham, NC 27701, USA; (A.M.); (X.G.); (N.D.)
| | - Nhi Dinh
- FHI 360, Durham, NC 27701, USA; (A.M.); (X.G.); (N.D.)
| | - Gia Linh Hoang
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Thi Bich Phuong Nguyen
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Viet Hien Vu
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Tuan Ho Thanh Luu
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Thi Thu Trang Le
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Victoria Lebrun
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Van Chinh Trieu
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Nghi Do Bao Tran
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| | - Zhi Zhen Qin
- Stop TB Partnership, Grand-Saconnex, 1218 Geneva, Switzerland;
| | - Huy Minh Pham
- United States Agency for International Development/Vietnam, Hanoi 10000, Vietnam;
| | - Van Luong Dinh
- Vietnam National Lung Hospital, Hanoi 10000, Vietnam; (V.L.D.); (B.H.N.); (T.T.H.T.); (V.C.N.); (V.N.N.)
| | - Binh Hoa Nguyen
- Vietnam National Lung Hospital, Hanoi 10000, Vietnam; (V.L.D.); (B.H.N.); (T.T.H.T.); (V.C.N.); (V.N.N.)
| | - Thi Thanh Huyen Truong
- Vietnam National Lung Hospital, Hanoi 10000, Vietnam; (V.L.D.); (B.H.N.); (T.T.H.T.); (V.C.N.); (V.N.N.)
| | - Van Cu Nguyen
- Vietnam National Lung Hospital, Hanoi 10000, Vietnam; (V.L.D.); (B.H.N.); (T.T.H.T.); (V.C.N.); (V.N.N.)
| | - Viet Nhung Nguyen
- Vietnam National Lung Hospital, Hanoi 10000, Vietnam; (V.L.D.); (B.H.N.); (T.T.H.T.); (V.C.N.); (V.N.N.)
- Pulmonology Department, University of Medicine and Pharmacy, Vietnam National University, Hanoi 10000, Vietnam
| | - Thu Hien Mai
- FHI 360 Vietnam, Hanoi 10000, Vietnam; (G.L.H.); (T.B.P.N.); (V.H.V.); (T.H.T.L.); (T.T.T.L.); (V.L.); (V.C.T.); (N.D.B.T.); (T.H.M.)
| |
Collapse
|