1
|
Uddin MG, Rahman A, Rosa Taghikhah F, Olbert AI. Data-driven evolution of water quality models: An in-depth investigation of innovative outlier detection approaches-A case study of Irish Water Quality Index (IEWQI) model. WATER RESEARCH 2024; 255:121499. [PMID: 38552494 DOI: 10.1016/j.watres.2024.121499] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 02/09/2024] [Accepted: 03/19/2024] [Indexed: 04/24/2024]
Abstract
Recently, there has been a significant advancement in the water quality index (WQI) models utilizing data-driven approaches, especially those integrating machine learning and artificial intelligence (ML/AI) technology. Although, several recent studies have revealed that the data-driven model has produced inconsistent results due to the data outliers, which significantly impact model reliability and accuracy. The present study was carried out to assess the impact of data outliers on a recently developed Irish Water Quality Index (IEWQI) model, which relies on data-driven techniques. To the author's best knowledge, there has been no systematic framework for evaluating the influence of data outliers on such models. For the purposes of assessing the outlier impact of the data outliers on the water quality (WQ) model, this was the first initiative in research to introduce a comprehensive approach that combines machine learning with advanced statistical techniques. The proposed framework was implemented in Cork Harbour, Ireland, to evaluate the IEWQI model's sensitivity to outliers in input indicators to assess the water quality. In order to detect the data outlier, the study utilized two widely used ML techniques, including Isolation Forest (IF) and Kernel Density Estimation (KDE) within the dataset, for predicting WQ with and without these outliers. For validating the ML results, the study used five commonly used statistical measures. The performance metric (R2) indicates that the model performance improved slightly (R2 increased from 0.92 to 0.95) in predicting WQ after removing the data outlier from the input. But the IEWQI scores revealed that there were no statistically significant differences among the actual values, predictions with outliers, and predictions without outliers, with a 95 % confidence interval at p < 0.05. The results of model uncertainty also revealed that the model contributed <1 % uncertainty to the final assessment results for using both datasets (with and without outliers). In addition, all statistical measures indicated that the ML techniques provided reliable results that can be utilized for detecting outliers and their impacts on the IEWQI model. The findings of the research reveal that although the data outliers had no significant impact on the IEWQI model architecture, they had moderate impacts on the rating schemes' of the model. This finding indicated that detecting the data outliers could improve the accuracy of the IEWQI model in rating WQ as well as be helpful in mitigating the model eclipsing problem. In addition, the results of the research provide evidence of how the data outliers influenced the data-driven model in predicting WQ and reliability, particularly since the study confirmed that the IEWQI model's could be effective for accurately rating WQ despite the presence of the data outliers in the input. It could occur due to the spatio-temporal variability inherent in WQ indicators. However, the research assesses the influence of data input outliers on the IEWQI model and underscores important areas for future investigation. These areas include expanding temporal analysis using multi-year data, examining spatial outlier patterns, and evaluating detection methods. Moreover, it is essential to explore the real-world impacts of revised rating categories, involve stakeholders in outlier management, and fine-tune model parameters. Analysing model performance across varying temporal and spatial resolutions and incorporating additional environmental data can significantly enhance the accuracy of WQ assessment. Consequently, this study offers valuable insights to strengthen the IEWQI model's robustness and provides avenues for enhancing its utility in broader WQ assessment applications. Moreover, the study successfully adopted the framework for evaluating how data input outliers affect the data-driven model, such as the IEWQI model. The current study has been carried out in Cork Harbour for only a single year of WQ data. The framework should be tested across various domains for evaluating the response of the IEWQI model's in terms of the spatio-temporal resolution of the domain. Nevertheless, the study recommended that future research should be conducted to adjust or revise the IEWQI model's rating schemes and investigate the practical effects of data outliers on updated rating categories. However, the study provides potential recommendations for enhancing the IEWQI model's adaptability and reveals its effectiveness in expanding its applicability in more general WQ assessment scenarios.
Collapse
Affiliation(s)
- Md Galal Uddin
- School of Engineering, University of Galway, Ireland; Ryan Institute, University of Galway, Ireland; MaREI Research Centre, University of Galway, Ireland; Eco-HydroInformatics Research Group (EHIRG), Civil Engineering, National University of Ireland Galway, Ireland.
| | - Azizur Rahman
- School of Computing, Mathematics and Engineering, Charles Sturt University, Wagga, Australia; The Gulbali Institute of Agriculture, Water and Environment, Charles Sturt University, Wagga, Australia
| | | | - Agnieszka I Olbert
- School of Engineering, University of Galway, Ireland; Ryan Institute, University of Galway, Ireland; MaREI Research Centre, University of Galway, Ireland; Eco-HydroInformatics Research Group (EHIRG), Civil Engineering, National University of Ireland Galway, Ireland
| |
Collapse
|
2
|
Niraula D, Cuneo KC, Dinov ID, Gonzalez BD, Jamaluddin JB, Jin JJ, Luo Y, Matuszak MM, Ten Haken RK, Bryant AK, Dilling TJ, Dykstra MP, Frakes JM, Liveringhouse CL, Miller SR, Mills MN, Palm RF, Regan SN, Rishi A, Torres-Roca JF, Yu HHM, El Naqa I. Intricacies of Human-AI Interaction in Dynamic Decision-Making for Precision Oncology: A Case Study in Response-Adaptive Radiotherapy. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.04.27.24306434. [PMID: 38746238 PMCID: PMC11092730 DOI: 10.1101/2024.04.27.24306434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
Background Adaptive treatment strategies that can dynamically react to individual cancer progression can provide effective personalized care. Longitudinal multi-omics information, paired with an artificially intelligent clinical decision support system (AI-CDSS) can assist clinicians in determining optimal therapeutic options and treatment adaptations. However, AI-CDSS is not perfectly accurate, as such, clinicians' over/under reliance on AI may lead to unintended consequences, ultimately failing to develop optimal strategies. To investigate such collaborative decision-making process, we conducted a Human-AI interaction case study on response-adaptive radiotherapy (RT). Methods We designed and conducted a two-phase study for two disease sites and two treatment modalities-adaptive RT for non-small cell lung cancer (NSCLC) and adaptive stereotactic body RT for hepatocellular carcinoma (HCC)-in which clinicians were asked to consider mid-treatment modification of the dose per fraction for a number of retrospective cancer patients without AI-support (Unassisted Phase) and with AI-assistance (AI-assisted Phase). The AI-CDSS graphically presented trade-offs in tumor control and the likelihood of toxicity to organs at risk, provided an optimal recommendation, and associated model uncertainties. In addition, we asked for clinicians' decision confidence level and trust level in individual AI recommendations and encouraged them to provide written remarks. We enrolled 13 evaluators (radiation oncology physicians and residents) from two medical institutions located in two different states, out of which, 4 evaluators volunteered in both NSCLC and HCC studies, resulting in a total of 17 completed evaluations (9 NSCLC, and 8 HCC). To limit the evaluation time to under an hour, we selected 8 treated patients for NSCLC and 9 for HCC, resulting in a total of 144 sets of evaluations (72 from NSCLC and 72 from HCC). Evaluation for each patient consisted of 8 required inputs and 2 optional remarks, resulting in up to a total of 1440 data points. Results AI-assistance did not homogeneously influence all experts and clinical decisions. From NSCLC cohort, 41 (57%) decisions and from HCC cohort, 34 (47%) decisions were adjusted after AI assistance. Two evaluations (12%) from the NSCLC cohort had zero decision adjustments, while the remaining 15 (88%) evaluations resulted in at least two decision adjustments. Decision adjustment level positively correlated with dissimilarity in decision-making with AI [NSCLC: ρ = 0.53 ( p < 0.001); HCC: ρ = 0.60 ( p < 0.001)] indicating that evaluators adjusted their decision closer towards AI recommendation. Agreement with AI-recommendation positively correlated with AI Trust Level [NSCLC: ρ = 0.59 ( p < 0.001); HCC: ρ = 0.7 ( p < 0.001)] indicating that evaluators followed AI's recommendation if they agreed with that recommendation. The correlation between decision confidence changes and decision adjustment level showed an opposite trend [NSCLC: ρ = -0.24 ( p = 0.045), HCC: ρ = 0.28 ( p = 0.017)] reflecting the difference in behavior due to underlying differences in disease type and treatment modality. Decision confidence positively correlated with the closeness of decisions to the standard of care (NSCLC: 2 Gy/fx; HCC: 10 Gy/fx) indicating that evaluators were generally more confident in prescribing dose fractionations more similar to those used in standard clinical practice. Inter-evaluator agreement increased with AI-assistance indicating that AI-assistance can decrease inter-physician variability. The majority of decisions were adjusted to achieve higher tumor control in NSCLC and lower normal tissue complications in HCC. Analysis of evaluators' remarks indicated concerns for organs at risk and RT outcome estimates as important decision-making factors. Conclusions Human-AI interaction depends on the complex interrelationship between expert's prior knowledge and preferences, patient's state, disease site, treatment modality, model transparency, and AI's learned behavior and biases. The collaborative decision-making process can be summarized as follows: (i) some clinicians may not believe in an AI system, completely disregarding its recommendation, (ii) some clinicians may believe in the AI system but will critically analyze its recommendations on a case-by-case basis; (iii) when a clinician finds that the AI recommendation indicates the possibility for better outcomes they will adjust their decisions accordingly; and (iv) When a clinician finds that the AI recommendation indicate a worse possible outcome they will disregard it and seek their own alternative approach.
Collapse
|
3
|
Vens C, van Luijk P, Vogelius RI, El Naqa I, Humbert-Vidan L, von Neubeck C, Gomez-Roman N, Bahn E, Brualla L, Böhlen TT, Ecker S, Koch R, Handeland A, Pereira S, Possenti L, Rancati T, Todor D, Vanderstraeten B, Van Heerden M, Ullrich W, Jackson M, Alber M, Marignol L. A joint physics and radiobiology DREAM team vision - Towards better response prediction models to advance radiotherapy. Radiother Oncol 2024; 196:110277. [PMID: 38670264 DOI: 10.1016/j.radonc.2024.110277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 03/21/2024] [Accepted: 04/11/2024] [Indexed: 04/28/2024]
Abstract
Radiotherapy developed empirically through experience balancing tumour control and normal tissue toxicities. Early simple mathematical models formalized this practical knowledge and enabled effective cancer treatment to date. Remarkable advances in technology, computing, and experimental biology now create opportunities to incorporate this knowledge into enhanced computational models. The ESTRO DREAM (Dose Response, Experiment, Analysis, Modelling) workshop brought together experts across disciplines to pursue the vision of personalized radiotherapy for optimal outcomes through advanced modelling. The ultimate vision is leveraging quantitative models dynamically during therapy to ultimately achieve truly adaptive and biologically guided radiotherapy at the population as well as individual patient-based levels. This requires the generation of models that inform response-based adaptations, individually optimized delivery and enable biological monitoring to provide decision support to clinicians. The goal is expanding to models that can drive the realization of personalized therapy for optimal outcomes. This position paper provides their propositions that describe how innovations in biology, physics, mathematics, and data science including AI could inform models and improve predictions. It consolidates the DREAM team's consensus on scientific priorities and organizational requirements. Scientifically, it stresses the need for rigorous, multifaceted model development, comprehensive validation and clinical applicability and significance. Organizationally, it reinforces the prerequisites of interdisciplinary research and collaboration between physicians, medical physicists, radiobiologists, and computational scientists throughout model development. Solely by a shared understanding of clinical needs, biological mechanisms, and computational methods, more informed models can be created. Future research environment and support must facilitate this integrative method of operation across multiple disciplines.
Collapse
Affiliation(s)
- C Vens
- School of Cancer Science, University of Glasgow, Glasgow, UK; Department of Head and Neck Oncology and Surgery, The Netherlands Cancer Institute/Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands.
| | - P van Luijk
- Department of Biomedical Sciences of Cells and Systems, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands; Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.
| | - R I Vogelius
- Department of Oncology, Rigshospitalet, Copenhagen, Denmark; Faculty of Health and Medical Sciences, University of Copenhagen, Denmark.
| | - I El Naqa
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI 48103, United States.
| | - L Humbert-Vidan
- University of Texas MD Anderson Cancer Centre, Houston, TX, United States; Department of MedicalPhysics, Guy's and St Thomas' NHS Foundation Trust, London, UK; School of Cancer and Pharmaceutical Sciences, Comprehensive Cancer Centre, King's College London, London, UK
| | - C von Neubeck
- Department of Particle Therapy, University Hospital Essen, University of Duisburg-Essen, Essen 45147, Germany
| | - N Gomez-Roman
- Strathclyde Institute of Phrmacy and Biomedical Sciences, University of Strathclyde, Glasgow, UK
| | - E Bahn
- Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany; Heidelberg Institute of Radiation Oncology (HIRO), Heidelberg, Germany; National Center for Tumor Diseases (NCT), Heidelberg, Germany; Clinical Cooperation Unit Radiation Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - L Brualla
- West German Proton Therapy Centre Essen (WPE), Essen, Germany; Faculty of Medicine, University of Duisburg-Essen, Germany
| | - T T Böhlen
- Institute of Radiation Physics, Lausanne University Hospital and Lausanne University, Lausanne, Switzerland
| | - S Ecker
- Department of Radiation Oncology, Medical University of Wien, Austria
| | - R Koch
- Department of Particle Therapy, University Hospital Essen, University of Duisburg-Essen, Essen 45147, Germany
| | - A Handeland
- Department of Oncology and Medical Physics, Haukeland University Hospital, Bergen, Norway; Department of Physics and Technology, University of Bergen, Bergen, Norway
| | - S Pereira
- Neolys Diagnostics, 7 Allée de l'Europe, 67960 Entzheim, France
| | - L Possenti
- Data Science Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - T Rancati
- Data Science Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - D Todor
- Department of Radiation Oncology, Virginia Commonwealth University, United States
| | - B Vanderstraeten
- Department of Radiotherapy-Oncology, Ghent University Hospital, Gent, Belgium; Department of Human Structure and Repair, Ghent University, Gent, Belgium
| | - M Van Heerden
- Center for Proton Therapy, Paul Scherrer Institute, Villigen, Switzerland
| | | | - M Jackson
- School of Cancer Science, University of Glasgow, Glasgow, UK
| | - M Alber
- Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany; Heidelberg Institute of Radiation Oncology (HIRO), Heidelberg, Germany
| | - L Marignol
- Applied Radiation Therapy Trinity (ARTT), Discipline of Radiation Therapy, School of Medicine, Trinity St. James's Cancer Institute, Trinity College Dublin, University of Dublin, Dublin, Ireland
| |
Collapse
|
4
|
Vrudhula A, Kwan AC, Ouyang D, Cheng S. Machine Learning and Bias in Medical Imaging: Opportunities and Challenges. Circ Cardiovasc Imaging 2024; 17:e015495. [PMID: 38377237 PMCID: PMC10883605 DOI: 10.1161/circimaging.123.015495] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
Bias in health care has been well documented and results in disparate and worsened outcomes for at-risk groups. Medical imaging plays a critical role in facilitating patient diagnoses but involves multiple sources of bias including factors related to access to imaging modalities, acquisition of images, and assessment (ie, interpretation) of imaging data. Machine learning (ML) applied to diagnostic imaging has demonstrated the potential to improve the quality of imaging-based diagnosis and the precision of measuring imaging-based traits. Algorithms can leverage subtle information not visible to the human eye to detect underdiagnosed conditions or derive new disease phenotypes by linking imaging features with clinical outcomes, all while mitigating cognitive bias in interpretation. Importantly, however, the application of ML to diagnostic imaging has the potential to either reduce or propagate bias. Understanding the potential gain as well as the potential risks requires an understanding of how and what ML models learn. Common risks of propagating bias can arise from unbalanced training, suboptimal architecture design or selection, and uneven application of models. Notwithstanding these risks, ML may yet be applied to improve gain from imaging across all 3A's (access, acquisition, and assessment) for all patients. In this review, we present a framework for understanding the balance of opportunities and challenges for minimizing bias in medical imaging, how ML may improve current approaches to imaging, and what specific design considerations should be made as part of efforts to maximize the quality of health care for all.
Collapse
Affiliation(s)
- Amey Vrudhula
- Icahn School of Medicine at Mount Sinai, New York
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center
| | - Alan C Kwan
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center
| | - David Ouyang
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center
- Division of Artificial Intelligence in Medicine, Department of Medicine, Cedars-Sinai Medical Center
| | - Susan Cheng
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center
| |
Collapse
|
5
|
Shbeer AM, Robadi IA. liquid biopsy holds a promising approach for the early detection of cancer: Current information and future perspectives. Pathol Res Pract 2024; 254:155082. [PMID: 38246032 DOI: 10.1016/j.prp.2023.155082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 12/24/2023] [Accepted: 12/30/2023] [Indexed: 01/23/2024]
Abstract
Cancer is becoming a global pandemic, and its occurrence is increasing rapidly, putting a strain on people's families, health systems, and finances, in addition to their physical, mental, and emotional well-being. Many cancer types lack screening programs, and many people at high risk of developing cancer do not follow recommended medical screening regimens because of the nature of currently available screening tests and other compliance issues, despite cancer being the second leading cause of death worldwide. Furthermore, a lot of liquid biopsy methods for early cancer screening are not sensitive enough to catch cancer early. Cancer treatment costs increase with the time it takes to diagnose the disease; therefore, early detection is essential to enhance the quality of life and survival rates. The current status of the liquid biopsy sector is examined in this paper.
Collapse
Affiliation(s)
- Abdullah M Shbeer
- Department of Surgery, Faculty of Medicine, Jazan University, Jazan, Saudi Arabia.
| | - Ibrahim Ahmed Robadi
- Department of Pathology, Faculty of Medicine, Jazan University, Jazan, Saudi Arabia.
| |
Collapse
|
6
|
Grizzi F, Bax C, Capelli L, Taverna G. Editorial: Reshaping the diagnostic process in oncology: science versus technology. Front Oncol 2023; 13:1321688. [PMID: 37941548 PMCID: PMC10628722 DOI: 10.3389/fonc.2023.1321688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2023] [Accepted: 10/18/2023] [Indexed: 11/10/2023] Open
Affiliation(s)
- Fabio Grizzi
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital, Rozzano, Italy
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
| | - Carmen Bax
- Politecnico di Milano, Department of Chemistry, Materials and Chemical Engineering “Giulio Natta”, Milano, Italy
| | - Laura Capelli
- Politecnico di Milano, Department of Chemistry, Materials and Chemical Engineering “Giulio Natta”, Milano, Italy
| | - Gianluigi Taverna
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Department of Urology, Humanitas Mater Domini, Castellanza, Italy
| |
Collapse
|
7
|
Fico N, Di Grezia G, Cuccurullo V, Salvia AAH, Iacomino A, Sciarra A, Gatta G. Breast Imaging Physics in Mammography (Part I). Diagnostics (Basel) 2023; 13:3227. [PMID: 37892053 PMCID: PMC10606465 DOI: 10.3390/diagnostics13203227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 10/04/2023] [Indexed: 10/29/2023] Open
Abstract
Breast cancer is the most frequently diagnosed neoplasm in women in Italy. There are several risk factors, but thanks to screening and increased awareness, most breast cancers are diagnosed at an early stage when surgical treatment can most often be conservative and the adopted therapy is more effective. Regular screening is essential but advanced technology is needed to achieve quality diagnoses. Mammography is the gold standard for early detection of breast cancer. It is a specialized technique for detecting breast cancer and, thus, distinguishing normal tissue from cancerous breast tissue. Mammography techniques are based on physical principles: through the proper use of X-rays, the structures of different tissues can be observed. This first part of the paper attempts to explain the physical principles used in mammography. In particular, we will see how a mammogram is composed and what physical principles are used to obtain diagnostic images.
Collapse
Affiliation(s)
- Noemi Fico
- Department of Physics Ettore Pancini, Università di Napoli Federico II, 80126 Naples, Italy
| | | | - Vincenzo Cuccurullo
- Nuclear Medicine Unit, Department of Precision Medicine, Università della Campania Luigi Vanvitelli, 81100 Napoli, Italy;
| | | | - Aniello Iacomino
- Department of Human Science, Guglielmo Marconi University, 00193 Rome, Italy;
| | - Antonella Sciarra
- Department of Experimental Medicine, University of Campania Luigi Vanvitelli, 80138 Napoli, Italy;
| | - Gianluca Gatta
- Department of Precision Medicine, Università della Campania Luigi Vanvitelli, 81100 Napoli, Italy; (A.A.H.S.); (G.G.)
| |
Collapse
|
8
|
Naqa IE, Drukker K. AI in imaging and therapy: innovations, ethics, and impact - introductory editorial. Br J Radiol 2023; 96:20239004. [PMID: 38011226 PMCID: PMC10546442 DOI: 10.1259/bjr.20239004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2023] Open
|