1
|
Straw I, Rees G, Nachev P. Sex-Based Performance Disparities in Machine Learning Algorithms for Cardiac Disease Prediction: Exploratory Study. J Med Internet Res 2024; 26:e46936. [PMID: 39186324 PMCID: PMC11384168 DOI: 10.2196/46936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 10/13/2023] [Accepted: 05/04/2024] [Indexed: 08/27/2024] Open
Abstract
BACKGROUND The presence of bias in artificial intelligence has garnered increased attention, with inequities in algorithmic performance being exposed across the fields of criminal justice, education, and welfare services. In health care, the inequitable performance of algorithms across demographic groups may widen health inequalities. OBJECTIVE Here, we identify and characterize bias in cardiology algorithms, looking specifically at algorithms used in the management of heart failure. METHODS Stage 1 involved a literature search of PubMed and Web of Science for key terms relating to cardiac machine learning (ML) algorithms. Papers that built ML models to predict cardiac disease were evaluated for their focus on demographic bias in model performance, and open-source data sets were retained for our investigation. Two open-source data sets were identified: (1) the University of California Irvine Heart Failure data set and (2) the University of California Irvine Coronary Artery Disease data set. We reproduced existing algorithms that have been reported for these data sets, tested them for sex biases in algorithm performance, and assessed a range of remediation techniques for their efficacy in reducing inequities. Particular attention was paid to the false negative rate (FNR), due to the clinical significance of underdiagnosis and missed opportunities for treatment. RESULTS In stage 1, our literature search returned 127 papers, with 60 meeting the criteria for a full review and only 3 papers highlighting sex differences in algorithm performance. In the papers that reported sex, there was a consistent underrepresentation of female patients in the data sets. No papers investigated racial or ethnic differences. In stage 2, we reproduced algorithms reported in the literature, achieving mean accuracies of 84.24% (SD 3.51%) for data set 1 and 85.72% (SD 1.75%) for data set 2 (random forest models). For data set 1, the FNR was significantly higher for female patients in 13 out of 16 experiments, meeting the threshold of statistical significance (-17.81% to -3.37%; P<.05). A smaller disparity in the false positive rate was significant for male patients in 13 out of 16 experiments (-0.48% to +9.77%; P<.05). We observed an overprediction of disease for male patients (higher false positive rate) and an underprediction of disease for female patients (higher FNR). Sex differences in feature importance suggest that feature selection needs to be demographically tailored. CONCLUSIONS Our research exposes a significant gap in cardiac ML research, highlighting that the underperformance of algorithms for female patients has been overlooked in the published literature. Our study quantifies sex disparities in algorithmic performance and explores several sources of bias. We found an underrepresentation of female patients in the data sets used to train algorithms, identified sex biases in model error rates, and demonstrated that a series of remediation techniques were unable to address the inequities present.
Collapse
Affiliation(s)
- Isabel Straw
- University College London, London, United Kingdom
| | - Geraint Rees
- University College London, London, United Kingdom
| | | |
Collapse
|
2
|
Chinni BK, Manlhiot C. Emerging Analytical Approaches for Personalized Medicine Using Machine Learning In Pediatric and Congenital Heart Disease. Can J Cardiol 2024:S0828-282X(24)00585-3. [PMID: 39097187 DOI: 10.1016/j.cjca.2024.07.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Revised: 07/25/2024] [Accepted: 07/26/2024] [Indexed: 08/05/2024] Open
Abstract
Precision and personalized medicine, the process by which patient management is tailored to individual circumstances, are now terms that are familiar to cardiologists, despite it still being an emerging field. Although precision medicine relies most often on the underlying biology and pathophysiology of a patient's condition, personalized medicine relies on digital biomarkers generated through algorithms. Given the complexity of the underlying data, these digital biomarkers are most often generated through machine-learning algorithms. There are a number of analytic considerations regarding the creation of digital biomarkers that are discussed in this review, including data preprocessing, time dependency and gating, dimensionality reduction, and novel methods, both in the realm of supervised and unsupervised machine learning. Some of these considerations, such as sample size requirements and measurements of model performance, are particularly challenging in small and heterogeneous populations with rare outcomes such as children with congenital heart disease. Finally, we review analytic considerations for the deployment of digital biomarkers in clinical settings, including the emerging field of clinical artificial intelligence (AI) operations, computational needs for deployment, efforts to increase the explainability of AI, algorithmic drift, and the needs for distributed surveillance and federated learning. We conclude this review by discussing a recent simulation study that shows that, despite these analytic challenges and complications, the use of digital biomarkers in managing clinical care might have substantial benefits regarding individual patient outcomes.
Collapse
Affiliation(s)
- Bhargava K Chinni
- The Blalock-Taussig-Thomas Pediatric and Congenital Heart Center, Department of Pediatrics, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
| | - Cedric Manlhiot
- The Blalock-Taussig-Thomas Pediatric and Congenital Heart Center, Department of Pediatrics, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA; Research Institute, SickKids Hospital, Department of Pediatrics, University of Toronto, Toronto, Ontario, Canada.
| |
Collapse
|
3
|
Federico CA, Trotsyuk AA. Biomedical Data Science, Artificial Intelligence, and Ethics: Navigating Challenges in the Face of Explosive Growth. Annu Rev Biomed Data Sci 2024; 7:1-14. [PMID: 38598860 DOI: 10.1146/annurev-biodatasci-102623-104553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/12/2024]
Abstract
Advances in biomedical data science and artificial intelligence (AI) are profoundly changing the landscape of healthcare. This article reviews the ethical issues that arise with the development of AI technologies, including threats to privacy, data security, consent, and justice, as they relate to donors of tissue and data. It also considers broader societal obligations, including the importance of assessing the unintended consequences of AI research in biomedicine. In addition, this article highlights the challenge of rapid AI development against the backdrop of disparate regulatory frameworks, calling for a global approach to address concerns around data misuse, unintended surveillance, and the equitable distribution of AI's benefits and burdens. Finally, a number of potential solutions to these ethical quandaries are offered. Namely, the merits of advocating for a collaborative, informed, and flexible regulatory approach that balances innovation with individual rights and public welfare, fostering a trustworthy AI-driven healthcare ecosystem, are discussed.
Collapse
Affiliation(s)
- Carole A Federico
- Center for Biomedical Ethics, Stanford University School of Medicine, Stanford, California, USA; ,
| | - Artem A Trotsyuk
- Center for Biomedical Ethics, Stanford University School of Medicine, Stanford, California, USA; ,
| |
Collapse
|
4
|
Montomoli J, Bitondo MM, Cascella M, Rezoagli E, Romeo L, Bellini V, Semeraro F, Gamberini E, Frontoni E, Agnoletti V, Altini M, Benanti P, Bignami EG. Algor-ethics: charting the ethical path for AI in critical care. J Clin Monit Comput 2024; 38:931-939. [PMID: 38573370 PMCID: PMC11297831 DOI: 10.1007/s10877-024-01157-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2023] [Accepted: 03/22/2024] [Indexed: 04/05/2024]
Abstract
The integration of Clinical Decision Support Systems (CDSS) based on artificial intelligence (AI) in healthcare is groundbreaking evolution with enormous potential, but its development and ethical implementation, presents unique challenges, particularly in critical care, where physicians often deal with life-threating conditions requiring rapid actions and patients unable to participate in the decisional process. Moreover, development of AI-based CDSS is complex and should address different sources of bias, including data acquisition, health disparities, domain shifts during clinical use, and cognitive biases in decision-making. In this scenario algor-ethics is mandatory and emphasizes the integration of 'Human-in-the-Loop' and 'Algorithmic Stewardship' principles, and the benefits of advanced data engineering. The establishment of Clinical AI Departments (CAID) is necessary to lead AI innovation in healthcare, ensuring ethical integrity and human-centered development in this rapidly evolving field.
Collapse
Affiliation(s)
- Jonathan Montomoli
- Department of Anesthesia and Intensive Care, Infermi Hospital, Romagna Local Health Authority, Viale Settembrini 2, Rimini, 47923, Italy.
- Health Services Research, Evaluation and Policy Unit, Romagna Local Health Authority, Viale Settembrini 2, Rimini, 47923, Italy.
| | - Maria Maddalena Bitondo
- Department of Anesthesia and Intensive Care, Infermi Hospital, Romagna Local Health Authority, Viale Settembrini 2, Rimini, 47923, Italy
| | - Marco Cascella
- Unit of Anesthesia and Pain Medicine, Department of Medicine, Surgery and Dentistry "Scuola Medica Salernitana, " University of Salerno, Baronissi, Salerno, Italy
| | - Emanuele Rezoagli
- School of Medicine and Surgery, University of Milano-Bicocca, Via Cadore, 48, Monza, 20900, Italy
- Dipartimento di Emergenza e Urgenza, Terapia intensiva e Semintensiva adulti e pediatrica, Fondazione IRCCS San Gerardo dei Tintori, Via Pergolesi, 33, Monza, 20900, Italy
| | - Luca Romeo
- Department of Economics and Law, University of Macerata, Macerata, 62100, Italy
| | - Valentina Bellini
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Via Gramsci 14, Parma, 43125, Italy
| | - Federico Semeraro
- Department of Anesthesia, Intensive Care and Prehospital Emergency, Ospedale Maggiore Carlo Alberto Pizzardi, Largo Bartolo Nigrisoli, 2, Bologna, 40133, Italy
| | - Emiliano Gamberini
- Department of Anesthesia and Intensive Care, Infermi Hospital, Romagna Local Health Authority, Viale Settembrini 2, Rimini, 47923, Italy
| | - Emanuele Frontoni
- Department of Political Sciences, Communication and International Relations, University of Macerata, Macerata, 62100, Italy
| | - Vanni Agnoletti
- Department of Surgery and Trauma, Anesthesia and Intensive Care Unit, Maurizio Bufalini Hospital, Romagna Local Health Authority, Viale Giovanni Ghirotti, 286, Cesena, 47521, Italy
| | - Mattia Altini
- Hospital Care Sector, Emilia-Romagna Region, Via Aldo Moro, 21, Bologna, 40127, Italy
| | - Paolo Benanti
- Pontifical Gregorian University, Piazza della Pilotta 4, Roma, 00187, Italy
| | - Elena Giovanna Bignami
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Via Gramsci 14, Parma, 43125, Italy
| |
Collapse
|
5
|
Hong Y, Zhu B, Chen C, Qiu M, Liu L. Belief in a just world and fair behavior among clinical nurses: a moderated mediation model of empathy and observer justice sensitivity. BMC Nurs 2024; 23:475. [PMID: 39010062 PMCID: PMC11247869 DOI: 10.1186/s12912-024-02140-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 07/01/2024] [Indexed: 07/17/2024] Open
Abstract
BACKGROUND Exploration of the relationship between nursing staffs' justice in belief world and fair behavior is important to promote equity and access to health services in health organizations, as well as to enhance the quality of care. In order to further dissect the influencing factors of fair behavior among clinical nurses, the current study aims to investigate how belief in a just world influences the fair behavior among nurses. Based on the belief in a just world theory, the empathy-altruism theory and the protective-protective model, the current study aimed to provide a deeper understanding of the effect of belief in a just world on fair behavior by investigating the mediating role of empathy and the moderating role of observer justice sensitivity. METHOD This was a cross-sectional study. 571 registered clinical nurses were included from five hospitals in Fuzhou through a convenience sampling method. Measurements included Chinese translations of belief in a just world scale, empathy scale, observer justice sensitivity scale, fair behavior scale. SPSS 22.0 was used to describe descriptive statistics and the variables' Pearson correlation coefficient. SPSS PROCESS macro Model 4 and model 14 were used to examine the mediation and the moderation between the relationship of belief in a just world and fairness behavior. RESULT The results shower that fairness behavior was positively correlated with one's belief in a just world (r = 0.26, p < 0.01); (2)empathy mediated the relationship between belief in a just world and fair behavior. The mediation model explains 20.83%; (3) Observer justice sensitivity moderated the relationship between empathy and fair behavior. CONCLUSIONS Belief in a just world, empathy, and observer justice sensitivity were motivations for nurses' fair behavior. Nursing administrators should focus on cultivating nurses' belief in a just world, their empathy abilities, and positive qualities of justice sensitivity to enhance fair behavior in a healthcare setting.
Collapse
Affiliation(s)
- Youjuan Hong
- School of Nursing, Fujian Medical University, Fuzhou, China.
| | - Bo Zhu
- School of Marxism, Fujian Medical University, Fuzhou, China
| | - Caimei Chen
- Critical Care Department, Longyan People's Hospital, Longyan, Fujian, China
| | - Meichai Qiu
- Center for information Management, Fujian Provincial Hospital, Fuzhou, Fujian, China
| | - Liting Liu
- School of Management Studies, Shanghai University of Engineering Science, Shanghai, China
| |
Collapse
|
6
|
Maurya R, Chug I, Vudatha V, Palma AM. Applications of spatial transcriptomics and artificial intelligence to develop integrated management of pancreatic cancer. Adv Cancer Res 2024; 163:107-136. [PMID: 39271261 DOI: 10.1016/bs.acr.2024.06.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/15/2024]
Abstract
Cancer is a complex disease intrinsically associated with cellular processes and gene expression. With the development of techniques such as single-cell sequencing and sequential fluorescence in situ hybridization (seqFISH), it was possible to map the location of cells based on their gene expression with more precision. Moreover, in recent years, many tools have been developed to analyze these extensive datasets by integrating machine learning and artificial intelligence in a comprehensive manner. Since these tools analyze sequencing data, they offer the chance to analyze any tissue regardless of its origin. By applying this to cancer settings, spatial transcriptomic analysis based on artificial intelligence may help us understand cell-cell communications within the tumor microenvironment. Another advantage of this analysis is the identification of new biomarkers and therapeutic targets. The integration of such analysis with other omics data and with routine exams such as magnetic resonance imaging can help physicians with the earlier diagnosis of tumors as well as establish a more personalized treatment for pancreatic cancer patients. In this review, we give an overview description of pancreatic cancer, describe how spatial transcriptomics and artificial intelligence have been used to study pancreatic cancer and provide examples of how integrating these tools may help physicians manage pancreatic cancer in a more personalized approach.
Collapse
Affiliation(s)
- Rishabh Maurya
- Department of Surgery, Virginia Commonwealth University, School of Medicine, Richmond, VA, United States; Massey Comprehensive Cancer Center, Virginia Commonwealth University, Richmond, VA, United States
| | - Isha Chug
- Massey Comprehensive Cancer Center, Virginia Commonwealth University, Richmond, VA, United States
| | - Vignesh Vudatha
- Department of Surgery, Virginia Commonwealth University, School of Medicine, Richmond, VA, United States; Massey Comprehensive Cancer Center, Virginia Commonwealth University, Richmond, VA, United States
| | - António M Palma
- Massey Comprehensive Cancer Center, Virginia Commonwealth University, Richmond, VA, United States; VCU Institute of Molecular Medicine, Department of Human and Molecular Genetics, Virginia Commonwealth University, School of Medicine, Richmond, VA, United States; Department of Human and Molecular Genetics, Virginia Commonwealth University, School of Medicine, Richmond, VA, United States.
| |
Collapse
|
7
|
Murphy A, Bowen K, Naqa IME, Yoga B, Green BL. Bridging Health Disparities in the Data-Driven World of Artificial Intelligence: A Narrative Review. J Racial Ethn Health Disparities 2024:10.1007/s40615-024-02057-2. [PMID: 38955956 DOI: 10.1007/s40615-024-02057-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 10/27/2023] [Accepted: 06/17/2024] [Indexed: 07/04/2024]
Abstract
BACKGROUND Artificial intelligence (AI) holds exciting potential to revolutionize healthcare delivery in the United States. However, there are concerns about its potential to perpetuate disparities among historically marginalized populations. OBJECTIVE Following the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses, we conducted a narrative review of current literature on AI and health disparities in the United States. We aimed to answer the question, Does AI have the potential to reduce or eliminate health disparities, or will its use further exacerbate these disparities? METHODS We searched the Ovid MEDLINE electronic database to identify and retrieve publications discussing AI and its impact on racial/ethnic health disparities. Articles were included if they discussed AI as a tool to mitigate racial health disparities with or without bias in developing and using AI. RESULTS This review included 65 articles. We identified six themes of limitations in AI that impact health equity: (1) biases in AI can perpetuate and exacerbate racial and ethnic inequities; (2) equity in algorithms should be a priority; (3) lack of diversity in the field of AI is concerning; (4) the need for regulation and testing algorithms for accuracy; (5) ethical standards for AI in health care are needed; and (6) the importance of promoting transparency and accountability. CONCLUSIONS While AI promises to enhance healthcare outcomes and address equity concerns, risks and challenges are associated with its implementation. To maximize the use of AI, it must be approached with an equity lens during all phases of development.
Collapse
Affiliation(s)
- Anastasia Murphy
- Department of Health Outcomes and Behavior, Moffitt Cancer Center, 12902 Magnolia Drive, Tampa, FL, 33612, USA.
| | - Kuan Bowen
- Department of Health Outcomes and Behavior, Moffitt Cancer Center, 12902 Magnolia Drive, Tampa, FL, 33612, USA
| | - Isaam M El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, USA
| | | | - B Lee Green
- Department of Health Outcomes and Behavior, Moffitt Cancer Center, 12902 Magnolia Drive, Tampa, FL, 33612, USA
| |
Collapse
|
8
|
Papini S, Hsin H, Kipnis P, Liu VX, Lu Y, Girard K, Sterling SA, Iturralde EM. Validation of a Multivariable Model to Predict Suicide Attempt in a Mental Health Intake Sample. JAMA Psychiatry 2024; 81:700-707. [PMID: 38536187 PMCID: PMC10974695 DOI: 10.1001/jamapsychiatry.2024.0189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 01/16/2024] [Indexed: 07/04/2024]
Abstract
Importance Given that suicide rates have been increasing over the past decade and the demand for mental health care is at an all-time high, targeted prevention efforts are needed to identify individuals seeking to initiate mental health outpatient services who are at high risk for suicide. Suicide prediction models have been developed using outpatient mental health encounters, but their performance among intake appointments has not been directly examined. Objective To assess the performance of a predictive model of suicide attempts among individuals seeking to initiate an episode of outpatient mental health care. Design, Setting, and Participants This prognostic study tested the performance of a previously developed machine learning model designed to predict suicide attempts within 90 days of any mental health outpatient visit. All mental health intake appointments scheduled between January 1, 2012, and April 1, 2022, at Kaiser Permanente Northern California, a large integrated health care delivery system serving over 4.5 million patients, were included. Data were extracted and analyzed from August 9, 2022, to July 31, 2023. Main Outcome and Measures Suicide attempts (including completed suicides) within 90 days of the appointment, determined by diagnostic codes and government databases. All predictors were extracted from electronic health records. Results The study included 1 623 232 scheduled appointments from 835 616 unique patients. There were 2800 scheduled appointments (0.17%) followed by a suicide attempt within 90 days. The mean (SD) age across appointments was 39.7 (15.8) years, and most appointments were for women (1 103 184 [68.0%]). The model had an area under the receiver operating characteristic curve of 0.77 (95% CI, 0.76-0.78), an area under the precision-recall curve of 0.02 (95% CI, 0.02-0.02), an expected calibration error of 0.0012 (95% CI, 0.0011-0.0013), and sensitivities of 37.2% (95% CI, 35.5%-38.9%) and 18.8% (95% CI, 17.3%-20.2%) at specificities of 95% and 99%, respectively. The 10% of appointments at the highest risk level accounted for 48.8% (95% CI, 47.0%-50.6%) of the appointments followed by a suicide attempt. Conclusions and Relevance In this prognostic study involving mental health intakes, a previously developed machine learning model of suicide attempts showed good overall classification performance. Implementation research is needed to determine appropriate thresholds and interventions for applying the model in an intake setting to target high-risk cases in a manner that is acceptable to patients and clinicians.
Collapse
Affiliation(s)
- Santiago Papini
- Division of Research, Kaiser Permanente Division of Research, Oakland, California
- Department of Psychology, University of Hawaiʻi at Mānoa, Honolulu
| | - Honor Hsin
- The Permanente Medical Group, Kaiser Permanente, San Jose, California
| | - Patricia Kipnis
- Division of Research, Kaiser Permanente Division of Research, Oakland, California
| | - Vincent X. Liu
- Division of Research, Kaiser Permanente Division of Research, Oakland, California
| | - Yun Lu
- Division of Research, Kaiser Permanente Division of Research, Oakland, California
| | - Kristine Girard
- The Permanente Medical Group, Kaiser Permanente, San Jose, California
| | - Stacy A. Sterling
- Division of Research, Kaiser Permanente Division of Research, Oakland, California
| | - Esti M. Iturralde
- Division of Research, Kaiser Permanente Division of Research, Oakland, California
| |
Collapse
|
9
|
Yang Y, Zhang H, Gichoya JW, Katabi D, Ghassemi M. The limits of fair medical imaging AI in real-world generalization. Nat Med 2024:10.1038/s41591-024-03113-4. [PMID: 38942996 DOI: 10.1038/s41591-024-03113-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 06/05/2024] [Indexed: 06/30/2024]
Abstract
As artificial intelligence (AI) rapidly approaches human-level performance in medical imaging, it is crucial that it does not exacerbate or propagate healthcare disparities. Previous research established AI's capacity to infer demographic data from chest X-rays, leading to a key concern: do models using demographic shortcuts have unfair predictions across subpopulations? In this study, we conducted a thorough investigation into the extent to which medical AI uses demographic encodings, focusing on potential fairness discrepancies within both in-distribution training sets and external test sets. Our analysis covers three key medical imaging disciplines-radiology, dermatology and ophthalmology-and incorporates data from six global chest X-ray datasets. We confirm that medical imaging AI leverages demographic shortcuts in disease classification. Although correcting shortcuts algorithmically effectively addresses fairness gaps to create 'locally optimal' models within the original data distribution, this optimality is not true in new test settings. Surprisingly, we found that models with less encoding of demographic attributes are often most 'globally optimal', exhibiting better fairness during model evaluation in new test environments. Our work establishes best practices for medical imaging models that maintain their performance and fairness in deployments beyond their initial training contexts, underscoring critical considerations for AI clinical deployments across populations and sites.
Collapse
Affiliation(s)
- Yuzhe Yang
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
| | - Haoran Zhang
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Judy W Gichoya
- Department of Radiology, Emory University School of Medicine, Atlanta, GA, USA
| | - Dina Katabi
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Marzyeh Ghassemi
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
10
|
Ueda D, Walston SL, Fujita S, Fushimi Y, Tsuboyama T, Kamagata K, Yamada A, Yanagawa M, Ito R, Fujima N, Kawamura M, Nakaura T, Matsui Y, Tatsugami F, Fujioka T, Nozaki T, Hirata K, Naganawa S. Climate change and artificial intelligence in healthcare: Review and recommendations towards a sustainable future. Diagn Interv Imaging 2024:S2211-5684(24)00138-4. [PMID: 38918123 DOI: 10.1016/j.diii.2024.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2024] [Revised: 06/03/2024] [Accepted: 06/03/2024] [Indexed: 06/27/2024]
Abstract
The rapid advancement of artificial intelligence (AI) in healthcare has revolutionized the industry, offering significant improvements in diagnostic accuracy, efficiency, and patient outcomes. However, the increasing adoption of AI systems also raises concerns about their environmental impact, particularly in the context of climate change. This review explores the intersection of climate change and AI in healthcare, examining the challenges posed by the energy consumption and carbon footprint of AI systems, as well as the potential solutions to mitigate their environmental impact. The review highlights the energy-intensive nature of AI model training and deployment, the contribution of data centers to greenhouse gas emissions, and the generation of electronic waste. To address these challenges, the development of energy-efficient AI models, the adoption of green computing practices, and the integration of renewable energy sources are discussed as potential solutions. The review also emphasizes the role of AI in optimizing healthcare workflows, reducing resource waste, and facilitating sustainable practices such as telemedicine. Furthermore, the importance of policy and governance frameworks, global initiatives, and collaborative efforts in promoting sustainable AI practices in healthcare is explored. The review concludes by outlining best practices for sustainable AI deployment, including eco-design, lifecycle assessment, responsible data management, and continuous monitoring and improvement. As the healthcare industry continues to embrace AI technologies, prioritizing sustainability and environmental responsibility is crucial to ensure that the benefits of AI are realized while actively contributing to the preservation of our planet.
Collapse
Affiliation(s)
- Daiju Ueda
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Abeno-ku, Osaka 545-8585, Japan; Department of Artificial Intelligence, Graduate School of Medicine, Osaka Metropolitan University, Abeno-ku, Osaka 545-8585, Japan.
| | - Shannon L Walston
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Abeno-ku, Osaka 545-8585, Japan
| | - Shohei Fujita
- Department of Radiology, University of Tokyo, Bunkyo-ku, Tokyo 113-8655, Japan
| | - Yasutaka Fushimi
- Department of Diagnostic Imaging and Nuclear Medicine, Kyoto University Graduate School of Medicine, Sakyoku, Kyoto 606-8507, Japan
| | - Takahiro Tsuboyama
- Department of Radiology, Kobe University Graduate School of Medicine, Kobe, Hyogo 650-0017, Japan
| | - Koji Kamagata
- Department of Radiology, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo 113-8421, Japan
| | - Akira Yamada
- Medical Data Science Course, Shinshu University School of Medicine, Matsumoto, Nagano 390-8621, Japan
| | - Masahiro Yanagawa
- Department of Radiology, Graduate School of Medicine, Osaka University, Suita-city, Osaka 565-0871, Japan
| | - Rintaro Ito
- Department of Radiology, Nagoya University Graduate School of Medicine, Nagoya, Aichi 466-8550, Japan
| | - Noriyuki Fujima
- Department of Diagnostic and Interventional Radiology, Hokkaido University Hospital, Sapporo, Hokkaido 060-8648, Japan
| | - Mariko Kawamura
- Department of Radiology, Nagoya University Graduate School of Medicine, Nagoya, Aichi 466-8550, Japan
| | - Takeshi Nakaura
- Department of Diagnostic Radiology, Kumamoto University Graduate School of Medicine, Chuo-ku, Kumamoto 860-8556, Japan
| | - Yusuke Matsui
- Department of Radiology, Faculty of Medicine, Dentistry and Pharmaceutical Sciences, Okayama University, Kita-ku, Okayama 700-8558, Japan
| | - Fuminari Tatsugami
- Department of Diagnostic Radiology, Hiroshima University, Minami-ku, Hiroshima City, Hiroshima 734-8551, Japan
| | - Tomoyuki Fujioka
- Department of Diagnostic Radiology, Tokyo Medical and Dental University, Bunkyo-ku, Tokyo 113-8510, Japan
| | - Taiki Nozaki
- Department of Radiology, Keio University School of Medicine, Shinjuku-ku, Tokyo 160-8582, Japan
| | - Kenji Hirata
- Department of Diagnostic Imaging, Faculty of Medicine, Hokkaido University, Sapporo, Hokkaido 060-8638, Japan
| | - Shinji Naganawa
- Department of Radiology, Nagoya University Graduate School of Medicine, Nagoya, Aichi 466-8550, Japan
| |
Collapse
|
11
|
Slattery SM, Wilkinson J, Mittal A, Zheng C, Easton N, Singh S, Baker JJ, Rand CM, Khaytin I, Stewart TM, Demeter D, Weese-Mayer DE. Computer-aided diagnostic screen for Congenital Central Hypoventilation Syndrome with facial phenotype. Pediatr Res 2024; 95:1843-1850. [PMID: 38238566 DOI: 10.1038/s41390-023-02990-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 10/30/2023] [Accepted: 12/13/2023] [Indexed: 07/14/2024]
Abstract
BACKGROUND Congenital Central Hypoventilation Syndrome (CCHS) has devastating consequences if not diagnosed promptly. Despite identification of the disease-defining gene PHOX2B and a facial phenotype, CCHS remains underdiagnosed. This study aimed to incorporate automated techniques on facial photos to screen for CCHS in a diverse pediatric cohort to improve early case identification and assess a facial phenotype-PHOX2B genotype relationship. METHODS Facial photos of children and young adults with CCHS were control-matched by age, sex, race/ethnicity. After validating landmarks, principal component analysis (PCA) was applied with logistic regression (LR) for feature attribution and machine learning models for subject classification and assessment by PHOX2B pathovariant. RESULTS Gradient-based feature attribution confirmed a subtle facial phenotype and models were successful in classifying CCHS: neural network performed best (median sensitivity 90% (IQR 84%, 95%)) on 179 clinical photos (versus LR and XGBoost, both 85% (IQR 75-76%, 90%)). Outcomes were comparable stratified by PHOX2B genotype and with the addition of publicly available CCHS photos (n = 104) using PCA and LR (sensitivity 83-89% (IQR 67-76%, 92-100%). CONCLUSIONS Utilizing facial features, findings suggest an automated, accessible classifier may be used to screen for CCHS in children with the phenotype and support providers to seek PHOX2B testing to improve the diagnostics. IMPACT Facial landmarking and principal component analysis on a diverse pediatric and young adult cohort with PHOX2B pathovariants delineated a distinct, subtle CCHS facial phenotype. Automated, low-cost machine learning models can detect a CCHS facial phenotype with a high sensitivity in screening to ultimately refer for disease-defining PHOX2B testing, potentially addressing gaps in disease underdiagnosis and allow for critical, timely intervention.
Collapse
Affiliation(s)
- Susan M Slattery
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA.
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
| | - James Wilkinson
- Department of Computer Science, Northwestern University McCormick School of Engineering, Evanston, IL, USA
| | - Angeli Mittal
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
- Department of Computer Science, Northwestern University McCormick School of Engineering, Evanston, IL, USA
| | - Charlie Zheng
- Department of Computer Science, Northwestern University McCormick School of Engineering, Evanston, IL, USA
| | - Nicholas Easton
- Department of Computer Science, Northwestern University McCormick School of Engineering, Evanston, IL, USA
| | - Saumya Singh
- Department of Computer Science, Northwestern University McCormick School of Engineering, Evanston, IL, USA
| | - Joshua J Baker
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Genetics, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Casey M Rand
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
- Stanley Manne Children's Research Institute, Chicago, IL, USA
| | - Ilya Khaytin
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Tracey M Stewart
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - David Demeter
- Department of Computer Science, Northwestern University McCormick School of Engineering, Evanston, IL, USA
| | - Debra E Weese-Mayer
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Stanley Manne Children's Research Institute, Chicago, IL, USA
| |
Collapse
|
12
|
Farah L, Borget I, Martelli N, Vallee A. Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review. J Med Internet Res 2024; 26:e51514. [PMID: 38739911 PMCID: PMC11130781 DOI: 10.2196/51514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 12/17/2023] [Accepted: 12/28/2023] [Indexed: 05/16/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI)-based medical devices have garnered attention due to their ability to revolutionize medicine. Their health technology assessment framework is lacking. OBJECTIVE This study aims to analyze the suitability of each health technology assessment (HTA) domain for the assessment of AI-based medical devices. METHODS We conducted a scoping literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We searched databases (PubMed, Embase, and Cochrane Library), gray literature, and HTA agency websites. RESULTS A total of 10.1% (78/775) of the references were included. Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process. When it comes to implementing specialized HTA for AI-based medical devices, several practical challenges and potential barriers could be highlighted and should be taken into account (AI technological evolution timeline, data requirements, complexity and transparency, clinical validation and safety requirements, regulatory and ethical considerations, and economic evaluation). CONCLUSIONS The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into the effectiveness, cost-effectiveness, and societal impact of AI-based medical devices, guiding their responsible implementation and maximizing their benefits for patients and health care systems.
Collapse
Affiliation(s)
- Line Farah
- Innovation Center for Medical Devices Department, Foch Hospital, Suresnes, France
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé Department, University Paris-Saclay, Orsay, France
| | - Isabelle Borget
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé Department, University Paris-Saclay, Orsay, France
- Department of Biostatistics and Epidemiology, Gustave Roussy, University Paris-Saclay, Villejuif, France
- Oncostat U1018, Inserm, Équipe Labellisée Ligue Contre le Cancer, University Paris-Saclay, Villejuif, France
| | - Nicolas Martelli
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé Department, University Paris-Saclay, Orsay, France
- Pharmacy Department, Georges Pompidou European Hospital, Paris, France
| | - Alexandre Vallee
- Department of Epidemiology and Public Health, Foch Hospital, Suresnes, France
| |
Collapse
|
13
|
Colegate SP, Palipana A, Gecili E, Szczesniak RD, Brokamp C. Evaluating precision medicine tools in cystic fibrosis for racial and ethnic fairness. J Clin Transl Sci 2024; 8:e94. [PMID: 39220818 PMCID: PMC11362628 DOI: 10.1017/cts.2024.532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 03/28/2024] [Accepted: 04/23/2024] [Indexed: 09/04/2024] Open
Abstract
Introduction Patients with cystic fibrosis (CF) experience frequent episodes of acute decline in lung function called pulmonary exacerbations (PEx). An existing clinical and place-based precision medicine algorithm that accurately predicts PEx could include racial and ethnic biases in clinical and geospatial training data, leading to unintentional exacerbation of health inequities. Methods We estimated receiver operating characteristic curves based on predictions from a nonstationary Gaussian stochastic process model for PEx within 3, 6, and 12 months among 26,392 individuals aged 6 years and above (2003-2017) from the US CF Foundation Patient Registry. We screened predictors to identify reasons for discriminatory model performance. Results The precision medicine algorithm performed worse predicting a PEx among Black patients when compared with White patients or to patients of another race for all three prediction horizons. There was little to no difference in prediction accuracies among Hispanic and non-Hispanic patients for the same prediction horizons. Differences in F508del, smoking households, secondhand smoke exposure, primary and secondary road densities, distance and drive time to the CF center, and average number of clinical evaluations were key factors associated with race. Conclusions Racial differences in prediction accuracies from our PEx precision medicine algorithm exist. Misclassification of future PEx was attributable to several underlying factors that correspond to race: CF mutation, location where the patient lives, and clinical awareness. Associations of our proxies with race for CF-related health outcomes can lead to systemic racism in data collection and in prediction accuracies from precision medicine algorithms constructed from it.
Collapse
Affiliation(s)
- Stephen P. Colegate
- Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
| | | | - Emrah Gecili
- Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
- Department of Pediatrics, University of Cincinnati, Cincinnati, OH, USA
| | - Rhonda D. Szczesniak
- Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
- Department of Pediatrics, University of Cincinnati, Cincinnati, OH, USA
| | - Cole Brokamp
- Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
- Department of Pediatrics, University of Cincinnati, Cincinnati, OH, USA
| |
Collapse
|
14
|
Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, Ghassemi M, Liu X, Reitsma JB, van Smeden M, Boulesteix AL, Camaradou JC, Celi LA, Denaxas S, Denniston AK, Glocker B, Golub RM, Harvey H, Heinze G, Hoffman MM, Kengne AP, Lam E, Lee N, Loder EW, Maier-Hein L, Mateen BA, McCradden MD, Oakden-Rayner L, Ordish J, Parnell R, Rose S, Singh K, Wynants L, Logullo P. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024; 385:e078378. [PMID: 38626948 PMCID: PMC11019967 DOI: 10.1136/bmj-2023-078378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/17/2024] [Indexed: 04/19/2024]
Affiliation(s)
- Gary S Collins
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Karel G M Moons
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Paula Dhiman
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Andrew L Beam
- Department of Epidemiology, Harvard T H Chan School of Public Health, Boston, MA, USA
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Science, Leiden University Medical Centre, Leiden, Netherlands
| | - Marzyeh Ghassemi
- Department of Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Xiaoxuan Liu
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Johannes B Reitsma
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten van Smeden
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Anne-Laure Boulesteix
- Institute for Medical Information Processing, Biometry and Epidemiology, Faculty of Medicine, Ludwig-Maximilians-University of Munich and Munich Centre of Machine Learning, Germany
| | - Jennifer Catherine Camaradou
- Patient representative, Health Data Research UK patient and public involvement and engagement group
- Patient representative, University of East Anglia, Faculty of Health Sciences, Norwich Research Park, Norwich, UK
| | - Leo Anthony Celi
- Beth Israel Deaconess Medical Center, Boston, MA, USA
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA
| | - Spiros Denaxas
- Institute of Health Informatics, University College London, London, UK
- British Heart Foundation Data Science Centre, London, UK
| | - Alastair K Denniston
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Ben Glocker
- Department of Computing, Imperial College London, London, UK
| | - Robert M Golub
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Georg Heinze
- Section for Clinical Biometrics, Centre for Medical Data Science, Medical University of Vienna, Vienna, Austria
| | - Michael M Hoffman
- Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| | | | - Emily Lam
- Patient representative, Health Data Research UK patient and public involvement and engagement group
| | - Naomi Lee
- National Institute for Health and Care Excellence, London, UK
| | - Elizabeth W Loder
- The BMJ, London, UK
- Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Lena Maier-Hein
- Department of Intelligent Medical Systems, German Cancer Research Centre, Heidelberg, Germany
| | - Bilal A Mateen
- Institute of Health Informatics, University College London, London, UK
- Wellcome Trust, London, UK
- Alan Turing Institute, London, UK
| | - Melissa D McCradden
- Department of Bioethics, Hospital for Sick Children Toronto, ON, Canada
- Genetics and Genome Biology, SickKids Research Institute, Toronto, ON, Canada
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Johan Ordish
- Medicines and Healthcare products Regulatory Agency, London, UK
| | - Richard Parnell
- Patient representative, Health Data Research UK patient and public involvement and engagement group
| | - Sherri Rose
- Department of Health Policy and Center for Health Policy, Stanford University, Stanford, CA, USA
| | - Karandeep Singh
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
| | - Laure Wynants
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
| | - Patricia Logullo
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| |
Collapse
|
15
|
Verhoeven R, Hulscher JBF. Editorial: Artificial intelligence and machine learning in pediatric surgery. Front Pediatr 2024; 12:1404600. [PMID: 38659697 PMCID: PMC11042026 DOI: 10.3389/fped.2024.1404600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Accepted: 04/01/2024] [Indexed: 04/26/2024] Open
Affiliation(s)
- Rosa Verhoeven
- Department of Surgery, Division of Pediatric Surgery, University Medical Center Groningen, University of Groningen, Groningen, Netherlands
- Department of Neonatology, Beatrix Children’s Hospital, University Medical Center Groningen, University of Groningen, Groningen, Netherlands
| | - Jan B. F. Hulscher
- Department of Surgery, Division of Pediatric Surgery, University Medical Center Groningen, University of Groningen, Groningen, Netherlands
| |
Collapse
|
16
|
Vaidya A, Chen RJ, Williamson DFK, Song AH, Jaume G, Yang Y, Hartvigsen T, Dyer EC, Lu MY, Lipkova J, Shaban M, Chen TY, Mahmood F. Demographic bias in misdiagnosis by computational pathology models. Nat Med 2024; 30:1174-1190. [PMID: 38641744 DOI: 10.1038/s41591-024-02885-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2023] [Accepted: 02/23/2024] [Indexed: 04/21/2024]
Abstract
Despite increasing numbers of regulatory approvals, deep learning-based computational pathology systems often overlook the impact of demographic factors on performance, potentially leading to biases. This concern is all the more important as computational pathology has leveraged large public datasets that underrepresent certain demographic groups. Using publicly available data from The Cancer Genome Atlas and the EBRAINS brain tumor atlas, as well as internal patient data, we show that whole-slide image classification models display marked performance disparities across different demographic groups when used to subtype breast and lung carcinomas and to predict IDH1 mutations in gliomas. For example, when using common modeling approaches, we observed performance gaps (in area under the receiver operating characteristic curve) between white and Black patients of 3.0% for breast cancer subtyping, 10.9% for lung cancer subtyping and 16.0% for IDH1 mutation prediction in gliomas. We found that richer feature representations obtained from self-supervised vision foundation models reduce performance variations between groups. These representations provide improvements upon weaker models even when those weaker models are combined with state-of-the-art bias mitigation strategies and modeling choices. Nevertheless, self-supervised vision foundation models do not fully eliminate these discrepancies, highlighting the continuing need for bias mitigation efforts in computational pathology. Finally, we demonstrate that our results extend to other demographic factors beyond patient race. Given these findings, we encourage regulatory and policy agencies to integrate demographic-stratified evaluation into their assessment guidelines.
Collapse
Affiliation(s)
- Anurag Vaidya
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Richard J Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA, USA
| | - Andrew H Song
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Yuzhe Yang
- Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA
| | - Thomas Hartvigsen
- School of Data Science, University of Virginia, Charlottesville, VA, USA
| | - Emma C Dyer
- T.H. Chan School of Public Health, Harvard University, Cambridge, MA, USA
| | - Ming Y Lu
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA
| | - Jana Lipkova
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Muhammad Shaban
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Tiffany Y Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA.
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
- Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
17
|
Barwise AK, Curtis S, Diedrich DA, Pickering BW. Using artificial intelligence to promote equitable care for inpatients with language barriers and complex medical needs: clinical stakeholder perspectives. J Am Med Inform Assoc 2024; 31:611-621. [PMID: 38099504 PMCID: PMC10873784 DOI: 10.1093/jamia/ocad224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 11/14/2023] [Indexed: 02/18/2024] Open
Abstract
OBJECTIVES Inpatients with language barriers and complex medical needs suffer disparities in quality of care, safety, and health outcomes. Although in-person interpreters are particularly beneficial for these patients, they are underused. We plan to use machine learning predictive analytics to reliably identify patients with language barriers and complex medical needs to prioritize them for in-person interpreters. MATERIALS AND METHODS This qualitative study used stakeholder engagement through semi-structured interviews to understand the perceived risks and benefits of artificial intelligence (AI) in this domain. Stakeholders included clinicians, interpreters, and personnel involved in caring for these patients or for organizing interpreters. Data were coded and analyzed using NVIVO software. RESULTS We completed 49 interviews. Key perceived risks included concerns about transparency, accuracy, redundancy, privacy, perceived stigmatization among patients, alert fatigue, and supply-demand issues. Key perceived benefits included increased awareness of in-person interpreters, improved standard of care and prioritization for interpreter utilization; a streamlined process for accessing interpreters, empowered clinicians, and potential to overcome clinician bias. DISCUSSION This is the first study that elicits stakeholder perspectives on the use of AI with the goal of improved clinical care for patients with language barriers. Perceived benefits and risks related to the use of AI in this domain, overlapped with known hazards and values of AI but some benefits were unique for addressing challenges with providing interpreter services to patients with language barriers. CONCLUSION Artificial intelligence to identify and prioritize patients for interpreter services has the potential to improve standard of care and address healthcare disparities among patients with language barriers.
Collapse
Affiliation(s)
- Amelia K Barwise
- Biomedical Ethics Research Program, Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN 55902, United States
| | - Susan Curtis
- Biomedical Ethics Research Program, Mayo Clinic, Rochester, MN 55902, United States
| | - Daniel A Diedrich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN 55902, United States
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN 55902, United States
| |
Collapse
|
18
|
Hanneman K, Playford D, Dey D, van Assen M, Mastrodicasa D, Cook TS, Gichoya JW, Williamson EE, Rubin GD. Value Creation Through Artificial Intelligence and Cardiovascular Imaging: A Scientific Statement From the American Heart Association. Circulation 2024; 149:e296-e311. [PMID: 38193315 DOI: 10.1161/cir.0000000000001202] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2024]
Abstract
Multiple applications for machine learning and artificial intelligence (AI) in cardiovascular imaging are being proposed and developed. However, the processes involved in implementing AI in cardiovascular imaging are highly diverse, varying by imaging modality, patient subtype, features to be extracted and analyzed, and clinical application. This article establishes a framework that defines value from an organizational perspective, followed by value chain analysis to identify the activities in which AI might produce the greatest incremental value creation. The various perspectives that should be considered are highlighted, including clinicians, imagers, hospitals, patients, and payers. Integrating the perspectives of all health care stakeholders is critical for creating value and ensuring the successful deployment of AI tools in a real-world setting. Different AI tools are summarized, along with the unique aspects of AI applications to various cardiac imaging modalities, including cardiac computed tomography, magnetic resonance imaging, and positron emission tomography. AI is applicable and has the potential to add value to cardiovascular imaging at every step along the patient journey, from selecting the more appropriate test to optimizing image acquisition and analysis, interpreting the results for classification and diagnosis, and predicting the risk for major adverse cardiac events.
Collapse
|
19
|
Langs G. Artificial intelligence in medical imaging is a tool for clinical routine and scientific discovery. Semin Arthritis Rheum 2024; 64S:152321. [PMID: 38007360 DOI: 10.1016/j.semarthrit.2023.152321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 11/09/2023] [Indexed: 11/27/2023]
Abstract
The emergence of powerful machine learning methodology together with an increasing amount of data collected during clinical routine have fostered a growing role of artificial intelligence (AI) in medicine. Algorithms have become part of clinical care enhancing image reconstruction, detecting cancer or predicting individual risk to support treatment decisions and patient management. The entry into clinical care is determined by technological feasibility, integration into effective workflows, and immediacy of benefits. At the same time, research is advancing the integration of imaging data and other modalities such as genomics, and the linking of observations made at large scale with the understanding of underlying biological processes. AI will have impact in imaging and precision medicine not only because of the successful application of techniques established in other domains, but primarily because of the effective joint development of new technology and corresponding advance of diagnosis and care.
Collapse
Affiliation(s)
- Georg Langs
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Imaging, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Spitalgasse 23, Vienna 1090, Austria.
| |
Collapse
|
20
|
Chen K, Shao A, Burapacheep J, Li Y. Conversational AI and equity through assessing GPT-3's communication with diverse social groups on contentious topics. Sci Rep 2024; 14:1561. [PMID: 38238474 PMCID: PMC10796352 DOI: 10.1038/s41598-024-51969-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 01/11/2024] [Indexed: 01/22/2024] Open
Abstract
Autoregressive language models, which use deep learning to produce human-like texts, have surged in prevalence. Despite advances in these models, concerns arise about their equity across diverse populations. While AI fairness is discussed widely, metrics to measure equity in dialogue systems are lacking. This paper presents a framework, rooted in deliberative democracy and science communication studies, to evaluate equity in human-AI communication. Using it, we conducted an algorithm auditing study to examine how GPT-3 responded to different populations who vary in sociodemographic backgrounds and viewpoints on crucial science and social issues: climate change and the Black Lives Matter (BLM) movement. We analyzed 20,000 dialogues with 3290 participants differing in gender, race, education, and opinions. We found a substantively worse user experience among the opinion minority groups (e.g., climate deniers, racists) and the education minority groups; however, these groups changed attitudes toward supporting BLM and climate change efforts much more compared to other social groups after the chat. GPT-3 used more negative expressions when responding to the education and opinion minority groups. We discuss the social-technological implications of our findings for a conversational AI system that centralizes diversity, equity, and inclusion.
Collapse
Affiliation(s)
- Kaiping Chen
- Department of Life Sciences Communication, University of Wisconsin-Madison, Madison, USA.
| | - Anqi Shao
- Department of Life Sciences Communication, University of Wisconsin-Madison, Madison, USA
| | | | - Yixuan Li
- Department of Computer Sciences, University of Wisconsin-Madison, Madison, USA
| |
Collapse
|
21
|
Herington J, McCradden MD, Creel K, Boellaard R, Jones EC, Jha AK, Rahmim A, Scott PJH, Sunderland JJ, Wahl RL, Zuehlsdorff S, Saboury B. Ethical Considerations for Artificial Intelligence in Medical Imaging: Data Collection, Development, and Evaluation. J Nucl Med 2023; 64:1848-1854. [PMID: 37827839 PMCID: PMC10690124 DOI: 10.2967/jnumed.123.266080] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 09/12/2023] [Indexed: 10/14/2023] Open
Abstract
The development of artificial intelligence (AI) within nuclear imaging involves several ethically fraught components at different stages of the machine learning pipeline, including during data collection, model training and validation, and clinical use. Drawing on the traditional principles of medical and research ethics, and highlighting the need to ensure health justice, the AI task force of the Society of Nuclear Medicine and Molecular Imaging has identified 4 major ethical risks: privacy of data subjects, data quality and model efficacy, fairness toward marginalized populations, and transparency of clinical performance. We provide preliminary recommendations to developers of AI-driven medical devices for mitigating the impact of these risks on patients and populations.
Collapse
Affiliation(s)
- Jonathan Herington
- Department of Health Humanities and Bioethics and Department of Philosophy, University of Rochester, Rochester, New York
| | - Melissa D McCradden
- Department of Bioethics, Hospital for Sick Children, Toronto and Dana Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Kathleen Creel
- Department of Philosophy and Religion and Khoury College of Computer Sciences, Northeastern University, Boston, Massachusetts
| | - Ronald Boellaard
- Department of Radiology and Nuclear Medicine, Cancer Centre Amsterdam, Amsterdam University Medical Centres, Amsterdam, The Netherlands
| | - Elizabeth C Jones
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, Bethesda, Maryland
| | - Abhinav K Jha
- Department of Biomedical Engineering and Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri
| | - Arman Rahmim
- Departments of Radiology and Physics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Peter J H Scott
- Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan
| | - John J Sunderland
- Departments of Radiology and Physics, University of Iowa, Iowa City, Iowa
| | - Richard L Wahl
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri; and
| | | | - Babak Saboury
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, Bethesda, Maryland;
| |
Collapse
|
22
|
Arora A, Alderman JE, Palmer J, Ganapathi S, Laws E, McCradden MD, Oakden-Rayner L, Pfohl SR, Ghassemi M, McKay F, Treanor D, Rostamzadeh N, Mateen B, Gath J, Adebajo AO, Kuku S, Matin R, Heller K, Sapey E, Sebire NJ, Cole-Lewis H, Calvert M, Denniston A, Liu X. The value of standards for health datasets in artificial intelligence-based applications. Nat Med 2023; 29:2929-2938. [PMID: 37884627 PMCID: PMC10667100 DOI: 10.1038/s41591-023-02608-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Accepted: 09/22/2023] [Indexed: 10/28/2023]
Abstract
Artificial intelligence as a medical device is increasingly being applied to healthcare for diagnosis, risk stratification and resource allocation. However, a growing body of evidence has highlighted the risk of algorithmic bias, which may perpetuate existing health inequity. This problem arises in part because of systemic inequalities in dataset curation, unequal opportunity to participate in research and inequalities of access. This study aims to explore existing standards, frameworks and best practices for ensuring adequate data diversity in health datasets. Exploring the body of existing literature and expert views is an important step towards the development of consensus-based guidelines. The study comprises two parts: a systematic review of existing standards, frameworks and best practices for healthcare datasets; and a survey and thematic analysis of stakeholder views of bias, health equity and best practices for artificial intelligence as a medical device. We found that the need for dataset diversity was well described in literature, and experts generally favored the development of a robust set of guidelines, but there were mixed views about how these could be implemented practically. The outputs of this study will be used to inform the development of standards for transparency of data diversity in health datasets (the STANDING Together initiative).
Collapse
Affiliation(s)
- Anmol Arora
- School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Joseph E Alderman
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
| | - Joanne Palmer
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
| | | | - Elinor Laws
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
| | - Melissa D McCradden
- Department of Bioethics, The Hospital for Sick Children, Toronto, Ontario, Canada
- Genetics and Genome Biology, Peter Gilgan Centre for Research and Learning, Toronto, Ontario, Canada
- Dalla Lana School of Public Health, Toronto, Ontario, Canada
| | - Lauren Oakden-Rayner
- The Australian Institute for Machine Learning, University of Adelaide, Adelaide, South Australia, Australia
| | | | - Marzyeh Ghassemi
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Vector Institute, Toronto, Ontario, Canada
| | - Francis McKay
- The Ethox Centre and the Wellcome Centre for Ethics and Humanities, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Darren Treanor
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
- University of Leeds, Leeds, UK
- Department of Clinical Pathology and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden
- Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | | | - Bilal Mateen
- Institute for Health Informatics, University College London, London, UK
- Wellcome Trust, London, UK
| | - Jacqui Gath
- Patient and Public Involvement and Engagement (PPIE) Group, STANDING Together, Birmingham, UK
| | - Adewole O Adebajo
- Patient and Public Involvement and Engagement (PPIE) Group, STANDING Together, Birmingham, UK
| | | | - Rubeta Matin
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | | | - Elizabeth Sapey
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- PIONEER, HDR UK Hub in Acute Care, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
| | - Neil J Sebire
- National Institute for Health and Care Research, Great Ormond Street Hospital Biomedical Research Centre, London, UK
- Great Ormond Street Institute of Child Health, University Hospital London, London, UK
| | | | - Melanie Calvert
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research Applied Research Collaboration West Midlands, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research Birmingham-Oxford Blood and Transplant Research Unit in Precision Transplant and Cellular Therapeutics, University of Birmingham, Birmingham, UK
- DEMAND Hub, University of Birmingham, Birmingham, UK
- UK SPINE, University of Birmingham, Birmingham, UK
| | - Alastair Denniston
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research Biomedical Research Centre, Moorfields Eye Hospital/University College London, London, UK
| | - Xiaoxuan Liu
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK.
| |
Collapse
|
23
|
Omar A, Abd El-Hafeez T. Quantum computing and machine learning for Arabic language sentiment classification in social media. Sci Rep 2023; 13:17305. [PMID: 37828056 PMCID: PMC10570340 DOI: 10.1038/s41598-023-44113-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 10/03/2023] [Indexed: 10/14/2023] Open
Abstract
With the increasing amount of digital data generated by Arabic speakers, the need for effective and efficient document classification techniques is more important than ever. In recent years, both quantum computing and machine learning have shown great promise in the field of document classification. However, there is a lack of research investigating the performance of these techniques on the Arabic language. This paper presents a comparative study of quantum computing and machine learning for two datasets of Arabic language document classification. In the first dataset of 213,465 Arabic tweets, both classic machine learning (ML) and quantum computing approaches achieve high accuracy in sentiment analysis, with quantum computing slightly outperforming classic ML. Quantum computing completes the task in approximately 59 min, slightly faster than classic ML, which takes around 1 h. The precision, recall, and F1 score metrics indicate the effectiveness of both approaches in predicting sentiment in Arabic tweets. Classic ML achieves precision, recall, and F1 score values of 0.8215, 0.8175, and 0.8121, respectively, while quantum computing achieves values of 0.8239, 0.8199, and 0.8147, respectively. In the second dataset of 44,000 tweets, both classic ML (using the Random Forest algorithm) and quantum computing demonstrate significantly reduced processing times compared to the first dataset, with no substantial difference between them. Classic ML completes the analysis in approximately 2 min, while quantum computing takes approximately 1 min and 53 s. The accuracy of classic ML is higher at 0.9241 compared to 0.9205 for quantum computing. However, both approaches achieve high precision, recall, and F1 scores, indicating their effectiveness in accurately predicting sentiment in the dataset. Classic ML achieves precision, recall, and F1 score values of 0.9286, 0.9241, and 0.9249, respectively, while quantum computing achieves values of 0.92456, 0.9205, and 0.9214, respectively. The analysis of the metrics indicates that quantum computing approaches are effective in identifying positive instances and capturing relevant sentiment information in large datasets. On the other hand, traditional machine learning techniques exhibit faster processing times when dealing with smaller dataset sizes. This study provides valuable insights into the strengths and limitations of quantum computing and machine learning for Arabic document classification, emphasizing the potential of quantum computing in achieving high accuracy, particularly in scenarios where traditional machine learning techniques may encounter difficulties. These findings contribute to the development of more accurate and efficient document classification systems for Arabic data.
Collapse
Affiliation(s)
- Ahmed Omar
- Department of Computer Science, Faculty of Science, Minia University, EL-Minia, Egypt.
| | - Tarek Abd El-Hafeez
- Department of Computer Science, Faculty of Science, Minia University, EL-Minia, Egypt.
- Computer Science Unit, Deraya University, EL-Minia, Egypt.
| |
Collapse
|
24
|
Ahmed MI, Spooner B, Isherwood J, Lane M, Orrock E, Dennison A. A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare. Cureus 2023; 15:e46454. [PMID: 37927664 PMCID: PMC10623210 DOI: 10.7759/cureus.46454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/03/2023] [Indexed: 11/07/2023] Open
Abstract
Artificial intelligence (AI) is expected to improve healthcare outcomes by facilitating early diagnosis, reducing the medical administrative burden, aiding drug development, personalising medical and oncological management, monitoring healthcare parameters on an individual basis, and allowing clinicians to spend more time with their patients. In the post-pandemic world where there is a drive for efficient delivery of healthcare and manage long waiting times for patients to access care, AI has an important role in supporting clinicians and healthcare systems to streamline the care pathways and provide timely and high-quality care for the patients. Despite AI technologies being used in healthcare for some decades, and all the theoretical potential of AI, the uptake in healthcare has been uneven and slower than anticipated and there remain a number of barriers, both overt and covert, which have limited its incorporation. This literature review highlighted barriers in six key areas: ethical, technological, liability and regulatory, workforce, social, and patient safety barriers. Defining and understanding the barriers preventing the acceptance and implementation of AI in the setting of healthcare will enable clinical staff and healthcare leaders to overcome the identified hurdles and incorporate AI technologies for the benefit of patients and clinical staff.
Collapse
Affiliation(s)
- Molla Imaduddin Ahmed
- Paediatric Respiratory Medicine, University Hospitals of Leicester NHS Trust, Leicester, GBR
| | - Brendan Spooner
- Intensive Care and Anaesthesia, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, GBR
| | - John Isherwood
- Hepatobiliary and Pancreatic Surgery, University Hospitals of Leicester NHS Trust, Leicester, GBR
| | - Mark Lane
- Ophthalmology, Birmingham and Midland Eye Centre, Birmingham, GBR
| | - Emma Orrock
- Head of Clinical Senates, East and West Midlands Clinical Senate, Leicester, GBR
| | - Ashley Dennison
- Hepatobiliary and Pancreatic Surgery, University Hospitals of Leicester NHS Trust, Leicester, GBR
| |
Collapse
|
25
|
Herington J, McCradden MD, Creel K, Boellaard R, Jones EC, Jha AK, Rahmim A, Scott PJH, Sunderland JJ, Wahl RL, Zuehlsdorff S, Saboury B. Ethical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance. J Nucl Med 2023; 64:1509-1515. [PMID: 37620051 DOI: 10.2967/jnumed.123.266110] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 07/11/2023] [Indexed: 08/26/2023] Open
Abstract
The deployment of artificial intelligence (AI) has the potential to make nuclear medicine and medical imaging faster, cheaper, and both more effective and more accessible. This is possible, however, only if clinicians and patients feel that these AI medical devices (AIMDs) are trustworthy. Highlighting the need to ensure health justice by fairly distributing benefits and burdens while respecting individual patients' rights, the AI Task Force of the Society of Nuclear Medicine and Molecular Imaging has identified 4 major ethical risks that arise during the deployment of AIMD: autonomy of patients and clinicians, transparency of clinical performance and limitations, fairness toward marginalized populations, and accountability of physicians and developers. We provide preliminary recommendations for governing these ethical risks to realize the promise of AIMD for patients and populations.
Collapse
Affiliation(s)
- Jonathan Herington
- Department of Health Humanities and Bioethics and Department of Philosophy, University of Rochester, Rochester, New York
| | - Melissa D McCradden
- Department of Bioethics, Hospital for Sick Children, and Dana Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Kathleen Creel
- Department of Philosophy and Religion and Khoury College of Computer Sciences, Northeastern University, Boston, Massachusetts
| | - Ronald Boellaard
- Department of Radiology and Nuclear Medicine, Cancer Centre Amsterdam, Amsterdam University Medical Centres, Amsterdam, The Netherlands
| | - Elizabeth C Jones
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, Bethesda, Maryland
| | - Abhinav K Jha
- Department of Biomedical Engineering and Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri
| | - Arman Rahmim
- Departments of Radiology and Physics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Peter J H Scott
- Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan
| | - John J Sunderland
- Departments of Radiology and Physics, University of Iowa, Iowa City, Iowa
| | - Richard L Wahl
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri; and
| | | | - Babak Saboury
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, Bethesda, Maryland;
| |
Collapse
|
26
|
Sullivan SS, Ledwin KM, Hewner S. A clinical classification framework for identifying persons with high social and medical needs: The COMPLEXedex-SDH. Nurs Outlook 2023; 71:102044. [PMID: 37729813 PMCID: PMC10842584 DOI: 10.1016/j.outlook.2023.102044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 08/15/2023] [Accepted: 08/18/2023] [Indexed: 09/22/2023]
Abstract
BACKGROUND First-generation algorithms resulted in high-cost features as a representation of need but unintentionally introduced systemic bias based on prior ability to access care. Improved precision health approaches are needed to reduce bias and improve health equity. PURPOSE To integrate nursing expertise into a clinical definition of high-need cases and develop a clinical classification algorithm for implementing nursing interventions. METHODS Two-phase retrospective, descriptive cohort study using 2019 data to build the algorithm (n = 19,20,848) and 2021 data to test it in adults ≥18 years old (n = 15,99,176). DISCUSSION The COMPLEXedex-SDH algorithm identified the following populations: cross-cohort needs (10.9%); high-need persons (cross-cohort needs and other social determinants) (17.7%); suboptimal health care utilization for persons with medical complexity (13.8%); high need persons with suboptimal health care utilization (6.2%). CONCLUSION The COMPLEXedex-SDH enables the identification of high-need cases and value-based utilization into actionable cohorts to prioritize outreach calls to improve health equity and outcomes.
Collapse
Affiliation(s)
- Suzanne S Sullivan
- Department of Nursing, University at Buffalo, State University of New York, Buffalo, NY.
| | - Kathryn M Ledwin
- Department of Nursing, University at Buffalo, State University of New York, Buffalo, NY
| | - Sharon Hewner
- Department of Nursing, University at Buffalo, State University of New York, Buffalo, NY
| |
Collapse
|
27
|
McKee KR, Bai X, Fiske ST. Humans perceive warmth and competence in artificial intelligence. iScience 2023; 26:107256. [PMID: 37520710 PMCID: PMC10371826 DOI: 10.1016/j.isci.2023.107256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 05/04/2023] [Accepted: 06/27/2023] [Indexed: 08/01/2023] Open
Abstract
Artificial intelligence (A.I.) increasingly suffuses everyday life. However, people are frequently reluctant to interact with A.I. systems. This challenges both the deployment of beneficial A.I. technology and the development of deep learning systems that depend on humans for oversight, direction, and regulation. Nine studies (N = 3,300) demonstrate that social-cognitive processes guide human interactions across a diverse range of real-world A.I. systems. Across studies, perceived warmth and competence emerge prominently in participants' impressions of A.I. systems. Judgments of warmth and competence systematically depend on human-A.I. interdependence and autonomy. In particular, participants perceive systems that optimize interests aligned with human interests as warmer and systems that operate independently from human direction as more competent. Finally, a prisoner's dilemma game shows that warmth and competence judgments predict participants' willingness to cooperate with a deep-learning system. These results underscore the generality of intent detection to perceptions of a broad array of algorithmic actors.
Collapse
Affiliation(s)
| | - Xuechunzi Bai
- Department of Psychology, Princeton University, Princeton, NJ 08540, USA
- School of Public and International Affairs, Princeton University, Princeton, NJ 08540, USA
| | - Susan T. Fiske
- Department of Psychology, Princeton University, Princeton, NJ 08540, USA
- School of Public and International Affairs, Princeton University, Princeton, NJ 08540, USA
| |
Collapse
|
28
|
Prasad S, Farella M. Wearables for personalized monitoring of masticatory muscle activity - opportunities, challenges, and the future. Clin Oral Investig 2023; 27:4861-4867. [PMID: 37410151 DOI: 10.1007/s00784-023-05127-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Accepted: 06/20/2023] [Indexed: 07/07/2023]
Abstract
Wearable devices are worn on or remain in close proximity of the human body. The use of wearable devices specific to the orofacial region is steadily increasing. Orofacial applications of wearable devices include supplementing diagnosis, tracking treatment progress, monitoring patient compliance, and understanding oral parafunctional behaviours. In this short communication, the role of wearable devices in advancing personalized dental medicine are highlighted with a specific focus on masticatory muscle activity monitoring in naturalistic settings. Additionally, challenges, opportunities, as well as future research areas for successful use of wearable devices for precise, personalized care of muscle disorders are discussed.
Collapse
Affiliation(s)
- Sabarinath Prasad
- Department of Orthodontics, Hamdan Bin Mohammed College of Dental Medicine, Mohammed Bin Rashid University, Dubai, United Arab Emirates.
| | - Mauro Farella
- Discipline of Orthodontics, Faculty of Dentistry, University of Otago, Dunedin, New Zealand
- Discipline of Orthodontics and Pediatric Dentistry, Department of Surgical Science, University of Cagliari, Cagliari, Italy
| |
Collapse
|
29
|
Asif A, Rajpoot K, Graham S, Snead D, Minhas F, Rajpoot N. Unleashing the potential of AI for pathology: challenges and recommendations. J Pathol 2023; 260:564-577. [PMID: 37550878 PMCID: PMC10952719 DOI: 10.1002/path.6168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 06/21/2023] [Accepted: 06/22/2023] [Indexed: 08/09/2023]
Abstract
Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Amina Asif
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Kashif Rajpoot
- School of Computer ScienceUniversity of BirminghamBirminghamUK
| | - Simon Graham
- Histofy Ltd, Birmingham Business ParkBirminghamUK
| | - David Snead
- Histofy Ltd, Birmingham Business ParkBirminghamUK
- Department of PathologyUniversity Hospitals Coventry & Warwickshire NHS TrustCoventryUK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
- Cancer Research CentreUniversity of WarwickCoventryUK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
- Histofy Ltd, Birmingham Business ParkBirminghamUK
- Cancer Research CentreUniversity of WarwickCoventryUK
- The Alan Turing InstituteLondonUK
| |
Collapse
|
30
|
Piette JD, Thomas L, Newman S, Marinec N, Krauss J, Chen J, Wu Z, Bohnert ASB. An Automatically Adaptive Digital Health Intervention to Decrease Opioid-Related Risk While Conserving Counselor Time: Quantitative Analysis of Treatment Decisions Based on Artificial Intelligence and Patient-Reported Risk Measures. J Med Internet Res 2023; 25:e44165. [PMID: 37432726 PMCID: PMC10369305 DOI: 10.2196/44165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 04/04/2023] [Accepted: 05/17/2023] [Indexed: 07/12/2023] Open
Abstract
BACKGROUND Some patients prescribed opioid analgesic (OA) medications for pain experience serious side effects, including dependence, sedation, and overdose. As most patients are at low risk for OA-related harms, risk reduction interventions requiring multiple counseling sessions are impractical on a large scale. OBJECTIVE This study evaluates whether an intervention based on reinforcement learning (RL), a field of artificial intelligence, learned through experience to personalize interactions with patients with pain discharged from the emergency department (ED) and decreased self-reported OA misuse behaviors while conserving counselors' time. METHODS We used data representing 2439 weekly interactions between a digital health intervention ("Prescription Opioid Wellness and Engagement Research in the ED" [PowerED]) and 228 patients with pain discharged from 2 EDs who reported recent opioid misuse. During each patient's 12 weeks of intervention, PowerED used RL to select from 3 treatment options: a brief motivational message delivered via an interactive voice response (IVR) call, a longer motivational IVR call, or a live call from a counselor. The algorithm selected session types for each patient each week, with the goal of minimizing OA risk, defined in terms of a dynamic score reflecting patient reports during IVR monitoring calls. When a live counseling call was predicted to have a similar impact on future risk as an IVR message, the algorithm favored IVR to conserve counselor time. We used logit models to estimate changes in the relative frequency of each session type as PowerED gained experience. Poisson regression was used to examine the changes in self-reported OA risk scores over calendar time, controlling for the ordinal session number (1st to 12th). RESULTS Participants on average were 40 (SD 12.7) years of age; 66.7% (152/228) were women and 51.3% (117/228) were unemployed. Most participants (175/228, 76.8%) reported chronic pain, and 46.2% (104/225) had moderate to severe depressive symptoms. As PowerED gained experience through interactions over a period of 142 weeks, it delivered fewer live counseling sessions than brief IVR sessions (P=.006) and extended IVR sessions (P<.001). Live counseling sessions were selected 33.5% of the time in the first 5 weeks of interactions (95% CI 27.4%-39.7%) but only for 16.4% of sessions (95% CI 12.7%-20%) after 125 weeks. Controlling for each patient's changes during the course of treatment, this adaptation of treatment-type allocation led to progressively greater improvements in self-reported OA risk scores (P<.001) over calendar time, as measured by the number of weeks since enrollment began. Improvement in risk behaviors over time was especially pronounced among patients with the highest risk at baseline (P=.02). CONCLUSIONS The RL-supported program learned which treatment modalities worked best to improve self-reported OA risk behaviors while conserving counselors' time. RL-supported interventions represent a scalable solution for patients with pain receiving OA prescriptions. TRIAL REGISTRATION Clinicaltrials.gov NCT02990377; https://classic.clinicaltrials.gov/ct2/show/NCT02990377.
Collapse
Affiliation(s)
- John D Piette
- Ann Arbor Department of Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
- Department of Health Behavior Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Laura Thomas
- Ann Arbor Department of Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
- Department of Anesthesiology, School of Medicine, University of Michigan, Ann Arbor, MI, United States
| | - Sean Newman
- Ann Arbor Department of Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
- Department of Health Behavior Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Nicolle Marinec
- Ann Arbor Department of Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
- Department of Health Behavior Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Joel Krauss
- Department of Emergency Medicine, Trinity Health St. Joseph Mercy, Ann Arbor, MI, United States
| | - Jenny Chen
- Ann Arbor Department of Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
- Department of Health Behavior Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Zhenke Wu
- Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, MI, United States
| | - Amy S B Bohnert
- Ann Arbor Department of Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
- Department of Anesthesiology, School of Medicine, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
31
|
Chohlas-Wood A, Coots M, Goel S, Nyarko J. Designing equitable algorithms. NATURE COMPUTATIONAL SCIENCE 2023; 3:601-610. [PMID: 38177749 DOI: 10.1038/s43588-023-00485-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Accepted: 06/06/2023] [Indexed: 01/06/2024]
Abstract
Predictive algorithms are now commonly used to distribute society's resources and sanctions. But these algorithms can entrench and exacerbate inequities. To guard against this possibility, many have suggested that algorithms be subject to formal fairness constraints. Here we argue, however, that popular constraints-while intuitively appealing-often worsen outcomes for individuals in marginalized groups, and can even leave all groups worse off. We outline a more holistic path forward for improving the equity of algorithmically guided decisions.
Collapse
|
32
|
Graham M. Data for sale: trust, confidence and sharing health data with commercial companies. JOURNAL OF MEDICAL ETHICS 2023; 49:515-522. [PMID: 34330796 PMCID: PMC10359563 DOI: 10.1136/medethics-2021-107464] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Accepted: 07/13/2021] [Indexed: 06/13/2023]
Abstract
Powered by 'big health data' and enormous gains in computing power, artificial intelligence and related technologies are already changing the healthcare landscape. Harnessing the potential of these technologies will necessitate partnerships between health institutions and commercial companies, particularly as it relates to sharing health data. The need for commercial companies to be trustworthy users of data has been argued to be critical to the success of this endeavour. I argue that this approach is mistaken. Our interactions with commercial companies need not, and should not, be based on trust. Rather, they should be based on confidence. I begin by elucidating the differences between trust, reliability, and confidence, and argue that trust is not the appropriate attitude to adopt when it comes to sharing data with commercial companies. I argue that what we really should want is confidence in a system of data sharing. I then provide an outline of what a confidence-worthy system of data sharing with commercial companies might look like, and conclude with some remarks about the role of trust within this system.
Collapse
Affiliation(s)
- Mackenzie Graham
- Wellcome Centre for Ethics and Humanities, University of Oxford, Oxford, UK
| |
Collapse
|
33
|
Banda JM, Shah NH, Periyakoil VS. Characterizing subgroup performance of probabilistic phenotype algorithms within older adults: a case study for dementia, mild cognitive impairment, and Alzheimer's and Parkinson's diseases. JAMIA Open 2023; 6:ooad043. [PMID: 37397506 PMCID: PMC10307941 DOI: 10.1093/jamiaopen/ooad043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 06/06/2023] [Accepted: 06/22/2023] [Indexed: 07/04/2023] Open
Abstract
Objective Biases within probabilistic electronic phenotyping algorithms are largely unexplored. In this work, we characterize differences in subgroup performance of phenotyping algorithms for Alzheimer's disease and related dementias (ADRD) in older adults. Materials and methods We created an experimental framework to characterize the performance of probabilistic phenotyping algorithms under different racial distributions allowing us to identify which algorithms may have differential performance, by how much, and under what conditions. We relied on rule-based phenotype definitions as reference to evaluate probabilistic phenotype algorithms created using the Automated PHenotype Routine for Observational Definition, Identification, Training and Evaluation framework. Results We demonstrate that some algorithms have performance variations anywhere from 3% to 30% for different populations, even when not using race as an input variable. We show that while performance differences in subgroups are not present for all phenotypes, they do affect some phenotypes and groups more disproportionately than others. Discussion Our analysis establishes the need for a robust evaluation framework for subgroup differences. The underlying patient populations for the algorithms showing subgroup performance differences have great variance between model features when compared with the phenotypes with little to no differences. Conclusion We have created a framework to identify systematic differences in the performance of probabilistic phenotyping algorithms specifically in the context of ADRD as a use case. Differences in subgroup performance of probabilistic phenotyping algorithms are not widespread nor do they occur consistently. This highlights the great need for careful ongoing monitoring to evaluate, measure, and try to mitigate such differences.
Collapse
Affiliation(s)
- Juan M Banda
- Corresponding Author: Juan M. Banda, PhD, Department of Computer Science, College of Arts and Sciences, Georgia State University, 25 Park Place, Suite 752, Atlanta, GA 30303, USA;
| | - Nigam H Shah
- Stanford Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, California, USA
| | - Vyjeyanthi S Periyakoil
- Stanford Department of Medicine, Palo Alto, California, USA
- VA Palo Alto Health Care System, Palo Alto, California, USA
| |
Collapse
|
34
|
Fanconi C, de Hond A, Peterson D, Capodici A, Hernandez-Boussard T. A Bayesian approach to predictive uncertainty in chemotherapy patients at risk of acute care utilization. EBioMedicine 2023; 92:104632. [PMID: 37269570 PMCID: PMC10250586 DOI: 10.1016/j.ebiom.2023.104632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 05/09/2023] [Accepted: 05/11/2023] [Indexed: 06/05/2023] Open
Abstract
BACKGROUND Machine learning (ML) predictions are becoming increasingly integrated into medical practice. One commonly used method, ℓ1-penalised logistic regression (LASSO), can estimate patient risk for disease outcomes but is limited by only providing point estimates. Instead, Bayesian logistic LASSO regression (BLLR) models provide distributions for risk predictions, giving clinicians a better understanding of predictive uncertainty, but they are not commonly implemented. METHODS This study evaluates the predictive performance of different BLLRs compared to standard logistic LASSO regression, using real-world, high-dimensional, structured electronic health record (EHR) data from cancer patients initiating chemotherapy at a comprehensive cancer centre. Multiple BLLR models were compared against a LASSO model using an 80-20 random split using 10-fold cross-validation to predict the risk of acute care utilization (ACU) after starting chemotherapy. FINDINGS This study included 8439 patients. The LASSO model predicted ACU with an area under the receiver operating characteristic curve (AUROC) of 0.806 (95% CI: 0.775-0.834). BLLR with a Horseshoe+ prior and a posterior approximated by Metropolis-Hastings sampling showed similar performance: 0.807 (95% CI: 0.780-0.834) and offers the advantage of uncertainty estimation for each prediction. In addition, BLLR could identify predictions too uncertain to be automatically classified. BLLR uncertainties were stratified by different patient subgroups, demonstrating that predictive uncertainties significantly differ across race, cancer type, and stage. INTERPRETATION BLLRs are a promising yet underutilised tool that increases explainability by providing risk estimates while offering a similar level of performance to standard LASSO-based models. Additionally, these models can identify patient subgroups with higher uncertainty, which can augment clinical decision-making. FUNDING This work was supported in part by the National Library Of Medicine of the National Institutes of Health under Award Number R01LM013362. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Collapse
Affiliation(s)
- Claudio Fanconi
- Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, USA
| | - Anne de Hond
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, USA
- Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, the Netherlands
| | - Dylan Peterson
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, USA
| | - Angelo Capodici
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, USA
- Department of Biomedical and Neuromotor Science, University of Bologna, Bologna, Italy
| | | |
Collapse
|
35
|
MacMath D, Chen M, Khoury P. Artificial Intelligence: Exploring the Future of Innovation in Allergy Immunology. Curr Allergy Asthma Rep 2023; 23:351-362. [PMID: 37160554 PMCID: PMC10169188 DOI: 10.1007/s11882-023-01084-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/21/2023] [Indexed: 05/11/2023]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) has increasingly been used in healthcare. Given the capacity of AI to handle large data and complex relationships between variables, AI is well suited for applications in healthcare. Recently, AI has been applied to allergy research. RECENT FINDINGS In this article, we review how AI technologies have been utilized in basic science and clinical allergy research for asthma, atopic dermatitis, rhinology, adverse reactions to drugs and vaccines, food allergy, anaphylaxis, urticaria, and eosinophilic gastrointestinal disorders. We discuss barriers for AI adoption to improve the care of patients with atopic diseases. These studies demonstrate the utility of applying AI to the field of allergy to help investigators expand their understanding of disease pathogenesis, improve diagnostic accuracy, enable prediction for treatments and outcomes, and for drug discovery.
Collapse
Affiliation(s)
- Derek MacMath
- Department of Medicine, Department of Pediatrics, Baylor College of Medicine, Houston, TX, USA
| | - Meng Chen
- Division of Pulmonary, Allergy & Critical Care Medicine, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
| | - Paneez Khoury
- National Institutes of Allergic and Infectious Diseases, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD, USA.
| |
Collapse
|
36
|
Chen RJ, Wang JJ, Williamson DFK, Chen TY, Lipkova J, Lu MY, Sahai S, Mahmood F. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nat Biomed Eng 2023; 7:719-742. [PMID: 37380750 PMCID: PMC10632090 DOI: 10.1038/s41551-023-01056-8] [Citation(s) in RCA: 30] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 04/13/2023] [Indexed: 06/30/2023]
Abstract
In healthcare, the development and deployment of insufficiently fair systems of artificial intelligence (AI) can undermine the delivery of equitable care. Assessments of AI models stratified across subpopulations have revealed inequalities in how patients are diagnosed, treated and billed. In this Perspective, we outline fairness in machine learning through the lens of healthcare, and discuss how algorithmic biases (in data acquisition, genetic variation and intra-observer labelling variability, in particular) arise in clinical workflows and the resulting healthcare disparities. We also review emerging technology for mitigating biases via disentanglement, federated learning and model explainability, and their role in the development of AI-based software as a medical device.
Collapse
Affiliation(s)
- Richard J Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Judy J Wang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Boston University School of Medicine, Boston, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Tiffany Y Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Jana Lipkova
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Ming Y Lu
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Sharifa Sahai
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Systems Biology, Harvard Medical School, Boston, MA, USA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
- Cancer Program, Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA, USA.
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
- Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
37
|
Ratajczak R, Cockerill RG. Artificial Intelligence in Violence Risk Assessment: Addressing Racial Bias and Inequity. J Psychiatr Pract 2023; 29:239-245. [PMID: 37200144 DOI: 10.1097/pra.0000000000000713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
Although there has been no shortage of technological innovation in recent decades, a solution to sociodemographic disparities in the forensic setting has remained elusive. Artificial intelligence (AI) is a uniquely powerful emerging technology that is likely to either exacerbate or mitigate existing disparities and biases. This column argues that the implementation of AI in forensic settings is inevitable, and that practitioners and researchers should focus on developing AI systems that reduce bias and advance sociodemographic equity rather than attempt to impede the use of this novel technology.
Collapse
Affiliation(s)
- Robert Ratajczak
- RATAJCZAK and COCKERILL: Department of Psychiatry and Behavioral Neuroscience, University of Chicago Pritzker School of Medicine, Chicago, IL
| | | |
Collapse
|
38
|
Glocker B, Jones C, Bernhardt M, Winzeck S. Algorithmic encoding of protected characteristics in chest X-ray disease detection models. EBioMedicine 2023; 89:104467. [PMID: 36791660 PMCID: PMC10025760 DOI: 10.1016/j.ebiom.2023.104467] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 01/23/2023] [Accepted: 01/24/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND It has been rightfully emphasized that the use of AI for clinical decision making could amplify health disparities. An algorithm may encode protected characteristics, and then use this information for making predictions due to undesirable correlations in the (historical) training data. It remains unclear how we can establish whether such information is actually used. Besides the scarcity of data from underserved populations, very little is known about how dataset biases manifest in predictive models and how this may result in disparate performance. This article aims to shed some light on these issues by exploring methodology for subgroup analysis in image-based disease detection models. METHODS We utilize two publicly available chest X-ray datasets, CheXpert and MIMIC-CXR, to study performance disparities across race and biological sex in deep learning models. We explore test set resampling, transfer learning, multitask learning, and model inspection to assess the relationship between the encoding of protected characteristics and disease detection performance across subgroups. FINDINGS We confirm subgroup disparities in terms of shifted true and false positive rates which are partially removed after correcting for population and prevalence shifts in the test sets. We find that transfer learning alone is insufficient for establishing whether specific patient information is used for making predictions. The proposed combination of test-set resampling, multitask learning, and model inspection reveals valuable insights about the way protected characteristics are encoded in the feature representations of deep neural networks. INTERPRETATION Subgroup analysis is key for identifying performance disparities of AI models, but statistical differences across subgroups need to be taken into account when analyzing potential biases in disease detection. The proposed methodology provides a comprehensive framework for subgroup analysis enabling further research into the underlying causes of disparities. FUNDING European Research Council Horizon 2020, UK Research and Innovation.
Collapse
Affiliation(s)
- Ben Glocker
- Department of Computing, Imperial College London, London, SW7 2AZ, UK.
| | - Charles Jones
- Department of Computing, Imperial College London, London, SW7 2AZ, UK
| | - Mélanie Bernhardt
- Department of Computing, Imperial College London, London, SW7 2AZ, UK
| | - Stefan Winzeck
- Department of Computing, Imperial College London, London, SW7 2AZ, UK
| |
Collapse
|
39
|
Greenburg J, Lu Y, Lu S, Kamau U, Hamilton R, Pettus J, Preum S, Vaickus L, Levy J. Development of an interactive web dashboard to facilitate the reexamination of pathology reports for instances of underbilling of CPT codes. J Pathol Inform 2023; 14:100187. [PMID: 36700236 PMCID: PMC9867971 DOI: 10.1016/j.jpi.2023.100187] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Accepted: 01/03/2023] [Indexed: 01/13/2023] Open
Abstract
Current Procedural Terminology Codes is a numerical coding system used to bill for medical procedures and services and crucially, represents a major reimbursement pathway. Given that pathology services represent a consequential source of hospital revenue, understanding instances where codes may have been misassigned or underbilled is critical. Several algorithms have been proposed that can identify improperly billed CPT codes in existing datasets of pathology reports. Estimation of the fiscal impacts of these reports requires a coder (i.e., billing staff) to review the original reports and manually code them again. As the re-assignment of codes using machine learning algorithms can be done quickly, the bottleneck in validating these reassignments is in this manual re-coding process, which can prove cumbersome. This work documents the development of a rapidly deployable dashboard for examination of reports that the original coder may have misbilled. Our dashboard features the following main components: (1) a bar plot to show the predicted probabilities for each CPT code, (2) an interpretation plot showing how each word in the report combines to form the overall prediction, and (3) a place for the user to input the CPT code they have chosen to assign. This dashboard utilizes the algorithms developed to accurately identify CPT codes to highlight the codes missed by the original coders. In order to demonstrate the function of this web application, we recruited pathologists to utilize it to highlight reports that had codes incorrectly assigned. We expect this application to accelerate the validation of re-assigned codes through facilitating rapid review of false-positive pathology reports. In the future, we will use this technology to review thousands of past cases in order to estimate the impact of underbilling has on departmental revenue.
Collapse
Affiliation(s)
- Jack Greenburg
- Department of Computer Science, Middlebury College, Middlebury, VT, USA
| | - Yunrui Lu
- Program in Quantitative Biomedical Sciences, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
| | - Shuyang Lu
- Program in Quantitative Biomedical Sciences, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
| | - Uhuru Kamau
- Program in Quantitative Biomedical Sciences, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
| | - Robert Hamilton
- Department of Pathology, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Jason Pettus
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
| | - Sarah Preum
- Department of Computer Science, Dartmouth College, Hanover, NH, USA
| | - Louis Vaickus
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
| | - Joshua Levy
- Program in Quantitative Biomedical Sciences, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
- Department of Epidemiology, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Department of Dermatology, Dartmouth Health, Lebanon, NH, USA
| |
Collapse
|
40
|
Enodien B, Taha-Mehlitz S, Saad B, Nasser M, Frey DM, Taha A. The development of machine learning in bariatric surgery. Front Surg 2023; 10:1102711. [PMID: 36911599 PMCID: PMC9998495 DOI: 10.3389/fsurg.2023.1102711] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Accepted: 02/08/2023] [Indexed: 03/14/2023] Open
Abstract
Background Machine learning (ML), is an approach to data analysis that makes the process of analytical model building automatic. The significance of ML stems from its potential to evaluate big data and achieve quicker and more accurate outcomes. ML has recently witnessed increased adoption in the medical domain. Bariatric surgery, otherwise referred to as weight loss surgery, reflects the series of procedures performed on people demonstrating obesity. This systematic scoping review aims to explore the development of ML in bariatric surgery. Methods The study used the Preferred Reporting Items for Systematic and Meta-analyses for Scoping Review (PRISMA-ScR). A comprehensive literature search was performed of several databases including PubMed, Cochrane, and IEEE, and search engines namely Google Scholar. Eligible studies included journals published from 2016 to the current date. The PRESS checklist was used to evaluate the consistency demonstrated during the process. Results A total of seventeen articles qualified for inclusion in the study. Out of the included studies, sixteen concentrated on the role of ML algorithms in prediction, while one addressed ML's diagnostic capacity. Most articles (n = 15) were journal publications, whereas the rest (n = 2) were papers from conference proceedings. Most included reports were from the United States (n = 6). Most studies addressed neural networks, with convolutional neural networks as the most prevalent. Also, the data type used in most articles (n = 13) was derived from hospital databases, with very few articles (n = 4) collecting original data via observation. Conclusions This study indicates that ML has numerous benefits in bariatric surgery, however its current application is limited. The evidence suggests that bariatric surgeons can benefit from ML algorithms since they will facilitate the prediction and evaluation of patient outcomes. Also, ML approaches to enhance work processes by making data categorization and analysis easier. However, further large multicenter studies are required to validate results internally and externally as well as explore and address limitations of ML application in bariatric surgery.
Collapse
Affiliation(s)
- Bassey Enodien
- Department of Surgery, GZO-Hospital, Wetzikon, Switzerland
| | - Stephanie Taha-Mehlitz
- Clarunis, University Centre for Gastrointestinal and Liver Diseases, St. Clara Hospital and University Hospital, Basel, Switzerland
| | - Baraa Saad
- School of Medicine, St George's University of London, London, United Kingdom
| | - Maya Nasser
- School of Medicine, St George's University of London, London, United Kingdom
| | - Daniel M Frey
- Department of Biomedical Engineering, Faculty of Medicine, University of Basel, Allschwil, Switzerland
| | - Anas Taha
- Clarunis, University Centre for Gastrointestinal and Liver Diseases, St. Clara Hospital and University Hospital, Basel, Switzerland.,Department of Biomedical Engineering, Faculty of Medicine, University of Basel, Allschwil, Switzerland
| |
Collapse
|
41
|
Bowe AK, Lightbody G, Staines A, Murray DM. Big data, machine learning, and population health: predicting cognitive outcomes in childhood. Pediatr Res 2023; 93:300-307. [PMID: 35681091 PMCID: PMC7614199 DOI: 10.1038/s41390-022-02137-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 05/05/2022] [Accepted: 05/17/2022] [Indexed: 11/09/2022]
Abstract
The application of machine learning (ML) to address population health challenges has received much less attention than its application in the clinical setting. One such challenge is addressing disparities in early childhood cognitive development-a complex public health issue rooted in the social determinants of health, exacerbated by inequity, characterised by intergenerational transmission, and which will continue unabated without novel approaches to address it. Early life, the period of optimal neuroplasticity, presents a window of opportunity for early intervention to improve cognitive development. Unfortunately for many, this window will be missed, and intervention may never occur or occur only when overt signs of cognitive delay manifest. In this review, we explore the potential value of ML and big data analysis in the early identification of children at risk for poor cognitive outcome, an area where there is an apparent dearth of research. We compare and contrast traditional statistical methods with ML approaches, provide examples of how ML has been used to date in the field of neurodevelopmental disorders, and present a discussion of the opportunities and risks associated with its use at a population level. The review concludes by highlighting potential directions for future research in this area. IMPACT: To date, the application of machine learning to address population health challenges in paediatrics lags behind other clinical applications. This review provides an overview of the public health challenge we face in addressing disparities in childhood cognitive development and focuses on the cornerstone of early intervention. Recent advances in our ability to collect large volumes of data, and in analytic capabilities, provide a potential opportunity to improve current practices in this field. This review explores the potential role of machine learning and big data analysis in the early identification of children at risk for poor cognitive outcomes.
Collapse
Affiliation(s)
- Andrea K. Bowe
- grid.7872.a0000000123318773INFANT Research Centre, University College Cork, Cork, Ireland
| | - Gordon Lightbody
- grid.7872.a0000000123318773INFANT Research Centre, University College Cork, Cork, Ireland ,grid.7872.a0000000123318773Department of Electrical and Electronic Engineering, University College Cork, Cork, Ireland
| | - Anthony Staines
- grid.15596.3e0000000102380260School of Nursing, Psychotherapy, and Community Health, Dublin City University, Dublin, Ireland
| | - Deirdre M. Murray
- grid.7872.a0000000123318773INFANT Research Centre, University College Cork, Cork, Ireland
| |
Collapse
|
42
|
Al Meslamani AZ. How AI is advancing asthma management? Insights into economic and clinical aspects. J Med Econ 2023; 26:1489-1494. [PMID: 37902681 DOI: 10.1080/13696998.2023.2277072] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 10/26/2023] [Indexed: 10/31/2023]
Abstract
Asthma, an increasingly prevalent chronic respiratory condition, incurs significant economic costs worldwide. Artificial Intelligence (AI), particularly Machine Learning (ML), has been widely recognized as transformative when applied to asthma care. This commentary investigates how AI and ML may improve clinical outcomes while alleviating some of the costs associated with asthma care. AI's powerful analytical abilities could usher in an unprecedented era of preventive measures, particularly by identifying at-risk populations and anticipating environmental triggers. ML shows promise for enhancing real-time monitoring, early detection, and tailored treatment strategies in paediatric asthma, potentially reducing hospitalizations and emergency care costs. Emerging AI-powered wearable technologies are catalysing a revolutionary shift in patient monitoring, providing proactive interventions. Although optimistic, this commentary highlights a gap in empirical studies evaluating the cost-effectiveness of AI in asthma care and stresses the need for larger datasets to accurately represent the economic benefits of AI solutions. Additionally, this paper emphasizes the ethical considerations surrounding data privacy and algorithmic bias, which are vital for the successful and equitable integration of AI into healthcare settings. This editorial underscores the urgent necessity of conducting thorough analyses to assess all economic implications, facilitate optimized resource allocation, and foster a nuanced understanding of AI/ML technologies in asthma management that may reduce costs to healthcare systems.
Collapse
Affiliation(s)
- Ahmad Z Al Meslamani
- College of Pharmacy, Al Ain University, Abu Dhabi, United Arab Emirates
- AAU Health and Biomedical Research Center, Al Ain University, Abu Dhabi, United Arab Emirates
| |
Collapse
|
43
|
Terhorst Y, Sander LB, Ebert DD, Baumeister H. Optimizing the predictive power of depression screenings using machine learning. Digit Health 2023; 9:20552076231194939. [PMID: 37654715 PMCID: PMC10467308 DOI: 10.1177/20552076231194939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Accepted: 07/28/2023] [Indexed: 09/02/2023] Open
Abstract
Objective Mental health self-report and clinician-rating scales with diagnoses defined by sum-score cut-offs are often used for depression screening. This study investigates whether machine learning (ML) can detect major depressive episodes (MDE) based on screening scales with higher accuracy than best-practice clinical sum-score approaches. Methods Primary data was obtained from two RCTs on the treatment of depression. Ground truth were DSM 5 MDE diagnoses based on structured clinical interviews (SCID) and PHQ-9 self-report, clinician-rated QIDS-16, and HAM-D-17 were predictors. ML models were trained using 10-fold cross-validation. Performance was compared against best-practice sum-score cut-offs. Primary outcome was the Area Under the Curve (AUC) of the Receiver Operating Characteristic curve. DeLong's test with bootstrapping was used to test for differences in AUC. Secondary outcomes were balanced accuracy, precision, recall, F1-score, and number needed to diagnose (NND). Results A total of k = 1030 diagnoses (no diagnosis: k = 775; MDE: k = 255) were included. ML models achieved an AUCQIDS-16 = 0.94, AUCHAM-D-17 = 0.88, and AUCPHQ-9 = 0.83 in the testing set. ML AUC was significantly higher than sum-score cut-offs for QIDS-16 and PHQ-9 (ps ≤ 0.01; HAM_D-17: p = 0.847). Applying optimal prediction thresholds, QIDS-16 classifier achieved clinically relevant improvements (Δbalanced accuracy = 8%, ΔF1-score = 14%, ΔNND = 21%). Differences for PHQ_9 and HAM-D-17 were marginal. Conclusions ML augmented depression screenings could potentially make a major contribution to improving MDE diagnosis depending on questionnaire (e.g., QIDS-16). Confirmatory studies are needed before ML enhanced screening can be implemented into routine care practice.
Collapse
Affiliation(s)
- Yannik Terhorst
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, University Ulm, Ulm, Germany
| | - Lasse B Sander
- Medical Psychology and Medical Sociology, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - David D Ebert
- Department for Sport and Health Sciences, Chair for Psychology & Digital Mental Health Care, Technical University of Munich, Munich, Germany
| | - Harald Baumeister
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, University Ulm, Ulm, Germany
| |
Collapse
|
44
|
Cox M, Panagides JC, Tabari A, Kalva S, Kalpathy-Cramer J, Daye D. Risk stratification with explainable machine learning for 30-day procedure-related mortality and 30-day unplanned readmission in patients with peripheral arterial disease. PLoS One 2022; 17:e0277507. [PMID: 36409699 PMCID: PMC9678279 DOI: 10.1371/journal.pone.0277507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2022] [Accepted: 10/28/2022] [Indexed: 11/22/2022] Open
Abstract
Predicting 30-day procedure-related mortality risk and 30-day unplanned readmission in patients undergoing lower extremity endovascular interventions for peripheral artery disease (PAD) may assist in improving patient outcomes. Risk prediction of 30-day mortality can help clinicians identify treatment plans to reduce the risk of death, and prediction of 30-day unplanned readmission may improve outcomes by identifying patients who may benefit from readmission prevention strategies. The goal of this study is to develop machine learning models to stratify risk of 30-day procedure-related mortality and 30-day unplanned readmission in patients undergoing lower extremity infra-inguinal endovascular interventions. We used a cohort of 14,444 cases from the American College of Surgeons National Surgical Quality Improvement Program database. For each outcome, we developed and evaluated multiple machine learning models, including Support Vector Machines, Multilayer Perceptrons, and Gradient Boosting Machines, and selected a random forest as the best-performing model for both outcomes. Our 30-day procedure-related mortality model achieved an AUC of 0.75 (95% CI: 0.71-0.79) and our 30-day unplanned readmission model achieved an AUC of 0.68 (95% CI: 0.67-0.71). Stratification of the test set by race (white and non-white), sex (male and female), and age (≥65 years and <65 years) and subsequent evaluation of demographic parity by AUC shows that both models perform equally well across race, sex, and age groups. We interpret the model globally and locally using Gini impurity and SHapley Additive exPlanations (SHAP). Using the top five predictors for death and mortality, we demonstrate differences in survival for subgroups stratified by these predictors, which underscores the utility of our model.
Collapse
Affiliation(s)
- Meredith Cox
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America
| | - J. C. Panagides
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America
| | - Azadeh Tabari
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America
| | - Sanjeeva Kalva
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America
| | - Jayashree Kalpathy-Cramer
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America
| | - Dania Daye
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America
| |
Collapse
|
45
|
de Hond AAH, van Buchem MM, Hernandez-Boussard T. Picture a data scientist: a call to action for increasing diversity, equity, and inclusion in the age of AI. J Am Med Inform Assoc 2022; 29:2178-2181. [PMID: 36048021 PMCID: PMC9667164 DOI: 10.1093/jamia/ocac156] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 08/18/2022] [Accepted: 08/26/2022] [Indexed: 11/13/2022] Open
Abstract
The lack of diversity, equity, and inclusion continues to hamper the artificial intelligence (AI) field and is especially problematic for healthcare applications. In this article, we expand on the need for diversity, equity, and inclusion, specifically focusing on the composition of AI teams. We call to action leaders at all levels to make team inclusivity and diversity the centerpieces of AI development, not the afterthought. These recommendations take into consideration mitigation at several levels, including outreach programs at the local level, diversity statements at the academic level, and regulatory steps at the federal level.
Collapse
Affiliation(s)
- Anne A H de Hond
- Clinical AI Implementation and Research Lab, Leiden University Medical Center, Leiden, The Netherlands
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, California, USA
| | - Marieke M van Buchem
- Clinical AI Implementation and Research Lab, Leiden University Medical Center, Leiden, The Netherlands
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, California, USA
| | - Tina Hernandez-Boussard
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, California, USA
- Department of Biomedical Data Science, Stanford University, Stanford, California, USA
- Department of Epidemiology & Population Health (By Courtesy), Stanford University, Stanford, California, USA
| |
Collapse
|
46
|
Liu F, Demosthenes P. Real-world data: a brief review of the methods, applications, challenges and opportunities. BMC Med Res Methodol 2022; 22:287. [PMID: 36335315 PMCID: PMC9636688 DOI: 10.1186/s12874-022-01768-6] [Citation(s) in RCA: 86] [Impact Index Per Article: 43.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 10/22/2022] [Indexed: 11/07/2022] Open
Abstract
Abstract
Background
The increased adoption of the internet, social media, wearable devices, e-health services, and other technology-driven services in medicine and healthcare has led to the rapid generation of various types of digital data, providing a valuable data source beyond the confines of traditional clinical trials, epidemiological studies, and lab-based experiments.
Methods
We provide a brief overview on the type and sources of real-world data and the common models and approaches to utilize and analyze real-world data. We discuss the challenges and opportunities of using real-world data for evidence-based decision making This review does not aim to be comprehensive or cover all aspects of the intriguing topic on RWD (from both the research and practical perspectives) but serves as a primer and provides useful sources for readers who interested in this topic.
Results and Conclusions
Real-world hold great potential for generating real-world evidence for designing and conducting confirmatory trials and answering questions that may not be addressed otherwise. The voluminosity and complexity of real-world data also call for development of more appropriate, sophisticated, and innovative data processing and analysis techniques while maintaining scientific rigor in research findings, and attentions to data ethics to harness the power of real-world data.
Collapse
|
47
|
Carruthers R, Straw I, Ruffle JK, Herron D, Nelson A, Bzdok D, Fernandez-Reyes D, Rees G, Nachev P. Representational ethical model calibration. NPJ Digit Med 2022; 5:170. [PMID: 36333390 PMCID: PMC9636204 DOI: 10.1038/s41746-022-00716-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Accepted: 10/19/2022] [Indexed: 11/06/2022] Open
Abstract
Equity is widely held to be fundamental to the ethics of healthcare. In the context of clinical decision-making, it rests on the comparative fidelity of the intelligence - evidence-based or intuitive - guiding the management of each individual patient. Though brought to recent attention by the individuating power of contemporary machine learning, such epistemic equity arises in the context of any decision guidance, whether traditional or innovative. Yet no general framework for its quantification, let alone assurance, currently exists. Here we formulate epistemic equity in terms of model fidelity evaluated over learnt multidimensional representations of identity crafted to maximise the captured diversity of the population, introducing a comprehensive framework for Representational Ethical Model Calibration. We demonstrate the use of the framework on large-scale multimodal data from UK Biobank to derive diverse representations of the population, quantify model performance, and institute responsive remediation. We offer our approach as a principled solution to quantifying and assuring epistemic equity in healthcare, with applications across the research, clinical, and regulatory domains.
Collapse
Affiliation(s)
- Robert Carruthers
- Department of Computer Science, University College London, London, UK.
| | - Isabel Straw
- UCL Queen Square Institute of Neurology, University College London, London, UK
| | - James K Ruffle
- UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Daniel Herron
- Research and Development, NIHR University College London Hospitals Biomedical Research Centre, London, UK
| | - Amy Nelson
- UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Danilo Bzdok
- Department of Biomedical Engineering, Faculty of Medicine, McGill University, Montreal, Canada
| | | | - Geraint Rees
- UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Parashkev Nachev
- UCL Queen Square Institute of Neurology, University College London, London, UK.
| |
Collapse
|
48
|
Grote T. Randomised controlled trials in medical AI: ethical considerations. JOURNAL OF MEDICAL ETHICS 2022; 48:899-906. [PMID: 33990429 DOI: 10.1136/medethics-2020-107166] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 03/30/2021] [Accepted: 04/08/2021] [Indexed: 06/12/2023]
Abstract
In recent years, there has been a surge of high-profile publications on applications of artificial intelligence (AI) systems for medical diagnosis and prognosis. While AI provides various opportunities for medical practice, there is an emerging consensus that the existing studies show considerable deficits and are unable to establish the clinical benefit of AI systems. Hence, the view that the clinical benefit of AI systems needs to be studied in clinical trials-particularly randomised controlled trials (RCTs)-is gaining ground. However, an issue that has been overlooked so far in the debate is that, compared with drug RCTs, AI RCTs require methodological adjustments, which entail ethical challenges. This paper sets out to develop a systematic account of the ethics of AI RCTs by focusing on the moral principles of clinical equipoise, informed consent and fairness. This way, the objective is to animate further debate on the (research) ethics of medical AI.
Collapse
Affiliation(s)
- Thomas Grote
- Ethics and Philosophy Lab, Cluster of Excellence "Machine Learning: New Perspectives for Science", University of Tübingen, Tübingen D-72076, Germany
| |
Collapse
|
49
|
Ganapathi S, Palmer J, Alderman JE, Calvert M, Espinoza C, Gath J, Ghassemi M, Heller K, Mckay F, Karthikesalingam A, Kuku S, Mackintosh M, Manohar S, Mateen BA, Matin R, McCradden M, Oakden-Rayner L, Ordish J, Pearson R, Pfohl SR, Rostamzadeh N, Sapey E, Sebire N, Sounderajah V, Summers C, Treanor D, Denniston AK, Liu X. Tackling bias in AI health datasets through the STANDING Together initiative. Nat Med 2022; 28:2232-2233. [PMID: 36163296 DOI: 10.1038/s41591-022-01987-w] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023]
Affiliation(s)
- Shaswath Ganapathi
- College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Jo Palmer
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Joseph E Alderman
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.,Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Melanie Calvert
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.,Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.,NIHR Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK.,NIHR Surgical Reconstruction and Microbiology Research Centre, University of Birmingham, Birmingham, UK.,NIHR Applied Research Collaborative West Midlands University of Birmingham, Birmingham, UK
| | | | - Jacqui Gath
- Patient Partner, Birmingham, UK.,Patient Partner, Sheffield, UK
| | - Marzyeh Ghassemi
- Department of Electrical Engineering and Computer Science; Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | | | - Francis Mckay
- The Ethox Centre and the Wellcome Centre for Ethics and Humanities, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | | | - Stephanie Kuku
- Institute of Women's Health, University College London, London, UK.,Hardian Health, London, UK
| | | | | | - Bilal A Mateen
- Institute of Health Informatics, University College London, London, UK.,The Wellcome Trust, London, UK
| | - Rubeta Matin
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Melissa McCradden
- Department of Bioethics, Hospital for Sick Children, Toronto, Ontario, Canada.,Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, South Australia, Australia
| | - Johan Ordish
- Medicines and Healthcare Products Regulatory Agency, London, UK
| | - Russell Pearson
- Medicines and Healthcare Products Regulatory Agency, London, UK
| | | | | | - Elizabeth Sapey
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.,Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Neil Sebire
- Health Data Research, London, UK.,Great Ormond Street Hospital for Children, London, UK
| | - Viknesh Sounderajah
- Institute of Global Health Innovation, Imperial College London, London, UK.,Department of Surgery and Cancer, Imperial College London, London, UK
| | - Charlotte Summers
- Wolfson Lung Injury Unit, Heart and Lung Research Institute, University of Cambridge, Cambrdige, UK.,Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | - Darren Treanor
- Leeds Teaching Hospitals NHS Trust, Leeds, UK.,University of Leeds, Leeds, UK.,Department of Clinical Pathology, and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden.,Center for Medical Image Science and Visualization (CMIV), Linköping University, Linköping, Sweden
| | - Alastair K Denniston
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.,Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.,Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.,NIHR Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK.,Health Data Research, London, UK
| | - Xiaoxuan Liu
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. .,Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK. .,Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
| |
Collapse
|
50
|
Chavez-Yenter D, Goodman MS, Chen Y, Chu X, Bradshaw RL, Lorenz Chambers R, Chan PA, Daly BM, Flynn M, Gammon A, Hess R, Kessler C, Kohlmann WK, Mann DM, Monahan R, Peel S, Kawamoto K, Del Fiol G, Sigireddi M, Buys SS, Ginsburg O, Kaphingst KA. Association of Disparities in Family History and Family Cancer History in the Electronic Health Record With Sex, Race, Hispanic or Latino Ethnicity, and Language Preference in 2 Large US Health Care Systems. JAMA Netw Open 2022; 5:e2234574. [PMID: 36194411 PMCID: PMC9533178 DOI: 10.1001/jamanetworkopen.2022.34574] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 08/12/2022] [Indexed: 11/14/2022] Open
Abstract
Importance Clinical decision support (CDS) algorithms are increasingly being implemented in health care systems to identify patients for specialty care. However, systematic differences in missingness of electronic health record (EHR) data may lead to disparities in identification by CDS algorithms. Objective To examine the availability and comprehensiveness of cancer family history information (FHI) in patients' EHRs by sex, race, Hispanic or Latino ethnicity, and language preference in 2 large health care systems in 2021. Design, Setting, and Participants This retrospective EHR quality improvement study used EHR data from 2 health care systems: University of Utah Health (UHealth) and NYU Langone Health (NYULH). Participants included patients aged 25 to 60 years who had a primary care appointment in the previous 3 years. Data were collected or abstracted from the EHR from December 10, 2020, to October 31, 2021, and analyzed from June 15 to October 31, 2021. Exposures Prior collection of cancer FHI in primary care settings. Main Outcomes and Measures Availability was defined as having any FHI and any cancer FHI in the EHR and was examined at the patient level. Comprehensiveness was defined as whether a cancer family history observation in the EHR specified the type of cancer diagnosed in a family member, the relationship of the family member to the patient, and the age at onset for the family member and was examined at the observation level. Results Among 144 484 patients in the UHealth system, 53.6% were women; 74.4% were non-Hispanic or non-Latino and 67.6% were White; and 83.0% had an English language preference. Among 377 621 patients in the NYULH system, 55.3% were women; 63.2% were non-Hispanic or non-Latino, and 55.3% were White; and 89.9% had an English language preference. Patients from historically medically undeserved groups-specifically, Black vs White patients (UHealth: 17.3% [95% CI, 16.1%-18.6%] vs 42.8% [95% CI, 42.5%-43.1%]; NYULH: 24.4% [95% CI, 24.0%-24.8%] vs 33.8% [95% CI, 33.6%-34.0%]), Hispanic or Latino vs non-Hispanic or non-Latino patients (UHealth: 27.2% [95% CI, 26.5%-27.8%] vs 40.2% [95% CI, 39.9%-40.5%]; NYULH: 24.4% [95% CI, 24.1%-24.7%] vs 31.6% [95% CI, 31.4%-31.8%]), Spanish-speaking vs English-speaking patients (UHealth: 18.4% [95% CI, 17.2%-19.1%] vs 40.0% [95% CI, 39.7%-40.3%]; NYULH: 15.1% [95% CI, 14.6%-15.6%] vs 31.1% [95% CI, 30.9%-31.2%), and men vs women (UHealth: 30.8% [95% CI, 30.4%-31.2%] vs 43.0% [95% CI, 42.6%-43.3%]; NYULH: 23.1% [95% CI, 22.9%-23.3%] vs 34.9% [95% CI, 34.7%-35.1%])-had significantly lower availability and comprehensiveness of cancer FHI (P < .001). Conclusions and Relevance These findings suggest that systematic differences in the availability and comprehensiveness of FHI in the EHR may introduce informative presence bias as inputs to CDS algorithms. The observed differences may also exacerbate disparities for medically underserved groups. System-, clinician-, and patient-level efforts are needed to improve the collection of FHI.
Collapse
Affiliation(s)
- Daniel Chavez-Yenter
- Huntsman Cancer Institute, University of Utah, Salt Lake City
- Department of Communication, University of Utah, Salt Lake City
| | - Melody S. Goodman
- School of Global Public Health, New York University, New York, New York
| | - Yuyu Chen
- School of Global Public Health, New York University, New York, New York
| | - Xiangying Chu
- School of Global Public Health, New York University, New York, New York
| | - Richard L. Bradshaw
- Department of Biomedical Informatics, University of Utah, Salt Lake City
- School of Medicine, University of Utah Health, Salt Lake City, Utah
| | | | | | - Brianne M. Daly
- Huntsman Cancer Institute, University of Utah, Salt Lake City
| | - Michael Flynn
- School of Medicine, University of Utah Health, Salt Lake City, Utah
| | - Amanda Gammon
- Huntsman Cancer Institute, University of Utah, Salt Lake City
| | - Rachel Hess
- Department of Population Health Sciences, University of Utah, Salt Lake City
- Department of Internal Medicine, University of Utah, Salt Lake City
| | - Cecelia Kessler
- Huntsman Cancer Institute, University of Utah, Salt Lake City
| | | | - Devin M. Mann
- Department of Population Health, New York University Grossman School of Medicine, New York University, New York, New York
| | - Rachel Monahan
- Perlmutter Cancer Center, NYU Langone Health, New York, New York
- Department of Population Health, New York University Grossman School of Medicine, New York University, New York, New York
| | - Sara Peel
- Huntsman Cancer Institute, University of Utah, Salt Lake City
| | - Kensaku Kawamoto
- Department of Biomedical Informatics, University of Utah, Salt Lake City
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah, Salt Lake City
| | | | - Saundra S. Buys
- Huntsman Cancer Institute, University of Utah, Salt Lake City
- Department of Internal Medicine, University of Utah, Salt Lake City
| | - Ophira Ginsburg
- Center for Global Health, National Cancer Institute, Rockville, Maryland
| | - Kimberly A. Kaphingst
- Huntsman Cancer Institute, University of Utah, Salt Lake City
- Department of Communication, University of Utah, Salt Lake City
| |
Collapse
|