1
|
Hobensack M, Davoudi A, Song J, Cato K, Bowles KH, Topaz M. Comparing the influence of social risk factors on machine learning model performance across racial and ethnic groups in home healthcare. Nurs Outlook 2025; 73:102431. [PMID: 40339458 DOI: 10.1016/j.outlook.2025.102431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2024] [Revised: 04/12/2025] [Accepted: 04/15/2025] [Indexed: 05/10/2025]
Abstract
This study examined the impact of social risk factors on machine learning model performance for predicting hospitalization and emergency department visits in home healthcare. Using retrospective data from one U.S. home healthcare agency, four models were developed with unstructured social information documented in clinical notes. Performance was compared with and without social factors. A subgroup analyses was conducted by race and ethnicity to assess for fairness. LightGBM performed best overall. Social factors had a modest effect, but findings highlight the feasibility of integrating unstructured social information into machine learning models and the importance of fairness evaluation in home healthcare.
Collapse
Affiliation(s)
| | - Anahita Davoudi
- Center for Home Care Policy & Research, VNS Health, New York, NY
| | - Jiyoun Song
- University of Pennsylvania School of Nursing, Philadelphia, PA
| | - Kenrick Cato
- University of Pennsylvania School of Nursing, Philadelphia, PA; Children's Hospital of Philadelphia, Philadelphia, PA
| | - Kathryn H Bowles
- Center for Home Care Policy & Research, VNS Health, New York, NY; University of Pennsylvania School of Nursing, Philadelphia, PA
| | - Maxim Topaz
- Center for Home Care Policy & Research, VNS Health, New York, NY; Columbia University School of Nursing, New York, NY; Data Science Institute, Columbia University, New York, NY
| |
Collapse
|
2
|
Afshar M, Resnik F, Joyce C, Oguss M, Dligach D, Burnside ES, Sullivan AG, Churpek MM, Patterson BW, Salisbury-Afshar E, Liao FJ, Goswami C, Brown R, Mundt MP. Clinical implementation of AI-based screening for risk for opioid use disorder in hospitalized adults. Nat Med 2025:10.1038/s41591-025-03603-z. [PMID: 40181180 DOI: 10.1038/s41591-025-03603-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2024] [Accepted: 02/21/2025] [Indexed: 04/05/2025]
Abstract
Adults with opioid use disorder (OUD) are at increased risk for opioid-related complications and repeated hospital admissions. Routine screening for patients at risk for an OUD to prevent complications is not standard practice in many hospitals, leading to missed opportunities for intervention. The adoption of electronic health records (EHRs) and advancements in artificial intelligence (AI) offer a scalable approach to systematically identify at-risk patients for evidence-based care. This pre-post quasi-experimental study evaluated whether an AI-driven OUD screener embedded in the EHR was non-inferior to usual care in identifying patients for addiction medicine consultations, aiming to provide a similarly effective but more scalable alternative to human-led ad hoc consultations. The AI screener used a convolutional neural network to analyze EHR notes in real time, identifying patients at risk and recommending consultations. The primary outcome was the proportion of patients who completed a consultation with an addiction medicine specialist, which included interventions such as outpatient treatment referral, management of complicated withdrawal, medication management for OUD and harm reduction services. The study period consisted of a 16-month pre-intervention phase followed by an 8-month post-intervention phase, during which the AI screener was implemented to support hospital providers in identifying patients for consultation. Consultations did not change between periods (1.35% versus 1.51%, P < 0.001 for non-inferiority). In secondary outcome analysis, the AI screener was associated with a reduction in 30-day readmissions (odds ratio: 0.53, 95% confidence interval: 0.30-0.91, P = 0.02) with an incremental cost of US$6,801 per readmission avoided, demonstrating its potential as a scalable, cost-effective solution for OUD care. ClinicalTrials.gov registration: NCT05745480 .
Collapse
Affiliation(s)
- Majid Afshar
- Department of Medicine, University of Wisconsin-Madison, Madison, WI, USA.
| | - Felice Resnik
- Institute for Clinical and Translational Research, University of Wisconsin-Madison, Madison, WI, USA
| | - Cara Joyce
- Department of Public Health Sciences, Loyola University Chicago, Chicago, IL, USA
| | - Madeline Oguss
- Department of Medicine, University of Wisconsin-Madison, Madison, WI, USA
| | - Dmitriy Dligach
- Department of Computer Science, Loyola University Chicago, Chicago, IL, USA
| | - Elizabeth S Burnside
- Institute for Clinical and Translational Research, University of Wisconsin-Madison, Madison, WI, USA
| | - Anne Gravel Sullivan
- Institute for Clinical and Translational Research, University of Wisconsin-Madison, Madison, WI, USA
| | - Matthew M Churpek
- Department of Medicine, University of Wisconsin-Madison, Madison, WI, USA
| | - Brian W Patterson
- Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, USA
| | | | - Frank J Liao
- Information Systems and Informatics, University of Wisconsin Health System, Madison, WI, USA
| | - Cherodeep Goswami
- Information Systems and Informatics, University of Wisconsin Health System, Madison, WI, USA
| | - Randy Brown
- Department of Family Medicine and Community Health, University of Wisconsin-Madison, Madison, WI, USA
| | - Marlon P Mundt
- Department of Family Medicine and Community Health, University of Wisconsin-Madison, Madison, WI, USA
| |
Collapse
|
3
|
Rossetti SC, Dykes PC, Knaplund C, Cho S, Withall J, Lowenthal G, Albers D, Lee RY, Jia H, Bakken S, Kang MJ, Chang FY, Zhou L, Bates DW, Daramola T, Liu F, Schwartz-Dillard J, Tran M, Bokhari SMA, Thate J, Cato KD. Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial. Nat Med 2025:10.1038/s41591-025-03609-7. [PMID: 40175738 DOI: 10.1038/s41591-025-03609-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2024] [Accepted: 02/24/2025] [Indexed: 04/04/2025]
Abstract
The COmmunicating Narrative Concerns Entered by RNs (CONCERN) early warning system (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify deterioration risk. We conducted a 1-year, multisite, pragmatic trial with cluster-randomization of 74 clinical units (37 intervention; 37 usual care) across 2 health systems. Eligible adult hospital encounters were included. We tested if outcomes differed between patients whose care teams were and patients whose care teams were not informed by the CONCERN EWS. Coprimary outcomes were in-hospital mortality (examined as instantaneous risk) and length of stay. Secondary outcomes were cardiopulmonary arrest, sepsis, unanticipated intensive care unit transfers and 30-day hospital readmission. Among 60,893 hospital encounters (33,024 intervention; 27,869 usual care), intervention group encounters had 35.6% decreased instantaneous risk of death (adjusted hazard ratio (HR), 0.64; 95% confidence interval (CI), 0.53-0.78; P < 0.0001), 11.2% decreased length of stay (adjusted incidence rate ratio, 0.91; 95% CI, 0.90-0.93; P < 0.0001), 7.5% decreased instantaneous risk of sepsis (adjusted HR, 0.93; 95% CI, 0.86-0.99; P = 0.0317) and 24.9% increased instantaneous risk of unanticipated intensive care unit transfer (adjusted HR, 1.25; 95% CI, 1.09-1.43; P = 0.0011) compared with usual-care group encounters. No adverse events were reported. A machine learning-based EWS, modeled on nursing surveillance patterns, decreased inpatient deterioration risk with statistical significance. ClinicalTrials.gov registration: NCT03911687 .
Collapse
Affiliation(s)
- Sarah C Rossetti
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA.
- Columbia University Irving Medical Center, School of Nursing, New York, NY, USA.
| | - Patricia C Dykes
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Chris Knaplund
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
| | - Sandy Cho
- Newton-Wellesley Hospital, Newton, MA, USA
| | - Jennifer Withall
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
| | | | - David Albers
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
- University of Colorado, Anschutz Medical Campus, Department of Biomedical Informatics, Aurora, CO, USA
| | - Rachel Y Lee
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
| | - Haomiao Jia
- Columbia University Irving Medical Center, School of Nursing, New York, NY, USA
- Columbia University Irving Medical Center, Mailman School of Public Health, New York, NY, USA
| | - Suzanne Bakken
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
- Columbia University Irving Medical Center, School of Nursing, New York, NY, USA
| | - Min-Jeoung Kang
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | | | - Li Zhou
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - David W Bates
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Temiloluwa Daramola
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
| | - Fang Liu
- University of Pennsylvania, Philadelphia, PA, USA
| | - Jessica Schwartz-Dillard
- Columbia University Irving Medical Center, School of Nursing, New York, NY, USA
- Hospital for Special Surgery, New York, NY, USA
| | - Mai Tran
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA
| | | | | | - Kenrick D Cato
- University of Pennsylvania, Philadelphia, PA, USA
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
| |
Collapse
|
4
|
Tahtali MA, Snijders CCP, Dirne CWGM, Le Blanc PM. Prioritizing Trust in Podiatrists' Preference for AI in Supportive Roles Over Diagnostic Roles in Health Care: Qualitative Interview and Focus Group Study. JMIR Hum Factors 2025; 12:e59010. [PMID: 39983118 PMCID: PMC11890136 DOI: 10.2196/59010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2024] [Revised: 10/18/2024] [Accepted: 12/29/2024] [Indexed: 02/23/2025] Open
Abstract
BACKGROUND As artificial intelligence (AI) evolves, its roles have expanded from helping out with routine tasks to making complex decisions, once the exclusive domain of human experts. This shift is pronounced in health care, where AI aids in tasks ranging from image recognition in radiology to personalized treatment plans, demonstrating the potential to, at times, surpass human accuracy and efficiency. Despite AI's accuracy in some critical tasks, the adoption of AI in health care is a challenge, in part because of skepticism about being able to rely on AI decisions. OBJECTIVE This study aimed to identify and delve into more effective and acceptable ways of integrating AI into a broader spectrum of health care tasks. METHODS We included 2 qualitative phases to explore podiatrists' views on AI in health care. Initially, we interviewed 9 podiatrists (7 women and 2 men) with a mean age of 41 (SD 12) years and aimed to capture their sentiments regarding the use and role of AI in their work. Subsequently, a focus group with 5 podiatrists (4 women and 1 man) with a mean age of 54 (SD 10) years delved into AI's supportive and diagnostic roles on the basis of the interviews. All interviews were recorded, transcribed verbatim, and analyzed using Atlas.ti and QDA-Miner, using both thematic analysis for broad patterns and framework analysis for structured insights per established guidelines. RESULTS Our research unveiled 9 themes and 3 subthemes, clarifying podiatrists' nuanced views on AI in health care. Key overlapping insights in the 2 phases included a preference for using AI in supportive roles, such as triage, because of its efficiency and process optimization capabilities. There is a discernible hesitancy toward leveraging AI for diagnostic purposes, driven by concerns regarding its accuracy and the essential nature of human expertise. The need for transparency and explainability in AI systems emerged as a critical factor for fostering trust in both phases. CONCLUSIONS The findings highlight a complex view from podiatrists on AI, showing openness to its application in supportive roles while exercising caution with diagnostic use. This result is consistent with a careful introduction of AI into health care in roles, such as triage, in which there is initial trust, as opposed to roles that ask the AI for a complete diagnosis. Such strategic adoption can mitigate initial resistance, gradually building the confidence to explore AI's capabilities in more nuanced tasks, including diagnostics, where skepticism is currently more pronounced. Adopting AI stepwise could thus enhance trust and acceptance across a broader range of health care tasks, aligning technology integration with professional comfort and patient care standards.
Collapse
Affiliation(s)
- Mohammed A Tahtali
- Department of Industrial Engineering & Management, Fontys University of Applied Sciences, Eindhoven, The Netherlands
- Department of Industrial Engineering & Innovation Sciences, Human Technology Interaction group, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Chris C P Snijders
- Department of Industrial Engineering & Innovation Sciences, Human Technology Interaction group, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Corné W G M Dirne
- Department of Industrial Engineering & Management, Fontys University of Applied Sciences, Eindhoven, The Netherlands
| | - Pascale M Le Blanc
- Department of Industrial Engineering & Innovation Sciences, Human Performance Management group, Eindhoven University of Technology, Eindhoven, The Netherlands
| |
Collapse
|
5
|
Leivaditis V, Beltsios E, Papatriantafyllou A, Grapatsas K, Mulita F, Kontodimopoulos N, Baikoussis NG, Tchabashvili L, Tasios K, Maroulis I, Dahm M, Koletsis E. Artificial Intelligence in Cardiac Surgery: Transforming Outcomes and Shaping the Future. Clin Pract 2025; 15:17. [PMID: 39851800 PMCID: PMC11763739 DOI: 10.3390/clinpract15010017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2024] [Revised: 01/06/2025] [Accepted: 01/08/2025] [Indexed: 01/26/2025] Open
Abstract
Background: Artificial intelligence (AI) has emerged as a transformative technology in healthcare, with its integration into cardiac surgery offering significant advancements in precision, efficiency, and patient outcomes. However, a comprehensive understanding of AI's applications, benefits, challenges, and future directions in cardiac surgery is needed to inform its safe and effective implementation. Methods: A systematic review was conducted following PRISMA guidelines. Literature searches were performed in PubMed, Scopus, Cochrane Library, Google Scholar, and Web of Science, covering publications from January 2000 to November 2024. Studies focusing on AI applications in cardiac surgery, including risk stratification, surgical planning, intraoperative guidance, and postoperative management, were included. Data extraction and quality assessment were conducted using standardized tools, and findings were synthesized narratively. Results: A total of 121 studies were included in this review. AI demonstrated superior predictive capabilities in risk stratification, with machine learning models outperforming traditional scoring systems in mortality and complication prediction. Robotic-assisted systems enhanced surgical precision and minimized trauma, while computer vision and augmented cognition improved intraoperative guidance. Postoperative AI applications showed potential in predicting complications, supporting patient monitoring, and reducing healthcare costs. However, challenges such as data quality, validation, ethical considerations, and integration into clinical workflows remain significant barriers to widespread adoption. Conclusions: AI has the potential to revolutionize cardiac surgery by enhancing decision making, surgical accuracy, and patient outcomes. Addressing limitations related to data quality, bias, validation, and regulatory frameworks is essential for its safe and effective implementation. Future research should focus on interdisciplinary collaboration, robust testing, and the development of ethical and transparent AI systems to ensure equitable and sustainable advancements in cardiac surgery.
Collapse
Affiliation(s)
- Vasileios Leivaditis
- Department of Cardiothoracic and Vascular Surgery, WestpfalzKlinikum, 67655 Kaiserslautern, Germany; (V.L.); (A.P.); (M.D.)
| | - Eleftherios Beltsios
- Department of Anesthesiology and Intensive Care, Hannover Medical School, 30625 Hannover, Germany;
| | - Athanasios Papatriantafyllou
- Department of Cardiothoracic and Vascular Surgery, WestpfalzKlinikum, 67655 Kaiserslautern, Germany; (V.L.); (A.P.); (M.D.)
| | - Konstantinos Grapatsas
- Department of Thoracic Surgery and Thoracic Endoscopy, Ruhrlandklinik, West German Lung Center, University Hospital Essen, University Duisburg-Essen, 45141 Essen, Germany;
| | - Francesk Mulita
- Department of General Surgery, General University Hospital of Patras, 26504 Patras, Greece; (L.T.); (K.T.)
| | - Nikolaos Kontodimopoulos
- Department of Economics and Sustainable Development, Harokopio University, 17778 Athens, Greece;
| | - Nikolaos G. Baikoussis
- Department of Cardiac Surgery, Ippokrateio General Hospital of Athens, 11527 Athens, Greece;
| | - Levan Tchabashvili
- Department of General Surgery, General University Hospital of Patras, 26504 Patras, Greece; (L.T.); (K.T.)
| | - Konstantinos Tasios
- Department of General Surgery, General University Hospital of Patras, 26504 Patras, Greece; (L.T.); (K.T.)
| | - Ioannis Maroulis
- Department of General Surgery, General University Hospital of Patras, 26504 Patras, Greece; (L.T.); (K.T.)
| | - Manfred Dahm
- Department of Cardiothoracic and Vascular Surgery, WestpfalzKlinikum, 67655 Kaiserslautern, Germany; (V.L.); (A.P.); (M.D.)
| | - Efstratios Koletsis
- Department of Cardiothoracic Surgery, General University Hospital of Patras, 26504 Patras, Greece;
| |
Collapse
|
6
|
Binuya MAE, Linn SC, Boekhout AH, Schmidt MK, Engelhardt EG. Bridging the Gap: A Mixed-Methods Study on Factors Influencing Breast Cancer Clinicians' Decisions to Use Clinical Prediction Models. MDM Policy Pract 2025; 10:23814683251328377. [PMID: 40151468 PMCID: PMC11948560 DOI: 10.1177/23814683251328377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2024] [Accepted: 02/15/2025] [Indexed: 03/29/2025] Open
Abstract
Background. Clinical prediction models provide tailored risk estimates that can help guide decisions in breast cancer care. Despite their potential, few models are widely used in clinical practice. We aimed to identify the factors influencing breast cancer clinicians' decisions to adopt prediction models and assess their relative importance. Methods. We conducted a mixed-methods study, beginning with semi-structured interviews, followed by a nationwide online survey. Thematic analysis was used to qualitatively summarize the interviews and identify key factors. For the survey, we used descriptive analysis to characterize the sample and Mann-Whitney U and Kruskal-Wallis tests to explore differences in score (0 = not important to 10 = very important) distributions. Results. Interviews (N = 16) identified eight key factors influencing model use. Practical/methodological factors included accessibility, cost, understandability, objective accuracy, actionability, and clinical relevance. Perceptual factors included acceptability, subjective accuracy, and risk communication. In the survey (N = 146; 137 model users), clinicians ranked online accessibility (median score = 9 [interquartile range = 8-10]) as most important. Cost was also highly rated, with preferences for freely available models (9 [8-10]) and those with reimbursable tests (8 [8-10]). Formal regulatory approval (7 [5-8]) and direct integration with electronic health records (6 [3-8]) were considered less critical. Subgroup analysis revealed differences in score distributions; for example, clinicians from general hospitals prioritized inclusion of new biomarkers more than those in academic settings. Conclusions. Breast cancer clinicians' decisions to initiate use of prediction models are influenced by practical and perceptual factors, extending beyond technical metrics such as discrimination and calibration. Addressing these factors more holistically through collaborative efforts between model developers, clinicians, and communication and implementation experts, for instance, by developing clinician-friendly online tools that prioritize usability and local adaptability, could increase model uptake. Highlights Accessibility, cost, and practical considerations, such as ease of use and clinical utility, were prioritized slightly more than technical validation metrics, such as discrimination and calibration, when deciding to start using a clinical prediction model.Most breast cancer clinicians valued models with clear inputs (e.g., variable definitions, cutoffs) and outputs; few were interested in the exact model specifications.Perceptual or subjective factors, including perceived accuracy and peer acceptability, also influenced model adoption but were secondary to practical considerations.Sociodemographic variables, such as clinical specialization and hospital setting, influenced the importance of factors for model use.
Collapse
Affiliation(s)
- Mary Ann E. Binuya
- Division of Molecular Pathology, the Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands
- Department of Clinical Genetics, Leiden University Medical Center, Leiden, The Netherlands
| | - Sabine C. Linn
- Division of Molecular Pathology, the Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Division of Medical Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Annelies H. Boekhout
- Division of Psychosocial Research and Epidemiology, the Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - Marjanka K. Schmidt
- Division of Molecular Pathology, the Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Department of Clinical Genetics, Leiden University Medical Center, Leiden, The Netherlands
| | - Ellen G. Engelhardt
- Division of Molecular Pathology, the Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Division of Psychosocial Research and Epidemiology, the Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| |
Collapse
|
7
|
Owoyemi A, Okpara E, Salwei M, Boyd A. End user experience of a widely used artificial intelligence based sepsis system. JAMIA Open 2024; 7:ooae096. [PMID: 39386065 PMCID: PMC11458550 DOI: 10.1093/jamiaopen/ooae096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Revised: 06/27/2024] [Accepted: 09/05/2024] [Indexed: 10/12/2024] Open
Abstract
Objectives Research on the Epic Sepsis System (ESS) has predominantly focused on technical accuracy, neglecting the user experience of healthcare professionals. Understanding these experiences is crucial for the design of Artificial Intelligence (AI) systems in clinical settings. This study aims to explore the socio-technical dynamics affecting ESS adoption and use, based on user perceptions and experiences. Materials and Methods Resident doctors and nurses with recent ESS interaction were interviewed using purposive sampling until data saturation. A content analysis was conducted using Dedoose software, with codes generated from Sittig and Singh's and Salwei and Carayon's frameworks, supplemented by inductive coding for emerging themes. Results Interviews with 10 healthcare providers revealed mixed but generally positive or neutral perceptions of the ESS. Key discussion points included its workflow integration and usability. Findings were organized into 2 main domains: workflow fit, and usability and utility, highlighting the system's seamless electronic health record integration and identifying design gaps. Discussion This study offers insights into clinicians' experiences with the ESS, emphasizing the socio-technical factors that influence its adoption and effective use. The positive reception was tempered by identified design issues, with clinician perceptions varying by their professional experience and frequency of ESS interaction. Conclusion The findings highlight the need for ongoing ESS refinement, emphasizing a balance between technological advancement and clinical practicality. This research contributes to the understanding of AI system adoption in healthcare, suggesting improvements for future clinical AI tools.
Collapse
Affiliation(s)
- Ayomide Owoyemi
- Department of Biomedical and Health Informatics, University of Illinois at Chicago, Chicago, IL 60612, United States
| | - Ebere Okpara
- Department of Pharmacy Systems, Outcomes and Policy, University of Illinois at Chicago, Chicago, IL 60612, United States
| | - Megan Salwei
- Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN 37232, United States
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37232, United States
| | - Andrew Boyd
- Department of Biomedical and Health Informatics, University of Illinois at Chicago, Chicago, IL 60612, United States
| |
Collapse
|
8
|
Preti LM, Ardito V, Compagni A, Petracca F, Cappellaro G. Implementation of Machine Learning Applications in Health Care Organizations: Systematic Review of Empirical Studies. J Med Internet Res 2024; 26:e55897. [PMID: 39586084 PMCID: PMC11629039 DOI: 10.2196/55897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2023] [Revised: 07/07/2024] [Accepted: 10/03/2024] [Indexed: 11/27/2024] Open
Abstract
BACKGROUND There is a growing enthusiasm for machine learning (ML) among academics and health care practitioners. Despite the transformative potential of ML-based applications for patient care, their uptake and implementation in health care organizations are sporadic. Numerous challenges currently impede or delay the widespread implementation of ML in clinical practice, and limited knowledge is available regarding how these challenges have been addressed. OBJECTIVE This work aimed to (1) examine the characteristics of ML-based applications and the implementation process in clinical practice, using the Consolidated Framework for Implementation Research (CFIR) for theoretical guidance and (2) synthesize the strategies adopted by health care organizations to foster successful implementation of ML. METHODS A systematic literature review was conducted based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The search was conducted in PubMed, Scopus, and Web of Science over a 10-year period (2013-2023). The search strategy was built around 4 blocks of keywords (artificial intelligence, implementation, health care, and study type). Only empirical studies documenting the implementation of ML applications in clinical settings were considered. The implementation process was investigated using a thematic analysis and coding procedure. RESULTS Thirty-four studies were selected for data synthesis. Selected papers were relatively recent, with only 9% (3/34) of records published before 2019. ML-based applications were implemented mostly within hospitals (29/34, 85%). In terms of clinical workflow, ML-based applications supported mostly prognosis (20/34, 59%) and diagnosis (10/34, 29%). The implementation efforts were analyzed using CFIR domains. As for the inner setting domain, access to knowledge and information (12/34, 35%), information technology infrastructure (11/34, 32%), and organizational culture (9/34, 26%) were among the most observed dimensions influencing the success of implementation. As for the ML innovation itself, factors deemed relevant were its design (15/34, 44%), the relative advantage with respect to existing clinical practice (14/34, 41%), and perceived complexity (14/34, 41%). As for the other domains (ie, processes, roles, and outer setting), stakeholder engagement (12/34, 35%), reflecting and evaluating practices (11/34, 32%), and the presence of implementation leaders (9/34, 26%) were the main factors identified as important. CONCLUSIONS This review sheds some light on the factors that are relevant and that should be accounted for in the implementation process of ML-based applications in health care. While the relevance of ML-specific dimensions, like trust, emerges clearly across several implementation domains, the evidence from this review highlighted that relevant implementation factors are not necessarily specific for ML but rather transversal for digital health technologies. More research is needed to further clarify the factors that are relevant to implementing ML-based applications at the organizational level and to support their uptake within health care organizations. TRIAL REGISTRATION PROSPERO 403873; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=403873. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.2196/47971.
Collapse
Affiliation(s)
- Luigi M Preti
- Center for Research on Health and Social Care Management (CERGAS), SDA Bocconi School of Management, Milan, Italy
| | - Vittoria Ardito
- Center for Research on Health and Social Care Management (CERGAS), SDA Bocconi School of Management, Milan, Italy
| | - Amelia Compagni
- Center for Research on Health and Social Care Management (CERGAS), SDA Bocconi School of Management, Milan, Italy
- Department of Social and Political Sciences, Bocconi University, Milan, Italy
| | - Francesco Petracca
- Center for Research on Health and Social Care Management (CERGAS), SDA Bocconi School of Management, Milan, Italy
| | - Giulia Cappellaro
- Center for Research on Health and Social Care Management (CERGAS), SDA Bocconi School of Management, Milan, Italy
- Department of Social and Political Sciences, Bocconi University, Milan, Italy
| |
Collapse
|
9
|
Xu Z, Evans L, Song J, Chae S, Davoudi A, Bowles KH, McDonald MV, Topaz M. Exploring home healthcare clinicians' needs for using clinical decision support systems for early risk warning. J Am Med Inform Assoc 2024; 31:2641-2650. [PMID: 39302103 PMCID: PMC11491664 DOI: 10.1093/jamia/ocae247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 07/05/2024] [Accepted: 09/11/2024] [Indexed: 09/22/2024] Open
Abstract
OBJECTIVES To explore home healthcare (HHC) clinicians' needs for Clinical Decision Support Systems (CDSS) information delivery for early risk warning within HHC workflows. METHODS Guided by the CDS "Five-Rights" framework, we conducted semi-structured interviews with multidisciplinary HHC clinicians from April 2023 to August 2023. We used deductive and inductive content analysis to investigate informants' responses regarding CDSS information delivery. RESULTS Interviews with thirteen HHC clinicians yielded 16 codes mapping to the CDS "Five-Rights" framework (right information, right person, right format, right channel, right time) and 11 codes for unintended consequences and training needs. Clinicians favored risk levels displayed in color-coded horizontal bars, concrete risk indicators in bullet points, and actionable instructions in the existing EHR system. They preferred non-intrusive risk alerts requiring mandatory confirmation. Clinicians anticipated risk information updates aligned with patient's condition severity and their visit pace. Additionally, they requested training to understand the CDSS's underlying logic, and raised concerns about information accuracy and data privacy. DISCUSSION While recognizing CDSS's value in enhancing early risk warning, clinicians highlighted concerns about increased workload, alert fatigue, and CDSS misuse. The top risk factors identified by machine learning algorithms, especially text features, can be ambiguous due to a lack of context. Future research should ensure that CDSS outputs align with clinical evidence and are explainable. CONCLUSION This study identified HHC clinicians' expectations, preferences, adaptations, and unintended uses of CDSS for early risk warning. Our findings endorse operationalizing the CDS "Five-Rights" framework to optimize CDSS information delivery and integration into HHC workflows.
Collapse
Affiliation(s)
- Zidu Xu
- School of Nursing, Columbia University, New York, NY 10032, United States
| | - Lauren Evans
- Center for Home Care Policy & Research, VNS Health, New York, NY 10017, United States
| | - Jiyoun Song
- School of Nursing, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Sena Chae
- College of Nursing, The University of Iowa, Iowa City, IA 52242, United States
| | - Anahita Davoudi
- Center for Home Care Policy & Research, VNS Health, New York, NY 10017, United States
| | - Kathryn H Bowles
- Center for Home Care Policy & Research, VNS Health, New York, NY 10017, United States
- School of Nursing, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Margaret V McDonald
- Center for Home Care Policy & Research, VNS Health, New York, NY 10017, United States
| | - Maxim Topaz
- School of Nursing, Columbia University, New York, NY 10032, United States
- Center for Home Care Policy & Research, VNS Health, New York, NY 10017, United States
| |
Collapse
|
10
|
Bedford J, Fields KG, Collins GS, Lip GYH, Clifton DA, O’Brien B, Muehlschlegel JD, Watkinson PJ, Redfern OC. Atrial fibrillation after cardiac surgery: identifying candidate predictors through a Delphi process. BMJ Open 2024; 14:e086589. [PMID: 39322590 PMCID: PMC11425939 DOI: 10.1136/bmjopen-2024-086589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Accepted: 09/02/2024] [Indexed: 09/27/2024] Open
Abstract
OBJECTIVES This study was undertaken to identify potential predictors of atrial fibrillation after cardiac surgery (AFACS) through a modified Delphi process and expert consensus. These will supplement predictors identified through a systematic review and cohort study to inform the development of two AFACS prediction models as part of the PARADISE project (NCT05255224). Atrial fibrillation is a common complication after cardiac surgery. It is associated with worse postoperative outcomes. Reliable prediction of AFACS would enable risk stratification and targeted prevention. Systematic identification of candidate predictors is important to improve validity of AFACS prediction tools. DESIGN This study is a Delphi consensus exercise. SETTING This study was undertaken through remote participation. PARTICIPANTS The participants are an international multidisciplinary panel of experts selected through national research networks. INTERVENTIONS This is a two-stage consensus exercise consisting of generating a long list of variables, followed by refinement by voting and retaining variables selected by at least 40% of panel members. RESULTS The panel comprised 15 experts who participated in both stages, comprising cardiac intensive care physicians (n=3), cardiac anaesthetists (n=2), cardiac surgeons (n=1), cardiologists (n=4), cardiac pharmacists (n=1), critical care nurses (n=1), cardiac nurses (n=1) and patient representatives (n=2). Our Delphi process highlighted candidate AFACS predictors, including both patient factors and those related to the surgical intervention. We generated a final list of 72 candidate predictors. The final list comprised 3 demographic, 29 comorbidity, 4 vital sign, 13 intraoperative, 10 postoperative investigation and 13 postoperative intervention predictors. CONCLUSIONS A Delphi consensus exercise has the potential to highlight predictors beyond the scope of existing literature. This method proved effective in identifying a range of candidate AFACS predictors. Our findings will inform the development of future AFACS prediction tools as part of the larger PARADISE project. TRIAL REGISTRATION NUMBER NCT05255224.
Collapse
Affiliation(s)
- Jonathan Bedford
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Milton Keynes University Hospital NHS Foundation Trust, Milton Keynes, UK
| | - Kara G Fields
- Department of Anesthesiology, Perioperative and Pain Medicine, Harvard Medical School, Boston, Massachusetts, USA
| | | | - Gregory Y H Lip
- Liverpool Centre for Cardiovascular Science, University of Liverpool, Liverpool, Merseyside, UK
- Department of Clinical Medicine, Aalborg University, Aalborg, Region Nordjylland, Denmark
| | - David A Clifton
- Department of Engineering Science, University of Oxford, Oxford, UK
| | - Benjamin O’Brien
- Department of Cardiac Anesthesiology and Intensive Care Medicine, Charité Universitätsmedizin Berlin, Berlin, Germany
- Department of Perioperative Medicine, Barts Health NHS Trust, London, UK
| | - Jochen D Muehlschlegel
- Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, UK
| | - Peter J Watkinson
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Oliver C Redfern
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| |
Collapse
|
11
|
Liao X, Yao C, Jin F, Zhang J, Liu L. Barriers and facilitators to implementing imaging-based diagnostic artificial intelligence-assisted decision-making software in hospitals in China: a qualitative study using the updated Consolidated Framework for Implementation Research. BMJ Open 2024; 14:e084398. [PMID: 39260855 PMCID: PMC11409362 DOI: 10.1136/bmjopen-2024-084398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 08/26/2024] [Indexed: 09/13/2024] Open
Abstract
OBJECTIVES To identify the barriers and facilitators to the successful implementation of imaging-based diagnostic artificial intelligence (AI)-assisted decision-making software in China, using the updated Consolidated Framework for Implementation Research (CFIR) as a theoretical basis to develop strategies that promote effective implementation. DESIGN This qualitative study involved semistructured interviews with key stakeholders from both clinical settings and industry. Interview guide development, coding, analysis and reporting of findings were thoroughly informed by the updated CFIR. SETTING Four healthcare institutions in Beijing and Shanghai and two vendors of AI-assisted decision-making software for lung nodules detection and diabetic retinopathy screening were selected based on purposive sampling. PARTICIPANTS A total of 23 healthcare practitioners, 6 hospital informatics specialists, 4 hospital administrators and 7 vendors of the selected AI-assisted decision-making software were included in the study. RESULTS Within the 5 CFIR domains, 10 constructs were identified as barriers, 8 as facilitators and 3 as both barriers and facilitators. Major barriers included unsatisfactory clinical performance (Innovation); lack of collaborative network between primary and tertiary hospitals, lack of information security measures and certification (outer setting); suboptimal data quality, misalignment between software functions and goals of healthcare institutions (inner setting); unmet clinical needs (individuals). Key facilitators were strong empirical evidence of effectiveness, improved clinical efficiency (innovation); national guidelines related to AI, deployment of AI software in peer hospitals (outer setting); integration of AI software into existing hospital systems (inner setting) and involvement of clinicians (implementation process). CONCLUSIONS The study findings contributed to the ongoing exploration of AI integration in healthcare from the perspective of China, emphasising the need for a comprehensive approach considering both innovation-specific factors and the broader organisational and contextual dynamics. As China and other developing countries continue to advance in adopting AI technologies, the derived insights could further inform healthcare practitioners, industry stakeholders and policy-makers, guiding policies and practices that promote the successful implementation of imaging-based diagnostic AI-assisted decision-making software in healthcare for optimal patient care.
Collapse
Affiliation(s)
- Xiwen Liao
- Peking University First Hospital, Beijing, China
- Clinical Research Institute, Institute of Advanced Clinical Medicine, Peking University, Beijing, China
| | - Chen Yao
- Peking University First Hospital, Beijing, China
- Clinical Research Institute, Institute of Advanced Clinical Medicine, Peking University, Beijing, China
| | - Feifei Jin
- Trauma Medicine Center, Peking University People's Hospital, Beijing, China
- Key Laboratory of Trauma treatment and Neural Regeneration, Peking University, Ministry of Education, Beijing, China
| | - Jun Zhang
- MSD R&D (China) Co., Ltd, Beijing, China
| | - Larry Liu
- Merck & Co Inc, Rahway, New Jersey, USA
- Weill Cornell Medical College, New York City, New York, USA
| |
Collapse
|
12
|
Griffin AC, Wang KH, Leung TI, Facelli JC. Recommendations to promote fairness and inclusion in biomedical AI research and clinical use. J Biomed Inform 2024; 157:104693. [PMID: 39019301 PMCID: PMC11402591 DOI: 10.1016/j.jbi.2024.104693] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Revised: 06/25/2024] [Accepted: 07/14/2024] [Indexed: 07/19/2024]
Abstract
OBJECTIVE Understanding and quantifying biases when designing and implementing actionable approaches to increase fairness and inclusion is critical for artificial intelligence (AI) in biomedical applications. METHODS In this Special Communication, we discuss how bias is introduced at different stages of the development and use of AI applications in biomedical sciences and health care. We describe various AI applications and their implications for fairness and inclusion in sections on 1) Bias in Data Source Landscapes, 2) Algorithmic Fairness, 3) Uncertainty in AI Predictions, 4) Explainable AI for Fairness and Equity, and 5) Sociological/Ethnographic Issues in Data and Results Representation. RESULTS We provide recommendations to address biases when developing and using AI in clinical applications. CONCLUSION These recommendations can be applied to informatics research and practice to foster more equitable and inclusive health care systems and research discoveries.
Collapse
Affiliation(s)
- Ashley C Griffin
- Veterans Affairs Palo Alto Health Care System, Palo Alto, California and Stanford University School of Medicine, Stanford, California, USA.
| | - Karen H Wang
- Department of Internal Medicine and Equity Research and Innovation Center, Yale School of Medicine, USA.
| | - Tiffany I Leung
- Southern Illinois University School of Medicine, Scientific Editorial Director, JMIR Publications, USA.
| | - Julio C Facelli
- Department of Biomedical Informatics and Utah Center for Clinical and Translatinal Science, Spencer Fox Eccles School of Medicine, University of Utah, USA.
| |
Collapse
|
13
|
Johnson R, Li MM, Noori A, Queen O, Zitnik M. Graph Artificial Intelligence in Medicine. Annu Rev Biomed Data Sci 2024; 7:345-368. [PMID: 38749465 PMCID: PMC11344018 DOI: 10.1146/annurev-biodatasci-110723-024625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/23/2024]
Abstract
In clinical artificial intelligence (AI), graph representation learning, mainly through graph neural networks and graph transformer architectures, stands out for its capability to capture intricate relationships and structures within clinical datasets. With diverse data-from patient records to imaging-graph AI models process data holistically by viewing modalities and entities within them as nodes interconnected by their relationships. Graph AI facilitates model transfer across clinical tasks, enabling models to generalize across patient populations without additional parameters and with minimal to no retraining. However, the importance of human-centered design and model interpretability in clinical decision-making cannot be overstated. Since graph AI models capture information through localized neural transformations defined on relational datasets, they offer both an opportunity and a challenge in elucidating model rationale. Knowledge graphs can enhance interpretability by aligning model-driven insights with medical knowledge. Emerging graph AI models integrate diverse data modalities through pretraining, facilitate interactive feedback loops, and foster human-AI collaboration, paving the way toward clinically meaningful predictions.
Collapse
Affiliation(s)
- Ruth Johnson
- Berkowitz Family Living Laboratory, Harvard Medical School, Boston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Michelle M Li
- Bioinformatics and Integrative Genomics Program, Harvard Medical School, Boston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Ayush Noori
- Department of Computer Science, Harvard John A. Paulson School of Engineering and Applied Sciences, Allston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Owen Queen
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Marinka Zitnik
- Harvard Data Science Initiative, Cambridge, Massachusetts, USA
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Allston, Massachusetts, USA
- Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| |
Collapse
|
14
|
Secor AM, Justafort J, Torrilus C, Honoré J, Kiche S, Sandifer TK, Beima-Sofie K, Wagner AD, Pintye J, Puttkammer N. "Following the data": perceptions of and willingness to use clinical decision support tools to inform HIV care among Haitian clinicians. HEALTH POLICY AND TECHNOLOGY 2024; 13:100880. [PMID: 39555144 PMCID: PMC11567668 DOI: 10.1016/j.hlpt.2024.100880] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2024]
Abstract
Background Clinical decision support (CDS) tools can support HIV care, including through case tracking, treatment and medication monitoring, and promoting provider compliance with care guidelines. There has been limited research into the technical, organizational, and behavioral factors that impact perceptions of and willingness to use CDS tools at scale in resource-limited settings, including in Haiti. Methods Our sample included fifteen purposively chosen Haitian HIV program experts, including active clinicians and HIV program managers. Participants completed structured quantitative surveys and one-on-one qualitative semi-structured interviews. Results Study participants had high levels of familiarity and experience with CDS tools. The primary motivator for CDS tool use was a perceived benefit to quality of care, including improved provider time use, efficiency, and decision-making ability, and patient outcomes. Participants highlighted decision-making autonomy and how CDS tools could support provider decision making but should not supplant provider knowledge and experience. Participants highlighted the need for sufficient provider training/sensitization, inclusion of providers in the system design process, and prioritization of tool user-friendliness as key mechanisms to drive tool use and impact. Some participants noted that systemic issues, such as limited laboratory capacity, may reduce the usefulness of CDS alerts, particularly concerning differentiated care and priority viral load testing. Conclusion Respondents had largely positive perceptions of EMRs and CDS tools, particularly due to perceived improvements in quality of care. To improve tool use, stakeholders should prioritize tool user-friendliness and provider training. Addressing systemic health system issues is necessary to unlock the full potential of these tools.
Collapse
Affiliation(s)
- Andrew M Secor
- Department of Global Health, University of Washington, Seattle, WA, USA
| | - John Justafort
- Centre Haïtien pour le Renforcement du Système de Santé (CHARESS), Port-au-Prince, Haiti
| | - Chenet Torrilus
- Centre Haïtien pour le Renforcement du Système de Santé (CHARESS), Port-au-Prince, Haiti
| | - Jean Honoré
- Centre Haïtien pour le Renforcement du Système de Santé (CHARESS), Port-au-Prince, Haiti
| | - Sharon Kiche
- Department of Global Health, University of Washington, Seattle, WA, USA
| | - Tracy K Sandifer
- Department of Epidemiology, University of Washington, Seattle, WA, USA
| | | | - Anjuli D Wagner
- Department of Global Health, University of Washington, Seattle, WA, USA
| | - Jillian Pintye
- Department of Global Health, University of Washington, Seattle, WA, USA
| | - Nancy Puttkammer
- Department of Global Health, University of Washington, Seattle, WA, USA
- International Training and Education Center for Health (I-TECH), Seattle, WA, USA
| |
Collapse
|
15
|
Peek N, Capurro D, Rozova V, van der Veer SN. Bridging the Gap: Challenges and Strategies for the Implementation of Artificial Intelligence-based Clinical Decision Support Systems in Clinical Practice. Yearb Med Inform 2024; 33:103-114. [PMID: 40199296 PMCID: PMC12020628 DOI: 10.1055/s-0044-1800729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/10/2025] Open
Abstract
OBJECTIVES Despite the surge in development of artificial intelligence (AI) algorithms to support clinical decision-making, few of these algorithms are used in practice. We reviewed recent literature on clinical deployment of AI-based clinical decision support systems (AI-CDSS), and assessed the maturity of AI-CDSS implementation research. We also aimed to compare and contrast implementation of rule-based CDSS with implementation of AI-CDSS, and to give recommendations for future research in this area. METHODS We searched PubMed and Scopus for publications in 2022 and 2023 that focused on AI and/or CDSS, health care, and implementation research, and extracted: clinical setting; clinical task; translational research phase; study design; participants; implementation theory, model or framework used; and key findings. RESULTS We selected and described a total of 31 recent papers addressing implementation of AI-CDSS in clinical practice, categorised into four groups: (i) Implementation theories, frameworks, and models (4 papers); (ii) Stakeholder perspectives (22 papers); (iii) Implementation feasibility (three papers); and (iv) Technical infrastructure (2 papers). Stakeholders saw potential benefits of AI-CDSS, but emphasized the need for a strong evidence base and indicated that systems should fit into clinical workflows. There were clear similarities with rule-based CDSS, but also differences with respect to trust and transparency, knowledge, intellectual property, and regulation. CONCLUSIONS The field of AI-CDSS implementation research is still in its infancy. It can be strengthened by grounding studies in established theories, models and frameworks from implementation science, focusing on the perspectives of stakeholder groups other than healthcare professionals, conducting more real-world implementation feasibility studies, and through development of reusable technical infrastructure that facilitates rapid deployment of AI-CDSS in clinical practice.
Collapse
Affiliation(s)
- Niels Peek
- The Healthcare Improvement Studies Institute (THIS Institute), Department of Public Health and Primary Care, University of Cambridge. Cambridge, UK
| | - Daniel Capurro
- Centre for the Digital Transformation of Health, University of Melbourne & The Royal Melbourne Hospital. Melbourne, Australia
| | - Vlada Rozova
- Centre for the Digital Transformation of Health, University of Melbourne. Melbourne, Australia
| | - Sabine N van der Veer
- Centre for Health Informatics, Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester. Manchester, UK
| |
Collapse
|
16
|
Rossetti SC, Dykes PC, Knaplund C, Cho S, Withall J, Lowenthal G, Albers D, Lee R, Jia H, Bakken S, Kang MJ, Chang FY, Zhou L, Bates DW, Daramola T, Liu F, Schwartz-Dillard J, Tran M, Abbas Bokhari SM, Thate J, Cato KD. Multisite Pragmatic Cluster-Randomized Controlled Trial of the CONCERN Early Warning System. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.06.04.24308436. [PMID: 38883706 PMCID: PMC11177900 DOI: 10.1101/2024.06.04.24308436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/18/2024]
Abstract
Importance Late predictions of hospitalized patient deterioration, resulting from early warning systems (EWS) with limited data sources and/or a care team's lack of shared situational awareness, contribute to delays in clinical interventions. The COmmunicating Narrative Concerns Entered by RNs (CONCERN) Early Warning System (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify patients' deterioration risk up to 42 hours earlier than other EWSs. Objective To test our a priori hypothesis that patients with care teams informed by the CONCERN EWS intervention have a lower mortality rate and shorter length of stay (LOS) than the patients with teams not informed by CONCERN EWS. Design One-year multisite, pragmatic controlled clinical trial with cluster-randomization of acute and intensive care units to intervention or usual-care groups. Setting Two large U.S. health systems. Participants Adult patients admitted to acute and intensive care units, excluding those on hospice/palliative/comfort care, or with Do Not Resuscitate/Do Not Intubate orders. Intervention The CONCERN EWS intervention calculates patient deterioration risk based on nurses' concern levels measured by surveillance documentation patterns, and it displays the categorical risk score (low, increased, high) in the electronic health record (EHR) for care team members. Main Outcomes and Measures Primary outcomes: in-hospital mortality, LOS; survival analysis was used. Secondary outcomes: cardiopulmonary arrest, sepsis, unanticipated ICU transfers, 30-day hospital readmission. Results A total of 60 893 hospital encounters (33 024 intervention and 27 869 usual-care) were included. Both groups had similar patient age, race, ethnicity, and illness severity distributions. Patients in the intervention group had a 35.6% decreased risk of death (adjusted hazard ratio [HR], 0.644; 95% confidence interval [CI], 0.532-0.778; P<.0001), 11.2% decreased LOS (adjusted incidence rate ratio, 0.914; 95% CI, 0.902-0.926; P<.0001), 7.5% decreased risk of sepsis (adjusted HR, 0.925; 95% CI, 0.861-0.993; P=.0317), and 24.9% increased risk of unanticipated ICU transfer (adjusted HR, 1.249; 95% CI, 1.093-1.426; P=.0011) compared with patients in the usual-care group. Conclusions and Relevance A hospital-wide EWS based on nursing surveillance patterns decreased in-hospital mortality, sepsis, and LOS when integrated into the care team's EHR workflow. Trial Registration ClinicalTrials.gov Identifier: NCT03911687.
Collapse
Affiliation(s)
- Sarah C Rossetti
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
- Columbia University Irving Medical Center, School of Nursing, New York, NY
| | - Patricia C Dykes
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | - Chris Knaplund
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
| | - Sandy Cho
- Newton Wellesley Hospital, Newton, MA
| | - Jennifer Withall
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
| | | | - David Albers
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
- University of Colorado, Anschutz Medical Campus, Department of Biomedical Informatics
| | - Rachel Lee
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
| | - Haomiao Jia
- Columbia University Irving Medical Center, School of Nursing, New York, NY
- Columbia University Irving Medical Center, Mailman School of Public Health, New York, NY
| | - Suzanne Bakken
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
- Columbia University Irving Medical Center, School of Nursing, New York, NY
| | - Min-Jeoung Kang
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | | | - Li Zhou
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | - David W Bates
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | - Temiloluwa Daramola
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
| | - Fang Liu
- University of Pennsylvania, Philadelphia, PA
| | - Jessica Schwartz-Dillard
- Columbia University Irving Medical Center, School of Nursing, New York, NY
- Hospital for Special Surgery, New York, NY
| | - Mai Tran
- Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY
| | | | | | - Kenrick D Cato
- University of Pennsylvania, Philadelphia, PA
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
| |
Collapse
|
17
|
Bear Don't Walk OJ, Paullada A, Everhart A, Casanova-Perez R, Cohen T, Veinot T. Opportunities for incorporating intersectionality into biomedical informatics. J Biomed Inform 2024; 154:104653. [PMID: 38734158 PMCID: PMC11146624 DOI: 10.1016/j.jbi.2024.104653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2023] [Revised: 04/06/2024] [Accepted: 05/08/2024] [Indexed: 05/13/2024]
Abstract
Many approaches in biomedical informatics (BMI) rely on the ability to define, gather, and manipulate biomedical data to support health through a cyclical research-practice lifecycle. Researchers within this field are often fortunate to work closely with healthcare and public health systems to influence data generation and capture and have access to a vast amount of biomedical data. Many informaticists also have the expertise to engage with stakeholders, develop new methods and applications, and influence policy. However, research and policy that explicitly seeks to address the systemic drivers of health would more effectively support health. Intersectionality is a theoretical framework that can facilitate such research. It holds that individual human experiences reflect larger socio-structural level systems of privilege and oppression, and cannot be truly understood if these systems are examined in isolation. Intersectionality explicitly accounts for the interrelated nature of systems of privilege and oppression, providing a lens through which to examine and challenge inequities. In this paper, we propose intersectionality as an intervention into how we conduct BMI research. We begin by discussing intersectionality's history and core principles as they apply to BMI. We then elaborate on the potential for intersectionality to stimulate BMI research. Specifically, we posit that our efforts in BMI to improve health should address intersectionality's five key considerations: (1) systems of privilege and oppression that shape health; (2) the interrelated nature of upstream health drivers; (3) the nuances of health outcomes within groups; (4) the problematic and power-laden nature of categories that we assign to people in research and in society; and (5) research to inform and support social change.
Collapse
Affiliation(s)
- Oliver J Bear Don't Walk
- Department of Biomedical Informatics and Medical Education, University of Washington, United States.
| | - Amandalynne Paullada
- Department of Biomedical Informatics and Medical Education, University of Washington, United States
| | - Avery Everhart
- Department of Geography, Faculty of Arts, University of British Columbia, Canada
| | - Reggie Casanova-Perez
- Department of Biomedical Informatics and Medical Education, University of Washington, United States
| | - Trevor Cohen
- Department of Biomedical Informatics and Medical Education, University of Washington, United States
| | - Tiffany Veinot
- School of Information and School of Public Health, University of Michigan, United States
| |
Collapse
|
18
|
Kamboj N, Metcalfe K, Chu CH, Conway A. Designing the User Interface of a Nitroglycerin Dose Titration Decision Support System: User-Centered Design Study. Appl Clin Inform 2024; 15:583-599. [PMID: 39048084 PMCID: PMC11268987 DOI: 10.1055/s-0044-1787755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 05/14/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND Nurses adjust intravenous nitroglycerin infusions to provide acute relief for angina by manually increasing or decreasing the dosage. However, titration can pose challenges, as excessively high doses can lead to hypotension, and low doses may result in inadequate pain relief. Clinical decision support systems (CDSSs) that predict changes in blood pressure for nitroglycerin dose adjustments may assist nurses with titration. OBJECTIVE This study aimed to design a user interface for a CDSS for nitroglycerin dose titration (Nitroglycerin Dose Titration Decision Support System [nitro DSS]). METHODS A user-centered design (UCD) approach, consisting of an initial qualitative study with semistructured interviews to identify design specifications for prototype development, was used. This was followed by three iterative rounds of usability testing. Nurses with experience titrating nitroglycerin infusions in coronary care units participated. RESULTS A total of 20 nurses participated, including 7 during the qualitative study and 15 during usability testing (2 nurses participated in both phases). Analysis of the qualitative data revealed four themes for the interface design to be (1) clear and consistent, (2) vigilant, (3) interoperable, and (4) reliable. The major elements of the final prototype included a feature for viewing the predicted and actual blood pressure over time to determine the reliability of the predictions, a drop-down option to report patient side effects, a feature to report reasons for not accepting the prediction, and a visual alert indicating any systolic blood pressure predictions below 90 mm Hg. Nurses' ratings on the questionnaires indicated excellent usability and acceptability of the final nitro DSS prototype. CONCLUSION This study successfully applied a UCD approach to collaborate with nurses in developing a user interface for the nitro DSS that supports the clinical decision-making of nurses titrating nitroglycerin.
Collapse
Affiliation(s)
- Navpreet Kamboj
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada
| | - Kelly Metcalfe
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada
- Women's College Hospital Research and Innovation Institute, Toronto, Canada
| | - Charlene H. Chu
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, Canada
| | - Aaron Conway
- School of Nursing, Queensland University of Technology (QUT), Brisbane, Australia
| |
Collapse
|
19
|
Zhang T, Chung T, Dey A, Bae SW. Exploring Algorithmic Explainability: Generating Explainable AI Insights for Personalized Clinical Decision Support Focused on Cannabis Intoxication in Young Adults. 2024 INTERNATIONAL CONFERENCE ON ACTIVITY AND BEHAVIOR COMPUTING 2024; 2024:10.1109/abc61795.2024.10652070. [PMID: 39600343 PMCID: PMC11586775 DOI: 10.1109/abc61795.2024.10652070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/29/2024]
Abstract
As an increasing number of states adopt more permissive cannabis regulations, the necessity of gaining a comprehensive understanding of cannabis's effects on young adults has grown exponentially, driven by its escalating prevalence of use. By leveraging popular eXplainable Artificial Intelligence (XAI) techniques such as SHAP (SHapley Additive exPlanations), rule-based explanations, intrinsically interpretable models, and counterfactual explanations, we undertake an exploratory but in-depth examination of the impact of cannabis use on individual behavioral patterns and physiological states. This study explores the possibility of facilitating algorithmic decision-making by combining interpretable artificial intelligence (XAI) techniques with sensor data, with the aim of providing researchers and clinicians with personalized analyses of cannabis intoxication behavior. SHAP analyzes the importance and quantifies the impact of specific factors such as environmental noise or heart rate, enabling clinicians to pinpoint influential behaviors and environmental conditions. SkopeRules simplify the understanding of cannabis use for a specific activity or environmental use. Decision trees provide a clear visualization of how factors interact to influence cannabis consumption. Counterfactual models help identify key changes in behaviors or conditions that may alter cannabis use outcomes, to guide effective individualized intervention strategies. This multidimensional analytical approach not only unveils changes in behavioral and physiological states after cannabis use, such as frequent fluctuations in activity states, nontraditional sleep patterns, and specific use habits at different times and places, but also highlights the significance of individual differences in responses to cannabis use. These insights carry profound implications for clinicians seeking to gain a deeper understanding of the diverse needs of their patients and for tailoring precisely targeted intervention strategies. Furthermore, our findings highlight the pivotal role that XAI technologies could play in enhancing the transparency and interpretability of Clinical Decision Support Systems (CDSS), with a particular focus on substance misuse treatment. This research significantly contributes to ongoing initiatives aimed at advancing clinical practices that aim to prevent and reduce cannabis-related harms to health, positioning XAI as a supportive tool for clinicians and researchers alike.
Collapse
Affiliation(s)
- Tongze Zhang
- Stevens Institute of Technology, Hoboken, New Jersey
| | | | - Anind Dey
- University of Washington, Seattle, Washington
| | - Sang Won Bae
- Stevens Institute of Technology, Hoboken, New Jersey
| |
Collapse
|
20
|
Berkhout M, Smit K, Versendaal J. Decision discovery using clinical decision support system decision log data for supporting the nurse decision-making process. BMC Med Inform Decis Mak 2024; 24:100. [PMID: 38637792 PMCID: PMC11025262 DOI: 10.1186/s12911-024-02486-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 03/18/2024] [Indexed: 04/20/2024] Open
Abstract
BACKGROUND Decision-making in healthcare is increasingly complex; notably in hospital environments where the information density is high, e.g., emergency departments, oncology departments, and psychiatry departments. This study aims to discover decisions from logged data to improve the decision-making process. METHODS The Design Science Research Methodology (DSRM) was chosen to design an artifact (algorithm) for the discovery and visualization of decisions. The DSRM's different activities are explained, from the definition of the problem to the evaluation of the artifact. During the design and development activities, the algorithm itself is created. During the demonstration and evaluation activities, the algorithm was tested with an authentic synthetic dataset. RESULTS The results show the design and simulation of an algorithm for the discovery and visualization of decisions. A fuzzy classifier algorithm was adapted for (1) discovering decisions from a decision log and (2) visualizing the decisions using the Decision Model and Notation standard. CONCLUSIONS In this paper, we show that decisions can be discovered from a decision log and visualized for the improvement of the decision-making process of healthcare professionals or to support the periodic evaluation of protocols and guidelines.
Collapse
Affiliation(s)
- Matthijs Berkhout
- Digital Ethics, HU University of Applied Sciences Utrecht, Heidelberglaan 15, Utrecht, 3584 CS, The Netherlands.
| | - Koen Smit
- Digital Ethics, HU University of Applied Sciences Utrecht, Heidelberglaan 15, Utrecht, 3584 CS, The Netherlands
| | - Johan Versendaal
- Digital Ethics, HU University of Applied Sciences Utrecht, Heidelberglaan 15, Utrecht, 3584 CS, The Netherlands
- Open University of the Netherlands, Valkenburgerweg 177, Heerlen, 6419 AT, The Netherlands
| |
Collapse
|
21
|
Fritz BA, Pugazenthi S, Budelier TP, Tellor Pennington BR, King CR, Avidan MS, Abraham J. User-Centered Design of a Machine Learning Dashboard for Prediction of Postoperative Complications. Anesth Analg 2024; 138:804-813. [PMID: 37339083 PMCID: PMC10730770 DOI: 10.1213/ane.0000000000006577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/22/2023]
Abstract
BACKGROUND Machine learning models can help anesthesiology clinicians assess patients and make clinical and operational decisions, but well-designed human-computer interfaces are necessary for machine learning model predictions to result in clinician actions that help patients. Therefore, the goal of this study was to apply a user-centered design framework to create a user interface for displaying machine learning model predictions of postoperative complications to anesthesiology clinicians. METHODS Twenty-five anesthesiology clinicians (attending anesthesiologists, resident physicians, and certified registered nurse anesthetists) participated in a 3-phase study that included (phase 1) semistructured focus group interviews and a card sorting activity to characterize user workflows and needs; (phase 2) simulated patient evaluation incorporating a low-fidelity static prototype display interface followed by a semistructured interview; and (phase 3) simulated patient evaluation with concurrent think-aloud incorporating a high-fidelity prototype display interface in the electronic health record. In each phase, data analysis included open coding of session transcripts and thematic analysis. RESULTS During the needs assessment phase (phase 1), participants voiced that (a) identifying preventable risk related to modifiable risk factors is more important than nonpreventable risk, (b) comprehensive patient evaluation follows a systematic approach that relies heavily on the electronic health record, and (c) an easy-to-use display interface should have a simple layout that uses color and graphs to minimize time and energy spent reading it. When performing simulations using the low-fidelity prototype (phase 2), participants reported that (a) the machine learning predictions helped them to evaluate patient risk, (b) additional information about how to act on the risk estimate would be useful, and (c) correctable problems related to textual content existed. When performing simulations using the high-fidelity prototype (phase 3), usability problems predominantly related to the presentation of information and functionality. Despite the usability problems, participants rated the system highly on the System Usability Scale (mean score, 82.5; standard deviation, 10.5). CONCLUSIONS Incorporating user needs and preferences into the design of a machine learning dashboard results in a display interface that clinicians rate as highly usable. Because the system demonstrates usability, evaluation of the effects of implementation on both process and clinical outcomes is warranted.
Collapse
Affiliation(s)
- Bradley A Fritz
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO
| | - Sangami Pugazenthi
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO
| | - Thaddeus P Budelier
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO
| | | | - Christopher R King
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO
| | - Joanna Abraham
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO
- Institute for Informatics, Washington University School of Medicine, St. Louis, MO
| |
Collapse
|
22
|
Barwise AK, Curtis S, Diedrich DA, Pickering BW. Using artificial intelligence to promote equitable care for inpatients with language barriers and complex medical needs: clinical stakeholder perspectives. J Am Med Inform Assoc 2024; 31:611-621. [PMID: 38099504 PMCID: PMC10873784 DOI: 10.1093/jamia/ocad224] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 11/14/2023] [Indexed: 02/18/2024] Open
Abstract
OBJECTIVES Inpatients with language barriers and complex medical needs suffer disparities in quality of care, safety, and health outcomes. Although in-person interpreters are particularly beneficial for these patients, they are underused. We plan to use machine learning predictive analytics to reliably identify patients with language barriers and complex medical needs to prioritize them for in-person interpreters. MATERIALS AND METHODS This qualitative study used stakeholder engagement through semi-structured interviews to understand the perceived risks and benefits of artificial intelligence (AI) in this domain. Stakeholders included clinicians, interpreters, and personnel involved in caring for these patients or for organizing interpreters. Data were coded and analyzed using NVIVO software. RESULTS We completed 49 interviews. Key perceived risks included concerns about transparency, accuracy, redundancy, privacy, perceived stigmatization among patients, alert fatigue, and supply-demand issues. Key perceived benefits included increased awareness of in-person interpreters, improved standard of care and prioritization for interpreter utilization; a streamlined process for accessing interpreters, empowered clinicians, and potential to overcome clinician bias. DISCUSSION This is the first study that elicits stakeholder perspectives on the use of AI with the goal of improved clinical care for patients with language barriers. Perceived benefits and risks related to the use of AI in this domain, overlapped with known hazards and values of AI but some benefits were unique for addressing challenges with providing interpreter services to patients with language barriers. CONCLUSION Artificial intelligence to identify and prioritize patients for interpreter services has the potential to improve standard of care and address healthcare disparities among patients with language barriers.
Collapse
Affiliation(s)
- Amelia K Barwise
- Biomedical Ethics Research Program, Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN 55902, United States
| | - Susan Curtis
- Biomedical Ethics Research Program, Mayo Clinic, Rochester, MN 55902, United States
| | - Daniel A Diedrich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN 55902, United States
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN 55902, United States
| |
Collapse
|
23
|
Giddings R, Joseph A, Callender T, Janes SM, van der Schaar M, Sheringham J, Navani N. Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review. Lancet Digit Health 2024; 6:e131-e144. [PMID: 38278615 DOI: 10.1016/s2589-7500(23)00241-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 10/20/2023] [Accepted: 11/14/2023] [Indexed: 01/28/2024]
Abstract
Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.
Collapse
Affiliation(s)
- Rebecca Giddings
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK.
| | - Anabel Joseph
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Thomas Callender
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Sam M Janes
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Mihaela van der Schaar
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK; The Alan Turing Institute, London, UK
| | - Jessica Sheringham
- Department of Applied Health Research, University College London, London, UK
| | - Neal Navani
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| |
Collapse
|
24
|
Shevtsova D, Ahmed A, Boot IWA, Sanges C, Hudecek M, Jacobs JJL, Hort S, Vrijhoef HJM. Trust in and Acceptance of Artificial Intelligence Applications in Medicine: Mixed Methods Study. JMIR Hum Factors 2024; 11:e47031. [PMID: 38231544 PMCID: PMC10831593 DOI: 10.2196/47031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Revised: 09/25/2023] [Accepted: 11/20/2023] [Indexed: 01/18/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI)-powered technologies are being increasingly used in almost all fields, including medicine. However, to successfully implement medical AI applications, ensuring trust and acceptance toward such technologies is crucial for their successful spread and timely adoption worldwide. Although AI applications in medicine provide advantages to the current health care system, there are also various associated challenges regarding, for instance, data privacy, accountability, and equity and fairness, which could hinder medical AI application implementation. OBJECTIVE The aim of this study was to identify factors related to trust in and acceptance of novel AI-powered medical technologies and to assess the relevance of those factors among relevant stakeholders. METHODS This study used a mixed methods design. First, a rapid review of the existing literature was conducted, aiming to identify various factors related to trust in and acceptance of novel AI applications in medicine. Next, an electronic survey including the rapid review-derived factors was disseminated among key stakeholder groups. Participants (N=22) were asked to assess on a 5-point Likert scale (1=irrelevant to 5=relevant) to what extent they thought the various factors (N=19) were relevant to trust in and acceptance of novel AI applications in medicine. RESULTS The rapid review (N=32 papers) yielded 110 factors related to trust and 77 factors related to acceptance toward AI technology in medicine. Closely related factors were assigned to 1 of the 19 overarching umbrella factors, which were further grouped into 4 categories: human-related (ie, the type of institution AI professionals originate from), technology-related (ie, the explainability and transparency of AI application processes and outcomes), ethical and legal (ie, data use transparency), and additional factors (ie, AI applications being environment friendly). The categorized 19 umbrella factors were presented as survey statements, which were evaluated by relevant stakeholders. Survey participants (N=22) represented researchers (n=18, 82%), technology providers (n=5, 23%), hospital staff (n=3, 14%), and policy makers (n=3, 14%). Of the 19 factors, 16 (84%) human-related, technology-related, ethical and legal, and additional factors were considered to be of high relevance to trust in and acceptance of novel AI applications in medicine. The patient's gender, age, and education level were found to be of low relevance (3/19, 16%). CONCLUSIONS The results of this study could help the implementers of medical AI applications to understand what drives trust and acceptance toward AI-powered technologies among key stakeholders in medicine. Consequently, this would allow the implementers to identify strategies that facilitate trust in and acceptance of medical AI applications among key stakeholders and potential users.
Collapse
Affiliation(s)
- Daria Shevtsova
- Panaxea bv, Den Bosch, Netherlands
- Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | | | | | | | | | | | - Simon Hort
- Fraunhofer Institute for Production Technology, Aachen, Germany
| | | |
Collapse
|
25
|
Högberg C, Larsson S, Lång K. Engaging with artificial intelligence in mammography screening: Swedish breast radiologists' views on trust, information and expertise. Digit Health 2024; 10:20552076241287958. [PMID: 39381821 PMCID: PMC11459539 DOI: 10.1177/20552076241287958] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Accepted: 09/10/2024] [Indexed: 10/10/2024] Open
Abstract
Objectives Lack of trust and transparency is stressed as a challenge for clinical implementation of artificial intelligence (AI). In breast cancer screening, AI-supported reading shows promising results but more research is needed on how medical experts, which are facing the integration of AI into their work, reason about trust and information needs. From a sociotechnical information practice perspective, we add to this knowledge by a Swedish case study. This study aims to: (1) clarify Swedish breast radiologists' views on trust, information and expertise pertaining to AI in mammography screening and (2) analytically address ideas about medical professionals' critical engagement with AI and motivations for trust in AI. Method An online survey was distributed to Swedish breast radiologists. Survey responses were analysed by descriptive statistical method, correlation analysis and qualitative content analysis. The results were used as foundation for analysing trust and information as parts of critical engagements with AI. Results Of the Swedish breast radiologists (n = 105), 47 answered the survey (response rate = 44.8%). 53.2% (n = 25) of the respondents would to a high/somewhat high degree trust AI assessments. To a great extent, additional information would support the respondents' trust evaluations. What type of critical engagement medical professionals are expected to perform on AI as decision support remains unclear. Conclusions There is a demand for enhanced information, explainability and transparency of AI-supported mammography. Further discussion and agreement are needed considering what the desired goals for trust in AI should be and how it relates to medical professionals' critical evaluation of AI-made claims in medical decision support.
Collapse
Affiliation(s)
- Charlotte Högberg
- Department of Technology and Society, Faculty of Engineering, Lund University, Lund, Sweden
| | - Stefan Larsson
- Department of Technology and Society, Faculty of Engineering, Lund University, Lund, Sweden
| | - Kristina Lång
- Department of Translational Medicine, Diagnostic Radiology, Lund University, Lund, Sweden
- Unilabs Mammography Unit, Skane University Hospital, Malmö, Sweden
| |
Collapse
|
26
|
Bergquist M, Rolandsson B, Gryska E, Laesser M, Hoefling N, Heckemann R, Schneiderman JF, Björkman-Burtscher IM. Trust and stakeholder perspectives on the implementation of AI tools in clinical radiology. Eur Radiol 2024; 34:338-347. [PMID: 37505245 PMCID: PMC10791850 DOI: 10.1007/s00330-023-09967-5] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Revised: 04/22/2023] [Accepted: 05/26/2023] [Indexed: 07/29/2023]
Abstract
OBJECTIVES To define requirements that condition trust in artificial intelligence (AI) as clinical decision support in radiology from the perspective of various stakeholders and to explore ways to fulfil these requirements. METHODS Semi-structured interviews were conducted with twenty-five respondents-nineteen directly involved in the development, implementation, or use of AI applications in radiology and six working with AI in other areas of healthcare. We designed the questions to explore three themes: development and use of AI, professional decision-making, and management and organizational procedures connected to AI. The transcribed interviews were analysed in an iterative coding process from open coding to theoretically informed thematic coding. RESULTS We identified four aspects of trust that relate to reliability, transparency, quality verification, and inter-organizational compatibility. These aspects fall under the categories of substantial and procedural requirements. CONCLUSIONS Development of appropriate levels of trust in AI in healthcare is complex and encompasses multiple dimensions of requirements. Various stakeholders will have to be involved in developing AI solutions for healthcare and radiology to fulfil these requirements. CLINICAL RELEVANCE STATEMENT For AI to achieve advances in radiology, it must be given the opportunity to support, rather than replace, human expertise. Support requires trust. Identification of aspects and conditions for trust allows developing AI implementation strategies that facilitate advancing the field. KEY POINTS • Dimensions of procedural and substantial demands that need to be fulfilled to foster appropriate levels of trust in AI in healthcare are conditioned on aspects related to reliability, transparency, quality verification, and inter-organizational compatibility. •Creating the conditions for trust to emerge requires the involvement of various stakeholders, who will have to compensate the problem's inherent complexity by finding and promoting well-defined solutions.
Collapse
Affiliation(s)
- Magnus Bergquist
- School of Information Technology, Halmstad University, Halmstad, Sweden
| | - Bertil Rolandsson
- Department of Sociology and Work Science, University of Gothenburg, Gothenburg, Sweden
- Department of Sociology, Lund University, Lund, Sweden
| | - Emilia Gryska
- Department of Radiology, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
| | - Mats Laesser
- Department of Radiology, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Department of Radiology, Sahlgrenska University Hospital, Region Västra Götaland, Gothenburg, Sweden
| | - Nickoleta Hoefling
- Department of Radiology, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Department of Radiology, Sahlgrenska University Hospital, Region Västra Götaland, Gothenburg, Sweden
| | - Rolf Heckemann
- Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Justin F Schneiderman
- Department of Clinical Neuroscience, Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Isabella M Björkman-Burtscher
- Department of Radiology, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Department of Radiology, Sahlgrenska University Hospital, Region Västra Götaland, Gothenburg, Sweden
| |
Collapse
|
27
|
Magrabi F, Lyell D, Coiera E. Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings. Yearb Med Inform 2023; 32:115-126. [PMID: 38147855 PMCID: PMC10751141 DOI: 10.1055/s-0043-1768733] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2023] Open
Abstract
AIMS AND OBJECTIVES To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings. METHOD PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy. Reported effects on user experience, decision-making, care delivery and outcomes were summarised. RESULTS AI technologies are being applied in a wide variety of clinical areas. Most contemporary systems utilise deep learning, use routinely collected data, support diagnosis and triage, are assistive (requiring users to confirm or approve AI provided information or decisions), and are used by doctors in acute care settings in high-income nations. AI systems are integrated and used within existing clinical information systems including electronic medical records. There is limited support for One Health goals. Evaluation is largely based on quantitative methods measuring effects on decision-making. CONCLUSION AI systems are being implemented and evaluated in many clinical areas. There remain many opportunities to understand patterns of routine use and evaluate effects on decision-making, care delivery and patient outcomes using mixed-methods. Support for One Health including integrating data about environmental factors and social determinants needs further exploration.
Collapse
Affiliation(s)
- Farah Magrabi
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - David Lyell
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - Enrico Coiera
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| |
Collapse
|
28
|
Choi A, Choi SY, Chung K, Chung HS, Song T, Choi B, Kim JH. Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department. Sci Rep 2023; 13:8561. [PMID: 37237057 DOI: 10.1038/s41598-023-35617-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 05/21/2023] [Indexed: 05/28/2023] Open
Abstract
This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.
Collapse
Affiliation(s)
- Arom Choi
- Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
- Institute for Innovation in Digital Healthcare, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - So Yeon Choi
- Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - Kyungsoo Chung
- Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
- Institute for Innovation in Digital Healthcare, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - Hyun Soo Chung
- Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - Taeyoung Song
- LG Electronics, 128 Yeoui-daero, Yeongdeungpo-gu, Seoul, 07336, Republic of Korea
| | - Byunghun Choi
- LG Electronics, 128 Yeoui-daero, Yeongdeungpo-gu, Seoul, 07336, Republic of Korea
| | - Ji Hoon Kim
- Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
- Institute for Innovation in Digital Healthcare, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
| |
Collapse
|
29
|
van der Vegt AH, Scott IA, Dermawan K, Schnetler RJ, Kalke VR, Lane PJ. Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. J Am Med Inform Assoc 2023:7161075. [PMID: 37172264 DOI: 10.1093/jamia/ocad075] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 04/04/2023] [Accepted: 04/23/2023] [Indexed: 05/14/2023] Open
Abstract
OBJECTIVE To retrieve and appraise studies of deployed artificial intelligence (AI)-based sepsis prediction algorithms using systematic methods, identify implementation barriers, enablers, and key decisions and then map these to a novel end-to-end clinical AI implementation framework. MATERIALS AND METHODS Systematically review studies of clinically applied AI-based sepsis prediction algorithms in regard to methodological quality, deployment and evaluation methods, and outcomes. Identify contextual factors that influence implementation and map these factors to the SALIENT implementation framework. RESULTS The review identified 30 articles of algorithms applied in adult hospital settings, with 5 studies reporting significantly decreased mortality post-implementation. Eight groups of algorithms were identified, each sharing a common algorithm. We identified 14 barriers, 26 enablers, and 22 decision points which were able to be mapped to the 5 stages of the SALIENT implementation framework. DISCUSSION Empirical studies of deployed sepsis prediction algorithms demonstrate their potential for improving care and reducing mortality but reveal persisting gaps in existing implementation guidance. In the examined publications, key decision points reflecting real-word implementation experience could be mapped to the SALIENT framework and, as these decision points appear to be AI-task agnostic, this framework may also be applicable to non-sepsis algorithms. The mapping clarified where and when barriers, enablers, and key decisions arise within the end-to-end AI implementation process. CONCLUSIONS A systematic review of real-world implementation studies of sepsis prediction algorithms was used to validate an end-to-end staged implementation framework that has the ability to account for key factors that warrant attention in ensuring successful deployment, and which extends on previous AI implementation frameworks.
Collapse
Affiliation(s)
- Anton H van der Vegt
- Queensland Digital Health Centre, The University of Queensland, Brisbane, Queensland, Australia
| | - Ian A Scott
- Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Australia
| | - Krishna Dermawan
- Centre for Information Resilience, The University of Queensland, St Lucia, Australia
| | - Rudolf J Schnetler
- School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Australia
| | - Vikrant R Kalke
- Patient Safety and Quality, Clinical Excellence Queensland, Queensland Health, Brisbane, Australia
| | - Paul J Lane
- Safety Quality & Innovation, The Prince Charles Hospital, Queensland Health, Brisbane, Australia
| |
Collapse
|
30
|
Kim JH, Kim B, Kim MJ, Hyun H, Kim HC, Chang HJ. Prediction of inappropriate pre-hospital transfer of patients with suspected cardiovascular emergency diseases using machine learning: a retrospective observational study. BMC Med Inform Decis Mak 2023; 23:56. [PMID: 37024872 PMCID: PMC10080868 DOI: 10.1186/s12911-023-02149-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 03/15/2023] [Indexed: 04/08/2023] Open
Abstract
BACKGROUND This study aimed to develop a prediction model for transferring patients to an inappropriate hospital for suspected cardiovascular emergency diseases at the pre-hospital stage, using variables obtained from an integrated nationwide dataset, and to assess the performance of this model. METHODS We integrated three nationwide datasets and developed a two-step prediction model utilizing a machine learning algorithm. Ninety-eight clinical characteristics of patients identified at the pre-hospital stage and 13 hospital components were used as input data for the model. The primary endpoint of the model was the prediction of transfer to an inappropriate hospital. RESULTS A total of 94,256 transferred patients in the public pre-hospital care system matched the National Emergency Department Information System data of patients with a pre-hospital cardiovascular registry created in South Korea between July 2017 and December 2018. Of these, 1,770 (6.26%) patients failed to be transferred to a capable hospital. The area under the receiver operating characteristic curve of the final predictive model was 0.813 (0.800-0.825), and the area under the receiver precision-recall curve was 0.286 (0.265-0.308). CONCLUSIONS Our prediction model used machine learning to show favorable performance in transferring patients with suspected cardiovascular disease to a capable hospital. For our results to lead to changes in the pre-hospital care system, a digital platform for sharing real-time information should be developed.
Collapse
Affiliation(s)
- Ji Hoon Kim
- Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - Bomgyeol Kim
- Department of Preventive Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - Min Joung Kim
- Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea
| | - Heejung Hyun
- AITRICS, 28 Hyoryeong-ro 77-gil, Seocho-gu, Seoul, 06627, Republic of Korea
| | - Hyeon Chang Kim
- Department of Preventive Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
| | - Hyuk-Jae Chang
- Department of Cardiology, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun- gu, Seoul, 03722, Republic of Korea
| |
Collapse
|