Clinical use of artificial intelligence requires AI-capable organizations

Opportunities, challenges, and requirements for Artificial Intelligence (AI) implementation in Primary Health Care (PHC): a systematic review

BACKGROUND

Artificial Intelligence (AI) has significantly reshaped Primary Health Care (PHC), offering various possibilities and complexities across all functional dimensions. The objective is to review and synthesize available evidence on the opportunities, challenges, and requirements of AI implementation in PHC based on the Primary Care Evaluation Tool (PCET).

METHODS

We conducted a systematic review, following the Cochrane Collaboration method, to identify the latest evidence regarding AI implementation in PHC. A comprehensive search across eight databases- PubMed, Web of Science, Scopus, Science Direct, Embase, CINAHL, IEEE, and Cochrane was conducted using MeSH terms alongside the SPIDER framework to pinpoint quantitative and qualitative literature published from 2000 to 2024. Two reviewers independently applied inclusion and exclusion criteria, guided by the SPIDER framework, to review full texts and extract data. We synthesized extracted data from the study characteristics, opportunities, challenges, and requirements, employing thematic-framework analysis, according to the PCET model. The quality of the studies was evaluated using the JBI critical appraisal tools.

RESULTS

In this review, we included a total of 109 articles, most of which were conducted in North America (n = 49, 44%), followed by Europe (n = 36, 33%). The included studies employed a diverse range of study designs. Using the PCET model, we categorized AI-related opportunities, challenges, and requirements across four key dimensions. The greatest opportunities for AI integration in PHC were centered on enhancing comprehensive service delivery, particularly by improving diagnostic accuracy, optimizing screening programs, and advancing early disease prediction. However, the most challenges emerged within the stewardship and resource generation functions, with key concerns related to data security and privacy, technical performance issues, and limitations in data accessibility. Ensuring successful AI integration requires a robust stewardship function, strategic investments in resource generation, and a collaborative approach that fosters co-development, scientific advancements, and continuous evaluation.

CONCLUSIONS

Successful AI integration in PHC requires a coordinated, multidimensional approach, with stewardship, resource generation, and financing playing key roles in enabling service delivery. Addressing existing knowledge gaps, examining interactions among these dimensions, and fostering a collaborative approach in developing AI solutions among stakeholders are essential steps toward achieving an equitable and efficient AI-driven PHC system.

PROTOCOL

Registered in Open Science Framework (OSF) ( https://doi.org/10.17605/OSF.IO/HG2DV ).

AI-driven report-generation tools in mental healthcare: A review of commercial tools

Artificial intelligence (AI) systems are increasingly being integrated in clinical care, including for AI-powered note-writing. We aimed to develop and apply a scale for assessing mental health electronic health records (EHRs) that use large language models (LLMs) for note-writing, focusing on their features, security, and ethics. The assessment involved analyzing product information and directly querying vendors about their systems. On their websites, the majority of vendors provided comprehensive information on data protection, privacy measures, multi-platform availability, patient access features, software update history, and Meaningful Use compliance. Most products clearly indicated the LLM's capabilities in creating customized reports or functioning as a co-pilot. However, critical information was often absent, including details on LLM training methodologies, the specific LLM used, bias correction techniques, and methods for evaluating the evidence base. The lack of transparency regarding LLM specifics and bias mitigation strategies raises concerns about the ethical implementation and reliability of these systems in clinical practice. While LLM-enhanced EHRs show promise in alleviating the documentation burden for mental health professionals, there is a pressing need for greater transparency and standardization in reporting LLM-related information. We propose recommendations for the future development and implementation of these systems to ensure they meet the highest standards of security, ethics, and clinical care.

Approaches to identify scenarios for data science implementations within healthcare settings: recommendations based on experiences at multiple academic institutions

Objectives

To describe successful and unsuccessful approaches to identify scenarios for data science implementations within healthcare settings and to provide recommendations for future scenario identification procedures.

Materials and methods

Representatives from seven Toronto academic healthcare institutions participated in a one-day workshop. Each institution was asked to provide an introduction to their clinical data science program and to provide an example of a successful and unsuccessful approach to scenario identification at their institution. Using content analysis, common observations were summarized.

Results

Observations were coalesced to idea generation and value proposition, prioritization, approval and champions. Successful experiences included promoting a portfolio of ideas, articulating value proposition, ensuring alignment with organization priorities, ensuring approvers can adjudicate feasibility and identifying champions willing to take ownership over the projects.

Conclusion

Based on academic healthcare data science program experiences, we provided recommendations for approaches to identify scenarios for data science implementations within healthcare settings.

Factors Associated With the Accuracy of Large Language Models in Basic Medical Science Examinations: Cross-Sectional Study

Background

Artificial intelligence (AI) has become widely applied across many fields, including medical education. Content validation and its answers are based on training datasets and the optimization of each model. The accuracy of large language model (LLMs) in basic medical examinations and factors related to their accuracy have also been explored.

Objective

We evaluated factors associated with the accuracy of LLMs (GPT-3.5, GPT-4, Google Bard, and Microsoft Bing) in answering multiple-choice questions from basic medical science examinations.

Methods

We used questions that were closely aligned with the content and topic distribution of Thailand's Step 1 National Medical Licensing Examination. Variables such as the difficulty index, discrimination index, and question characteristics were collected. These questions were then simultaneously input into ChatGPT (with GPT-3.5 and GPT-4), Microsoft Bing, and Google Bard, and their responses were recorded. The accuracy of these LLMs and the associated factors were analyzed using multivariable logistic regression. This analysis aimed to assess the effect of various factors on model accuracy, with results reported as odds ratios (ORs).

Results

The study revealed that GPT-4 was the top-performing model, with an overall accuracy of 89.07% (95% CI 84.76%-92.41%), significantly outperforming the others (P<.001). Microsoft Bing followed with an accuracy of 83.69% (95% CI 78.85%-87.80%), GPT-3.5 at 67.02% (95% CI 61.20%-72.48%), and Google Bard at 63.83% (95% CI 57.92%-69.44%). The multivariable logistic regression analysis showed a correlation between question difficulty and model performance, with GPT-4 demonstrating the strongest association. Interestingly, no significant correlation was found between model accuracy and question length, negative wording, clinical scenarios, or the discrimination index for most models, except for Google Bard, which showed varying correlations.

Conclusions

The GPT-4 and Microsoft Bing models demonstrated equal and superior accuracy compared to GPT-3.5 and Google Bard in the domain of basic medical science. The accuracy of these models was significantly influenced by the item's difficulty index, indicating that the LLMs are more accurate when answering easier questions. This suggests that the more accurate models, such as GPT-4 and Bing, can be valuable tools for understanding and learning basic medical science concepts.

ChatGPT vs Expert-Guided Care Pathways for Postesophagectomy Symptom Management

Background

The objective of this study was to compare generative artificial intelligence-initiated care pathways, using ChatGPT, with expert-guided consensus-initiated care pathways from AskMayoExpert (AME) for symptom management of esophageal cancer patients after esophagectomy.

Methods

A formal protocol for development of 9 AME care pathways was followed for specific patient-identified domains after esophagectomy for esophageal cancer. Domain scores were measured and assessed through the Upper Digestive Disease tool. These care pathways were developed by experts validated by a consensus-driven methodology. ChatGPT was used to answer specific questions similar to the AME care pathway on April 9, 2023, and March 28, 2024. To compare outcomes, answers were recorded, and algorithms were compared with a survey tool composed of 5 questions.

Results

Both modalities were able to provide a clear definition with multidirectional management options for all 9 domains: dysphagia, generalized dumping, gastrointestinal dumping, pain, regurgitation, heartburn, nausea, physical health, and mental health. When provided with a simple prompt, ChatGPT 3.5 failed to provide a comprehensive stepwise approach for providers, any testing recommendations, or any form of triage process. However, ChatGPT 4.0 provided plans, similar to AME care pathways, when a sophisticated prompt was used.

Conclusions

Generative artificial intelligence-initiated care pathways can be used by physicians as a supplementary tool to guide provider management of patients with complex symptoms after esophagectomy. This technology will continue to advance but is currently insufficient to solely guide clinical management of complex patients with severe symptoms.

Addressing ethical issues in healthcare artificial intelligence using a lifecycle-informed process

Objectives

Artificial intelligence (AI) proceeds through an iterative and evaluative process of development, use, and refinement which may be characterized as a lifecycle. Within this context, stakeholders can vary in their interests and perceptions of the ethical issues associated with this rapidly evolving technology in ways that can fail to identify and avert adverse outcomes. Identifying issues throughout the AI lifecycle in a systematic manner can facilitate better-informed ethical deliberation.

Materials and Methods

We analyzed existing lifecycles from within the current literature for ethical issues of AI in healthcare to identify themes, which we relied upon to create a lifecycle that consolidates these themes into a more comprehensive lifecycle. We then considered the potential benefits and harms of AI through this lifecycle to identify ethical questions that can arise at each step and to identify where conflicts and errors could arise in ethical analysis. We illustrated the approach in 3 case studies that highlight how different ethical dilemmas arise at different points in the lifecycle.

Results Discussion and Conclusion

Through case studies, we show how a systematic lifecycle-informed approach to the ethical analysis of AI enables mapping of the effects of AI onto different steps to guide deliberations on benefits and harms. The lifecycle-informed approach has broad applicability to different stakeholders and can facilitate communication on ethical issues for patients, healthcare professionals, research participants, and other stakeholders.

Artificial intelligence predictive analytics in heart failure: results of the pilot phase of a pragmatic randomized clinical trial

OBJECTIVES

We conducted an implementation planning process during the pilot phase of a pragmatic trial, which tests an intervention guided by artificial intelligence (AI) analytics sourced from noninvasive monitoring data in heart failure patients (LINK-HF2).

MATERIALS AND METHODS

A mixed-method analysis was conducted at 2 pilot sites. Interviews were conducted with 12 of 27 enrolled patients and with 13 participating clinicians. iPARIHS constructs were used for interview construction to identify workflow, communication patterns, and clinician's beliefs. Interviews were transcribed and analyzed using inductive coding protocols to identify key themes. Behavioral response data from the AI-generated notifications were collected.

RESULTS

Clinicians responded to notifications within 24 hours in 95% of instances, with 26.7% resulting in clinical action. Four implementation themes emerged: (1) High anticipatory expectations for reliable patient communications, reduced patient burden, and less proactive provider monitoring. (2) The AI notifications required a differential and tailored balance of trust and action advice related to role. (3) Clinic experience with other home-based programs influenced utilization. (4) Responding to notifications involved significant effort, including electronic health record (EHR) review, patient contact, and consultation with other clinicians.

DISCUSSION

Clinician's use of AI data is a function of beliefs regarding the trustworthiness and usefulness of the data, the degree of autonomy in professional roles, and the cognitive effort involved.

CONCLUSION

The implementation planning analysis guided development of strategies that addressed communication technology, patient education, and EHR integration to reduce clinician and patient burden in the subsequent main randomized phase of the trial. Our results provide important insights into the unique implications of implementing AI analytics into clinical workflow.

Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review

BACKGROUND

Generative artificial intelligence tools and applications (GenAI) are being increasingly used in health care. Physicians, specialists, and other providers have started primarily using GenAI as an aid or tool to gather knowledge, provide information, train, or generate suggestive dialogue between physicians and patients or between physicians and patients' families or friends. However, unless the use of GenAI is oriented to be helpful in clinical service encounters that can improve the accuracy of diagnosis, treatment, and patient outcomes, the expected potential will not be achieved. As adoption continues, it is essential to validate the effectiveness of the infusion of GenAI as an intelligent technology in service encounters to understand the gap in actual clinical service use of GenAI.

OBJECTIVE

This study synthesizes preliminary evidence on how GenAI assists, guides, and automates clinical service rendering and encounters in health care The review scope was limited to articles published in peer-reviewed medical journals.

METHODS

We screened and selected 0.38% (161/42,459) of articles published between January 1, 2020, and May 31, 2023, identified from PubMed. We followed the protocols outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select highly relevant studies with at least 1 element on clinical use, evaluation, and validation to provide evidence of GenAI use in clinical services. The articles were classified based on their relevance to clinical service functions or activities using the descriptive and analytical information presented in the articles.

RESULTS

Of 161 articles, 141 (87.6%) reported using GenAI to assist services through knowledge access, collation, and filtering. GenAI was used for disease detection (19/161, 11.8%), diagnosis (14/161, 8.7%), and screening processes (12/161, 7.5%) in the areas of radiology (17/161, 10.6%), cardiology (12/161, 7.5%), gastrointestinal medicine (4/161, 2.5%), and diabetes (6/161, 3.7%). The literature synthesis in this study suggests that GenAI is mainly used for diagnostic processes, improvement of diagnosis accuracy, and screening and diagnostic purposes using knowledge access. Although this solves the problem of knowledge access and may improve diagnostic accuracy, it is oriented toward higher value creation in health care.

CONCLUSIONS

GenAI informs rather than assisting or automating clinical service functions in health care. There is potential in clinical service, but it has yet to be actualized for GenAI. More clinical service-level evidence that GenAI is used to streamline some functions or provides more automated help than only information retrieval is needed. To transform health care as purported, more studies related to GenAI applications must automate and guide human-performed services and keep up with the optimism that forward-thinking health care organizations will take advantage of GenAI.

Generative AI in healthcare: an implementation science informed translational path on application, integration and governance

BACKGROUND

Artificial intelligence (AI), particularly generative AI, has emerged as a transformative tool in healthcare, with the potential to revolutionize clinical decision-making and improve health outcomes. Generative AI, capable of generating new data such as text and images, holds promise in enhancing patient care, revolutionizing disease diagnosis and expanding treatment options. However, the utility and impact of generative AI in healthcare remain poorly understood, with concerns around ethical and medico-legal implications, integration into healthcare service delivery and workforce utilisation. Also, there is not a clear pathway to implement and integrate generative AI in healthcare delivery.

METHODS

This article aims to provide a comprehensive overview of the use of generative AI in healthcare, focusing on the utility of the technology in healthcare and its translational application highlighting the need for careful planning, execution and management of expectations in adopting generative AI in clinical medicine. Key considerations include factors such as data privacy, security and the irreplaceable role of clinicians' expertise. Frameworks like the technology acceptance model (TAM) and the Non-Adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) model are considered to promote responsible integration. These frameworks allow anticipating and proactively addressing barriers to adoption, facilitating stakeholder participation and responsibly transitioning care systems to harness generative AI's potential.

RESULTS

Generative AI has the potential to transform healthcare through automated systems, enhanced clinical decision-making and democratization of expertise with diagnostic support tools providing timely, personalized suggestions. Generative AI applications across billing, diagnosis, treatment and research can also make healthcare delivery more efficient, equitable and effective. However, integration of generative AI necessitates meticulous change management and risk mitigation strategies. Technological capabilities alone cannot shift complex care ecosystems overnight; rather, structured adoption programs grounded in implementation science are imperative.

CONCLUSIONS

It is strongly argued in this article that generative AI can usher in tremendous healthcare progress, if introduced responsibly. Strategic adoption based on implementation science, incremental deployment and balanced messaging around opportunities versus limitations helps promote safe, ethical generative AI integration. Extensive real-world piloting and iteration aligned to clinical priorities should drive development. With conscientious governance centred on human wellbeing over technological novelty, generative AI can enhance accessibility, affordability and quality of care. As these models continue advancing rapidly, ongoing reassessment and transparent communication around their strengths and weaknesses remain vital to restoring trust, realizing positive potential and, most importantly, improving patient outcomes.

Comprehensiveness, Accuracy, and Readability of Exercise Recommendations Provided by an AI-Based Chatbot: Mixed Methods Study

BACKGROUND

Regular physical activity is critical for health and disease prevention. Yet, health care providers and patients face barriers to implement evidence-based lifestyle recommendations. The potential to augment care with the increased availability of artificial intelligence (AI) technologies is limitless; however, the suitability of AI-generated exercise recommendations has yet to be explored.

OBJECTIVE

The purpose of this study was to assess the comprehensiveness, accuracy, and readability of individualized exercise recommendations generated by a novel AI chatbot.

METHODS

A coding scheme was developed to score AI-generated exercise recommendations across ten categories informed by gold-standard exercise recommendations, including (1) health condition-specific benefits of exercise, (2) exercise preparticipation health screening, (3) frequency, (4) intensity, (5) time, (6) type, (7) volume, (8) progression, (9) special considerations, and (10) references to the primary literature. The AI chatbot was prompted to provide individualized exercise recommendations for 26 clinical populations using an open-source application programming interface. Two independent reviewers coded AI-generated content for each category and calculated comprehensiveness (%) and factual accuracy (%) on a scale of 0%-100%. Readability was assessed using the Flesch-Kincaid formula. Qualitative analysis identified and categorized themes from AI-generated output.

RESULTS

AI-generated exercise recommendations were 41.2% (107/260) comprehensive and 90.7% (146/161) accurate, with the majority (8/15, 53%) of inaccuracy related to the need for exercise preparticipation medical clearance. Average readability level of AI-generated exercise recommendations was at the college level (mean 13.7, SD 1.7), with an average Flesch reading ease score of 31.1 (SD 7.7). Several recurring themes and observations of AI-generated output included concern for liability and safety, preference for aerobic exercise, and potential bias and direct discrimination against certain age-based populations and individuals with disabilities.

CONCLUSIONS

There were notable gaps in the comprehensiveness, accuracy, and readability of AI-generated exercise recommendations. Exercise and health care professionals should be aware of these limitations when using and endorsing AI-based technologies as a tool to support lifestyle change involving exercise.

Experiences of using artificial intelligence in healthcare: a qualitative study of UK clinician and key stakeholder perspectives

OBJECTIVES

Artificial intelligence (AI) is a rapidly developing field in healthcare, with tools being developed across various specialties to support healthcare professionals and reduce workloads. It is important to understand the experiences of professionals working in healthcare to ensure that future AI tools are acceptable and effectively implemented. The aim of this study was to gain an in-depth understanding of the experiences and perceptions of UK healthcare workers and other key stakeholders about the use of AI in the National Health Service (NHS).

DESIGN

A qualitative study using semistructured interviews conducted remotely via MS Teams. Thematic analysis was carried out.

SETTING

NHS and UK higher education institutes.

PARTICIPANTS

Thirteen participants were recruited, including clinical and non-clinical participants working for the NHS and researchers working to develop AI tools for healthcare settings.

RESULTS

Four core themes were identified: positive perceptions of AI; potential barriers to using AI in healthcare; concerns regarding AI use and steps needed to ensure the acceptability of future AI tools. Overall, we found that those working in healthcare were generally open to the use of AI and expected it to have many benefits for patients and facilitate access to care. However, concerns were raised regarding the security of patient data, the potential for misdiagnosis and that AI could increase the burden on already strained healthcare staff.

CONCLUSION

This study found that healthcare staff are willing to engage with AI research and incorporate AI tools into care pathways. Going forward, the NHS and AI developers will need to collaborate closely to ensure that future tools are suitable for their intended use and do not negatively impact workloads or patient trust. Future AI studies should continue to incorporate the views of key stakeholders to improve tool acceptability.

TRIAL REGISTRATION NUMBER

NCT05028179; ISRCTN15113915; IRAS ref: 293515.