Clinical considerations when applying machine learning to decision-support tasks versus automation

Artificial Intelligence for Clinical Decision Support in Acute Ischemic Stroke: A Systematic Review

BACKGROUND

Established randomized trial-based parameters for acute ischemic stroke group patients into generic treatment groups, leading to attempts using various artificial intelligence (AI) methods to directly correlate patient characteristics to outcomes and thereby provide decision support to stroke clinicians. We review AI-based clinical decision support systems in the development stage, specifically regarding methodological robustness and constraints for clinical implementation.

METHODS

Our systematic review included full-text English language publications proposing a clinical decision support system using AI techniques for direct decision support in acute ischemic stroke cases in adult patients. We (1) describe data and outcomes used in those systems, (2) estimate the systems' benefits compared with traditional stroke diagnosis and treatment, and (3) reported concordance with reporting standards for AI in healthcare.

RESULTS

One hundred twenty-one studies met our inclusion criteria. Sixty-five were included for full extraction. In our sample, utilized data sources, methods, and reporting practices were highly heterogeneous.

CONCLUSIONS

Our results suggest significant validity threats, dissonance in reporting practices, and challenges to clinical translation. We outline practical recommendations for the successful implementation of AI research in acute ischemic stroke treatment and diagnosis.

Machine learning predicts the short-term requirement for invasive ventilation among Australian critically ill COVID-19 patients

OBJECTIVE(S)

To use machine learning (ML) to predict short-term requirements for invasive ventilation in patients with COVID-19 admitted to Australian intensive care units (ICUs).

DESIGN

A machine learning study within a national ICU COVID-19 registry in Australia.

PARTICIPANTS

Adult patients who were spontaneously breathing and admitted to participating ICUs with laboratory-confirmed COVID-19 from 20 February 2020 to 7 March 2021. Patients intubated on day one of their ICU admission were excluded.

MAIN OUTCOME MEASURES

Six machine learning models predicted the requirement for invasive ventilation by day three of ICU admission from variables recorded on the first calendar day of ICU admission; (1) random forest classifier (RF), (2) decision tree classifier (DT), (3) logistic regression (LR), (4) K neighbours classifier (KNN), (5) support vector machine (SVM), and (6) gradient boosted machine (GBM). Cross-validation was used to assess the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity of machine learning models.

RESULTS

300 ICU admissions collected from 53 ICUs across Australia were included. The median [IQR] age of patients was 59 [50-69] years, 109 (36%) were female and 60 (20%) required invasive ventilation on day two or three. Random forest and Gradient boosted machine were the best performing algorithms, achieving mean (SD) AUCs of 0.69 (0.06) and 0.68 (0.07), and mean sensitivities of 77 (19%) and 81 (17%), respectively.

CONCLUSION

Machine learning can be used to predict subsequent ventilation in patients with COVID-19 who were spontaneously breathing and admitted to Australian ICUs.

AI support for ethical decision-making around resuscitation: proceed with care

Artificial intelligence (AI) systems are increasingly being used in healthcare, thanks to the high level of performance that these systems have proven to deliver. So far, clinical applications have focused on diagnosis and on prediction of outcomes. It is less clear in what way AI can or should support complex clinical decisions that crucially depend on patient preferences. In this paper, we focus on the ethical questions arising from the design, development and deployment of AI systems to support decision-making around cardiopulmonary resuscitation and the determination of a patient's Do Not Attempt to Resuscitate status (also known as code status). The COVID-19 pandemic has made us keenly aware of the difficulties physicians encounter when they have to act quickly in stressful situations without knowing what their patient would have wanted. We discuss the results of an interview study conducted with healthcare professionals in a university hospital aimed at understanding the status quo of resuscitation decision processes while exploring a potential role for AI systems in decision-making around code status. Our data suggest that (1) current practices are fraught with challenges such as insufficient knowledge regarding patient preferences, time pressure and personal bias guiding care considerations and (2) there is considerable openness among clinicians to consider the use of AI-based decision support. We suggest a model for how AI can contribute to improve decision-making around resuscitation and propose a set of ethically relevant preconditions-conceptual, methodological and procedural-that need to be considered in further development and implementation efforts.

Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review

Machine Learning and Artificial Intelligence (AI) more broadly have great immediate and future potential for transforming almost all aspects of medicine. However, in many applications, even outside medicine, a lack of transparency in AI applications has become increasingly problematic. This is particularly pronounced where users need to interpret the output of AI systems. Explainable AI (XAI) provides a rationale that allows users to understand why a system has produced a given output. The output can then be interpreted within a given context. One area that is in great need of XAI is that of Clinical Decision Support Systems (CDSSs). These systems support medical practitioners in their clinic decision-making and in the absence of explainability may lead to issues of under or over-reliance. Providing explanations for how recommendations are arrived at will allow practitioners to make more nuanced, and in some cases, life-saving decisions. The need for XAI in CDSS, and the medical field in general, is amplified by the need for ethical and fair decision-making and the fact that AI trained with historical data can be a reinforcement agent of historical actions and biases that should be uncovered. We performed a systematic literature review of work to-date in the application of XAI in CDSS. Tabular data processing XAI-enabled systems are the most common, while XAI-enabled CDSS for text analysis are the least common in literature. There is more interest in developers for the provision of local explanations, while there was almost a balance between post-hoc and ante-hoc explanations, as well as between model-specific and model-agnostic techniques. Studies reported benefits of the use of XAI such as the fact that it could enhance decision confidence for clinicians, or generate the hypothesis about causality, which ultimately leads to increased trustworthiness and acceptability of the system and potential for its incorporation in the clinical workflow. However, we found an overall distinct lack of application of XAI in the context of CDSS and, in particular, a lack of user studies exploring the needs of clinicians. We propose some guidelines for the implementation of XAI in CDSS and explore some opportunities, challenges, and future research needs.

Predicted Influences of Artificial Intelligence on the Domains of Nursing: Scoping Review

BACKGROUND

Artificial intelligence (AI) is set to transform the health system, yet little research to date has explored its influence on nurses-the largest group of health professionals. Furthermore, there has been little discussion on how AI will influence the experience of person-centered compassionate care for patients, families, and caregivers.

OBJECTIVE

This review aims to summarize the extant literature on the emerging trends in health technologies powered by AI and their implications on the following domains of nursing: administration, clinical practice, policy, and research. This review summarizes the findings from 3 research questions, examining how these emerging trends might influence the roles and functions of nurses and compassionate nursing care over the next 10 years and beyond.

METHODS

Using an established scoping review methodology, MEDLINE, CINAHL, EMBASE, PsycINFO, Cochrane Database of Systematic Reviews, Cochrane Central, Education Resources Information Center, Scopus, Web of Science, and ProQuest databases were searched. In addition to the electronic database searches, a targeted website search was performed to access relevant gray literature. Abstracts and full-text studies were independently screened by 2 reviewers using prespecified inclusion and exclusion criteria. Included articles focused on nursing and digital health technologies that incorporate AI. Data were charted using structured forms and narratively summarized.

RESULTS

A total of 131 articles were retrieved from the scoping review for the 3 research questions that were the focus of this manuscript (118 from database sources and 13 from targeted websites). Emerging AI technologies discussed in the review included predictive analytics, smart homes, virtual health care assistants, and robots. The results indicated that AI has already begun to influence nursing roles, workflows, and the nurse-patient relationship. In general, robots are not viewed as replacements for nurses. There is a consensus that health technologies powered by AI may have the potential to enhance nursing practice. Consequently, nurses must proactively define how person-centered compassionate care will be preserved in the age of AI.

CONCLUSIONS

Nurses have a shared responsibility to influence decisions related to the integration of AI into the health system and to ensure that this change is introduced in a way that is ethical and aligns with core nursing values such as compassionate care. Furthermore, nurses must advocate for patient and nursing involvement in all aspects of the design, implementation, and evaluation of these technologies.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/17490.

Barriers to Creating Scalable Business Models for Digital Health Innovation in Public Systems: Qualitative Case Study

Background

Health systems are increasingly looking toward the private sector to provide digital solutions to address health care demands. Innovation in digital health is largely driven by small- and medium-sized enterprises (SMEs), yet these companies experience significant barriers to entry, especially in public health systems. Complex and fragmented care models, alongside a myriad of relevant stakeholders (eg, purchasers, providers, and producers of health care products), make developing value propositions for digital solutions highly challenging.

Objective

This study aims to identify areas for health system improvement to promote the integration of innovative digital health technologies developed by SMEs.

Methods

This paper qualitatively analyzes a series of case studies to identify health system barriers faced by SMEs developing digital health technologies in Canada and proposed solutions to encourage a more innovative ecosystem. The Women’s College Hospital Institute for Health System Solutions and Virtual Care established a consultation program for SMEs to help them increase their innovation capacity and take their ideas to market. The consultation involved the SME filling out an onboarding form and review of this information by an expert advisory committee using guided considerations, leading to a recommendation report provided to the SME. This paper reports on the characteristics of 25 SMEs who completed the program and qualitatively analyzed their recommendation reports to identify common barriers to digital health innovation.

Results

A total of 2 central themes were identified, each with 3 subthemes. First, a common barrier to system integration was the lack of formal evaluation, with SMEs having limited resources and opportunities to conduct such an evaluation. Second, the health system’s current structure does not create incentives for clinicians to use digital technologies, which threatens the sustainability of SMEs’ business models. SMEs faced significant challenges in engaging users and payers from the public system due to perverse economic incentives. Physicians are compensated by in-person visits, which actively works against the goals of many digital health solutions of keeping patients out of clinics and hospitals.

Conclusions

There is a significant disconnect between the economic incentives that drive clinical behaviors and the use of digital technologies that would benefit patients’ well-being. To encourage the use of digital health technologies, publicly funded health systems need to dedicate funding for the evaluation of digital solutions and streamlined pathways for clinical integration.

Quality of Care Perceived by Older Patients and Caregivers in Integrated Care Pathways With Interviewing Assistance From a Social Robot: Noninferiority Randomized Controlled Trial

Background

Society is facing a global shortage of 17 million health care workers, along with increasing health care demands from a growing number of older adults. Social robots are being considered as solutions to part of this problem.

Objective

Our objective is to evaluate the quality of care perceived by patients and caregivers for an integrated care pathway in an outpatient clinic using a social robot for patient-reported outcome measure (PROM) interviews versus the currently used professional interviews.

Methods

A multicenter, two-parallel-group, nonblinded, randomized controlled trial was used to test for noninferiority of the quality of care delivered through robot-assisted care. The randomization was performed using a computer-generated table. The setting consisted of two outpatient clinics, and the study took place from July to December 2019. Of 419 patients who visited the participating outpatient clinics, 110 older patients met the criteria for recruitment. Inclusion criteria were the ability to speak and read Dutch and being assisted by a participating health care professional. Exclusion criteria were serious hearing or vision problems, serious cognitive problems, and paranoia or similar psychiatric problems. The intervention consisted of a social robot conducting a 36-item PROM. As the main outcome measure, the customized Consumer Quality Index (CQI) was used, as reported by patients and caregivers for the outpatient pathway of care.

Results

In total, 75 intermediately frail older patients were included in the study, randomly assigned to the intervention and control groups, and processed: 36 female (48%) and 39 male (52%); mean age 77.4 years (SD 7.3), range 60-91 years. There was no significant difference in the total patient CQI scores between the patients included in the robot-assisted care pathway (mean 9.27, SD 0.65, n=37) and those in the control group (mean 9.00, SD 0.70, n=38): P=.08, 95% CI –0.04 to 0.58. There was no significant difference in the total CQI scores between caregivers in the intervention group (mean 9.21, SD 0.76, n=30) and those in the control group (mean 9.09, SD 0.60, n=35): P=.47, 95% CI –0.21 to 0.46. No harm or unintended effects occurred.

Conclusions

Geriatric patients and their informal caregivers valued robot-assisted and nonrobot-assisted care pathways equally.

Trial Registration

ClinicalTrials.gov NCT03857789; https://clinicaltrials.gov/ct2/show/NCT03857789

Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study

Background

Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature.

Objective

This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care.

Methods

In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch.

Results

Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch.

Conclusions

Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.

Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study

OBJECTIVE

In this study, we used human factors (HF) methods and principles to design a clinical decision support (CDS) that provides cognitive support to the pulmonary embolism (PE) diagnostic decision-making process in the emergency department. We hypothesised that the application of HF methods and principles will produce a more usable CDS that improves PE diagnostic decision-making, in particular decision about appropriate clinical pathway.

MATERIALS AND METHODS

We conducted a scenario-based simulation study to compare a HF-based CDS (the so-called CDS for PE diagnosis (PE-Dx CDS)) with a web-based CDS (MDCalc); 32 emergency physicians performed various tasks using both CDS. PE-Dx integrated HF design principles such as automating information acquisition and analysis, and minimising workload. We assessed all three dimensions of usability using both objective and subjective measures: effectiveness (eg, appropriate decision regarding the PE diagnostic pathway), efficiency (eg, time spent, perceived workload) and satisfaction (perceived usability of CDS).

RESULTS

Emergency physicians made more appropriate diagnostic decisions (94% with PE-Dx; 84% with web-based CDS; p<0.01) and performed experimental tasks faster with the PE-Dx CDS (on average 96 s per scenario with PE-Dx; 117 s with web-based CDS; p<0.001). They also reported lower workload (p<0.001) and higher satisfaction (p<0.001) with PE-Dx.

CONCLUSIONS

This simulation study shows that HF methods and principles can improve usability of CDS and diagnostic decision-making. Aspects of the HF-based CDS that provided cognitive support to emergency physicians and improved diagnostic performance included automation of information acquisition (eg, auto-populating risk scoring algorithms), minimisation of workload and support of decision selection (eg, recommending a clinical pathway). These HF design principles can be applied to the design of other CDS technologies to improve diagnostic safety.