The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support

Use of artificial intelligence in critical care: opportunities and obstacles

BACKGROUND

Perhaps nowhere else in the healthcare system than in the intensive care unit environment are the challenges to create useful models with direct time-critical clinical applications more relevant and the obstacles to achieving those goals more massive. Machine learning-based artificial intelligence (AI) techniques to define states and predict future events are commonplace activities of modern life. However, their penetration into acute care medicine has been slow, stuttering and uneven. Major obstacles to widespread effective application of AI approaches to the real-time care of the critically ill patient exist and need to be addressed.

MAIN BODY

Clinical decision support systems (CDSSs) in acute and critical care environments support clinicians, not replace them at the bedside. As will be discussed in this review, the reasons are many and include the immaturity of AI-based systems to have situational awareness, the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address and technical barriers to the timely access to valid data and its display in a fashion useful for clinical workflow. The inherent "black-box" nature of many predictive algorithms and CDSS makes trustworthiness and acceptance by the medical community difficult. Logistically, collating and curating in real-time multidimensional data streams of various sources needed to inform the algorithms and ultimately display relevant clinical decisions support format that adapt to individual patient responses and signatures represent the efferent limb of these systems and is often ignored during initial validation efforts. Similarly, legal and commercial barriers to the access to many existing clinical databases limit studies to address fairness and generalizability of predictive models and management tools.

CONCLUSIONS

AI-based CDSS are evolving and are here to stay. It is our obligation to be good shepherds of their use and further development.

Implementation of an Electronic Clinical Decision Support System for the Early Recognition and Management of Dysglycemia in an Inpatient Mental Health Setting Using CogStack: Protocol for a Pilot Hybrid Type 3 Effectiveness-Implementation Randomized Controlled Cluster Trial

BACKGROUND

Severe mental illnesses (SMIs), including schizophrenia, bipolar affective disorder, and major depressive disorder, are associated with an increased risk of physical health comorbidities and premature mortality from conditions including cardiovascular disease and diabetes. Digital technologies such as electronic clinical decision support systems (eCDSSs) could play a crucial role in improving the clinician-led management of conditions such as dysglycemia (deranged blood sugar levels) and associated conditions such as diabetes in people with a diagnosis of SMI in mental health settings.

OBJECTIVE

We have developed a real-time eCDSS using CogStack, an information retrieval and extraction platform, to automatically alert clinicians with National Health Service Trust-approved, guideline-based recommendations for dysglycemia monitoring and management in secondary mental health care. This novel system aims to improve the management of dysglycemia and associated conditions, such as diabetes, in SMI. This protocol describes a pilot study to explore the acceptability, feasibility, and evaluation of its implementation in a mental health inpatient setting.

METHODS

This will be a pilot hybrid type 3 effectiveness-implementation randomized controlled cluster trial in inpatient mental health wards. A ward will be the unit of recruitment, where it will be randomly allocated to receive either access to the eCDSS plus usual care or usual care alone over a 4-month period. We will measure implementation outcomes, including the feasibility and acceptability of the eCDSS to clinicians, as primary outcomes, alongside secondary outcomes relating to the process of care measures such as dysglycemia screening rates. An evaluation of other implementation outcomes relating to the eCDSS will be conducted, identifying facilitators and barriers based on established implementation science frameworks.

RESULTS

Enrollment of wards began in April 2022, after which clinical staff were recruited to take part in surveys and interviews. The intervention period of the trial began in February 2023, and subsequent data collection was completed in August 2023. Data are currently being analyzed, and results are expected to be available in June 2024.

CONCLUSIONS

An eCDSS can have the potential to improve clinician-led management of dysglycemia in inpatient mental health settings. If found to be feasible and acceptable, then, in combination with the results of the implementation evaluation, the system can be refined and improved to support future successful implementation. A larger and more definitive effectiveness trial should then be conducted to assess its impact on clinical outcomes and to inform scalability and application to other conditions in wider mental health care settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04792268; https://clinicaltrials.gov/study/NCT04792268.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/49548.

Developing and Testing the Usability of a Novel Child Abuse Clinical Decision Support System: Mixed Methods Study

BACKGROUND

Despite the impact of physical abuse on children, it is often underdiagnosed, especially among children evaluated in emergency departments (EDs). Electronic clinical decision support (CDS) can improve the recognition of child physical abuse.

OBJECTIVE

We aimed to develop and test the usability of a natural language processing-based child abuse CDS system, known as the Child Abuse Clinical Decision Support (CA-CDS), to alert ED clinicians about high-risk injuries suggestive of abuse in infants' charts.

METHODS

Informed by available evidence, a multidisciplinary team, including an expert in user design, developed the CA-CDS prototype that provided evidence-based recommendations for the evaluation and management of suspected child abuse when triggered by documentation of a high-risk injury. Content was customized for medical versus nursing providers and initial versus subsequent exposure to the alert. To assess the usability of and refine the CA-CDS, we interviewed 24 clinicians from 4 EDs about their interactions with the prototype. Interview transcripts were coded and analyzed using conventional content analysis.

RESULTS

Overall, 5 main categories of themes emerged from the study. CA-CDS benefits included providing an extra layer of protection, providing evidence-based recommendations, and alerting the entire clinical ED team. The user-centered, workflow-compatible design included soft-stop alert configuration, editable and automatic documentation, and attention-grabbing formatting. Recommendations for improvement included consolidating content, clearer design elements, and adding a hyperlink with additional resources. Barriers to future implementation included alert fatigue, hesitancy to change, and concerns regarding documentation. Facilitators of future implementation included stakeholder buy-in, provider education, and sharing the test characteristics. On the basis of user feedback, iterative modifications were made to the prototype.

CONCLUSIONS

With its user-centered design and evidence-based content, the CA-CDS can aid providers in the real-time recognition and evaluation of infant physical abuse and has the potential to reduce the number of missed cases.

Using Clinical Decision Support Systems to Decrease Intravenous Acetaminophen Use: Implementation and Lessons Learned

BACKGROUND

Clinical decision support systems (CDSS) can enhance medical decision-making by providing targeted information to providers. While they have the potential to improve quality of care and reduce costs, they are not universally effective and can lead to unintended harm.

OBJECTIVES

To describe the implementation of an unsuccessful interruptive CDSS that aimed to promote appropriate use of intravenous (IV) acetaminophen at an academic pediatric hospital, with an emphasis on lessons learned.

METHODS

Quality improvement methodology was used to study the effect of an interruptive CDSS, which set a mandatory expiry time of 24 hours for all IV acetaminophen orders. This CDSS was implemented on April 5, 2021. The primary outcome measure was number of IV acetaminophen administrations per 1,000 patient days, measured pre- and postimplementation. Process measures were the number of IV acetaminophen orders placed per 1,000 patient days. Balancing measures were collected via survey data and included provider and nursing acceptability and unintended consequences of the CDSS.

RESULTS

There was no special cause variation in hospital-wide IV acetaminophen administrations and orders after CDSS implementation, nor when the CDSS was removed. A total of 88 participants completed the survey. Nearly half (40/88) of respondents reported negative issues with the CDSS, with the majority stating that this affected patient care (39/40). Respondents cited delays in patient care and reduced efficiency as the most common negative effects.

CONCLUSION

This study underscores the significance of monitoring CDSS implementations and including end user acceptability as an outcome measure. Teams should be prepared to modify or remove CDSS that do not achieve their intended goal or are associated with low end user acceptability. CDSS holds promise for improving clinical practice, but careful implementation and ongoing evaluation are crucial for maximizing their benefits and minimizing potential harm.

An Electronic Medical Record Intervention to Increase Pharmacologic Prophylaxis for Venous Thromboembolism in Emergency Department Observation Patients

STUDY OBJECTIVE

The role of venous thromboembolism (VTE) prophylaxis among patients receiving emergency department (ED) observation unit care is unclear. We investigated an electronic health record-based clinical decision support tool aimed at increasing pharmacologic VTE prophylaxis use among at-risk patients placed in ED observation units.

METHODS

We conducted an interrupted time-series study of an Epic-based best practice advisory implemented in May 2019 at a health care system comprising 2 academic medical centers and 4 community hospitals with dedicated ED observation units. The best practice advisory alerted staff at 24 hours to conduct a risk assessment and linked to a VTE prophylaxis order set. We used an interrupted time series, Bayesian structured time series, and a multivariable mixed-effect regression model to estimate the intervention effect.

RESULTS

Prior to the best practice advisory implementation, there were 8,895 ED observation unit patients with a length of stay more than or equal to 24 hours, and 0.9% received pharmacologic VTE prophylaxis. Afterward, there were 12,664 ED observation unit patients with a length of stay more than or equal to 24 hours, and 4.8% received pharmacologic VTE prophylaxis. The interrupted time series and causal impact analysis showed a statistically significant increase in VTE prophylaxis (eg, absolute percent difference 3.8%, 95% confidence interval 3.5 to 4.1). In a multivariable model, only the intervention was significantly associated with receiving VTE prophylaxis (odds ratio 4.56, 95% confidence interval 2.22 to 9.37).

CONCLUSION

An electronic health record-based alert helped to prompt staff caring for ED observation unit patients at risk for VTE with prolonged visits to order recommended pharmacologic prophylaxis. The best risk assessment model to use and the true incidence of VTE events in this population are unclear.

Improving computerized decision support system interventions: a qualitative study combining the theoretical domains framework with the GUIDES Checklist

BACKGROUND

Computerized clinical decision support systems (CDSSs) can improve care by bridging knowledge to practice gaps. However, the real-world uptake of such systems in health care settings has been suboptimal. We sought to: (1) use the Theoretical Domains Framework (TDF) to identify determinants (barriers/enablers) of uptake of the Electronic Asthma Management System (eAMS) CDSS; (2) match identified TDF belief statements to elements in the Guideline Implementation with Decision Support (GUIDES) Checklist; and (3) explore the relationship between the TDF and GUIDES frameworks and the usefulness of this sequential approach for identifying opportunities to improve CDSS uptake.

METHODS

In Phase 1, we conducted semistructured interviews with primary care physicians in Toronto, Canada regarding the uptake of the eAMS CDSS. Using content analysis, two coders independently analyzed interview transcripts guided by the TDF to generate themes representing barriers and enablers to CDSS uptake. In Phase 2, the same reviewers independently mapped each belief statement to a GUIDES domain and factor. We calculated the proportion of TDF belief statements that linked to each GUIDES domain and the proportion of TDF domains that linked to GUIDES factors (and vice-versa) and domains.

RESULTS

We interviewed 10 participants before data saturation. In Phase 1, we identified 53 belief statements covering 12 TDF domains; 18 (34.0%) were barriers, and 35 (66.0%) were enablers. In Phase 2, 41 statements (77.4%) linked to at least one GUIDES factor, while 12 (22.6%) did not link to any specific factor. The GUIDES Context Domain was linked to the largest number of belief statements (19/53; 35.8%). Each TDF domain linked to one or more GUIDES factor, with 6 TDF domains linking to more than 1 factor and 8 TDF domains linking to more than 1 GUIDES domain.

CONCLUSIONS

The TDF provides unique insights into barriers and enablers to CDSS uptake, which can then be mapped to GUIDES domains and factors to identify required changes to CDSS context, content, and system. This can be followed by conventional mapping of TDF domains to behaviour change techniques to optimize CDSS implementation. This novel step-wise approach combines two established frameworks to optimize CDSS interventions, and requires prospective validation.

The barriers and facilitators for the implementation of clinical practice guidelines in healthcare: an umbrella review of qualitative and quantitative literature

OBJECTIVES

To identify barriers and facilitators of clinical practice guidelines (CPGs) implementation, and map those factors to the theoretical domains framework (TDF) and behavior change wheel (BCW).

METHODS

We conducted an umbrella review of systematic reviews. PubMed, Embase, and the Cochrane Library were searched. Two investigators independently screened the studies, extracted the data, and assessed the methodological quality. The identified barriers and facilitators of CPG implementation were categorized and mapped to the TDF domains and BCW components.

RESULTS

Thirty-seven studies were included, and 193 barriers and 140 facilitators were identified. Intrinsic aspects (35 barriers and 28 facilitators) mainly included the CPGs' impracticality, complexity, and inaccessibility. Extrinsic aspects (158 barriers and 113 facilitators) mainly included lack of resources, training, funding, or awareness of CPG content in barriers; audits and feedback; strong leadership and management support; and educating and training about CPGs in facilitators. Environmental context and resources (n = 97, 19.48%) were the most reported barriers in TDF domains. Physical opportunity and social opportunity were the most frequently mentioned models in BCW.

CONCLUSION

Multiple barriers and facilitators for healthcare CPG implementation are identified, with further links to TDF and BCW. Future knowledge translation strategies should be developed accordingly in specified health care settings.

Which computable biomedical knowledge objects will be regulated? Results of a UK workshop discussing the regulation of knowledge libraries and software as a medical device

Introduction

To understand when knowledge objects in a computable biomedical knowledge library are likely to be subject to regulation as a medical device in the United Kingdom.

Methods

A briefing paper was circulated to a multi-disciplinary group of 25 including regulators, lawyers and others with insights into device regulation. A 1-day workshop was convened to discuss questions relating to our aim. A discussion paper was drafted by lead authors and circulated to other authors for their comments and contributions.

Results

This article reports on those deliberations and describes how UK device regulators are likely to treat the different kinds of knowledge objects that may be stored in computable biomedical knowledge libraries. While our focus is the likely approach of UK regulators, our analogies and analysis will also be relevant to the approaches taken by regulators elsewhere. We include a table examining the implications for each of the four knowledge levels described by Boxwala in 2011 and propose an additional level.

Conclusions

If a knowledge object is described as directly executable for a medical purpose to provide decision support, it will generally be in scope of UK regulation as "software as a medical device." However, if the knowledge object consists of an algorithm, a ruleset, pseudocode or some other representation that is not directly executable and whose developers make no claim that it can be used for a medical purpose, it is not likely to be subject to regulation. We expect similar reasoning to be applied by regulators in other countries.

Electronic safety-netting tool features considered important by UK general practice staff: an interview and Delphi consensus study

BACKGROUND

The potential of the electronic health record to support safety netting has been recognised and a number of electronic safety-netting (E-SN) tools developed.

AIM

To establish the most important features of E-SN tools.

DESIGN & SETTING

User-experience interviews followed by a Delphi study in a primary care setting in the UK.

METHOD

The user-experience interviews were carried out remotely with primary care staff who had trialled the EMIS E-SN toolkit for suspected cancer. An electronic modified Delphi approach was used, with primary care staff involved in safety netting in any capacity, to measure consensus on tool features.

RESULTS

Thirteen user-experience interviews were carried out and features of E-SN tools seen as important formed the majority of the features included in the Delphi study. Three rounds of Delphi survey were administered. Sixteen responders (64%) completed all three rounds, and 28 out of 44 (64%) features reached consensus. Primary care staff preferred tools that were general in scope.

CONCLUSION

Primary care staff indicated that tools that were not specific to cancer or any other disease, and had features that promoted their flexible, efficient, and integrated use, were important. However, when the important features were discussed with the patient and public involvement (PPI) group, they expressed disappointment that features they believed would make E-SN tools robust and provide a safety net that is difficult to fall through did not reach consensus. The successful adoption of E-SN tools will rely on an evidence base of their effectiveness. Efforts should be made to assess the impact of these tools on patient outcomes.

Improving decision accuracy using a clinical decision support system for medical students during history-taking: a randomized clinical trial

BACKGROUND

A clinical diagnostic support system (CDSS) can support medical students and physicians in providing evidence-based care. In this study, we investigate diagnostic accuracy based on the history of present illness between groups of medical students using a CDSS, Google, and neither (control). Further, the degree of diagnostic accuracy of medical students using a CDSS is compared with that of residents using neither a CDSS nor Google.

METHODS

This study is a randomized educational trial. The participants comprised 64 medical students and 13 residents who rotated in the Department of General Medicine at Chiba University Hospital from May to December 2020. The medical students were randomly divided into the CDSS group (n = 22), Google group (n = 22), and control group (n = 20). Participants were asked to provide the three most likely diagnoses for 20 cases, mainly a history of a present illness (10 common and 10 emergent diseases). Each correct diagnosis was awarded 1 point (maximum 20 points). The mean scores of the three medical student groups were compared using a one-way analysis of variance. Furthermore, the mean scores of the CDSS, Google, and residents' (without CDSS or Google) groups were compared.

RESULTS

The mean scores of the CDSS (12.0 ± 1.3) and Google (11.9 ± 1.1) groups were significantly higher than those of the control group (9.5 ± 1.7; p = 0.02 and p = 0.03, respectively). The residents' group's mean score (14.7 ± 1.4) was higher than the mean scores of the CDSS and Google groups (p = 0.01). Regarding common disease cases, the mean scores were 7.4 ± 0.7, 7.1 ± 0.7, and 8.2 ± 0.7 for the CDSS, Google, and residents' groups, respectively. There were no significant differences in mean scores (p = 0.1).

CONCLUSIONS

Medical students who used the CDSS and Google were able to list differential diagnoses more accurately than those using neither. Furthermore, they could make the same level of differential diagnoses as residents in the context of common diseases.

TRIAL REGISTRATION

This study was retrospectively registered with the University Hospital Medical Information Network Clinical Trials Registry on 24/12/2020 (unique trial number: UMIN000042831).

Re-Aiming Equity Evaluation in Clinical Decision Support: A Scoping Review of Equity Assessments in Surgical Decision Support Systems

OBJECTIVE

We critically evaluated the surgical literature to explore the prevalence and describe how equity assessments occur when using clinical decision support systems.

BACKGROUND

Clinical decision support (CDS) systems are increasingly used to facilitate surgical care delivery. Despite formal recommendations to do so, equity evaluations are not routinely performed on CDS systems and underrepresented populations are at risk of harm and further health disparities. We explored surgical literature to determine frequency and rigor of CDS equity assessments and offer recommendations to improve CDS equity by appending existing frameworks.

METHODS

We performed a scoping review up to Augus 25, 2021 using PubMed and Google Scholar for the following search terms: clinical decision support, implementation, RE-AIM, Proctor, Proctor's framework, equity, trauma, surgery, surgical. We identified 1415 citations and 229 abstracts met criteria for review. A total of 84 underwent full review after 145 were excluded if they did not assess outcomes of an electronic CDS tool or have a surgical use case.

RESULTS

Only 6% (5/84) of surgical CDS systems reported equity analyses, suggesting that current methods for optimizing equity in surgical CDS are inadequate. We propose revising the RE-AIM framework to include an Equity element (RE 2 -AIM) specifying that CDS foundational analyses and algorithms are performed or trained on balanced datasets with sociodemographic characteristics that accurately represent the CDS target population and are assessed by sensitivity analyses focused on vulnerable subpopulations.

CONCLUSION

Current surgical CDS literature reports little with respect to equity. Revising the RE-AIM framework to include an Equity element (RE 2 -AIM) promotes the development and implementation of CDS systems that, at minimum, do not worsen healthcare disparities and possibly improve their generalizability.

De-implementing opioid prescribing in a dental group practice: Lessons learned

BACKGROUND

Drug overdose has become a leading cause of accidental death in the United States. Between 2000 and 2015, the rate of deaths from drug overdoses increased 137%, including a 200% increase in the rate of overdose deaths involving opioids (including opioid pain relievers and heroin). Unnecessary opioid prescribing is one of the factors driving this epidemic.

OBJECTIVES

The primary objective of this paper is to share lessons learned while conducting a randomized trial to de-implement opioids for post-extraction pain management utilizing clinical decision support (CDS) with and without patient education. The lessons learned from conducting this trial in a real-world setting can be applied to future dissemination and implementation oral health research.

METHODS

The sources informing lessons learned were generated from qualitative interviews conducted with 20 of the forty-nine dental providers involved in the study following the implementation phase of the trial. Ongoing policy, social and environmental factors were tracked throughout the study.

RESULTS

Dental providers in the trial identified the impact of training that involved health professionals sharing information about the personal impact of pain and opioid use. Additionally, they found utility in being presented with a dashboard detailing their prescribing patterns related to other dentists. For the 30 general dentists with access to the CDS, use of its portal varied widely, with most using it 10%-49% of the time related to extractions.

CONCLUSIONS

In the context of a downward trend in opioid prescribing and considering the influence of the COVID pandemic during the trial, dental providers indicated benefit in training about negative personal impacts of prescribing opioids, and personally relevant feedback about their prescribing patterns. Only modest use of the CDS was realized. Implementation of this trial was impacted by governmental and health system policies and the COVID pandemic, prompt the consideration of implications regarding continuing ways to limit opioid prescribing among dental providers.

Clinician Perspectives on Barriers and Enablers to Implementing an Inpatient Oncology Early Warning System: A Mixed-Methods Study

PURPOSE

To elicit end-user and stakeholder perceptions regarding design and implementation of an inpatient clinical deterioration early warning system (EWS) for oncology patients to better fit routine clinical practices and enhance clinical impact.

METHODS

In an explanatory-sequential mixed-methods study, we evaluated a stakeholder-informed oncology early warning system (OncEWS) using surveys and semistructured interviews. Stakeholders were physicians, advanced practice providers (APPs), and nurses. For qualitative data, we used grounded theory and thematic content analysis via the constant comparative method to identify determinants of OncEWS implementation.

RESULTS

Survey respondents generally agreed that an oncology-focused EWS could add value beyond clinical judgment, with nurses endorsing this notion significantly more strongly than other clinicians (nurse: median 5 on a 6-point scale [6 = strongly agree], interquartile range 4-5; doctors/advanced practice providers: 4 [4-5]; P = .005). However, some respondents would not trust an EWS to identify risk accurately (n = 36 [42%] somewhat or very concerned), while others were concerned that institutional culture would not embrace such an EWS (n = 17 [28%]).Interviews highlighted important aspects of the EWS and the local context that might facilitate implementation, including (1) a model tailored to the subtleties of oncology patients, (2) transparent model information, and (3) nursing-centric workflows. Interviewees raised the importance of sepsis as a common and high-risk deterioration syndrome.

CONCLUSION

Stakeholders prioritized maximizing the degree to which the OncEWS is understandable, informative, actionable, and workflow-complementary, and perceived these factors to be key for translation into clinical benefit.

Implementation of an Electronic Health Records-Based Safe Contrast Limit for Preventing Contrast-Associated Acute Kidney Injury After Percutaneous Coronary Intervention

BACKGROUND

Contrast-associated acute kidney injury (CA-AKI) after percutaneous coronary intervention is associated with increased mortality. We assessed the effectiveness of an electronic health records safe contrast limit tool in predicting CA-AKI risk and reducing contrast use and CA-AKI.

METHODS

We created an alert displaying the safe contrast limit to cardiac catheterization laboratory staff prior to percutaneous coronary intervention. The alert used risk factors automatically extracted from the electronic health records. We included procedures from June 1, 2020 to October 1, 2021; the intervention went live February 10, 2021. Using difference-in-differences analysis, we evaluated changes in contrast volume and CA-AKI rates after contrast limit tool implementation compared to control hospitals. Cardiologists were surveyed prior to and 9 months after alert implementation on beliefs, practice patterns, and safe contrast estimates for example patients.

RESULTS

At the one intervention site, there were 508 percutaneous coronary interventions before and 531 after tool deployment. At 15 control sites, there were 3550 and 3979 percutaneous coronary interventions, respectively. The contrast limit predicted CA-AKI with an accuracy of 64.1%, negative predictive value of 93.3%, and positive predictive value of 18.7%. After implementation, in high/modifiable risk patients (defined as having a calculated contrast limit <500ml) there was a small but significant -4.60 mL/month (95% CI, -8.24 to -1.00) change in average contrast use but no change in CA-AKI rates (odds ratio, 0.96 [95% CI, 0.84-1.10]). Low-risk patients had no change in contrast use (-0.50 mL/month [95% CI, -7.49 to 6.49]) or CA-AKI (odds ratio, 1.24 [95% CI, 0.79-1.93]). In assessing CA-AKI risk, clinicians heavily weighted age and diabetes but often did not consider anemia, cardiogenic shock, and heart failure.

CONCLUSIONS

Clinicians often used a simplified assessment of CA-AKI risk that did not include important risk factors, leading to risk estimations inconsistent with established models. Despite clinician skepticism, an electronic health records-based contrast limit tool more accurately predicted CA-AKI risk and was associated with a small decrease in contrast use during percutaneous coronary intervention but no change in CA-AKI rates.

Artificial Intelligence Implementation in Healthcare: A Theory-Based Scoping Review of Barriers and Facilitators

There is a large proliferation of complex data-driven artificial intelligence (AI) applications in many aspects of our daily lives, but their implementation in healthcare is still limited. This scoping review takes a theoretical approach to examine the barriers and facilitators based on empirical data from existing implementations. We searched the major databases of relevant scientific publications for articles related to AI in clinical settings, published between 2015 and 2021. Based on the theoretical constructs of the Consolidated Framework for Implementation Research (CFIR), we used a deductive, followed by an inductive, approach to extract facilitators and barriers. After screening 2784 studies, 19 studies were included in this review. Most of the cited facilitators were related to engagement with and management of the implementation process, while the most cited barriers dealt with the intervention's generalizability and interoperability with existing systems, as well as the inner settings' data quality and availability. We noted per-study imbalances related to the reporting of the theoretic domains. Our findings suggest a greater need for implementation science expertise in AI implementation projects, to improve both the implementation process and the quality of scientific reporting.

How are ontologies implemented to represent clinical practice guidelines in clinical decision support systems: protocol for a systematic review

BACKGROUND

Clinical practice guidelines are statements which are based on the best available evidence, and their goal is to improve the quality of patient care. Integrating clinical practice guidelines into computer systems can help physicians reduce medical errors and help them to have the best possible practice. Guideline-based clinical decision support systems play a significant role in supporting physicians in their decisions. Meantime, system errors are the most critical concerns in designing decision support systems that can affect their performance and efficacy. A well-developed ontology can be helpful in this matter. The proposed systematic review will specify the methods, components, language of rules, and evaluation methods of current ontology-driven guideline-based clinical decision support systems.

METHODS

This review will identify literature through searching MEDLINE (via Ovid), PubMed, EMBASE, Cochrane Library, CINAHL, ScienceDirect, IEEEXplore, and ACM Digital Library. Gray literature, reference lists, and citing articles of the included studies will be searched. The quality of the included studies will be assessed by the mixed methods appraisal tool (MMAT-version 2018). At least two independent reviewers will perform the screening, quality assessment, and data extraction. A third reviewer will resolve any disagreements. Proper data analysis will be performed based on the type of system and ontology engineering evaluation data.

DISCUSSION

The study will provide evidence regarding applying ontologies in guideline-based clinical decision support systems. The findings of this systematic review will be a guide for decision support system designers and developers, technologists, system providers, policymakers, and stakeholders. Ontology builders can use the information in this review to build well-structured ontologies for personalized medicine.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018106501.

Barriers and facilitators to implementing a nursing clinical decision support system in a tertiary hospital setting: A qualitative study using the FITT framework

OBJECTIVE

Clinical decision support systems (CDSSs) have been increasingly introduced to health care settings; however, their adoption is far from ideal. Guided by the FITT framework, this study aims to explore barriers and facilitators to the implementation of a CDSS from the perspective of nurses.

METHODS

A qualitative study with 200 h of participatory observation and 21 semi structured interviews was conducted from February to August 2021 in four medical-surgical wards in a 2000-bed tertiary hospital in Shanghai, China. The field notes were typed and the audio-recorded interviews were transcribed to texts verbatim and were coded with a four-step approach. We used the FITT framework to interpret our findings based on the technology, individual and task attributes and the fit between them.

RESULTS

A total of twelve categories were identified, which were integrated into two themes: barriers and facilitators to system implementation. All categories but one can be mapped to the three attributes of the FITT framework: technology, individual and task. We assumed that management has a vital role to play in the following areas: addressing user resistance, improving system usability, setting standards on practice and, finally, building connectivity between nurses and the technical staff to improve the fit between the technology, individual and task attribute and thus promote system implementation.

CONCLUSION

Barriers and facilitators to CDSS implementation include system-related, user-related and organizational factors which can largely be fit io the FITT framework. There is potential to extend the FITT framework to represent management intervention on inter-disciplinary collaboration. Future empirical studies on facilitating strategies from the management to improve user experience and willingness of CDSS adoption are needed.

Evidence-Based Clinical Decision Support to Improve Care for Patients Hospitalized With Acute Myocardial Infarction

Clinical decision support in the EHR is an innovation that can support guideline adherence in acute myocardial infarction. Cardiac rehabilitation referral and left ventricular systolic function assessment are part of evidence-based clinical practice guidelines associated with reduced morbidity and mortality following acute myocardial infarction. Effective clinical decision support is sustained by evidence-based principles for design and implementation. This quality improvement project evaluated the impact of practice advisories designed using principles of effective clinical decision support design to improve performance of left ventricular systolic function assessment and ambulatory referral to cardiac rehabilitation for patients hospitalized with acute myocardial infarction. Performance in cardiac rehabilitation referral and left ventricular systolic function assessment was measured for a 3-month interval pre- and post-intervention. Pre-implementation, cardiac rehabilitation referral or valid documented reason for non-referral was 80.3%. Rehabilitation referral or documented valid reason for non-referral increased to 98.4% post-implementation (P < .001). Left ventricular systolic function assessment increased from 94.2% to 100% following clinical decision support implementation (P = .120). This quality improvement project supports the positive impact of effective clinical decision support design and implementation to improve outcomes for patients hospitalized with acute myocardial infarction.

Optimization of a Quality Improvement Tool for Cancer Diagnosis in Primary Care: Qualitative Study

Background

The most common route to a diagnosis of cancer is through primary care. Delays in diagnosing cancer occur when an opportunity to make a timely diagnosis is missed and is evidenced by patients visiting the general practitioner (GP) on multiple occasions before referral to a specialist. Tools that minimize prolonged diagnostic intervals and reduce missed opportunities to investigate patients for cancer are therefore a priority.

Objective

This study aims to explore the usefulness and feasibility of a novel quality improvement (QI) tool in which algorithms flag abnormal test results that may be indicative of undiagnosed cancer. This study allows for the optimization of the cancer recommendations before testing the efficacy in a randomized controlled trial.

Methods

GPs, practice nurses, practice managers, and consumers were recruited to participate in individual interviews or focus groups. Participants were purposively sampled as part of a pilot and feasibility study, in which primary care practices were receiving recommendations relating to the follow-up of abnormal test results for prostate-specific antigen, thrombocytosis, and iron-deficiency anemia. The Clinical Performance Feedback Intervention Theory (CP-FIT) was applied to the analysis using a thematic approach.

Results

A total of 17 interviews and 3 focus groups (n=18) were completed. Participant themes were mapped to CP-FIT across the constructs of context, recipient, and feedback variables. The key facilitators to use were alignment with workflow, recognized need, the perceived importance of the clinical topic, and the GPs’ perception that the recommendations were within their control. Barriers to use included competing priorities, usability and complexity of the recommendations, and knowledge of the clinical topic. There was consistency between consumer and practitioner perspectives, reporting language concerns associated with the word cancer, the need for more patient-facing resources, and time constraints of the consultation to address patients’ worries.

Conclusions

There was a recognized need for the QI tool to support the diagnosis of cancer in primary care, but barriers were identified that hindered the usability and actionability of the recommendations in practice. In response, the tool has been refined and is currently being evaluated as part of a randomized controlled trial. Successful and effective implementation of this QI tool could support the detection of patients at risk of undiagnosed cancer in primary care and assist in preventing unnecessary delays.

Initial experience with AI Pathway Companion: Evaluation of dashboard-enhanced clinical decision making in prostate cancer screening

Purpose

Rising complexity of patients and the consideration of heterogeneous information from various IT systems challenge the decision-making process of urological oncologists. Siemens AI Pathway Companion is a decision support tool that provides physicians with comprehensive patient information from various systems. In the present study, we examined the impact of providing organized patient information in comprehensive dashboards on information quality, effectiveness, and satisfaction of physicians in the clinical decision-making process.

Methods

Ten urologists in our department performed the entire diagnostic workup to treatment decision for 10 patients in the prostate cancer screening setting. Expenditure of time, information quality, and user satisfaction during the decision-making process with AI Pathway Companion were recorded and compared to the current workflow.

Results

A significant reduction in the physician’s expenditure of time for the decision-making process by -59.9% (p < 0,001) was found using the software. System usage showed a high positive effect on evaluated information quality parameters completeness (Cohen’s d of 2.36), format (6.15), understandability (2.64), as well as user satisfaction (4.94).

Conclusion

The software demonstrated that comprehensive organization of information improves physician’s effectiveness and satisfaction in the clinical decision-making process. Further development is needed to map more complex patient pathways, such as the follow-up treatment of prostate cancer.

The development and implementation of a guideline-based clinical decision support system to improve empirical antibiotic prescribing

Background

To describe and evaluate a clinical decision support system (CDSS) for empirical antibiotic therapy using a systematic framework.

Methods

A reporting framework for behavior change intervention implementation was used, which includes several domains: development, evaluation and implementation. Within the development domain a description is given of the engagement of stakeholders, a rationale for how the CDSS may influence antibiotic prescribing and a detailed outline of how the system was developed. Within the evaluation domain a technical validation is performed and the interaction between potential users and the CDSS is analyzed. Within the domain of implementation a description is given on how the CDSS was tested in the real world and the strategies that were used for implementation and adoption of the CDSS.

Results

Development: a CDSS was developed, with the involvement of stakeholders, to assist empirical antibiotic prescribing by physicians. Evaluation: Technical problems were determined during the validation process and corrected in a new CDSS version. A usability study was performed to assess problems in the system-user interaction. Implementation: In 114 patients the antibiotic advice that was generated by the CDSS was followed. For 54 patients the recommendations were not adhered to.

Conclusions

This study describes the development and validation of a CDSS for empirical antibiotic therapy and shows the usefulness of the systematic framework for reporting CDSS interventions. In addition it shows that CDSS recommendations are not always adhered to which is associated with incorrect use of the system.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01860-3.

Implementation of a Web-Based Tool for Shared Decision-making in Lung Cancer Screening: Mixed Methods Quality Improvement Evaluation

Background

Lung cancer risk and life expectancy vary substantially across patients eligible for low-dose computed tomography lung cancer screening (LCS), which has important consequences for optimizing LCS decisions for different patients. To account for this heterogeneity during decision-making, web-based decision support tools are needed to enable quick calculations and streamline the process of obtaining individualized information that more accurately informs patient-clinician LCS discussions. We created DecisionPrecision, a clinician-facing web-based decision support tool, to help tailor the LCS discussion to a patient’s individualized lung cancer risk and estimated net benefit.

Objective

The objective of our study is to test two strategies for implementing DecisionPrecision in primary care at eight Veterans Affairs medical centers: a quality improvement (QI) training approach and academic detailing (AD).

Methods

Phase 1 comprised a multisite, cluster randomized trial comparing the effectiveness of standard implementation (adding a link to DecisionPrecision in the electronic health record vs standard implementation plus the Learn, Engage, Act, and Process [LEAP] QI training program). The primary outcome measure was the use of DecisionPrecision at each site before versus after LEAP QI training. The second phase of the study examined the potential effectiveness of AD as an implementation strategy for DecisionPrecision at all 8 medical centers. Outcomes were assessed by comparing absolute tool use before and after AD visits and conducting semistructured interviews with a subset of primary care physicians (PCPs) following the AD visits.

Results

Phase 1 findings showed that sites that participated in the LEAP QI training program used DecisionPrecision significantly more often than the standard implementation sites (tool used 190.3, SD 174.8 times on average over 6 months at LEAP sites vs 3.5 SD 3.7 at standard sites; P<.001). However, this finding was confounded by the lack of screening coordinators at standard implementation sites. In phase 2, there was no difference in the 6-month tool use between before and after AD (95% CI −5.06 to 6.40; P=.82). Follow-up interviews with PCPs indicated that the AD strategy increased provider awareness and appreciation for the benefits of the tool. However, other priorities and limited time prevented PCPs from using them during routine clinical visits.

Conclusions

The phase 1 findings did not provide conclusive evidence of the benefit of a QI training approach for implementing a decision support tool for LCS among PCPs. In addition, phase 2 findings showed that our light-touch, single-visit AD strategy did not increase tool use. To enable tool use by PCPs, prediction-based tools must be fully automated and integrated into electronic health records, thereby helping providers personalize LCS discussions among their many competing demands. PCPs also need more time to engage in shared decision-making discussions with their patients.

Trial Registration

ClinicalTrials.gov NCT02765412; https://clinicaltrials.gov/ct2/show/NCT02765412

A Clinical Decision Support System for Increasing Compliance with Protocols in Chemotherapy of Children with Acute Lymphoblastic Leukemia

Objective:

In this survey, a protocol-based Chemotherapy Prescription Decision Support System (CPDSS) was designed and evaluated to reduce medication errors in the chemotherapy process of children with ALL.

Methods:

The CPDSS algorithm was extracted by the software development team based on the protocol used by doctors to treat children with ALL. The ASP.Net MVC and SQL Server 2016 programming languages were used to develop the system. A 3-step evaluation (technical, retrospective, and user satisfaction) was performed on CPDSS designed at 2 children’s hospitals in Tehran. The data were analyzed using descriptive statistics. At the technical evaluation step, users provided recommendations included in the system.

Results:

In the retrospective CPDSS evaluation step, 1281 prescribed doses of the drugs related to 30 patients were entered into the system. CPDSS detected 735 cases of protocol deviations and 57 (95%, CI = 1.25-2.55) errors in prescribed chemotherapy for children with ALL. In the user satisfaction evaluation, the users approved two dimensions of the user interface and functionality of the system.

Conclusions:

With the provision of alerts, the CPDSS can help increase compliance with chemotherapy protocols and decrease the chemotherapy prescribing errors that can improve patient safety.

Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake

BACKGROUND

Computerized clinical decision support systems (CDSSs) are a promising knowledge translation tool, but often fail to meaningfully influence the outcomes they target. Low CDSS provider uptake is a potential contributor to this problem but has not been systematically studied. The objective of this systematic review and meta-regression was to determine reported CDSS uptake and identify which CDSS features may influence uptake.

METHODS

Medline, Embase, CINAHL, and the Cochrane Database of Controlled Trials were searched from January 2000 to August 2020. Randomized, non-randomized, and quasi-experimental trials reporting CDSS uptake in any patient population or setting were included. The main outcome extracted was CDSS uptake, reported as a raw proportion, and representing the number of times the CDSS was used or accessed over the total number of times it could have been interacted with. We also extracted context, content, system, and implementation features that might influence uptake, for each CDSS. Overall weighted uptake was calculated using random-effects meta-analysis and determinants of uptake were investigated using multivariable meta-regression.

RESULTS

Among 7995 citations screened, 55 studies involving 373,608 patients and 3607 providers met full inclusion criteria. Meta-analysis revealed that overall CDSS uptake was 34.2% (95% CI 23.2 to 47.1%). Uptake was only reported in 12.4% of studies that otherwise met inclusion criteria. Multivariable meta-regression revealed the following factors significantly associated with uptake: (1) formally evaluating the availability and quality of the patient data needed to inform CDSS advice; and (2) identifying and addressing other barriers to the behaviour change targeted by the CDSS.

CONCLUSIONS AND RELEVANCE

System uptake was seldom reported in CDSS trials. When reported, uptake was low. This represents a major and potentially modifiable barrier to overall CDSS effectiveness. We found that features relating to CDSS context and implementation strategy best predicted uptake. Future studies should measure the impact of addressing these features as part of the CDSS implementation strategy. Uptake reporting must also become standard in future studies reporting CDSS intervention effects.

REGISTRATION

Pre-registered on PROSPERO, CRD42018092337.

Clinical Decision Support with or without Shared Decision Making to Improve Preventive Cancer Care: A Cluster-Randomized Trial

BACKGROUND

Innovative interventions are needed to address gaps in preventive cancer care, especially in rural areas. This study evaluated the impact of clinical decision support (CDS) with and without shared decision making (SDM) on cancer-screening completion.

METHODS

In this 3-arm, parallel-group, cluster-randomized trial conducted at a predominantly rural medical group, 34 primary care clinics were randomized to clinical decision support (CDS), CDS plus shared decision making (CDS+SDM), or usual care (UC). The CDS applied web-based clinical algorithms identifying patients overdue for United States Preventive Services Task Force-recommended preventive cancer care and presented evidence-based recommendations to patients and providers on printouts and on the electronic health record interface. Patients in the CDS+SDM clinic also received shared decision-making tools (SDMTs). The primary outcome was a composite indicator of the proportion of patients overdue for breast, cervical, or colorectal cancer screening at index who were up to date on these 1 y later.

RESULTS

From August 1, 2018, to March 15, 2019, 69,405 patients aged 21 to 74 y had visits at study clinics and 25,198 were overdue for 1 or more cancer screening tests at an index visit. At 12-mo follow-up, 9,543 of these (37.9%) were up to date on the composite endpoint. The adjusted, model-derived percentage of patients up to date was 36.5% (95% confidence interval [CI]: 34.0-39.1) in the UC group, 38.1% (95% CI: 35.5-40.9) in the CDS group, and 34.4% (95% CI: 31.8-37.2) in the CDS+SDM group. For all comparisons, the screening rates were higher than UC in the CDS group and lower than UC in the CDS+SDM group, although these differences did not reach statistical significance.

CONCLUSION

The CDS did not significantly increase cancer-screening rates. Exploratory analyses suggest a deeper understanding of how SDM and CDS interact to affect cancer prevention decisions is needed. Trial registration: ClinicalTrials.gov ID: NCT02986230, December 6, 2016.

Primary care clinicians' opinions before and after implementation of cancer screening and prevention clinical decision support in a clinic cluster-randomized control trial: a survey research study

BACKGROUND

Electronic health record (EHR)-linked clinical decision support (CDS) may impact primary care clinicians' (PCCs') clinical care opinions. As part of a clinic cluster-randomized control trial (RCT) testing a cancer prevention and screening CDS system with patient and PCC printouts (with or without shared decision-making tools [SDMT]) for patients due for breast, cervical, colorectal, and lung cancer screening and/or human papillomavirus (HPV) vaccination compared to usual care (UC), we surveyed PCCs at study clinics pre- and post-CDS implementation. Our primary aim was to learn if PCCs' opinions changed over time within study arms. Secondary aims including examining whether PCCs' opinions in study arms differed both pre- and post-implementation, and gauging PCCs' opinions on the CDS in the two intervention arms.

METHODS

This study was conducted within a healthcare system serving an upper Midwestern population. We administered pre-implementation (11/2/2017-1/24/2018) and post-implementation (2/2/2020-4/9/2020) cross-sectional electronic surveys to PCCs practicing within a RCT arm: UC; CDS; or CDS + SDMT. Bivariate analyses compared responses between study arms at both time periods and longitudinally within study arms.

RESULTS

Pre-implementation (53%, n = 166) and post-implementation (57%, n = 172) response rates were similar. No significant differences in PCC responses were seen between study arms on cancer prevention and screening questions pre-implementation, with few significant differences found between study arms post-implementation. However, significantly fewer intervention arm clinic PCCs reported being very comfortable with discussing breast cancer screening options with patients compared to UC post-implementation, as well as compared to the same intervention arms pre-implementation. Other significant differences were noted within arms longitudinally. For intervention arms, these differences related to CDS areas like EHR alerts, risk calculators, and ordering screening. Most intervention arm PCCs noted the CDS provided overdue screening alerts to which they were unaware. Few PCCs reported using the CDS, but most would recommend it to colleagues, expressed high CDS satisfaction rates, and thought patients liked the CDS's information and utility.

CONCLUSIONS

While appreciated by PCCs with high satisfaction rates, the CDS may lower PCCs' confidence regarding discussing patients' breast cancer screening options and may be used irregularly. Future research will evaluate the impact of the CDS on cancer prevention and screening rates.

TRIAL REGISTRATION

clinicaltrials.gov , NCT02986230, December 6, 2016.

Changing the medication documentation process for discharge: impact on clinical routine and documentation quality-a process analysis

OBJECTIVES

In 2017, an in-house best-practice process for medication documentation was developed and implemented to meet the new German legal requirements concerning the management of patient discharge from the hospital. Because this law regulates the common steps of good discharge practices (eg, specification of discharge mediation documentation), we used its implementation to assess the impact of such a measure on the quality of medication documentation and related workflows in clinical routine.

METHODS

By observing workflows and interviewing the affected employees, we analysed the medication workflow processes from admission to discharge of seven representative departments of a large university hospital before and early after implementation of a newly defined best-practice process. To investigate the implementation impact, following measures were determined overall and for five key process steps: quality of medication documentation as measured by predefined criteria, the adherence to the best-practice process (range 0%-100%), workload and potential shifts in responsibilities.

RESULTS

Already early after implementation, all departments met the legal requirements and the quality of the medication documentation increased from low to high quality in most departments. Mean adherence to the best-practice process was 77% (range 60%-100%) with strictest adherence of 100% in one department. Thereby, the number of process steps and hence, likely also the workload increased in all departments. New tasks were mainly performed by physicians and in one department by pharmacists.

CONCLUSIONS

The new lawful best-practice process led to a higher quality in medication documentation at the cost of a higher workload for physicians, potentially limiting time for other care tasks. Therefore, it could be important to define areas of the medication documentation process in which physicians could be supported by other professions or new tools facilitating accurate medication documentation as the basis of continuity of care.

Clinical Decision Support for Laboratory Testing

BACKGROUND

As technology enables new and increasingly complex laboratory tests, test utilization presents a growing challenge for healthcare systems. Clinical decision support (CDS) refers to digital tools that present providers with clinically relevant information and recommendations, which have been shown to improve test utilization. Nevertheless, individual CDS applications often fail, and implementation remains challenging.

CONTENT

We review common classes of CDS tools grounded in examples from the literature as well as our own institutional experience. In addition, we present a practical framework and specific recommendations for effective CDS implementation.

SUMMARY

CDS encompasses a rich set of tools that have the potential to drive significant improvements in laboratory testing, especially with respect to test utilization. Deploying CDS effectively requires thoughtful design and careful maintenance, and structured processes focused on quality improvement and change management play an important role in achieving these goals.

Clinical validation of clinical decision support systems for medication review: A scoping review

The aim of this scoping review is to summarize approaches and outcomes of clinical validation studies of clinical decision support systems (CDSSs) to support (part of) a medication review. A literature search was conducted in Embase and Medline. In total, 30 articles validating a CDSS were ultimately included. Most of the studies focused on detection of adverse drug events, potentially inappropriate medications and drug‐related problems. We categorized the included articles in three groups: studies subjectively reviewing the clinical relevance of CDSS's output (21/30 studies) resulting in a positive predictive value (PPV) for clinical relevance of 4–80%; studies determining the relationship between alerts and actual events (10/30 studies) resulting in a PPV for actual events of 5–80%; and studies comparing output of CDSSs to chart/medication reviews in the whole study population (10/30 studies) resulting in a sensitivity of 28–85% and specificity of 42–75%. We found heterogeneity in the methods used and in the outcome measures. The validation studies did not report the use of a published CDSS validation strategy. To improve the effectiveness and uptake of CDSSs supporting a medication review, future research would benefit from a more systematic and comprehensive validation strategy.

How senior paramedics decide to cease resuscitation in pulseless electrical activity out of hospital cardiac arrest: a mixed methods study

BACKGROUND

Evidenced-based guidelines on when to cease resuscitation for pulseless electrical activity are limited and support for paramedics typically defaults to the senior clinician. Senior clinicians include paramedics employed to work beyond the scope of clinical guidelines as there may be a point at which it is reasonable to cease resuscitation. To support these decisions, one ambulance service has applied a locally derived cessation of resuscitation checklist. This study aimed to describe the patient, clinical and system factors and examine senior clinician experiences when ceasing resuscitation for pulseless electrical activity.

DESIGN AND METHODS

An explanatory sequential mixed method study was conducted in one ambulance service in the South West of England. A consecutive sample of checklist data for adult pulseless electrical activity were retrieved from 1st December 2015 to 31st December 2018. Unexpected results which required exploration were identified and developed into semi-structured interview questions. A purposive sample of senior clinicians who ceased resuscitation and applied the checklist were interviewed. Content framework analysis was applied to the qualitative findings.

RESULTS

Senior clinicians ceased resuscitation for 50 patients in the presence of factors known to optimise survival: Witnessed cardiac arrest (n = 37, 74%), bystander resuscitation (n = 30, 60%), defibrillation (n = 22, 44%), return of spontaneous circulation (n = 8, 16%). Significant association was found between witnessed cardiac arrest and bystander resuscitation (p = .00). Six senior clinicians were interviewed, and analysis resulted in four themes: defining resuscitation futility, the impact of ceasing resuscitation, conflicting views and clinical decision tools. In the local context, senior clinicians applied their clinical judgement to balance survivability. Multiple factors were considered as the decision to cease resuscitation was not always clear. Senior clinicians deviated from the checklist when the patient was perceived as non-survivable.

CONCLUSION

Senior clinicians applied clinical judgement to assess patients as non-survivable or when continued resuscitation was considered harmful with no patient benefit. Senior clinicians perceived pre-existing factors with duration of resuscitation and clinical factors known to optimise patient survival. Future practice could look beyond a set criteria in which to cease resuscitation, however, it would be helpful to investigate the value or threshold of factors associated with patient outcome.

A systematic approach to public health - Novel application of the human factors analysis and classification system to public health and COVID-19

In this article, we argue for a novel adaptation of the Human Factors Analysis and Classification System (HFACS) to proactive incidence prevention in the public health and in particular, during and in response to COVID-19. HFACS is a framework of causal categories of human errors typically applied for systematic retrospective incident analysis in high-risk domains. By leveraging this approach proactively, appropriate, and targeted measures can be quickly identified and established to mitigate potential errors at different levels within the public health system (from tertiary and secondary healthcare workers to primary public health officials, regulators, and policymakers).

Effect of Interventions With a Clinical Decision Support System for Hospitalized Older Patients: Systematic Review Mapping Implementation and Design Factors

Background

Clinical decision support systems (CDSSs) form an implementation strategy that can facilitate and support health care professionals in the care of older hospitalized patients.

Objective

Our study aims to systematically review the effects of CDSS interventions in older hospitalized patients. As a secondary aim, we aim to summarize the implementation and design factors described in effective and ineffective interventions and identify gaps in the current literature.

Methods

We conducted a systematic review with a search strategy combining the categories older patients, geriatric topic, hospital, CDSS, and intervention in the databases MEDLINE, Embase, and SCOPUS. We included controlled studies, extracted data of all reported outcomes, and potentially beneficial design and implementation factors. We structured these factors using the Grol and Wensing Implementation of Change model, the GUIDES (Guideline Implementation with Decision Support) checklist, and the two-stream model. The risk of bias of the included studies was assessed using the Cochrane Collaboration’s Effective Practice and Organisation of Care risk of bias approach.

Results

Our systematic review included 18 interventions, of which 13 (72%) were effective in improving care. Among these interventions, 8 (6 effective) focused on medication review, 8 (6 effective) on delirium, 7 (4 effective) on falls, 5 (4 effective) on functional decline, 4 (3 effective) on discharge or aftercare, and 2 (0 effective) on pressure ulcers. In 77% (10/13) effective interventions, the effect was based on process-related outcomes, in 15% (2/13) interventions on both process- and patient-related outcomes, and in 8% (1/13) interventions on patient-related outcomes. The following implementation and design factors were potentially associated with effectiveness: a priori problem or performance analyses (described in 9/13, 69% effective vs 0/5, 0% ineffective interventions), multifaceted interventions (8/13, 62% vs 1/5, 20%), and consideration of the workflow (9/13, 69% vs 1/5, 20%).

Conclusions

CDSS interventions can improve the hospital care of older patients, mostly on process-related outcomes. We identified 2 implementation factors and 1 design factor that were reported more frequently in articles on effective interventions. More studies with strong designs are needed to measure the effect of CDSS on relevant patient-related outcomes, investigate personalized (data-driven) interventions, and quantify the impact of implementation and design factors on CDSS effectiveness.

Trial Registration

PROSPERO (International Prospective Register of Systematic Reviews): CRD42019124470; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=124470.

Translating an evidence-based clinical pathway into shareable CDS: developing a systematic process using publicly available tools

OBJECTIVE

To develop a process for translating semi-structured clinical decision support (CDS) into shareable, computer-readable CDS.

MATERIALS AND METHODS

We developed a systematic and transparent process using publicly available tools (eGLIA, GEM Cutter, VSAC, and the CDS Authoring Tool) to translate an evidence-based clinical pathway (CP) into a Clinical Quality Language (CQL)-encoded CDS artifact.

RESULTS

We produced a 4-phase process for translating a CP into a CQL-based CDS artifact. CP content was extracted using GEM into discrete clinical concepts, encoded using standard terminologies into value sets on VSAC, evaluated against workflows using a wireframe, and finally structured as a computer readable CDS artifact using CQL. This process included a quality control step and intermediate products to support transparency and reuse by other CDS developers.

DISCUSSION

Translating a CP into a shareable, computer-readable CDS artifact was accomplished through a systematic process. Our process identified areas of ambiguity and gaps in the CP, which generated improvements in the CP. Collaboration with clinical subject experts and the CP development team was essential for translation. Publicly available tools were sufficient to support most translation steps, but expression of certain complex concepts required manual encoding.

CONCLUSION

Standardized development of CDS from a CP is feasible using a systematic 4-phase process. CPs represent a potential reservoir for developers of evidence-based CDS. Aspects of CP development simplified portions of the CDS translation process. Publicly available tools can facilitate CDS development; however, enhanced tool features are needed to model complex CDS statements.

Electronic clinical decision support for children with minor head trauma and intracranial injuries: a sociotechnical analysis

BACKGROUND

Current management of children with minor head trauma (MHT) and intracranial injuries is not evidence-based and may place some children at risk of harm. Evidence-based electronic clinical decision support (CDS) for management of these children may improve patient safety and decrease resource use. To guide these efforts, we evaluated the sociotechnical environment impacting the implementation of electronic CDS, including workflow and communication, institutional culture, and hardware and software infrastructure, among other factors.

METHODS

Between March and May, 2020 semi-structured qualitative focus group interviews were conducted to identify sociotechnical influences on CDS implementation. Physicians from neurosurgery, emergency medicine, critical care, and pediatric general surgery were included, along with information technology specialists. Participants were recruited from nine health centers in the United States. Focus group transcripts were coded and analyzed using thematic analysis. The final themes were then cross-referenced with previously defined sociotechnical dimensions.

RESULTS

We included 28 physicians and four information technology specialists in seven focus groups (median five participants per group). Five physicians were trainees and 10 had administrative leadership positions. Through inductive thematic analysis, we identified five primary themes: (1) clinical impact; (2) stakeholders and users; (3) tool content; (4) clinical practice integration; and (5) post-implementation evaluation measures. Participants generally supported using CDS to determine an appropriate level-of-care for these children. However, some had mixed feelings regarding how the tool could best be used by different specialties (e.g. use by neurosurgeons versus non-neurosurgeons). Feedback from the interviews helped refine the tool content and also highlighted potential technical and workflow barriers to address prior to implementation.

CONCLUSIONS

We identified key factors impacting the implementation of electronic CDS for children with MHT and intracranial injuries. These results have informed our implementation strategy and may also serve as a template for future efforts to implement health information technology in a multidisciplinary, emergency setting.

Quality Improvement in Otolaryngology-Head and Neck Surgery: Developing Registry-Enabled Quality Measures From Guidelines for Cerumen Impaction and Allergic Rhinitis Through a Transparent and Systematic Process

BACKGROUND AND SIGNIFICANCE

Quality measurement can drive improvement in clinical care and allow for easy reporting of quality care by clinicians, but creating quality measures is a time-consuming and costly process. ECRI (formerly Emergency Care Research Institute) has pioneered a process to support systematic translation of clinical practice guidelines into electronic quality measures using a transparent and reproducible pathway. This process could be used to augment or support the development of electronic quality measures of the American Academy of Otolaryngology-Head and Neck Surgery Foundation (AAO-HNSF) and others as the Centers for Medicare and Medicaid Services transitions from the Merit-Based Incentive Payment System (MIPS) to the MIPS Value Pathways for quality reporting.

METHODS

We used a transparent and reproducible process to create electronic quality measures based on recommendations from 2 AAO-HNSF clinical practice guidelines (cerumen impaction and allergic rhinitis). Steps of this process include source material review, electronic content extraction, logic development, implementation barrier analysis, content encoding and structuring, and measure formalization. Proposed measures then go through the standard publication process for AAO-HNSF measures.

RESULTS

The 2 guidelines contained 29 recommendation statements, of which 7 were translated into electronic quality measures and published. Intermediate products of the guideline conversion process facilitated development and were retained to support review, updating, and transparency. Of the 7 initially published quality measures, 6 were approved as 2018 MIPS measures, and 2 continued to demonstrate a gap in care after a year of data collection.

CONCLUSION

Developing high-quality, registry-enabled measures from guidelines via a rigorous reproducible process is feasible. The streamlined process was effective in producing quality measures for publication in a timely fashion. Efforts to better identify gaps in care and more quickly recognize recommendations that would not translate well into quality measures could further streamline this process.

Clinician acceptance of complex clinical decision support systems for treatment allocation of patients with chronic low back pain

Background

The uptake of complex clinical decision support systems (CDSS) in daily practice remains low, despite the proven potential to reduce medical errors and to improve the quality of care. To improve successful implementation of a complex CDSS this study aims to identify the factors that hinder, or alleviate the acceptance of, clinicians toward the use of a complex CDSS for treatment allocation of patients with chronic low back pain.

Methods

We tested a research model in which the intention to use a CDSS by clinicians is influenced by the perceived usefulness; this usefulness, in turn is influenced by the perceived service benefits and perceived service risks. An online survey was created to test our research model and the data was analysed using Partial Least Squares Structural Equation Modelling. The study population consisted of clinicians. The online questionnaire started with demographic questions and continued with a video animation of the complex CDSS followed by the set of measurement items. The online questionnaire ended with two open questions enquiring the reasons to use and not use, a complex CDSS.

Results

Ninety-eight participants (46% general practitioners, 25% primary care physical therapists, and 29% clinicians at a rehabilitation centre) fully completed the questionnaire. Fifty-two percent of the respondents were male. The average age was 48 years (SD ± 12.2). The causal model suggests that perceived usefulness is the main factor contributing to the intention to use a complex CDSS. Perceived service benefits and risks are both significant antecedents of perceived usefulness and perceived service risks are affected by the perceived threat to autonomy and trusting beliefs, particularly benevolence and competence.

Conclusions

To improve the acceptance of complex CDSSs it is important to address the risks, but the main focus during the implementation phase should be on the expected improvements in patient outcomes and the overall gain for clinicians. Our results will help the development of complex CDSSs that fit more into the daily clinical practice of clinicians.

Exploring approaches to identify, incorporate and report patient preferences in clinical guidelines: Qualitative interviews with guideline developers

BACKGROUND

Clinical guidelines informed by patient preferences are more likely to be used and widely advocated, yet research shows that few guidelines reflect patient preferences.

OBJECTIVE

Explore how developers generate guidelines informed by patient preferences.

PATIENT INVOLVEMENT

Seventeen patients were involved as interview participants.

METHODS

Using a basic descriptive approach, we conducted and analyzed semi-structured telephone interviews with 50 participants who were involved in developing guidelines on various topics. The sample included 17 patients, 16 clinicians and 17 managers from a total of 7 countries.

RESULTS

Participants used one or more approaches to identify preferences, patient panelists, focus groups, surveys and review of published research, despite acknowledging they identified similar preferences. Participants said they incorporated preferences in all guideline development steps, but provided little detail of specific processes. Few participants said their guidelines explicitly reported how patients were engaged, preferences identified, or how preferences influenced development processes or the guideline. Enablers were patient and clinician training, supportive coordinators and chairs, involving experienced patients, and assistance from qualitative and review experts. Barriers were finding and preparing patients, clinician skepticism about benefits, and token patient involvement. Participants recommended research on how to generate preference-informed guidelines.

DISCUSSION

Ideal approaches to identify, incorporate and report patient preferences in guidelines are unclear and unproven.

PRACTICAL VALUE

Findings revealed specific ways that developers can enhance their processes (e.g. patient training, supportive coordinators and chairs, involve experts in qualitative researcher and systematic reviews) and key issues that warrant ongoing research (e.g. how best to incorporate and report preferences).

Consistency of decision support software-integrated telephone triage and associated factors: a systematic review

Background

In the recent decades, the use of computerized decision support software (CDSS)-integrated telephone triage (TT) has become an important tool for managing rising healthcare demands and overcrowding in the emergency department. Though these services have generally been shown to be effective, large gaps in the literature exist with regards to the overall quality of these systems. In the current systematic review, we aim to document the consistency of decisions that are generated in CDSS-integrated TT. Furthermore, we also seek to map those factors in the literature that have been identified to have an impact on the consistency of generated triage decisions.

Methods

As part of the TRANS-SENIOR international training and research network, a systematic review of the literature was conducted in November 2019. PubMed, Web of Science, CENTRAL, and the CINAHL database were searched. Quantitative articles including a CDSS component and addressing consistency of triage decisions and/or factors associated with triage decisions were eligible for inclusion in the current review. Studies exploring the use of other types of digital support systems for triage (i.e. web chat, video conferencing) were excluded. Quality appraisal of included studies were performed independently by two authors using the Methodological Index for Non-Randomized Studies.

Results

From a total of 1551 records that were identified, 39 full-texts were assessed for eligibility and seven studies were included in the review. All of the studies (n = 7) identified as part of our search were observational and were based on nurse-led telephone triage. Scientific efforts investigating our first aim was very limited. In total, two articles were found to investigate the consistency of decisions that are generated in CDSS-integrated TT. Research efforts were targeted largely towards the second aim of our study—all of the included articles reported factors related to the operator- (n = 6), patient- (n = 1), and/or CDSS-integrated (n = 2) characteristics to have an influence on the consistency of CDSS-integrated TT decisions.

Conclusion

To date, some efforts have been made to better understand how the use of CDSS-integrated TT systems may vary across settings. In general, however, the evidence-base surrounding this field of literature is largely inconclusive. Further evaluations must be prompted to better understand this area of research.

Protocol registration

The protocol for this study is registered in the PROSPERO database (registration number: CRD42020146323).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01472-3.

A systematic review of theoretical constructs in CDS literature

Background

Studies that examine the adoption of clinical decision support (CDS) by healthcare providers have generally lacked a theoretical underpinning. The Unified Theory of Acceptance and Use of Technology (UTAUT) model may provide such a theory-based explanation; however, it is unknown if the model can be applied to the CDS literature.

Objective

Our overall goal was to develop a taxonomy based on UTAUT constructs that could reliably characterize CDS interventions.

Methods

We used a two-step process: (1) identified randomized controlled trials meeting comparative effectiveness criteria, e.g., evaluating the impact of CDS interventions with and without specific features or implementation strategies; (2) iteratively developed and validated a taxonomy for characterizing differential CDS features or implementation strategies using three raters.

Results

Twenty-five studies with 48 comparison arms were identified. We applied three constructs from the UTAUT model and added motivational control to characterize CDS interventions. Inter-rater reliability was as follows for model constructs: performance expectancy (κ = 0.79), effort expectancy (κ = 0.85), social influence (κ = 0.71), and motivational control (κ = 0.87).

Conclusion

We found that constructs from the UTAUT model and motivational control can reliably characterize features and associated implementation strategies. Our next step is to examine the quantitative relationships between constructs and CDS adoption.

Integrating the Practical Robust Implementation and Sustainability Model With Best Practices in Clinical Decision Support Design: Implementation Science Approach

Background

Clinical decision support (CDS) design best practices are intended to provide a narrative representation of factors that influence the success of CDS tools. However, they provide incomplete direction on evidence-based implementation principles.

Objective

This study aims to describe an integrated approach toward applying an existing implementation science (IS) framework with CDS design best practices to improve the effectiveness, sustainability, and reproducibility of CDS implementations.

Methods

We selected the Practical Robust Implementation and Sustainability Model (PRISM) IS framework. We identified areas where PRISM and CDS design best practices complemented each other and defined methods to address each. Lessons learned from applying these methods were then used to further refine the integrated approach.

Results

Our integrated approach to applying PRISM with CDS design best practices consists of 5 key phases that iteratively interact and inform each other: multilevel stakeholder engagement, designing the CDS, design and usability testing, thoughtful deployment, and performance evaluation and maintenance. The approach is led by a dedicated implementation team that includes clinical informatics and analyst builder expertise.

Conclusions

Integrating PRISM with CDS design best practices extends user-centered design and accounts for the multilevel, interacting, and dynamic factors that influence CDS implementation in health care. Integrating PRISM with CDS design best practices synthesizes the many known contextual factors that can influence the success of CDS tools, thereby enhancing the reproducibility and sustainability of CDS implementations. Others can adapt this approach to their situation to maximize and sustain CDS implementation success.

Knowledge-Based Decision Support in Healthcare via Near Field Communication

The benefits of automatic identification technologies in healthcare have been largely recognized. Nevertheless, unlocking their potential to support the most knowledge-intensive medical tasks requires to go beyond mere item identification. This paper presents an innovative Decision Support System (DSS), based on a semantic enhancement of Near Field Communication (NFC) standard. Annotated descriptions of medications and patient’s case history are stored in NFC transponders and used to help caregivers providing the right therapy. The proposed framework includes a lightweight reasoning engine to infer possible incompatibilities in treatment, suggesting substitute therapies. A working prototype is presented in a rheumatology case study and preliminary performance tests are reported. The approach is independent from back-end infrastructures. The proposed DSS framework is validated in a limited but realistic case study, and performance evaluation of the prototype supports its practical feasibility. Automated reasoning on knowledge fragments extracted via NFC enables effective decision support not only in hospital centers, but also in pervasive IoT-based healthcare contexts such as first aid, ambulance transport, rehabilitation facilities and home care.