A systematic review of trials evaluating success factors of interventions with computerised clinical decision support

Promoting meaningful activities by occupational therapy in elderly care in Belgium: the ProMOTE intervention

BACKGROUND

Older people want to age in place. Despite advancing functional limitations and their desire of aging in place, they are not always faithful to therapy that maintains independence and promotes safety. Occupational therapists can facilitate aging in place. Occupational therapy is defined as the therapeutic use of everyday life occupations with persons, groups, or populations for the purpose of enhancing or enabling participation.

AIM

To describe the content a high-adherence-to-therapy and evidence-based occupational therapy intervention to optimize functional performance and social participation of home-based physically frail older adults and wellbeing of their informal caregiver, and the research activities undertaken to design this intervention.

METHODS

A roadmap was created to develop the occupational therapy intervention. This roadmap is based on the Medical Research Council (MRC) framework and is supplemented with elements of the Intervention Mapping approach. The TIDieR checklist is applied to describe the intervention in detail. A systematic review and two qualitative studies substantiated the content of the intervention scientifically.

RESULTS

The application of the first two phases of the MRC framework resulted in the ProMOTE intervention (Promoting Meaningful activities by Occupational Therapy in Elderly). The ProMOTE intervention is a high-adherence-to-therapy occupational therapy intervention that consists of six steps and describes in detail the evidence-based components that are required to obtain an operational intervention for occupational therapy practice.

CONCLUSION

This study transparently reflects on the process of a high-quality occupational therapy intervention to optimize the functional performance and social participation of the home-based physically frail older adult and describes the ProMOTE intervention in detail. The ProMOTE intervention contributes to safely aging in place and to maintaining social participation. The designed intervention goes beyond a description of the 'what'. The added value lies in the interweaving of the 'why' and 'how'. By describing the 'how', our study makes the concept of 'therapeutic use-of-self' operational throughout the six steps of the occupational therapy intervention. A further rigorous study of the effect of the ProMOTE intervention on adherence, functional performance and social participation is recommended based to facilitate the implementation of this intervention on a national level in Belgium.

Evaluation of clinical practice guideline-derived clinical decision support systems using a novel quality model

Over the last decade, clinical practice guidelines (CPGs) have become an important asset for daily life in healthcare organizations. Efficient management and digitization of CPGs help achieve organizational objectives and improve patient care and healthcare quality by reducing variability. However, digitizing CPGs is a difficult, complex task because they are usually expressed as text, and this often leads to the development of partial software solutions. At present, different research proposals and CPG-derived CDSS (clinical decision support system) do exist for managing CPG digitalization lifecycles (from modeling to deployment and execution), but they do not all provide full lifecycle support, making it more difficult to choose solutions or proposals that fully meet the needs of a healthcare organization. This paper proposes a method based on quality models to uniformly compare and evaluate technological tools, providing a rigorous method that uses qualitative and quantitative analysis of technological aspects. In addition, this paper also presents how this method has been instantiated to evaluate and compare CPG-derived CDSS by highlighting each phase of the CPG digitization lifecycle. Finally, discussion and analysis of currently available tools are presented, identifying gaps and limitations.

Clinical Decision Support System in laboratory medicine

Clinical Decision Support Systems (CDSS) have been implemented in almost all healthcare settings. Laboratory medicine (LM), is one of the most important structured health data stores, but efforts are still needed to clarify the use and scope of these tools, especially in the laboratory setting. The aim is to clarify CDSS concept in LM, in the last decade. There is no consensus on the definition of CDSS in LM. A theoretical definition of CDSS in LM should capture the aim of driving significant improvements in LM mission, prevention, diagnosis, monitoring, and disease treatment. We identified the types, workflow and data sources of CDSS. The main applications of CDSS in LM were diagnostic support and clinical management, patient safety, workflow improvements, and cost containment. Laboratory professionals, with their expertise in quality improvement and quality assurance, have a chance to be leaders in CDSS.

Implementing patient decision aids into general practice clinical decision support systems: Feasibility study in cardiovascular disease prevention

Objective

Patient decision aids (DA) facilitate shared decision making, but implementation remains a challenge. This study tested the feasibility of integrating a cardiovascular disease (CVD) prevention DA into general practice software.

Methods

We developed a desktop computer application (app) to auto-populate a CVD prevention DA from general practice medical records. 4 practices received monthly practice reports from July-Nov 2021, and 2 practices use the app with limited engagement. CVD risk assessment data and app use were monitored.

Results

The proportion of eligible patients with complete CVD risk assessment data ranged from 59 to 94%. Monthly app use ranged from 0 to 285 sessions by 13 individual practice staff including GPs and nurses, with staff using the app an average of 67 sessions during the study period. High users in the 5-month study period continued to use the app for 10 months. Low use was attributed to reduced staff capacity during COVID-19 and technical issues.

Conclusion

High users sustained interest in the app, but additional strategies are required for low users. The study will inform implementation plans for new guidelines.

Innovation

This study showed it is feasible to integrate patient decision aids with Australian general practice software, despite the challenges of COVID-19 at the time of the study.

Examining primary care provider experiences with using a clinical decision support tool for pain management

Objective

To evaluate primary care provider (PCP) experiences using a clinical decision support (CDS) tool over 16 months following a user-centered design process and implementation.

Materials and Methods

We conducted a qualitative evaluation of the Chronic Pain OneSheet (OneSheet), a chronic pain CDS tool. OneSheet provides pain- and opioid-related risks, benefits, and treatment information for patients with chronic pain to PCPs. Using the 5 Rights of CDS framework, we conducted and analyzed semi-structured interviews with 19 PCPs across 2 academic health systems.

Results

PCPs stated that OneSheet mostly contained the right information required to treat patients with chronic pain and was correctly located in the electronic health record. PCPs used OneSheet for distinct subgroups of patients with chronic pain, including patients prescribed opioids, with poorly controlled pain, or new to a provider or clinic. PCPs reported variable workflow integration and selective use of certain OneSheet features driven by their preferences and patient population. PCPs recommended broadening OneSheet access to clinical staff and patients for data entry to address clinician time constraints.

Discussion

Differences in patient subpopulations and workflow preferences had an outsized effect on CDS tool use even when the CDS contained the right information identified in a user-centered design process.

Conclusions

To increase adoption and use, CDS design and implementation processes may benefit from increased tailoring that accommodates variation and dynamics among patients, visits, and providers.

A prospective observational concordance study to evaluate computational model-driven clinical practice guidelines for Type 2 diabetes mellitus

BACKGROUND

Clinical Practice Guidelines (CPGs) provide healthcare professionals with performance and decision-making support during the treatment of patients. Sometimes, however, they are poorly implemented. The IDE4ICDS platform was developed and validated with CPGs for type 2 diabetes mellitus (T2DM).

OBJECTIVE

The main objective of this paper is to present the results of the clinical validation of the IDE4ICDS platform in a real clinical environment at two health clinics in the Andalusian Public Health System (SSPA) in the southern Spanish region of Andalusia.

METHODS

National and international knowledge sources on T2DM were selected and reviewed and used to define a diabetes CPG model on the IDE4ICDS platform. Once the diabetes CPG was configured and deployed, it was validated. A total of 506 patients were identified as meeting the inclusion criteria, of whom 130 could be recruited and 89 attended the appointment.

RESULTS

A concordance analysis was performed with the kappa value. Overall agreement between the recommendations provided by the system and those recorded in each patient's EHR was good (0.61 - 0.80) with a total kappa index of 0.701, leading to the conclusion that the system provided appropriate recommendations for each patient and was therefore well-functioning.

CONCLUSIONS

A series of possible improvements were identified based on the limitations for the recovery of variables related to the quality of these recolected variables, the detection of duplicate recommendations based on different input variables for the same patient, and clinical usability, such as the capacity to generate reports based on the recommendations generated. Nevertheless, the project resulted in the IDE4ICDS platform: a Clinical Decision Support System (CDSS) capable of providing appropriate recommendations for improving the management and quality of patient care and optimizing health outcomes. The result of this validation is a safe and effective pathway for developing and adopting digital transformation at the regional scale of the use of biomedical knowledge in real healthcare.

Complex implementation mechanisms in primary care: do physicians' beliefs about the effectiveness of innovation play a mediating role? Applying a realist inquiry and structural equation modeling approach in a formative evaluation study

BACKGROUND

The adoption of digital health technologies can improve the quality of care for polypharmacy patients, if the underlying complex implementation mechanisms are better understood. Context effects play a critical role in relation to implementation mechanisms. In primary care research, evidence on the effects of context in the adoption of digital innovation for polypharmacy management is lacking.

STUDY AIM

This study aims to identify contextual factors relevant to physician behavior and how they might mediate the adoption process.

METHODS

The physicians who participated in this formative evaluation study (n = 218) were part of the intervention group in a cluster-randomized controlled trial (AdAM). The intervention group implemented a digital innovation for clinical decision making in polypharmacy. A three-step methodological approach was used: (1) a realist inquiry approach, which involves the description of a context-mechanism-outcome configuration for the primary care setting; (2) a belief elicitation approach, which involves qualitative content analysis and the development of a quantitative latent contextualized scale; and (3) a mediation analysis using structural equation modeling (SEM) based on quantitative survey data from physicians to assess the mediating role of the contextualized scale (n = 179).

RESULTS

The key dimensions of a (1) context-mechanism-outcome model were mapped and refined. A (2) latent construct of the physicians' innovation beliefs related to the effectiveness of polypharmacy management practices was identified. Innovation beliefs play a (3) mediating role between the organizational readiness to implement change (p < 0.01) and the desired behavioral intent of physicians to adopt digital innovation (p < 0.01; R² = 0.645). Our contextualized model estimated significant mediation, with a relative size of 38% for the mediation effect. Overall, the model demonstrated good fit indices (CFI = 0.985, RMSEA = 0.034).

CONCLUSION

Physician adoption is directly affected by the readiness of primary care organizations for the implementation of change. In addition, the mediation analysis revealed that this relationship is indirectly influenced by primary care physicians' beliefs regarding the effectiveness of digital innovation. Both individual physician beliefs and practice organizational capacity could be equally prioritized in developing implementation strategies. The methodological approach used is suitable for the evaluation of complex implementation mechanisms. It has been proven to be an advantageous approach for formative evaluation.

TRIAL REGISTRATION

NCT03430336 . First registration: 12/02/2018.

CLINICALTRIALS

gov.

The effects of computerised decision support systems on nursing and allied health professional performance and patient outcomes: a systematic review and user contextualisation

Background

Computerised decision support systems (CDSS) are widely used by nurses and allied health professionals but their effect on clinical performance and patient outcomes is uncertain.

Objectives

Evaluate the effects of clinical decision support systems use on nurses', midwives' and allied health professionals' performance and patient outcomes and sense-check the results with developers and users.

Eligibility criteria

Comparative studies (randomised controlled trials (RCTs), non-randomised trials, controlled before-and-after (CBA) studies, interrupted time series (ITS) and repeated measures studies comparing) of CDSS versus usual care from nurses, midwives or other allied health professionals.

Information sources

Nineteen bibliographic databases searched October 2019 and February 2021.

Risk of bias

Assessed using structured risk of bias guidelines; almost all included studies were at high risk of bias.

Synthesis of results

Heterogeneity between interventions and outcomes necessitated narrative synthesis and grouping by: similarity in focus or CDSS-type, targeted health professionals, patient group, outcomes reported and study design.

Included studies

Of 36,106 initial records, 262 studies were assessed for eligibility, with 35 included: 28 RCTs (80%), 3 CBA studies (8.6%), 3 ITS (8.6%) and 1 non-randomised trial, a total of 1318 health professionals and 67,595 patient participants. Few studies were multi-site and most focused on decision-making by nurses (71%) or paramedics (5.7%). Standalone, computer-based CDSS featured in 88.7% of the studies; only 8.6% of the studies involved 'smart' mobile or handheld technology. Care processes - including adherence to guidance - were positively influenced in 47% of the measures adopted. For example, nurses' adherence to hand disinfection guidance, insulin dosing, on-time blood sampling, and documenting care were improved if they used CDSS. Patient care outcomes were statistically - if not always clinically - significantly improved in 40.7% of indicators. For example, lower numbers of falls and pressure ulcers, better glycaemic control, screening of malnutrition and obesity, and accurate triaging were features of professionals using CDSS compared to those who were not.

Evidence limitations

Allied health professionals (AHPs) were underrepresented compared to nurses; systems, studies and outcomes were heterogeneous, preventing statistical aggregation; very wide confidence intervals around effects meant clinical significance was questionable; decision and implementation theory that would have helped interpret effects - including null effects - was largely absent; economic data were scant and diverse, preventing estimation of overall cost-effectiveness.

Interpretation

CDSS can positively influence selected aspects of nurses', midwives' and AHPs' performance and care outcomes. Comparative research is generally of low quality and outcomes wide ranging and heterogeneous. After more than a decade of synthesised research into CDSS in healthcare professions other than medicine, the effect on processes and outcomes remains uncertain. Higher-quality, theoretically informed, evaluative research that addresses the economics of CDSS development and implementation is still required.

Future work

Developing nursing CDSS and primary research evaluation.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in Health and Social Care Delivery Research; 2023. See the NIHR Journals Library website for further project information.

Registration

PROSPERO [number: CRD42019147773].

Automated feedback modestly improves perioperative treatment adherence of postoperative nausea and vomiting

STUDY OBJECTIVE

Extensive evidence demonstrates that medical record modernization and a vast amount of available data have not overcome the gap between recommended and delivered care. This study aimed to evaluate the use of clinical decision support (CDS) in conjunction with feedback (post-hoc reporting) to improve PONV medication administration compliance and postoperative nausea and vomiting (PONV) outcomes.

DESIGN

Single center, prospective observational study between January 1, 2015, and June 30, 2017.

SETTING

Perioperative care at a university-affiliated tertiary care center.

PATIENTS

57,401 adult patients who received general anesthesia in a non-emergency setting.

INTERVENTION

A multi-phased intervention that consisted of post-hoc reporting for individual providers by email about PONV occurrences in their patients, followed by directive CDS through preoperative daily case emails that provided therapeutic PONV prophylaxis recommendations based on patients' PONV risk scores.

MEASUREMENT

Compliance with PONV medication recommendations, as well as hospital rates of PONV were measured.

MAIN RESULT

Over the study period, there was a 5.5% (95% CI, 4.2% to 6.4%; p < 0.001) improvement in the compliance of PONV medication administration along with an 8.7% (95% CI, 7.1% to 10.2%, p < 0.001) reduction in PONV rescue medication administration in the PACU. However, there was no statistically or clinically significant reduction in the prevalence of PONV in the PACU. The prevalence of PONV rescue medication administration decreased during the Intervention Rollout Period (odds ratio 0.95 [per month]; 95% CI, 0.91 to 0.99; p = 0.017), and during the Feedback with CDS Recommendation Period (odds ratio, 0.96 [per month]; 95% CI, 0.94 to 0.99; p = 0.013).

CONCLUSION

PONV medication administration compliance modestly improves with CDS in conjunction with post-hoc reporting; however, no improvement in PACU rates of PONV occurred.

Evaluating the costs and consequences of computerized clinical decision support systems in hospitals: a scoping review and recommendations for future practice

OBJECTIVE

Sustainable investment in computerized decision support systems (CDSS) requires robust evaluation of their economic impacts compared with current clinical workflows. We reviewed current approaches used to evaluate the costs and consequences of CDSS in hospital settings and presented recommendations to improve the generalizability of future evaluations.

MATERIALS AND METHODS

A scoping review of peer-reviewed research articles published since 2010. Searches were completed in the PubMed, Ovid Medline, Embase, and Scopus databases (last searched February 14, 2023). All studies reported the costs and consequences of a CDSS-based intervention compared with current hospital workflows. Findings were summarized using narrative synthesis. Individual studies were further appraised against the Consolidated Health Economic Evaluation and Reporting (CHEERS) 2022 checklist.

RESULTS

Twenty-nine studies published since 2010 were included. Studies evaluated CDSS for adverse event surveillance (5 studies), antimicrobial stewardship (4 studies), blood product management (8 studies), laboratory testing (7 studies), and medication safety (5 studies). All studies evaluated costs from a hospital perspective but varied based on the valuation of resources affected by CDSS implementation, and the measurement of consequences. We recommend future studies follow guidance from the CHEERS checklist; use study designs that adjust for confounders; consider both the costs of CDSS implementation and adherence; evaluate consequences that are directly or indirectly affected by CDSS-initiated behavior change; examine the impacts of uncertainty and differences in outcomes across patient subgroups.

DISCUSSION AND CONCLUSION

Improving consistency in the conduct and reporting of evaluations will enable detailed comparisons between promising initiatives, and their subsequent uptake by decision-makers.

Pros and cons of streamlining and use of computerised clinical decision support systems to future-proof oncological multidisciplinary team meetings

Introduction

Nowadays nearly every patient with cancer is discussed in a multidisciplinary team meeting (MDTM) to determine an optimal treatment plan. The growth in the number of patients to be discussed is unsustainable. Streamlining and use of computerised clinical decision support systems (CCDSSs) are two major ways to restructure MDTMs. Streamlining is the process of selecting the patients who need to be discussed and in which type of MDTM. Using CCDSSs, patient data is automatically loaded into the minutes and a guideline-based treatment proposal is generated. We aimed to identify the pros and cons of streamlining and CCDSSs.

Methods

Semi-structured interviews were conducted with Dutch MDTM participants. With purposive sampling we maximised variation in participants' characteristics. Interview data were thematically analysed.

Results

Thirty-five interviews were analysed. All interviewees agreed on the need to change the current MDTM workflow. Streamlining suggestions were thematised based on standard and complex cases and the location of the MDTM (i.e. local, regional or nationwide). Interviewees suggested easing the pressure on MDTMs by discussing standard cases briefly, not at all, or outside the MDTM with only two to three specialists. Complex cases should be discussed in tumour-type-specific regional MDTMs and highly complex cases by regional/nationwide expert teams. Categorizing patients as standard or complex was found to be the greatest challenge of streamlining. CCDSSs were recognised as promising, although none of the interviewees had made use of them. The assumed advantage was their capacity to generate protocolised treatment proposals based on automatically uploaded patient data, to unify treatment proposals and to facilitate research. However, they were thought to limit the freedom to deviate from the treatment advice.

Conclusion

To make oncological MDTMs sustainable, methods of streamlining should be developed and introduced. Physicians still have doubts about the value of CCDSSs.

Scale up of implementation of a multidimensional intervention to enhance hypertension and diabetes care at the primary care setting: A protocol for a cluster-randomized study in Brazil

BACKGROUND

Hypertension and diabetes mellitus (DM) are highly prevalent in low and middle-income countries (LMICs), and the proportion of patients with uncontrolled diseases is higher than in high-income countries. Innovative strategies are required to surpass barriers of low sources, distance and quality of health care. Our aim is to assess the uptake and effectiveness of the implementation of an integrated multidimensional strategy in the primary care setting, for the management of people with hypertension and diabetes mellitus in Brazil.

METHODS

This scale up implementation study called Control of Hypertension and diAbetes in MINas Gerais (CHArMING) Project has mixed-methods, and comprehends 4 steps: (1) needs assessment, including a standardized structured questionnaire and focus groups with health care practitioners; (2) baseline period, 3 months before the implementation of the intervention; (3) cluster randomized controlled trial (RCT) with a 12-months follow-up period; and (4) a qualitative study after the end of follow-up. The cluster RCT will randomize 35 centers to intervention (n = 18) or usual care (n = 17). Patients ≥18 years old, with diagnosis of hypertension and/or DM, of 5 Brazilian cities in a resource-constrained area will be enrolled. The intervention consists of a multifaceted strategy, with a multidisciplinary approach, including telehealth tools (decision support systems, short message service, telediagnosis), continued education with an approach to issues related to the care of people with hypertension and diabetes in primary care, including pharmacological and non-pharmacological treatment and behavioral change. The project has actions focused on professionals and patients.

CONCLUSIONS

This study consists of a multidimensional strategy with multidisciplinary approach using digital health to improve the control of hypertension and/or DM in the primary health care setting. We expect to provide the basis for implementing an innovative management program for hypertension and DM in Brazil, aiming to reduce the present and future burden of these diseases in Brazil and other LMICs.

CLINICAL TRIAL IDENTIFIER

This study was registered in ClinicalTrials.gov. (NCT05660928).

Facilitators and barriers to conducting an efficient, competent and high-quality oncological multidisciplinary team meeting

BACKGROUND

Optimal oncological care nowadays requires discussing every patient in a multidisciplinary team meeting (MDTM). The number of patients to be discussed is rising rapidly due to the increasing incidence and prevalence of cancer and the emergence of new multidisciplinary treatment options. This puts MDTMs under considerable time pressure. The aim of this study is therefore to identify the facilitators and barriers with regard to performing an efficient, competent and high-quality MDTM.

METHODS

Semistructured interviews were conducted with Dutch medical specialists and residents participating in oncological MDTMs. Purposive sampling was used to maximise variation in participants' professional and demographic characteristics (eg, sex, medical specialist vs resident, specialty, type and location of affiliated hospital). Interview data were systematically analysed according to the principles of thematic content analysis.

RESULTS

Sixteen medical specialists and 19 residents were interviewed. All interviewees agreed that attending and preparing MDTMs is time-consuming and indicated the need for optimal execution in order to ensure that MDTMs remain feasible in the near future. Four themes emerged that are relevant to achieving an optimal MDTM: (1) organisational aspects; (2) participants' responsibilities and requirements; (3) competences, behaviour and team dynamics and (4) meeting content. Good organisation, a sound structure and functioning information and communication technology facilitate high-quality MDTMs. Multidisciplinary collaboration and adequate communication are essential competences for participants; a lack thereof and the existence of a hierarchy are hindering factors.

CONCLUSION

Conducting an efficient, competent and high-quality oncological MDTM is facilitated and hindered by many factors. Being aware of these factors provides opportunities for optimising MDTMs, which are under pressure due to the increase in the number of patients to discuss.

Clinical Relevance of Drug-Drug Interactions With Antibiotics as Listed in a National Medication Formulary: Results From Two Large Population-Based Case-Control Studies in Patients Aged 65-100 Years Using Linked English Primary Care and Hospital Data

This study evaluated drug-drug interactions (DDIs) between antibiotic and nonantibiotic drugs listed with warnings of severe outcomes in the British National Formulary based on adverse drug reaction (ADR) detectable with routine International Classification of Diseases, Tenth Revision coding. Data sources were Clinical Practice Research Databank GOLD and Aurum anonymized electronic health records from English general practices linked to hospital admission records. In propensity-matched case-control study, outcomes were ADR or emergency admissions. Analyzed were 121,546 ADR-related admission cases matched to 638,238 controls. For most antibiotics, adjusted odds ratios (aORs) for ADR-related hospital admission were large (aOR for trimethoprim 4.13; 95% confidence interval (CI), 3.97-4.30). Of the 51 DDIs evaluated for ADR-related admissions, 38 DDIs (74.5%) had statistically increased aORs of concomitant exposure compared with nonexposure (mean aOR 3.96; range 1.59-11.42); for the 89 DDIs for emergency hospital admission, the results were 75 (84.3%) and mean aOR 2.40; range 1.43-4.17. Changing reference group to single antibiotic exposure reduced aORs for concomitant exposure by 76.5% and 83.0%, respectively. Medicines listed to cause nephrotoxicity substantially increased risks that were related to number of medicines (aOR was 2.55 (95% CI, 2.46-2.64) for current use of 1 and 10.44 (95% CI, 7.36-14.81) for 3 or more medicines). In conclusion, no evidence of substantial risk was found for multiple DDIs with antibiotics despite warnings of severe outcomes in a national formulary and flagging in electronic health record software. It is proposed that the evidence base for inclusion of DDIs in national formularies be strengthened and made publicly accessible and indiscriminate flagging, which compounds alert fatigue, be reduced.

The use and impact of digital COVID-19 tracking in adult social care: a prospective cohort study of care homes in Greater Manchester

BACKGROUND

To support proactive care during the coronavirus pandemic, a digital COVID-19 symptom tracker was deployed in Greater Manchester (UK) care homes. This study aimed to understand what factors were associated with the post-uptake use of the tracker and whether the tracker had any effects in controlling the spread of COVID-19.

METHODS

Daily data on COVID-19, tracker uptake and use, and other key indicators such as staffing levels, the number of staff self-isolating, availability of personal protective equipment, bed occupancy levels, and any problems in accepting new residents were analysed for 547 care homes across Greater Manchester for the period April 2020 to April 2021. Differences in tracker use across local authorities, types of care homes, and over time were assessed using correlated effects logistic regressions. Differences in numbers of COVID-19 cases in homes adopting versus not adopting the tracker were compared via event design difference-in-difference estimations.

RESULTS

Homes adopting the tracker used it on 44% of days post-adoption. Use decreased by 88% after one year of uptake (odds ratio 0.12; 95% confidence interval 0.06-0.28). Use was highest in the locality initiating the project (odds ratio 31.73; 95% CI 3.76-268.05). Care homes owned by a chain had lower use (odds ratio 0.30; 95% CI 0.14-0.63 versus single ownership care homes), and use was not associated with COVID-19 or staffing levels. Tracker uptake had no impact on controlling COVID-19 spread. Staff self-isolating and local area COVID-19 cases were positively associated with lagged COVID-19 spread in care homes (relative risks 1.29; 1.2-1.4 and 1.05; 1.0-1.1, respectively).

CONCLUSIONS

The use of the COVID-19 symptom tracker in care homes was not maintained except in Locality 1 and did not appear to reduce the COVID-19 spread. COVID-19 cases in care homes were mainly driven by care home local-area COVID-19 cases and infections among the staff members. Digital deterioration trackers should be co-produced with care home staff, and local authorities should provide long-term support in their adoption and use.

Reflection Machines: Supporting Effective Human Oversight Over Medical Decision Support Systems

Human decisions are increasingly supported by decision support systems (DSS). Humans are required to remain "on the loop," by monitoring and approving/rejecting machine recommendations. However, use of DSS can lead to overreliance on machines, reducing human oversight. This paper proposes "reflection machines" (RM) to increase meaningful human control. An RM provides a medical expert not with suggestions for a decision, but with questions that stimulate reflection about decisions. It can refer to data points or suggest counterarguments that are less compatible with the planned decision. RMs think against the proposed decision in order to increase human resistance against automation complacency. Building on preliminary research, this paper will (1) make a case for deriving a set of design requirements for RMs from EU regulations, (2) suggest a way how RMs could support decision-making, (3) describe the possibility of how a prototype of an RM could apply to the medical domain of chronic low back pain, and (4) highlight the importance of exploring an RM's functionality and the experiences of users working with it.

Artificial Intelligence Implementation in Healthcare: A Theory-Based Scoping Review of Barriers and Facilitators

There is a large proliferation of complex data-driven artificial intelligence (AI) applications in many aspects of our daily lives, but their implementation in healthcare is still limited. This scoping review takes a theoretical approach to examine the barriers and facilitators based on empirical data from existing implementations. We searched the major databases of relevant scientific publications for articles related to AI in clinical settings, published between 2015 and 2021. Based on the theoretical constructs of the Consolidated Framework for Implementation Research (CFIR), we used a deductive, followed by an inductive, approach to extract facilitators and barriers. After screening 2784 studies, 19 studies were included in this review. Most of the cited facilitators were related to engagement with and management of the implementation process, while the most cited barriers dealt with the intervention's generalizability and interoperability with existing systems, as well as the inner settings' data quality and availability. We noted per-study imbalances related to the reporting of the theoretic domains. Our findings suggest a greater need for implementation science expertise in AI implementation projects, to improve both the implementation process and the quality of scientific reporting.

Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation

Background

Clinical decision support (CDS) is increasingly used to facilitate chronic disease care. Despite increased availability of electronic health records and the ongoing development of new CDS technologies, uptake of CDS into routine clinical settings is inconsistent. This qualitative systematic review seeks to synthesise healthcare provider experiences of CDS—exploring the barriers and enablers to implementing, using, evaluating, and sustaining chronic disease CDS systems.

Methods

A search was conducted in Medline, CINAHL, APA PsychInfo, EconLit, and Web of Science from 2011 to 2021. Primary research studies incorporating qualitative findings were included if they targeted healthcare providers and studied a relevant chronic disease CDS intervention. Relevant CDS interventions were electronic health record-based and addressed one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolaemia. Qualitative findings were synthesised using a meta-aggregative approach.

Results

Thirty-three primary research articles were included in this qualitative systematic review. Meta-aggregation of qualitative data revealed 177 findings and 29 categories, which were aggregated into 8 synthesised findings. The synthesised findings related to clinical context, user, external context, and technical factors affecting CDS uptake. Key barriers to uptake included CDS systems that were simplistic, had limited clinical applicability in multimorbidity, and integrated poorly into existing workflows. Enablers to successful CDS interventions included perceived usefulness in providing relevant clinical knowledge and structured chronic disease care; user confidence gained through training and post training follow-up; external contexts comprised of strong clinical champions, allocated personnel, and technical support; and CDS technical features that are both highly functional, and attractive.

Conclusion

This systematic review explored healthcare provider experiences, focussing on barriers and enablers to CDS use for chronic diseases. The results provide an evidence-base for designing, implementing, and sustaining future CDS systems. Based on the findings from this review, we highlight actionable steps for practice and future research.

Trial registration

PROSPERO CRD42020203716

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00326-x.

A Personalized Ontology-Based Decision Support System for Complex Chronic Patients: Retrospective Observational Study

BACKGROUND

Due to an increase in life expectancy, the prevalence of chronic diseases is also on the rise. Clinical practice guidelines (CPGs) provide recommendations for suitable interventions regarding different chronic diseases, but a deficiency in the implementation of these CPGs has been identified. The PITeS-TiiSS (Telemedicine and eHealth Innovation Platform: Information Communications Technology for Research and Information Challenges in Health Services) tool, a personalized ontology-based clinical decision support system (CDSS), aims to reduce variability, prevent errors, and consider interactions between different CPG recommendations, among other benefits.

OBJECTIVE

The aim of this study is to design, develop, and validate an ontology-based CDSS that provides personalized recommendations related to drug prescription. The target population is older adult patients with chronic diseases and polypharmacy, and the goal is to reduce complications related to these types of conditions while offering integrated care.

METHODS

A study scenario about atrial fibrillation and treatment with anticoagulants was selected to validate the tool. After this, a series of knowledge sources were identified, including CPGs, PROFUND index, LESS/CHRON criteria, and STOPP/START criteria, to extract the information. Modeling was carried out using an ontology, and mapping was done with Health Level 7 Fast Healthcare Interoperability Resources (HL7 FHIR) and Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT; International Health Terminology Standards Development Organisation). Once the CDSS was developed, validation was carried out by using a retrospective case study.

RESULTS

This project was funded in January 2015 and approved by the Virgen del Rocio University Hospital ethics committee on November 24, 2015. Two different tasks were carried out to test the functioning of the tool. First, retrospective data from a real patient who met the inclusion criteria were used. Second, the analysis of an adoption model was performed through the study of the requirements and characteristics that a CDSS must meet in order to be well accepted and used by health professionals. The results are favorable and allow the proposed research to continue to the next phase.

CONCLUSIONS

An ontology-based CDSS was successfully designed, developed, and validated. However, in future work, validation in a real environment should be performed to ensure the tool is usable and reliable.

Integration of Risk Scores and Integration Capability in Electronic Patient Records

BACKGROUND

Digital availability of patient data is continuously improving with the increasing implementation of electronic patient records in physician practices. The emergence of digital health data defines new fields of application for data analytics applications, which in turn offer extensive options of using data. Common areas of data analytics applications include decision support, administration, and fraud detection. Risk scores play an important role in compiling algorithms that underlay tools for decision support.

OBJECTIVES

This study aims to identify the current state of risk score integration and integration capability in electronic patient records for cardiovascular disease and diabetes in German primary care practices.

METHODS

We developed an evaluation framework to determine the current state of risk score integration and future integration options for four cardiovascular disease risk scores (arriba, Pooled Cohort Equations, QRISK3, and Systematic Coronary Risk Evaluation) and two diabetes risk scores (Finnish Diabetes Risk Score and German Diabetes Risk Score). We then used this framework to evaluate the integration of risk scores in common practice software solutions by examining the software and inquiring the respective software contact person.

RESULTS

Our evaluation showed that the most widely integrated risk score is arriba, as recommended by German medical guidelines. Every software version in our sample provided either an interface to arriba or the option to implement one. Our assessment of integration capability revealed a more nuanced picture. Results on data availability were mixed. Each score contains at least one variable, which requires laboratory diagnostics. Our analysis of data standardization showed that only one score documented all variables in a standardized way.

CONCLUSION

Our assessment revealed that the current state of risk score integration in physician practice software is rather low. Integration capability currently faces some obstacles. Future research should develop a comprehensive framework that considers the reasonable integration of risk scores into practice workflows, disease prevention programs, and the awareness of physicians and patients.

Comparing "people-like-me" and linear mixed model predictions of functional recovery following knee arthroplasty

OBJECTIVE

Prediction models can be useful tools for monitoring patient status and personalizing treatment in health care. The goal of this study was to compare the relative strengths and weaknesses of 2 different approaches for predicting functional recovery after knee arthroplasty: a neighbors-based "people-like-me" (PLM) approach and a linear mixed model (LMM) approach.

MATERIALS AND METHODS

We used 2 distinct datasets to train and then test PLM and LMM prediction approaches for functional recovery following knee arthroplasty. We used the Timed Up and Go (TUG)-a common test of mobility-to operationalize physical function. Both approaches used patient characteristics and baseline postoperative TUG values to predict TUG recovery from days 1-425 following surgery. We then compared the accuracy and precision of PLM and LMM predictions.

RESULTS

A total of 317 patient records with 1379 TUG observations were used to train PLM and LMM approaches, and 456 patient records with 1244 TUG observations were used to test the predictions. The approaches performed similarly in terms of mean squared error and bias, but the PLM approach provided more accurate and precise estimates of prediction uncertainty.

DISCUSSION AND CONCLUSION

Overall, the PLM approach more accurately and precisely predicted TUG recovery following knee arthroplasty. These results suggest PLM predictions may be more clinically useful for monitoring recovery and personalizing care following knee arthroplasty. However, clinicians and organizations seeking to use predictions in practice should consider additional factors (eg, resource requirements) when selecting a prediction approach.

Design, effectiveness, and economic outcomes of contemporary chronic disease clinical decision support systems: a systematic review and meta-analysis

Objectives

Electronic health record-based clinical decision support (CDS) has the potential to improve health outcomes. This systematic review investigates the design, effectiveness, and economic outcomes of CDS targeting several common chronic diseases.

Material and Methods

We conducted a search in PubMed (Medline), EBSCOHOST (CINAHL, APA PsychInfo, EconLit), and Web of Science. We limited the search to studies from 2011 to 2021. Studies were included if the CDS was electronic health record-based and targeted one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolemia. Studies with effectiveness or economic outcomes were considered for inclusion, and a meta-analysis was conducted.

Results

The review included 76 studies with effectiveness outcomes and 9 with economic outcomes. Of the effectiveness studies, 63% described a positive outcome that favored the CDS intervention group. However, meta-analysis demonstrated that effect sizes were heterogenous and small, with limited clinical and statistical significance. Of the economic studies, most full economic evaluations (n = 5) used a modeled analysis approach. Cost-effectiveness of CDS varied widely between studies, with an estimated incremental cost-effectiveness ratio ranging between USD$2192 to USD$151 955 per QALY.

Conclusion

We summarize contemporary chronic disease CDS designs and evaluation results. The effectiveness and cost-effectiveness results for CDS interventions are highly heterogeneous, likely due to differences in implementation context and evaluation methodology. Improved quality of reporting, particularly from modeled economic evaluations, would assist decision makers to better interpret and utilize results from these primary research studies.

Registration

PROSPERO (CRD42020203716)

The development and implementation of a guideline-based clinical decision support system to improve empirical antibiotic prescribing

Background

To describe and evaluate a clinical decision support system (CDSS) for empirical antibiotic therapy using a systematic framework.

Methods

A reporting framework for behavior change intervention implementation was used, which includes several domains: development, evaluation and implementation. Within the development domain a description is given of the engagement of stakeholders, a rationale for how the CDSS may influence antibiotic prescribing and a detailed outline of how the system was developed. Within the evaluation domain a technical validation is performed and the interaction between potential users and the CDSS is analyzed. Within the domain of implementation a description is given on how the CDSS was tested in the real world and the strategies that were used for implementation and adoption of the CDSS.

Results

Development: a CDSS was developed, with the involvement of stakeholders, to assist empirical antibiotic prescribing by physicians. Evaluation: Technical problems were determined during the validation process and corrected in a new CDSS version. A usability study was performed to assess problems in the system-user interaction. Implementation: In 114 patients the antibiotic advice that was generated by the CDSS was followed. For 54 patients the recommendations were not adhered to.

Conclusions

This study describes the development and validation of a CDSS for empirical antibiotic therapy and shows the usefulness of the systematic framework for reporting CDSS interventions. In addition it shows that CDSS recommendations are not always adhered to which is associated with incorrect use of the system.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01860-3.

Clinical decision support for familial hypercholesterolemia (CDS-FH): Rationale and design of a cluster randomized trial in primary care

BACKGROUND

Familial hypercholesterolemia (FH) is an underdiagnosed and undertreated genetic disorder with high risk of premature atherosclerotic cardiovascular disease and death. Clinical decision support (CDS) systems have the potential to aid in the identification and management of patients with FH. Prior studies using computer-based systems to screen patients for FH have shown promising results, but there has been no randomized controlled trial conducted. The aim of the current cluster randomized study is to evaluate if a CDS can increase the identification of FH.

METHODS

We have developed a CDS integrated in the electronic health records that will be activated in patients with elevated cholesterol levels (total cholesterol >8 mmol/L or low-density lipoprotein-cholesterol >5.5 mmol/L, adjusted for age, ongoing lipid lowering therapy and presence of premature coronary artery disease) at increased risk for FH. When activated, the CDS will urge the physician to send an automatically generated referral to the local lipid clinic for further evaluation. To evaluate the effects of the CDS, all primary care clinics will be cluster randomized 1:1 to either CDS intervention or standard care in a Swedish region with almost 500,000 inhabitants. The primary endpoint will be the number of patients diagnosed with FH at 30 months. Resource use and long-term health consequences will be estimated to assess the cost-effectiveness of the intervention.

CONCLUSION

Despite increasing awareness of FH, the condition remains underdiagnosed and undertreated. The present study will investigate whether a CDS can increase the number of patients being diagnosed with FH.

Identification of Uncontrolled Symptoms in Cancer Patients Using Natural Language Processing

CONTEXT

For patients with cancer, uncontrolled pain and other symptoms are the leading cause of unplanned hospitalizations. Early access to specialty palliative care (PC) is effective to reduce symptom burden, but more efficient approaches are needed for rapid identification and referral. Information on symptom burden largely exists in free-text notes, limiting its utility as a trigger for best practice alerts or automated referrals.

OBJECTIVES

To evaluate whether natural language processing (NLP) can be used to identify uncontrolled symptoms (pain, dyspnea, or nausea/vomiting) in the electronic health record (EHR) among hospitalized cancer patients with advanced disease.

METHODS

The dataset included 1,644 hospitalization encounters for cancer patients admitted from 1/2017 -6/2019. We randomly sampled 296 encounters, which included 15,580 clinical notes. We manually reviewed the notes and recorded symptom severity. The primary endpoint was an indicator for whether a symptom was labeled as "controlled" (none, mild, not reported) or as "uncontrolled" (moderate or severe). We randomly split the data into training and test sets and used the Random Forest algorithm to evaluate final model performance.

RESULTS

Our models predicted presence of an uncontrolled symptom with the following performance: pain with 61% accuracy, 69% sensitivity, and 46% specificity (F1: 69.5); nausea/vomiting with 68% accuracy, 21% sensitivity, and 90% specificity (F1: 29.4); and dyspnea with 80% accuracy, 22% sensitivity, and 88% specificity (F1: 21.1).

CONCLUSION

This study demonstrated initial feasibility of using NLP to identify hospitalized cancer patients with uncontrolled symptoms. Further model development is needed before these algorithms could be implemented to trigger early access to PC.

Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake

BACKGROUND

Computerized clinical decision support systems (CDSSs) are a promising knowledge translation tool, but often fail to meaningfully influence the outcomes they target. Low CDSS provider uptake is a potential contributor to this problem but has not been systematically studied. The objective of this systematic review and meta-regression was to determine reported CDSS uptake and identify which CDSS features may influence uptake.

METHODS

Medline, Embase, CINAHL, and the Cochrane Database of Controlled Trials were searched from January 2000 to August 2020. Randomized, non-randomized, and quasi-experimental trials reporting CDSS uptake in any patient population or setting were included. The main outcome extracted was CDSS uptake, reported as a raw proportion, and representing the number of times the CDSS was used or accessed over the total number of times it could have been interacted with. We also extracted context, content, system, and implementation features that might influence uptake, for each CDSS. Overall weighted uptake was calculated using random-effects meta-analysis and determinants of uptake were investigated using multivariable meta-regression.

RESULTS

Among 7995 citations screened, 55 studies involving 373,608 patients and 3607 providers met full inclusion criteria. Meta-analysis revealed that overall CDSS uptake was 34.2% (95% CI 23.2 to 47.1%). Uptake was only reported in 12.4% of studies that otherwise met inclusion criteria. Multivariable meta-regression revealed the following factors significantly associated with uptake: (1) formally evaluating the availability and quality of the patient data needed to inform CDSS advice; and (2) identifying and addressing other barriers to the behaviour change targeted by the CDSS.

CONCLUSIONS AND RELEVANCE

System uptake was seldom reported in CDSS trials. When reported, uptake was low. This represents a major and potentially modifiable barrier to overall CDSS effectiveness. We found that features relating to CDSS context and implementation strategy best predicted uptake. Future studies should measure the impact of addressing these features as part of the CDSS implementation strategy. Uptake reporting must also become standard in future studies reporting CDSS intervention effects.

REGISTRATION

Pre-registered on PROSPERO, CRD42018092337.

Implementing clinical decision support for reducing women Veterans' cardiovascular risk in VA: A mixed-method, longitudinal study of context, adaptation, and uptake

Evaluations of clinical decision support (CDS) implementation often struggle to measure and explain heterogeneity in uptake over time and across settings, and to account for the impact of context and adaptation on implementation success. In 2017-2020, the EMPOWER QUERI implemented a cardiovascular toolkit using a computerized template aimed at reducing women Veterans' cardiovascular risk across five Veterans Healthcare Administration (VA) sites, using an enhanced Replicating Effective Programs (REP) implementation approach. In this study, we used longitudinal joint displays of qualitative and quantitative findings to explore (1) how contextual factors emerged across sites, (2) how the template and implementation strategies were adapted in response to contextual factors, and (3) how contextual factors and adaptations coincided with template uptake across sites and over time. We identified site structure, staffing changes, relational authority of champions, and external leadership as important contextual factors. These factors gave rise to adaptations such as splitting the template into multiple parts, pairing the template with a computerized reminder, conducting academic detailing, creating cheat sheets, and using small-scale pilot testing. All five sites exhibited variability in utilization over the months of implementation, though later sites exhibited higher template utilization immediately post-launch, possibly reflecting a "preloading" of adaptations from previous sites. These findings underscore the importance of adaptive approaches to implementation, with intentional shifts in intervention and strategy to meet the needs of individual sites, as well as the value of integrating mixed-method data sources in conducting longitudinal evaluation of implementation efforts.

A clinical decision support system is associated with reduced loss to follow-up among patients receiving HIV treatment in Kenya: a cluster randomized trial

Background

Loss to follow-up (LFTU) among HIV patients remains a major obstacle to achieving treatment goals with the risk of failure to achieve viral suppression and thereby increased HIV transmission. Although use of clinical decision support systems (CDSS) has been shown to improve adherence to HIV clinical guidance, to our knowledge, this is among the first studies conducted to show its effect on LTFU in low-resource settings.

Methods

We analyzed data from a cluster randomized controlled trial in adults and children (aged ≥ 18 months) who were receiving antiretroviral therapy at 20 HIV clinics in western Kenya between Sept 1, 2012 and Jan 31, 2014. Participating clinics were randomly assigned, via block randomization. Clinics in the control arm had electronic health records (EHR) only while the intervention arm had an EHR with CDSS. The study objectives were to assess the effects of a CDSS, implemented as alerts on an EHR system, on: (1) the proportion of patients that were LTFU, (2) LTFU patients traced and successfully linked back to treatment, and (3) time from enrollment on the study to documentation of LTFU.

Results

Among 5901 eligible patients receiving ART, 40.6% (n = 2396) were LTFU during the study period. CDSS was associated with lower LTFU among the patients (Adjusted Odds Ratio—aOR 0.70 (95% CI 0.65–0.77)). The proportions of patients linked back to treatment were 25.8% (95% CI 21.5–25.0) and 30.6% (95% CI 27.9–33.4)) in EHR only and EHR with CDSS sites respectively. CDSS was marginally associated with reduced time from enrollment on the study to first documentation of LTFU (adjusted Hazard Ratio—aHR 0.85 (95% CI 0.78–0.92)).

Conclusion

A CDSS can potentially improve quality of care through reduction and early detection of defaulting and LTFU among HIV patients and their re-engagement in care in a resource-limited country. Future research is needed on how CDSS can best be combined with other interventions to reduce LTFU.

Trial registration NCT01634802. Registered at www.clinicaltrials.gov on 12-Jul-2012. Registered prospectively.

Clinical Decision Support for Laboratory Testing

BACKGROUND

As technology enables new and increasingly complex laboratory tests, test utilization presents a growing challenge for healthcare systems. Clinical decision support (CDS) refers to digital tools that present providers with clinically relevant information and recommendations, which have been shown to improve test utilization. Nevertheless, individual CDS applications often fail, and implementation remains challenging.

CONTENT

We review common classes of CDS tools grounded in examples from the literature as well as our own institutional experience. In addition, we present a practical framework and specific recommendations for effective CDS implementation.

SUMMARY

CDS encompasses a rich set of tools that have the potential to drive significant improvements in laboratory testing, especially with respect to test utilization. Deploying CDS effectively requires thoughtful design and careful maintenance, and structured processes focused on quality improvement and change management play an important role in achieving these goals.

Artificial intelligence in the GPs office: a retrospective study on diagnostic accuracy

OBJECTIVE

Machine learning (ML) is expected to play an increasing role within primary health care (PHC) in coming years. No peer-reviewed studies exist that evaluate the diagnostic accuracy of ML models compared to general practitioners (GPs). The aim of this study was to evaluate the diagnostic accuracy of an ML classifier on primary headache diagnoses in PHC, compare its performance to GPs, and examine the most impactful signs and symptoms when making a prediction.

DESIGN

A retrospective study on diagnostic accuracy, using electronic health records from the database of the Primary Health Care Service of the Capital Area (PHCCA) in Iceland.

SETTING

Fifteen primary health care centers of the PHCCA.

SUBJECTS

All patients that consulted a physician, from 1 January 2006 to 30 April 2020, and received one of the selected diagnoses.

MAIN OUTCOME MEASURES

Sensitivity, Specificity, Positive Predictive Value, Matthews Correlation Coefficient, Receiver Operating Characteristic (ROC) curve, and Area under the ROC curve (AUROC) score for primary headache diagnoses, as well as Shapley Additive Explanations (SHAP) values of the ML classifier.

RESULTS

The classifier outperformed the GPs on all metrics except specificity. The SHAP values indicate that the classifier uses the same signs and symptoms (features) as a physician would, when distinguishing between headache diagnoses.

CONCLUSION

In a retrospective comparison, the diagnostic accuracy of the ML classifier for primary headache diagnoses is superior to GPs. According to SHAP values, the ML classifier relies on the same signs and symptoms as a physician when making a diagnostic prediction.KeypointsLittle is known about the diagnostic accuracy of machine learning (ML) in the context of primary health care, despite its considerable potential to aid in clinical work. This novel research sheds light on the diagnostic accuracy of ML in a clinical context, as well as the interpretation of its predictions. If the vast potential of ML is to be utilized in primary health care, its performance, safety, and inner workings need to be understood by clinicians.

Utilizing a Human-Computer Interaction Approach to Evaluate the Design of Current Pharmacogenomics Clinical Decision Support

A formal assessment of pharmacogenomics clinical decision support (PGx-CDS) by providers is lacking in the literature. The objective of this study was to evaluate the usability of PGx-CDS tools that have been implemented in a healthcare setting. We enrolled ten prescribing healthcare providers and had them complete a 60-min usability session, which included interacting with two PGx-CDS scenarios using the “Think Aloud” technique, as well as completing the Computer System Usability Questionnaire (CSUQ). Providers reported positive comments, negative comments, and suggestions for the two PGx-CDS during the usability testing. Most provider comments were in favor of the current PGx-CDS design, with the exception of how the genotype and phenotype information is displayed. The mean CSUQ score for the PGx-CDS overall satisfaction was 6.3 ± 0.95, with seven strongly agreeing and one strongly disagreeing for overall satisfaction. The implemented PGx-CDS at our institution was well received by prescribing healthcare providers. The feedback collected from the session will guide future PGx-CDS designs for our healthcare system and provide a framework for other institutions implementing PGx-CDS.

A theory-based meta-regression of factors influencing clinical decision support adoption and implementation

OBJECTIVE

The purpose of the study was to explore the theoretical underpinnings of effective clinical decision support (CDS) factors using the comparative effectiveness results.

MATERIALS AND METHODS

We leveraged search results from a previous systematic literature review and updated the search to screen articles published from January 2017 to January 2020. We included randomized controlled trials and cluster randomized controlled trials that compared a CDS intervention with and without specific factors. We used random effects meta-regression procedures to analyze clinician behavior for the aggregate effects. The theoretical model was the Unified Theory of Acceptance and Use of Technology (UTAUT) model with motivational control.

RESULTS

Thirty-four studies were included. The meta-regression models identified the importance of effort expectancy (estimated coefficient = -0.162; P = .0003); facilitating conditions (estimated coefficient = 0.094; P = .013); and performance expectancy with motivational control (estimated coefficient = 1.029; P = .022). Each of these factors created a significant impact on clinician behavior. The meta-regression model with the multivariate analysis explained a large amount of the heterogeneity across studies (R2 = 88.32%).

DISCUSSION

Three positive factors were identified: low effort to use, low controllability, and providing more infrastructure and implementation strategies to support the CDS. The multivariate analysis suggests that passive CDS could be effective if users believe the CDS is useful and/or social expectations to use the CDS intervention exist.

CONCLUSIONS

Overall, a modified UTAUT model that includes motivational control is an appropriate model to understand psychological factors associated with CDS effectiveness and to guide CDS design, implementation, and optimization.

Quality-in-use characteristics for clinical decision support system assessment

BACKGROUND

Clinical decision support systems (CDSSs) are developed to support healthcare practitioners with decision-making about therapy and diagnosis' confirmation, among others. Although there are many advantages of using CDSSs, there are still many challenges in their adoption. Therefore, it is essential to ensure the quality of the system, so that it can be used confidently and securely.

OBJECTIVE

This study aims to propose a set of (sub)characteristics which should be considered in evaluating the quality-in-use of CDSSs, based on the ISO/IEC 25010 standard and on existing literature.

METHODS

We reviewed the existing literature on CDSS assessment and presented a list of quality characteristics evaluated.

RESULTS

Ten quality characteristics and 56 sub-characteristics were identified and selected from the literature, in which usability was evaluated the most. An example of a scenario has been presented to illustrate our assessment approach of satisfaction and efficiency as important quality-in-use characteristics to be applied in the evaluation of a CDSS.

CONCLUSION

The proposed approach will contribute in bridging the gap between the quality of CDSSs and their adoption.

Effect of Interventions With a Clinical Decision Support System for Hospitalized Older Patients: Systematic Review Mapping Implementation and Design Factors

Background

Clinical decision support systems (CDSSs) form an implementation strategy that can facilitate and support health care professionals in the care of older hospitalized patients.

Objective

Our study aims to systematically review the effects of CDSS interventions in older hospitalized patients. As a secondary aim, we aim to summarize the implementation and design factors described in effective and ineffective interventions and identify gaps in the current literature.

Methods

We conducted a systematic review with a search strategy combining the categories older patients, geriatric topic, hospital, CDSS, and intervention in the databases MEDLINE, Embase, and SCOPUS. We included controlled studies, extracted data of all reported outcomes, and potentially beneficial design and implementation factors. We structured these factors using the Grol and Wensing Implementation of Change model, the GUIDES (Guideline Implementation with Decision Support) checklist, and the two-stream model. The risk of bias of the included studies was assessed using the Cochrane Collaboration’s Effective Practice and Organisation of Care risk of bias approach.

Results

Our systematic review included 18 interventions, of which 13 (72%) were effective in improving care. Among these interventions, 8 (6 effective) focused on medication review, 8 (6 effective) on delirium, 7 (4 effective) on falls, 5 (4 effective) on functional decline, 4 (3 effective) on discharge or aftercare, and 2 (0 effective) on pressure ulcers. In 77% (10/13) effective interventions, the effect was based on process-related outcomes, in 15% (2/13) interventions on both process- and patient-related outcomes, and in 8% (1/13) interventions on patient-related outcomes. The following implementation and design factors were potentially associated with effectiveness: a priori problem or performance analyses (described in 9/13, 69% effective vs 0/5, 0% ineffective interventions), multifaceted interventions (8/13, 62% vs 1/5, 20%), and consideration of the workflow (9/13, 69% vs 1/5, 20%).

Conclusions

CDSS interventions can improve the hospital care of older patients, mostly on process-related outcomes. We identified 2 implementation factors and 1 design factor that were reported more frequently in articles on effective interventions. More studies with strong designs are needed to measure the effect of CDSS on relevant patient-related outcomes, investigate personalized (data-driven) interventions, and quantify the impact of implementation and design factors on CDSS effectiveness.

Trial Registration

PROSPERO (International Prospective Register of Systematic Reviews): CRD42019124470; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=124470.

The impact of personalized clinical decision support on primary care patients' views of cancer prevention and screening: a cross-sectional survey

BACKGROUND

Few studies have assessed the impact of clinical decision support (CDS), with or without shared decision-making tools (SDMTs), on patients' perceptions of cancer screening or prevention in primary care settings. This cross-sectional survey was conducted to understand primary care patient's perceptions on cancer screening or prevention.

METHODS

We mailed surveys (10/2018-1/2019) to 749 patients aged 18 to 75 years within 15 days after an index clinical encounter at 36 primary care clinics participating in a clinic-randomized control trial of a CDS system for cancer prevention. All patients were overdue for cancer screening or human papillomavirus vaccination. The survey compared respondents' answers by study arm: usual care; CDS; or CDS + SDMT.

RESULTS

Of 387 respondents (52% response rate), 73% reported having enough time to discuss cancer prevention options with their primary care provider (PCP), 64% reported their PCP explained the benefits of the cancer screening choice very well, and 32% of obese patients reported discussing weight management, with two-thirds reporting selecting a weight management intervention. Usual care respondents were significantly more likely to decide on colorectal cancer screening than CDS respondents (p < 0.01), and on tobacco cessation than CDS + SDMT respondents (p = 0.02) and both CDS and CDS + SDMT respondents (p < 0.001).

CONCLUSIONS

Most patients reported discussing cancer prevention needs with PCPs, with few significant differences between the three study arms in patient-reported cancer prevention care. Upcoming research will assess differences in screening and vaccination rates between study arms during the post-intervention follow-up period.

TRIAL REGISTRATION

clinicaltrials.gov , NCT02986230 , December 6, 2016.

Medication risk management and health equity in New Zealand general practice: a retrospective cross-sectional study

BACKGROUND

Despite an overt commitment to equity, health inequities are evident throughout Aotearoa New Zealand. A general practice electronic alert system was developed to notify clinicians about their patient's risk of harm due to their pre-existing medical conditions or current medication. We aimed to determine whether there were any disparities in clinician action taken on the alert based on patient ethnicity or other demographic factors.

METHODS

Sixty-six New Zealand general practices from throughout New Zealand participated. Data were available for 1611 alerts detected for 1582 patients between 1 and 2018 and 1 July 2019. The primary outcome was whether action was taken following an alert or not. Logistic regression was used to assess if patients of one ethnicity group were more or less likely to have action taken. Potential confounders considered in the analyses include patient age, gender, ethnicity, socio-economic deprivation, number of long term diagnoses and number of long term medications.

RESULTS

No evidence of a difference was found in the odds of having action taken amongst ethnicity groups, however the estimated odds for Māori and Pasifika patients were lower compared to the European group (Māori OR 0.88, 95 %CI 0.63-1.22; Pasifika OR 0.88, 95 %CI 0.52-1.49). Females had significantly lower odds of having action taken compared to males (OR 0.76, 95 %CI 0.59-0.96).

CONCLUSIONS

This analysis of data arising from a general practice electronic alert system in New Zealand found clinicians typically took action on those alerts. However, clinicians appear to take less action for women and Māori and Pasifika patients. Use of a targeted alert system has the potential to mitigate risk from medication-related harm. Recognising clinician biases may improve the equitability of health care provision.

The implementation, use and sustainability of a clinical decision support system for medication optimisation in primary care: A qualitative evaluation

Background

The quality and safety of prescribing in general practice is important, Clinical decision support (CDS) systems can be used which present alerts to health professionals when prescribing in order to identify patients at risk of potentially hazardous prescribing. It is known that such computerised alerts may improve the safety of prescribing in hospitals but their implementation and sustainable use in general practice is less well understood. We aimed to understand the factors that influenced the successful implementation and sustained use in primary care of a CDS system.

Methods

Participants were purposively recruited from Clinical Commissioning Groups (CCGs) and general practices in the North West and East Midlands regions of England and from the CDS developers. We conducted face-to-face and telephone-based semi-structured qualitative interviews with staff stakeholders. A selection of participants was interviewed longitudinally to explore the further sustainability 1–2 years after implementation of the CDS system. The analysis, informed by Normalisation Process Theory (NPT), was thematic, iterative and conducted alongside data collection.

Results

Thirty-nine interviews were conducted either individually or in groups, with 33 stakeholders, including 11 follow-up interviews. Eight themes were interpreted in alignment with the four NPT constructs: Coherence (The purpose of the CDS: Enhancing medication safety and improving cost effectiveness; Relationship of users to the technology; Engagement and communication between different stakeholders); Cognitive Participation (Management of the profile of alerts); Collective Action (Prescribing in general practice, patient and population characteristics and engagement with patients; Knowledge);and Reflexive Monitoring (Sustaining the use of the CDS through maintenance and customisation; Learning and behaviour change. Participants saw that the CDS could have a role in enhancing medication safety and in the quality of care. Engagement through communication and support for local primary care providers and management leaders was considered important for successful implementation. Management of prescribing alert profiles for general practices was a dynamic process evolving over time. At regional management levels, work was required to adapt, and modify the system to optimise its use in practice and fulfil local priorities. Contextual factors, including patient and population characteristics, could impact upon the decision-making processes of prescribers influencing the response to alerts. The CDS could operate as a knowledge base allowing prescribers access to evidence-based information that they otherwise would not have.

Conclusions

This qualitative evaluation utilised NPT to understand the implementation, use and sustainability of a widely deployed CDS system offering prescribing alerts in general practice. The system was understood as having a role in medication safety in providing relevant patient specific information to prescribers in a timely manner. Engagement between stakeholders was considered important for the intervention in ensuring prescribers continued to utilise its functionality. Sustained implementation might be enhanced by careful profile management of the suite of alerts in the system. Our findings suggest that the use and sustainability of the CDS was related to prescribers’ perceptions of the relevance of alerts. Shared understanding of the purpose of the CDS between CCGS and general practices particularly in balancing cost saving and safety messages could be beneficial.

A systematic review of theoretical constructs in CDS literature

Background

Studies that examine the adoption of clinical decision support (CDS) by healthcare providers have generally lacked a theoretical underpinning. The Unified Theory of Acceptance and Use of Technology (UTAUT) model may provide such a theory-based explanation; however, it is unknown if the model can be applied to the CDS literature.

Objective

Our overall goal was to develop a taxonomy based on UTAUT constructs that could reliably characterize CDS interventions.

Methods

We used a two-step process: (1) identified randomized controlled trials meeting comparative effectiveness criteria, e.g., evaluating the impact of CDS interventions with and without specific features or implementation strategies; (2) iteratively developed and validated a taxonomy for characterizing differential CDS features or implementation strategies using three raters.

Results

Twenty-five studies with 48 comparison arms were identified. We applied three constructs from the UTAUT model and added motivational control to characterize CDS interventions. Inter-rater reliability was as follows for model constructs: performance expectancy (κ = 0.79), effort expectancy (κ = 0.85), social influence (κ = 0.71), and motivational control (κ = 0.87).

Conclusion

We found that constructs from the UTAUT model and motivational control can reliably characterize features and associated implementation strategies. Our next step is to examine the quantitative relationships between constructs and CDS adoption.

Improving smoking cessation referrals among elective surgery clinics through electronic clinical decision support

INTRODUCTION

Preoperative visits are an exceptional opportunity to encourage smoking cessation, as studies demonstrate the experience of scheduling elective surgery produces an actionable incentive to quit. However, studies suggest surgeons do not regularly assess smoking behavior or offer cessation therapies. Clinical decision support (CDS) is a system in which providers are presented with clinically integrated tools to enhance decision-making.

METHODS

A CDS tool was designed to facilitate treatment referrals for smoking cessation services among patients seeking elective surgery. Two clinics were selected: the plastic and vascular surgeries. The study objectives were to assess the utilization rate and effectiveness of this system.

RESULTS

No smoking cessation referrals had been submitted by the plastic surgery or vascular surgery clinics in the year before CDS tool implementation. Providers at the plastic surgery clinic utilized the CDS tool in 95.0% (191 of 201) eligible patient encounters. Of these patients, 16.3% were identified as active smokers, and 16.1% of these smokers accepted treatment referrals. Providers at the vascular surgery clinic utilized the CDS tool in 50.3% (98 of 195) eligible patient encounters. Of these patients, 10.2% were identified as active smokers, and 30.0% of these smokers accepted treatment referrals.

CONCLUSIONS

The CDS tool improved the incidence of smoking cessation referrals in two surgical clinics from pretest baselines and achieved satisfactory utilization rates. This report demonstrates the feasibility of CDS tools to actualize the preoperative visit as an opportunity to promote smoking cessation.

Interests and needs of eye care providers in clinical decision support for glaucoma

Objective

To study whether clinicians who treat glaucoma are interested in using clinical decision support (CDS) tools for glaucoma, what glaucoma clinical decisions they feel would benefit from CDS, and what characteristics of CDS design they feel would be important in glaucoma clinical practice.

Methods and analysis

Working with the American Glaucoma Society, the Utah Ophthalmology Society and the Utah Optometric Association, we identified a group of clinicians who care for patients with glaucoma. We asked these clinicians about interest in CDS, what glaucoma clinical decisions would benefit from CDS, and what characteristics of CDS tool design would be important in glaucoma clinical practice.

Results

Of the 105 clinicians (31 optometrists, 10 general ophthalmologists and 64 glaucoma specialists), 93 (88.6%) were either ‘definitely’ or ‘probably’ interested in using CDS for glaucoma. There were no statistically significant differences in interest between clinical specialties (p=0.12), years in practice (p=0.85) or numbers of patients seen daily (p=0.99). Identifying progression of glaucoma was the clinical decision the largest number of clinicians felt would benefit from CDS (104/105, 99.1%). An easy to use interface was the CDS characteristic the largest number of clinicians felt would be ‘very important’ (93/105, 88.6%).

Conclusion

Of this group of clinicians who treat glaucoma, 88.6% were interested in using CDS for glaucoma and 99.1% felt that identification of glaucomatous progression could benefit from CDS. This level of interest supports future work to develop CDS for glaucoma.

Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening

OBJECTIVE

Evaluation of the effect of six optimization strategies in a clinical decision support system (CDSS) for drug-drug interaction (DDI) screening on alert burden and alert acceptance and description of clinical pharmacist intervention acceptance.

METHODS

Optimizations in the new CDSS were the customization of the knowledge base (with addition of 67 extra DDIs and changes in severity classification), a new alert design, required override reasons for the most serious alerts, the creation of DDI-specific screening intervals, patient-specific alerting, and a real-time follow-up system of all alerts by clinical pharmacists with interventions by telephone was introduced. The alert acceptance was evaluated both at the prescription level (i.e. prescription acceptance, was the DDI prescribed?) and at the administration level (i.e. administration acceptance, did the DDI actually take place?). Finally, the new follow-up system was evaluated by assessing the acceptance of clinical pharmacist's interventions.

RESULTS

In the pre-intervention period, 1087 alerts (92.0 % level 1 alerts) were triggered, accounting for 19 different DDIs. In the post-intervention period, 2630 alerts (38.4 % level 1 alerts) were triggered, representing 86 different DDIs. The relative risk forprescription acceptance in the post-intervention period compared to the pre-intervention period was 4.02 (95 % confidence interval (CI) 3.17-5.10; 25.5 % versus 6.3 %). The relative risk for administration acceptance was 1.16 (95 % CI 1.08-1.25; 54.4 % versus 46.7 %). Finally, 86.9 % of the clinical pharmacist interventions were accepted.

CONCLUSION

Six concurrently implemented CDSS optimization strategies resulted in a high alert acceptance and clinical pharmacist intervention acceptance. Administration acceptance was remarkably higher than prescription acceptance.

Decision support for Scotland's health and social care: learning from an outcomes-focused approach

This short report shares learning from the research and development phase of the national decision support programme in NHS Scotland. It outlines how the programme has adopted an outcomes-focused approach which has guided critical decisions on solution design, engagement of policy sponsors, clinical and management leaders, implementation and evaluation approach, technical architecture and technology development. It discusses how this outcomes-led approach positions decision support as catalyst for a learning health and care system that continuously refreshes the healthcare knowledge base through new insights generated by evaluating impact and outcomes.

Barriers and Facilitators for Implementation of a Computerized Clinical Decision Support System in Lung Cancer Multidisciplinary Team Meetings-A Qualitative Assessment

Simple Summary

Oncological computerized clinical decision support systems (CCDSSs) are currently being developed to facilitate workflows of multidisciplinary team meetings (MDTMs). To successfully implement these systems in MDTMs, the aim of this qualitative assessment was to identify barriers and facilitators for implementation and to provide actionable findings for an implementation strategy. The main facilitators for implementation of the CCDSS were considered to be easy access to well-structured data, and reducing time spent by clinicians on MDTM preparation and duration of the MDTMs. Main barriers for adoption were seen in incomplete or non-trustworthy output generated by the system and insufficient adaptability of the system to local and contextual needs. Actionable findings for an implementation strategy were a usability test and validation study involving key users in the organization’s real-life setting. Given the growing interest in CCDSSs in oncology care, insight in barriers and facilitators for successful implementation seems highly relevant.

Abstract

Background: Oncological computerized clinical decision support systems (CCDSSs) to facilitate workflows of multidisciplinary team meetings (MDTMs) are currently being developed. To successfully implement these CCDSSs in MDTMs, this study aims to: (a) identify barriers and facilitators for implementation for the use case of lung cancer; and (b) provide actionable findings for an implementation strategy. Methods: The Consolidated Framework for Implementation Science was used to create an interview protocol and to analyze the results. Semi-structured interviews were conducted among various health care professionals involved in MDTMs. The transcripts were analyzed using a thematic analysis following a deductive approach. Results: Twenty-six professionals participated in the interviews. The main facilitators for implementation of the CCDSS were considered to be easy access to well-structured patient data, and the resulting reduction of MDTM preparation time and of duration of MDTMs. Main barriers for adoption were seen in incomplete or non-trustworthy output generated by the system and insufficient adaptability of the system to local and contextual needs. Conclusion: Using a CCDSS in lung cancer MDTMs was expected to increase efficiency of workflows. Successful implementation was seen as dependent on the reliability and adaptability of the CCDSS and involvement of key users in the implementation process.

Future Health Today: codesign of an electronic chronic disease quality improvement tool for use in general practice using a service design approach

OBJECTIVE

To codesign an electronic chronic disease quality improvement tool for use in general practice.

DESIGN

Service design employing codesign strategies.

SETTING

General practice.

PARTICIPANTS

Seventeen staff (general practitioners, nurses and practice managers) from general practice in metropolitan Melbourne and regional Victoria and five patients from metropolitan Melbourne.

INTERVENTIONS

Codesign sessions with general practice staff, using a service design approach, were conducted to explore key design criteria and functionality of the audit and feedback and clinical decision support tools. Think aloud interviews were conducted in which participants articulated their thoughts of the resulting Future Health Today (FHT) prototype as they used it. One codesign session was held with patients. Using inductive and deductive coding, content and thematic analyses explored the development of a new technological platform and factors influencing implementation of the platform.

RESULTS

Participants identified that the prototype needed to work within their existing workflow to facilitate automated patient recall and track patients with or at-risk of specific conditions. It needed to be simple, provide visual snapshots of information and easy access to relevant guidelines and facilitate quality improvement activities. Successful implementation may be supported by: accuracy of the algorithms in FHT and data held in the practice; the platform supporting planned and spontaneous interactions with patients; the ability to hide tools; links to Medicare Benefits Schedule; and prefilled management plans. Participating patients supported the use of the platform in general practice. They suggested that use of the platform demonstrates a high level of patient care and could increase patient confidence in health practitioners.

CONCLUSION

Study participants worked together to design a platform that is clear, simple, accurate and useful and that sits within any given general practice setting. The resulting FHT platform is currently being piloted in general practices and will continue to be refined based on user feedback.

Association of Electronic Prescribing of Controlled Substances With Opioid Prescribing Rates

IMPORTANCE

The majority of US states have passed mandates requiring the use of electronic prescribing of controlled substances (EPCS) as a tool to reduce rates of opioid prescribing. It is not known whether increasing use of EPCS will have the intended effect.

OBJECTIVE

To assess the association between use of EPCS and trends in opioid prescribing.

DESIGN, SETTING, AND PARTICIPANTS

In this retrospective, longitudinal cohort study of all patients and prescribers in the 50 US states and the District of Columbia from 2010 to 2018, changes in state-level use of EPCS and concurrent changes in opioid prescribing in each state are described. Then the association between changes in the use of EPCS and opioid prescribing are estimated using state and year fixed-effects models that include covariates for policy change and state demographic change. Data Analysis was performed on May 5, 2020.

MAIN OUTCOMES AND MEASURES

The proportion of controlled substances in each state prescribed using EPCS based on opioid prescriptions per 100 persons and morphine milligram equivalents (MME) of opioids.

RESULTS

In 2018, the population-weighted percent of opioids prescribed using EPCS was 27%, up from 0% as of 2013. National rates of opioid prescriptions decreased from 78 prescriptions per 100 persons in 2013 to 53 in 2018. Over the same period, there was a decrease from 64 071 MME per 100 persons in 2013 to 40 906 MME per 100 persons in 2018, representing 36% of the 2013 level. By 2018, EPCS increased to 69.4% in states with mandates for its use and 23.6% in states without mandates. In multivariable models, a 10 percentage-point increase in the use of EPCS was associated with an additional 2 prescriptions per 100 persons (95% CI, 1.3-2.8) and a 0.8% (95% CI, 0.06%-1.5%) increase in MME per 100 persons.

CONCLUSIONS AND RELEVANCE

These data suggest that an increased use of EPCS was not associated with decreased opioid prescribing or a decrease in the amount prescribed and may have been associated with a small increase in opioid prescribing. Opioid prescribing is associated with a variety of social and public health factors, and thus, despite the appeal, EPCS adoption alone may be insufficient to reduce opioid prescribing. Policy makers should consider levers to ensure that EPCS is integrated with outside data and that information is actively used to inform prescribing decisions.

Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study

Background

Machine learning models have the potential to improve diagnostic accuracy and management of acute conditions. Despite growing efforts to evaluate and validate such models, little is known about how to best translate and implement these products as part of routine clinical care.

Objective

This study aims to explore the factors influencing the integration of a machine learning sepsis early warning system (Sepsis Watch) into clinical workflows.

Methods

We conducted semistructured interviews with 15 frontline emergency department physicians and rapid response team nurses who participated in the Sepsis Watch quality improvement initiative. Interviews were audio recorded and transcribed. We used a modified grounded theory approach to identify key themes and analyze qualitative data.

Results

A total of 3 dominant themes emerged: perceived utility and trust, implementation of Sepsis Watch processes, and workforce considerations. Participants described their unfamiliarity with machine learning models. As a result, clinician trust was influenced by the perceived accuracy and utility of the model from personal program experience. Implementation of Sepsis Watch was facilitated by the easy-to-use tablet application and communication strategies that were developed by nurses to share model outputs with physicians. Barriers included the flow of information among clinicians and gaps in knowledge about the model itself and broader workflow processes.

Conclusions

This study generated insights into how frontline clinicians perceived machine learning models and the barriers to integrating them into clinical workflows. These findings can inform future efforts to implement machine learning interventions in real-world settings and maximize the adoption of these interventions.