Making cognitive decision support work: Facilitating adoption, knowledge and behavior change through QI

Machine learning-based prediction of low-value care for hospitalized patients

Objective

Low-value care (i.e., costly health care treatments that provide little or no benefit) is an ongoing problem in United States hospitals. Traditional strategies for reducing low-value care are only moderately successful. Informed by behavioral science principles, we sought to use machine learning to inform a targeted prompting system that suggests preferred alternative treatments at the point of care but before clinicians have made a decision.

Methods

We used intravenous administration of albumin for fluid resuscitation in intensive care unit (ICU) patients as an exemplar of low-value care practice, identified using the electronic health record of a multi-hospital health system. We divided all ICU episodes into 4-h periods and defined a set of relevant clinical features at the period level. We then developed two machine learning models: a single-stage model that directly predicts if a patient will receive albumin in the next period; and a two-stage model that first predicts if any resuscitation fluid will be administered and then predicts albumin only among the patients with a high probability of fluid use.

Results

We examined 87,489 ICU episodes divided into approximately 1.5 million 4-h periods. The area under the receiver operating characteristic curve was 0.86 for both prediction models. The positive predictive value was 0.21 (95% confidence interval: 0.20, 0.23) for the single-stage model and 0.22 (0.20, 0.23) for the two-stage model. Applying either model in a targeted prompting system could prevent 10% of albumin administrations, with an attending physician receiving one prompt every 4.2 days of ICU service.

Conclusion

Prediction of low-value care is feasible and could enable a point-of-care, targeted prompting system that offers suggestions ahead of the moment of need before clinicians have already decided. A two-stage approach does not improve performance but does interject new levers for the calibration of such a system.

Development of a Long-Term Care Nurse Residency Program

Nurses working in the long-term care (LTC) setting provide increasingly complex patient care, often without formal training on the specific needs of LTC patients, which can lead to burnout and high turnover rates. Nurse residency programs (NRPs) have been used effectively to orient novice RNs to their work setting, address transition-to-practice challenges, and promote retention, yet few LTC NRPs have been developed. The University of Utah Geriatric Education Consortium Geriatric Workforce Enhancement Program created an online LTC NRP to provide LTC nurses with the knowledge and skills to succeed in the LTC environment. RNs with <1 year of LTC experience were paired with experienced nurse mentors working within the same LTC facility. Synchronous and asynchronous curricular modules addressed leadership and communication, caring for older adult patients, quality improvement, and the LTC regulatory environment. A distance-based LTC NRP allows nurses flexibility in gaining gerontological nursing and leadership expertise that supports their professional goals. [Journal of Gerontological Nursing, 47(2), 37-43.].

Interests and needs of eye care providers in clinical decision support for glaucoma

Objective

To study whether clinicians who treat glaucoma are interested in using clinical decision support (CDS) tools for glaucoma, what glaucoma clinical decisions they feel would benefit from CDS, and what characteristics of CDS design they feel would be important in glaucoma clinical practice.

Methods and analysis

Working with the American Glaucoma Society, the Utah Ophthalmology Society and the Utah Optometric Association, we identified a group of clinicians who care for patients with glaucoma. We asked these clinicians about interest in CDS, what glaucoma clinical decisions would benefit from CDS, and what characteristics of CDS tool design would be important in glaucoma clinical practice.

Results

Of the 105 clinicians (31 optometrists, 10 general ophthalmologists and 64 glaucoma specialists), 93 (88.6%) were either ‘definitely’ or ‘probably’ interested in using CDS for glaucoma. There were no statistically significant differences in interest between clinical specialties (p=0.12), years in practice (p=0.85) or numbers of patients seen daily (p=0.99). Identifying progression of glaucoma was the clinical decision the largest number of clinicians felt would benefit from CDS (104/105, 99.1%). An easy to use interface was the CDS characteristic the largest number of clinicians felt would be ‘very important’ (93/105, 88.6%).

Conclusion

Of this group of clinicians who treat glaucoma, 88.6% were interested in using CDS for glaucoma and 99.1% felt that identification of glaucomatous progression could benefit from CDS. This level of interest supports future work to develop CDS for glaucoma.

Special Commentary: Using Clinical Decision Support Systems to Bring Predictive Models to the Glaucoma Clinic

Advances in the field of predictive modeling using artificial intelligence and machine learning have the potential to improve clinical care and outcomes, but only if the results of these models are presented appropriately to clinicians at the time they make decisions for individual patients. Clinical decision support (CDS) systems could be used to accomplish this. Modern CDS systems are computer-based tools designed to improve clinician decision making for individual patients. However, not all CDS systems are effective. Four principles that have been shown in other medical fields to be important for successful CDS system implementation are (1) integration into clinician workflow, (2) user-centered interface design, (3) evaluation of CDS systems and rules, and (4) standards-based development so the tools can be deployed across health systems.

The use of the PARIHS framework in implementation research and practice-a citation analysis of the literature

BACKGROUND

The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.

METHODS

This citation analysis commenced from four core articles representing the key stages of the framework's development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.

RESULTS

The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.

CONCLUSIONS

In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.

Incorporating Guideline Adherence and Practice Implementation Issues into the Design of Decision Support for Beta-Blocker Titration for Heart Failure

BACKGROUND

The recognition of and response to undertreatment of heart failure (HF) patients can be complicated. A clinical reminder can facilitate use of guideline-concordant β-blocker titration for HF patients with depressed ejection fraction. However, the design must consider the cognitive demands on the providers and the context of the work.

OBJECTIVE

This study's purpose is to develop requirements for a clinical decision support tool (a clinical reminder) by analyzing the cognitive demands of the task along with the factors in the Cabana framework of physician adherence to guidelines, the health information technology (HIT) sociotechnical framework, and the Promoting Action on Research Implementation in Health Services (PARIHS) framework of health services implementation. It utilizes a tool that extracts information from medical records (including ejection fraction in free text reports) to identify qualifying patients at risk of undertreatment.

METHODS

We conducted interviews with 17 primary care providers, 5 PharmDs, and 5 Registered Nurses across three Veterans Health Administration outpatient clinics. The interviews were based on cognitive task analysis (CTA) methods and enhanced through the inclusion of the Cabana, HIT sociotechnical, and PARIHS frameworks. The analysis of the interview data led to the development of requirements and a prototype design for a clinical reminder. We conducted a small pilot usability assessment of the clinical reminder using realistic clinical scenarios.

RESULTS

We identified organizational challenges (such as time pressures and underuse of pharmacists), knowledge issues regarding the guideline, and information needs regarding patient history and treatment status. We based the design of the clinical reminder on how to best address these challenges. The usability assessment indicated the tool could help the decision and titration processes.

CONCLUSION

Through the use of CTA methods enhanced with adherence, sociotechnical, and implementation frameworks, we designed a decision support tool that considers important challenges in the decision and execution of β-blocker titration for qualifying HF patients at risk of undertreatment.