1
|
Slovis BH, Huang S, McArthur M, Martino C, Beers T, Labella M, Riggio JM, Pribitkin ED. Design and Implementation of an Opioid Scorecard for Hospital System-Wide Peer Comparison of Opioid Prescribing Habits: Observational Study. JMIR Hum Factors 2024; 11:e44662. [PMID: 39250214 PMCID: PMC11404392 DOI: 10.2196/44662] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 05/05/2023] [Accepted: 06/02/2024] [Indexed: 09/10/2024] Open
Abstract
Background Reductions in opioid prescribing by health care providers can lead to a decreased risk of opioid dependence in patients. Peer comparison has been demonstrated to impact providers' prescribing habits, though its effect on opioid prescribing has predominantly been studied in the emergency department setting. Objective The purpose of this study is to describe the development of an enterprise-wide opioid scorecard, the architecture of its implementation, and plans for future research on its effects. Methods Using data generated by the author's enterprise vendor-based electronic health record, the enterprise analytics software, and expertise from a dedicated group of informaticists, physicians, and analysts, the authors developed an opioid scorecard that was released on a quarterly basis via email to all opioid prescribers at our institution. These scorecards compare providers' opioid prescribing habits on the basis of established metrics to those of their peers within their specialty throughout the enterprise. Results At the time of this study's completion, 2034 providers have received at least 1 scorecard over a 5-quarter period ending in September 2021. Poisson regression demonstrated a 1.6% quarterly reduction in opioid prescribing, and chi-square analysis demonstrated pre-post reductions in the proportion of prescriptions longer than 5 days' duration and a morphine equivalent daily dose of >50. Conclusions To our knowledge, this is the first peer comparison effort with high-quality evidence-based metrics of this scale published in the literature. By sharing this process for designing the metrics and the process of distribution, the authors hope to influence other health systems to attempt to curb the opioid pandemic through peer comparison. Future research examining the effects of this intervention could demonstrate significant reductions in opioid prescribing, thus potentially reducing the progression of individual patients to opioid use disorder and the associated increased risk of morbidity and mortality.
Collapse
Affiliation(s)
- Benjamin Heritier Slovis
- Office of Clinical Informatics, Thomas Jefferson University, 111 S 11th St, Philadelphia, PA, 19107, United States, 1 (215) 955-6000
- Department of Emergency Medicine, Jefferson Health Northeast, Thomas Jefferson University, Philadelphia, PA, United States
| | - Soonyip Huang
- Enterprise Analytics, Thomas Jefferson University, Philadelphia, PA, United States
| | - Melanie McArthur
- Enterprise Analytics, Thomas Jefferson University, Philadelphia, PA, United States
| | - Cara Martino
- Enterprise Analytics, Thomas Jefferson University, Philadelphia, PA, United States
- Office of Quality and Patient Safety, Thomas Jefferson University, Philadelphia, PA, United States
| | - Tasia Beers
- Enterprise Analytics, Thomas Jefferson University, Philadelphia, PA, United States
| | - Meghan Labella
- Enterprise Analytics, Thomas Jefferson University, Philadelphia, PA, United States
| | - Jeffrey M Riggio
- Office of Clinical Informatics, Thomas Jefferson University, 111 S 11th St, Philadelphia, PA, 19107, United States, 1 (215) 955-6000
- Department of Medicine, Thomas Jefferson University, Philadelphia, PA, United States
| | - Edmund deAzevedo Pribitkin
- Office of the Chief Physician Executive, Thomas Jefferson University, Philadelphia, PA, United States
- Department of Otolaryngology, Thomas Jefferson University, Philadelphia, PA, United States
| |
Collapse
|
2
|
Barrington G, Davis K, Aandahl Z, Hose BA, Arthur M, Tran V. Influences of Software Changes on Oxycodone Prescribing at an Australian Tertiary Emergency Department: A Retrospective Review. PHARMACY 2024; 12:44. [PMID: 38525724 PMCID: PMC10961781 DOI: 10.3390/pharmacy12020044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 02/19/2024] [Accepted: 02/28/2024] [Indexed: 03/26/2024] Open
Abstract
Opioid prescribing and dispensing from emergency departments is a noteworthy issue given widespread opioid misuse and diversion in many countries, contributing both physical and economic harm to the population. High patient numbers and the stochastic nature of acute emergency presentations to emergency departments (EDs) introduce challenges for prescribers who are considering opioid stewardship principles. This study investigated the effect of changes to electronic prescribing software on prescriptions with an auto-populated quantity of oxycodone immediate release (IR) from an Australian tertiary emergency department following the implementation of national recommendations for reduced pack sizes. A retrospective review of oxycodone IR prescriptions over two six-month periods between 2019 and 2021 was undertaken, either side of a software adjustment to reduce the default quantities of tablets prescribed from 20 to 10. Patient demographic details were collected, and prescriber years of practice calculated for inclusion in linear mixed effects regression modelling. A reduction in the median number of tablets prescribed per prescription following the software changes (13.5 to 10.0, p < 0.001) with little change in the underlying characteristics of the patient or prescriber populations was observed, as well as an 11.65% reduction in the total number of tablets prescribed. The prescriber's years of practice, patient age and patient sex were found to influence increased prescription sizes. Reduced quantity of oxycodone tablets prescribed was achieved by alteration of prescribing software prefill parameters, providing further evidence to support systems-based policy interventions to influence health care providers behaviour and to act as a forcing function for prescribers to consider opioid stewardship principles.
Collapse
Affiliation(s)
- Giles Barrington
- Royal Hobart Hospital, Tasmanian Health Service, Hobart 7000, Australia; (B.-A.H.); (M.A.); (V.T.)
| | | | - Zach Aandahl
- School of Natural Sciences, University of Tasmania, Hobart 7000, Australia;
| | - Brodie-Anne Hose
- Royal Hobart Hospital, Tasmanian Health Service, Hobart 7000, Australia; (B.-A.H.); (M.A.); (V.T.)
| | - Mitchell Arthur
- Royal Hobart Hospital, Tasmanian Health Service, Hobart 7000, Australia; (B.-A.H.); (M.A.); (V.T.)
| | - Viet Tran
- Royal Hobart Hospital, Tasmanian Health Service, Hobart 7000, Australia; (B.-A.H.); (M.A.); (V.T.)
- School of Medicine, University of Tasmania, Hobart 7000, Australia
- Menzies Institute for Medical Research, University of Tasmania, Hobart 7000, Australia
| |
Collapse
|
3
|
Gangathimmaiah V, Drever N, Evans R, Moodley N, Sen Gupta T, Cardona M, Carlisle K. What works for and what hinders deimplementation of low-value care in emergency medicine practice? A scoping review. BMJ Open 2023; 13:e072762. [PMID: 37945299 PMCID: PMC10649718 DOI: 10.1136/bmjopen-2023-072762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 10/26/2023] [Indexed: 11/12/2023] Open
Abstract
OBJECTIVES Low-value care can harm patients and healthcare systems. Despite a decade of global endeavours, low value care has persisted. Identification of barriers and enablers is essential for effective deimplementation of low-value care. This scoping review is an evidence summary of barriers, enablers and features of effective interventions for deimplementation of low-value care in emergency medicine practice worldwide. DESIGN A mixed-methods scoping review was conducted using the Arksey and O'Malley framework. DATA SOURCES Medline, CINAHL, Embase, EMCare, Scopus and grey literature were searched from inception to 5 December 2022. ELIGIBILITY CRITERIA Primary studies which employed qualitative, quantitative or mixed-methods approaches to explore deimplementation of low-value care in an EM setting and reported barriers, enablers or interventions were included. Reviews, protocols, perspectives, comments, opinions, editorials, letters to editors, news articles, books, chapters, policies, guidelines and animal studies were excluded. No language limits were applied. DATA EXTRACTION AND SYNTHESIS Study selection, data collection and quality assessment were performed by two independent reviewers. Barriers, enablers and interventions were mapped to the domains of the Theoretical Domains Framework. The Mixed Methods Appraisal Tool was used for quality assessment. RESULTS The search yielded 167 studies. A majority were quantitative studies (90%, 150/167) that evaluated interventions (86%, 143/167). Limited provider abilities, diagnostic uncertainty, lack of provider insight, time constraints, fear of litigation, and patient expectations were the key barriers. Enablers included leadership commitment, provider engagement, provider training, performance feedback to providers and shared decision-making with patients. Interventions included one or more of the following facets: education, stakeholder engagement, audit and feedback, clinical decision support, nudge, clinical champions and training. Multifaceted interventions were more likely to be effective than single-faceted interventions. Effectiveness of multifaceted interventions was influenced by fidelity of the intervention facets. Use of behavioural change theories such as the Theoretical Domains Framework in the published studies appeared to enhance the effectiveness of interventions to deimplement low-value care. CONCLUSION High-fidelity, multifaceted interventions that incorporated education, stakeholder engagement, audit/feedback and clinical decision support, were administered daily and lasted longer than 1 year were most effective in achieving deimplementation of low-value care in emergency departments. This review contributes the best available evidence to date, but further rigorous, theory-informed, qualitative and mixed-methods studies are needed to supplement the growing body of evidence to effectively deimplement low-value care in emergency medicine practice.
Collapse
Affiliation(s)
- Vinay Gangathimmaiah
- Department of Emergency Medicine, Townsville University Hospital, Townsville, Queensland, Australia
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Natalie Drever
- Department of Obstetrics and Gynaecology, Cairns Hospital, Cairns, Queensland, Australia
- College of Medicine and Dentistry, James Cook University, Cairns, Queensland, Australia
| | - Rebecca Evans
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Nishila Moodley
- Department of Emergency Medicine, Townsville University Hospital, Townsville, Queensland, Australia
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Tarun Sen Gupta
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Magnolia Cardona
- A/Prof Implementation Science, Faculty of Health and Behavioural Sciences, School of Psychology, The University of Queensland, Brisbane, Queensland, Australia
- Honorary A/Prof of Research Translation, Institute for Evidence Based Healthcare, Bond University, Gold Coast, Queensland, Australia
| | - Karen Carlisle
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| |
Collapse
|
4
|
Jones CM, Coombs D, Lin CWC, Traeger A, Li Q, Abdel Shaheed C, Sharma S, Maher CG, Machado GC. Implementation of a model of care for low back pain produces sustained reduction in opioid use in emergency departments. Emerg Med J 2023; 40:359-360. [PMID: 37012024 DOI: 10.1136/emermed-2022-212874] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/19/2023] [Indexed: 04/05/2023]
Affiliation(s)
- Caitlin Mp Jones
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Danielle Coombs
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Chung-Wei Christine Lin
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Adrian Traeger
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Qiang Li
- The George Institute for Global Health, Faculty of Medicine, University of New South Wales, Sydney, New South Wales, Australia
| | - Christina Abdel Shaheed
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Sweekriti Sharma
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Chris G Maher
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| | - Gustavo C Machado
- Sydney Musculoskeletal Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
- Institute for Musculoskeletal Health, Sydney Local Health District, Sydney, New South Wales, Australia
| |
Collapse
|
5
|
Strayer RJ, Friedman BW, Haroz R, Ketcham E, Klein L, LaPietra AM, Motov S, Repanshek Z, Taylor S, Weiner SG, Nelson LS. Emergency Department Management of Patients With Alcohol Intoxication, Alcohol Withdrawal, and Alcohol Use Disorder: A White Paper Prepared for the American Academy of Emergency Medicine. J Emerg Med 2023; 64:517-540. [PMID: 36997435 DOI: 10.1016/j.jemermed.2023.01.010] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 01/06/2023] [Indexed: 03/30/2023]
Affiliation(s)
- Reuben J Strayer
- Department of Emergency Medicine, Maimonides Medical Center, Brooklyn, New York.
| | - Benjamin W Friedman
- Department of Emergency Medicine, Montefiore, Albert Einstein College of Medicine, Bronx, New York
| | - Rachel Haroz
- Cooper Medical School of Rowan University, Cooper University Healthcare, Camden, New Jersey
| | - Eric Ketcham
- Department of Emergency Medicine, Department of Behavioral Health, Addiction Medicine, Presbyterian Healthcare System, Santa Fe & Española, New Mexico
| | - Lauren Klein
- Department of Emergency Medicine, Good Samaritan Hospital, West Islip, New York
| | - Alexis M LaPietra
- Department of Emergency Medicine, Saint Joseph's Regional Medical Center, Paterson, New Jersey
| | - Sergey Motov
- Department of Emergency Medicine, Maimonides Medical Center, Brooklyn, New York
| | - Zachary Repanshek
- Department of Emergency Medicine, Lewis Katz School of Medicine, Temple University, Philadelphia, Pennsylvania
| | - Scott Taylor
- Department of Emergency Medicine, Royal Melbourne Hospital, Melbourne, Australia
| | - Scott G Weiner
- Department of Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Lewis S Nelson
- Department of Emergency Medicine, Rutgers New Jersey Medical School, Newark, New Jersey
| |
Collapse
|
6
|
Phinn K, Liu S, Patanwala AE, Penm J. Effectiveness of organizational interventions on appropriate opioid prescribing for noncancer pain upon hospital discharge: A systematic review. Br J Clin Pharmacol 2023; 89:982-1002. [PMID: 36495313 DOI: 10.1111/bcp.15633] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Revised: 11/24/2022] [Accepted: 12/04/2022] [Indexed: 12/14/2022] Open
Abstract
This study aims to summarize the effectiveness of organizational interventions on appropriate opioid prescribing for noncancer pain upon hospital discharge. A systematic search was conducted on 6 electronic databases by 2 independent reviewers. We included original research articles reporting on quantitative outcomes of organizational interventions targeting appropriate opioid prescribing on hospital discharge. Quality assessment was performed by 2 independent reviewers. The protocol for this review was prospectively registered on PROSPERO (ID: CRD42020156104). Out of 173 full texts assessed for eligibility, 43 were included in this review. The majority of studies had a moderate to serious risk of bias (33 out of 43). Most of the studies implemented a multifaceted organizational intervention (16 studies). Other interventions included guideline implementation, prescriber education and default opioid-prescribing quantity changes in electronic medical records. Multiple studies found that the dissemination of patient-specific and procedure-specific guidelines reduced the quantity of opioids prescribed by 44 to 57%. Prescriber education provided with feedback was implemented in 4 studies and resulted in a 33 to 44% decrease in prescribing rates. Lowering the default quantities in the electronic medical records produced a 40% decrease in opioids prescribed in 1 study. Guideline implementation, prescriber education and default opioid-prescribing quantity changes all appear effective in improving the appropriate prescribing of opioids on hospital discharge. However, the extent of reduction of opioid prescribing upon hospital discharge after the implementation of multifaceted intervention strategies appears similar to that of simpler interventions which require fewer resources.
Collapse
Affiliation(s)
- Katelyn Phinn
- Faculty of Medicine and Health, School of Pharmacy, The University of Sydney, Camperdown, New South Wales, Australia
| | - Shania Liu
- Faculty of Medicine and Health, School of Pharmacy, The University of Sydney, Camperdown, New South Wales, Australia.,Department of Pharmacy, Prince of Wales Hospital, Randwick, New South Wales, Australia
| | - Asad E Patanwala
- Faculty of Medicine and Health, School of Pharmacy, The University of Sydney, Camperdown, New South Wales, Australia.,Department of Pharmacy, Royal Prince Alfred Hospital, Sydney, New South Wales, Australia
| | - Jonathan Penm
- Faculty of Medicine and Health, School of Pharmacy, The University of Sydney, Camperdown, New South Wales, Australia.,Department of Pharmacy, Prince of Wales Hospital, Randwick, New South Wales, Australia
| |
Collapse
|
7
|
Walk CT, Nowak R, Parikh PP, Crawford TN, Woods RJ. Perception versus Reality: A Review of Narcotic Prescribing Habits After Common Laparoscopic Surgeries. J Surg Res 2023; 283:188-193. [PMID: 36410235 DOI: 10.1016/j.jss.2022.10.049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 10/10/2022] [Accepted: 10/18/2022] [Indexed: 11/19/2022]
Abstract
INTRODUCTION Data on how surgeons perceive their habits of prescribing narcotics compared to their actual practice are scarce. This study examines the perception and actual narcotic prescribing habits of surgeons and advanced practitioners. METHODS Surgical residents, attendings, and advanced practice providers (APPs) were surveyed to assess their perceived prescribing habits at discharge for laparoscopic appendectomy and laparoscopic cholecystectomy. Data on narcotics prescription for patients receiving either of the procedures from January 2017 to August 2020 were extracted from electronic health records. Prescribed narcotics were converted to morphine equivalent doses (MEQs) for comparison. RESULTS Of the 52 participants, the majority were residents (57.7%). Approximately 90% of residents, 72% of attendings, and 18% of APPs reported regularly prescribing narcotics at discharge. Approximately 67% (889/1332) of patients were discharged with narcotics. Of those, the majority of patients' narcotics were prescribed by surgery residents (71.2%). However, 72% of residents, 80% of attendings, and 72% of APPs were confident on prescribing the correct regimen of narcotics. There were no differences in average daily MEQs among the groups. However, the number of narcotics prescribed was higher among APPs compared to that in the other groups (P < 0.0001). CONCLUSIONS Most participants self-reported routinely prescribing narcotics at discharge. Although not the current recommendation, participants felt confident they were prescribing the correct regimen, but were observed to prescribe more than the recommended number of total narcotics which indicates a discrepancy between perception and actual habits of prescribing narcotics. Our findings suggest a need for education in the general surgery residency and continuing medical education setting.
Collapse
Affiliation(s)
- Casey T Walk
- Department of Surgery, Wright State University Boonshoft SOM, Dayton, Ohio.
| | - Rebecca Nowak
- Wright State University Boonshoft School of Medicine, Fairborn, Ohio
| | - Priti P Parikh
- Department of Surgery, Wright State University Boonshoft SOM, Dayton, Ohio
| | | | - Randy J Woods
- Department of Surgery, Wright State University Boonshoft SOM, Dayton, Ohio
| |
Collapse
|
8
|
Larson MJ, Bauer MR, Moresco N, Huntington N, Ritter G, Paul-Kagiri R, Hyppolite R, Richard P. Variation in prescribing of opioids for emergency department encounters: A cohort study in the Military Health System. J Eval Clin Pract 2022; 28:1157-1167. [PMID: 35666601 PMCID: PMC10281196 DOI: 10.1111/jep.13702] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Revised: 04/29/2022] [Accepted: 05/04/2022] [Indexed: 11/30/2022]
Abstract
RATIONALE, AIMS AND OBJECTIVES: Emergency department (ED) clinicians account for approximately 13% of all opioid prescriptions to opioid-naïve patients and variability in the rates of prescribing have been noted among individual clinicians and different EDs. This study elucidates the amount of variability within a unified health system (the U.S. Military Health System [MHS]) with the expectation that understanding the sources of variability will enable health system leaders to improve the quality of decision making. METHODS The design was a retrospective cohort study examining variation in opioid prescribing within EDs of the US MHS. Participants were Army soldiers who returned from a deployment and received care between October 2009 and September 2016. The exposure was ED encounters at a military treatment facility. Key measures were the proportion of ED encounters with an opioid prescription fill; total opioid dose of the fill (morphine milligram equivalent, MME); and total opioid days-supply of the fill. RESULTS The mean proportion of ED encounters with an opioid fill across providers was 19.7% (SD 8.8%), median proportion was 18.6%, and the distribution was close to symmetric with the 75th percentile provider prescribing opioids in 24.6% of their ED encounters and the 25th percentile provider prescribing in 13.4% of their encounters. The provider-level mean opioid dose per encounter was 113.1 MME (SD 56.0) with the 75th percentile (130.1) 50% higher than the 25th percentile (87.4). The mean opioid supply per encounter was 6.8 days (SD 3.9) with more than a twofold ratio between the 75th percentile (8.3) and the 25th (4.1). Using a series of multilevel regression models to examine opioid fills associated with ED encounters and their dose levels, the variation among providers within facilities was much larger in magnitude than the variation among facilities. CONCLUSION Among ED encounters of Army soldiers at military treatment facilities, there was substantial variation among providers in prescribing opioid prescriptions that were not explained by patient case-mix. These results suggest that programmes and protocols to address less than optimal prescribing in the ED should be initiated to improve the quality of care.
Collapse
Affiliation(s)
- Mary Jo Larson
- Institute for Behavioral Health, The Heller School for Social Policy and Management, Brandeis University, Waltham, MA
| | - Mark R. Bauer
- Institute for Behavioral Health, The Heller School for Social Policy and Management, Brandeis University, Waltham, MA
| | - Natalie Moresco
- Institute for Behavioral Health, The Heller School for Social Policy and Management, Brandeis University, Waltham, MA
| | - Nick Huntington
- Institute for Behavioral Health, The Heller School for Social Policy and Management, Brandeis University, Waltham, MA
| | - Grant Ritter
- Institute for Behavioral Health, The Heller School for Social Policy and Management, Brandeis University, Waltham, MA
| | | | - Regine Hyppolite
- Uniformed Services University of the Health Sciences, Bethesda, MD
| | - Patrick Richard
- Uniformed Services University of the Health Sciences, Bethesda, MD
| |
Collapse
|
9
|
Addressing an epidemic: Improving guideline-concordant opioid prescribing in surgical patients. Surgery 2022; 172:1407-1414. [PMID: 36088172 DOI: 10.1016/j.surg.2022.06.033] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 06/08/2022] [Accepted: 06/30/2022] [Indexed: 11/23/2022]
Abstract
BACKGROUND Excess postoperative opioid prescribing increases the risk of opioid abuse, diversion, and addiction. Clinicians receive variable training for opioid prescribing, and despite the availability of guidelines, wide variations in prescribing practices persist. This quality improvement initiative aimed to assess and improve institutional adherence to published guidelines. METHODS This study represented a quality improvement initiative at an academic medical center implemented over a 6-month period with data captured 1 year before and after implementation. The quality improvement initiative focused on prescribing education and monthly feedback reports for clinicians. All opioid-naïve, adult patients undergoing a reviewed procedure were included. Demographics, surgical details, hospital course, and opioid prescriptions were reviewed. Opioids prescribed on discharge were evaluated for concordance with recommendations based on published guidelines. Pre- and postimplementation cohorts were compared. RESULTS There were 4,905 patients included: 2,343 preimplementation and 2,562 postimplementation. There were similar distributions in patient demographics between the 2 cohorts. Guideline-concordant discharge prescriptions improved from 50.3% to 72.2% after the quality improvement initiative was implemented (P < .001). Adjusted analysis controlling for sex, age, discharge clinician, length of stay, outpatient surgery, and procedure demonstrated a 190% increase in odds of receiving a guideline-concordant opioid prescription on discharge in the postimplementation cohort (adjusted odds ratio 2.90; 95% confidence interval = 2.55-3.30). CONCLUSION This study represented a successful quality improvement initiative improving guideline-concordant opioid discharges and decreasing overprescribing. This study suggested published guidelines are insufficient without close attention to elements of effective change management including the critical importance of locally targeting educational efforts and suggested that real-time, data-driven feedback amplifies impact on prescribing behavior.
Collapse
|
10
|
Navathe AS, Liao JM, Yan XS, Delgado MK, Isenberg WM, Landa HM, Bond BL, Small DS, Rareshide CAL, Shen Z, Pepe RS, Refai F, Lei VJ, Volpp KG, Patel MS. The Effect Of Clinician Feedback Interventions On Opioid Prescribing. Health Aff (Millwood) 2022; 41:424-433. [PMID: 35254932 DOI: 10.1377/hlthaff.2021.01407] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
An initial opioid prescription with a greater number of pills is associated with a greater risk for future long-term opioid use, yet few interventions have reliably influenced individual clinicians' prescribing. Our objective was to evaluate the effect of feedback interventions for clinicians in reducing opioid prescribing. The interventions included feedback on a clinician's outlier prescribing (individual audit feedback), peer comparison, and both interventions combined. We conducted a four-arm factorial pragmatic cluster randomized trial at forty-eight emergency department (ED) and urgent care (UC) sites in the western US, including 263 ED and 175 UC clinicians with 294,962 patient encounters. Relative to usual care, there was a significant decrease in pills per prescription both for peer comparison feedback (-0.8) and for the combination of peer comparison and individual audit feedback (-1.2). This decrease was sustained during follow-up. There were no significant changes for individual audit feedback alone, and no interventions changed the proportion of encounters with an opioid prescription.
Collapse
Affiliation(s)
- Amol S Navathe
- Amol S. Navathe , Corporal Michael J. Cresencz Veterans Affairs Medical Center and University of Pennsylvania, Philadelphia, Pennsylvania
| | - Joshua M Liao
- Joshua M. Liao, University of Washington, Seattle, Washington, and University of Pennsylvania
| | - Xiaowei S Yan
- Xiaowei S. Yan, Sutter Health, Walnut Creek, California
| | | | | | | | - Barbara L Bond
- Barbara L. Bond, Sutter Health, Castro Valley, California
| | | | | | - Zijun Shen
- Zijun Shen, Sutter Health, San Francisco
| | | | | | | | | | - Mitesh S Patel
- Mitesh S. Patel, Corporal Michael J. Cresencz Veterans Affairs Medical Center and University of Pennsylvania
| |
Collapse
|
11
|
Alishahi ML, Olson K, Brooks-Russell A, Hoppe J, Runyan C. Provider Reactions to Opioid-Prescribing Report Cards. JOURNAL OF PUBLIC HEALTH MANAGEMENT AND PRACTICE 2022; 28:E518-E524. [PMID: 34016911 DOI: 10.1097/phh.0000000000001382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE To evaluate prescribers' reactions and self-reported intentions to change prescribing behavior in response to opioid-prescribing report cards. DESIGN We surveyed a sample of licensed prescribers in the state of Colorado registered with the state's prescription drug monitoring program (PDMP). SETTING In 2018, Colorado disseminated tailored opioid-prescribing report cards to increase use of the PDMP and improve opioid prescribing. Report cards reflected individual prescribing history and compared individuals with an aggregate of others in the same specialty. Surveys were sent to approximately 29 000 prescribers registered with the PDMP 12 weeks after report card distribution. If respondents were not sent a report card, they were shown a sample report. Respondents were asked about their perceptions of the usefulness of the information and intentions to change their prescribing. PARTICIPANTS A total of 3784 prescribers responded to the survey. MAIN OUTCOME MEASURES Respondents were asked about their attitudes and reactions to an opioid-prescribing report card. Answers were given in the form of a 5-point Likert scale or multiple-choice questions. RESULTS Of those who responded, 53.6% were male and nearly half (49.5%) had spent more than 20 years in practice. Among prescribers who recalled receiving a report card, most felt the reports were easy to understand (87.4%) and provided new information (82.8%). Two-thirds of prescribers who saw their reports felt the information accurately reflected their prescribing practices. Overall, 40.0% reported they planned to change their prescribing behaviors as a result of the information provided. The most useful metrics identified by prescribers were the number of patients with multiple providers and the number of patients receiving dangerous combination therapy. CONCLUSIONS Overall, perceptions of the usefulness and accuracy of the report cards were positive. Understanding how the reports are perceived is a key factor to their use and influence. Further tailoring of the report to prescribers of different specialties and experience may enhance the effectiveness of the report cards.
Collapse
Affiliation(s)
- Musheng L Alishahi
- Departments of Epidemiology (Ms Alishahi and Dr Runyan) and Community and Behavioral Health (Drs Brooks-Russell and Runyan), Colorado School of Public Health, Aurora, Colorado; Colorado Department of Public Health and Environment, Denver, Colorado (Ms Olson); and Department of Emergency Medicine, University of Colorado Anschutz, Aurora, Colorado (Dr Hoppe)
| | | | | | | | | |
Collapse
|
12
|
Oskvarek JJ, Aldeen A, Shawbell J, Venkat A, Zocchi MS, Pines JM. Opioid Prescription Reduction After Implementation of a Feedback Program in a National Emergency Department Group. Ann Emerg Med 2022; 79:420-432. [DOI: 10.1016/j.annemergmed.2021.12.009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2021] [Revised: 12/10/2021] [Accepted: 12/15/2021] [Indexed: 02/07/2023]
|
13
|
Larkin C, Sanseverino AM, Joseph J, Eisenhauer L, Reznek MA. Accuracy of emergency physicians' self-estimates of CT scan utilization and its potential effect on an audit and feedback intervention: a randomized trial. Implement Sci Commun 2021; 2:83. [PMID: 34315533 PMCID: PMC8317272 DOI: 10.1186/s43058-021-00182-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2020] [Accepted: 07/06/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Audit and feedback (A&F) has been used as a strategy to modify clinician behavior with moderate success. Although A&F is theorized to work by improving the accuracy of clinicians' estimates of their own behavior, few interventions have included assessment of clinicians' estimates at baseline to examine whether they account for intervention success or failure. We tested an A&F intervention to reduce computed tomography (CT) ordering by emergency physicians, while also examining the physicians' baseline estimates of their own behavior compared to peers. METHODS Our study was a prospective, multi-site, 20-month, randomized trial to examine the effect of an A&F intervention on CT ordering rates, overall and by test subtype. From the electronic health record, we obtained 12 months of baseline CT ordering per 100 patients treated for every physician from four emergency departments. Those who were randomized to receive A&F were shown a de-identified graph of the group's baseline CT utilization, asked to estimate wherein the distribution of their own CT order practices fell, and then shown their actual performance. All participants also received a brief educational intervention. CT ordering rates were collected for all physicians for 6 months after the intervention. Pre-post ordering rates were compared using independent and repeated measures t tests. RESULTS Fifty-one of 52 eligible physicians participated. The mean CT ordering rate increased significantly in both experimental conditions after the intervention (intervention pre = 35.7, post = 40.3, t = 4.13, p < 0.001; control pre = 33.9, post = 38.9, t = 3.94, p = 0.001), with no significant between-group difference observed at follow-up (t = 0.43, p = 0.67). Within the intervention group, physicians had poor accuracy in estimating their own ordering behavior at baseline: most overestimated and all guessed that they were in the upper half of the distribution of their peers. CT ordering increased regardless of self-estimate accuracy. CONCLUSIONS Our A&F intervention failed to reduce physician CT ordering: our feedback to the physicians showed most of them that they had overestimated their CT ordering behavior, and they were therefore unlikely to reduce it as a result. After "audit," it may be prudent to assess baseline clinician awareness of behavior before moving toward a feedback intervention.
Collapse
Affiliation(s)
- Celine Larkin
- University of Massachusetts Medical School, Worcester, USA.
| | | | - James Joseph
- University of Massachusetts Medical School, Worcester, USA
| | | | | |
Collapse
|
14
|
Suffoletto B, Landau A. Nudging Emergency Care Providers to Reduce Opioid Prescribing Using Peer Norm Comparison Feedback: A Pilot Randomized Trial. PAIN MEDICINE 2021; 21:1393-1399. [PMID: 31846029 DOI: 10.1093/pm/pnz314] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
OBJECTIVE To determine the feasibility, acceptability, and potential impact of using audit and feedback (A&F) with or without peer norm comparison on opioid prescribing by emergency medicine providers. METHODS A convenience sample of 37 emergency medicine providers were recruited from 16 emergency departments in Western Pennsylvania for a pilot randomized controlled trial. Participants completed a baseline survey, were randomly allocated to A&F (N = 17) or A&F with peer norm comparison (N = 20), and were asked to complete a postintervention survey. We matched each participant 1:1 to a control who was not exposed to either intervention. RESULTS At baseline, 57% of participants perceived that they prescribed opioids at the same frequency as their peers, whereas 32% perceived prescribing less than and 11% perceived prescribing more than their peers. Most participants rated the interventions as helpful, with no differences between conditions. For the A&F with peer norm comparison condition, from pre- to postintervention, there was a relative increase of 20% in the percentage of participants who perceived that they prescribed more opioids than their peers but no change in the A&F condition (P = 0.02). 56.8% of controls, 52.9% of A&F participants, and 75.5% of A&F with peer norm comparison participants reduced their opioid prescribing (P = 0.33). The mean reduction in opioid prescriptions (SD) was 3.3. (9.6) for controls, 3.9 (10.5) for A&F, and 7.3 (7.8) for A&F with peer norm comparison (P = 0.31). CONCLUSIONS Audit and feedback interventions with peer norm comparisons are helpful to providers, can alter perceptions about prescribing norms, and are a potentially effective way to alter ED providers' opioid prescribing behavior.
Collapse
Affiliation(s)
- Brian Suffoletto
- Department of Emergency Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | - Aaron Landau
- School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
15
|
Raman R, Fleming L. We Need to Talk About Codeine: an Implementation Study to reduce the number of Emergency Department patients discharged on high-strength co-codamol using the Behaviour Change Wheel. Emerg Med J 2021; 38:895-900. [PMID: 33658270 DOI: 10.1136/emermed-2020-209479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2020] [Revised: 02/08/2021] [Accepted: 02/13/2021] [Indexed: 11/03/2022]
Abstract
BACKGROUND The crisis of prescription opioid addiction in the USA is well-documented. Though opioid consumption per capita is lower in the UK, prescribing has increased dramatically in recent decades with an associated increase in deaths from prescription opioid overdose. At one Scottish Emergency Department high rates of prescribing of take-home co-codamol (30/500 mg) were observed, including for conditions where opioids are not recommended by national guidelines. An Implementation Science approach was adopted to investigate this. METHODS A Behaviour Change Wheel analysis suggested several factors contributing to high opioid prescribing: poor awareness of codeine addiction risk, poor knowledge of NICE (National Institute for Health and Care Excellence) guidelines on common painful conditions, mistaken assumptions about patient expectations and ready access to a large stock of take-home co-codamol. Based on this analysis a combined Education/Persuasion intervention was implemented over a 1-month period (January 2019) reaching 93% of prescribers. An Environmental Restructuring intervention was introduced at 4 months, and co-codamol prescriptions were monitored over a 12-month follow-up period. Unplanned re-attendances and complaints related to analgesia were monitored as balancing measures. RESULTS The Education/Persuasion intervention was associated with a 59% reduction in co-codamol prescribing that was maintained over 12 months. The Environmental Restructuring intervention was not associated with any further reduction in prescribing. No increase in unplanned re-attendances occurred during the study period and no complaints were received relating to pain control. CONCLUSIONS The increasing incidence of prescription opioid addiction in the UK suggests the need for all clinicians who write opioid prescriptions to re-evaluate their practice. This study suggests that knowledge of addiction risk and prescribing guidelines is poor among Emergency Department prescribers. We show that a rapid and sustained reduction in prescribing of take-home opioids is feasible in a UK Emergency Department, and that this reduction was not associated with any increase in unplanned re-attendances or complaints related to analgesia.
Collapse
Affiliation(s)
- Rajendra Raman
- Accident and Emergency, Victoria Hospital, Kirkcaldy, UK
| | - Laura Fleming
- Accident and Emergency, Victoria Hospital, Kirkcaldy, UK
| |
Collapse
|
16
|
Decreased Overall and Inappropriate Antibiotic Prescribing in a Veterans Affairs Hospital Emergency Department following a Peer Comparison-Based Stewardship Intervention. Antimicrob Agents Chemother 2020; 65:AAC.01660-20. [PMID: 33020159 DOI: 10.1128/aac.01660-20] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 10/01/2020] [Indexed: 12/19/2022] Open
Abstract
Antibiotic prescribing is very common in emergency departments (EDs). Optimal stewardship intervention strategies in EDs are not well defined. We conducted a prospective, observational cohort study in a Veterans Affairs ED in which clinician education and monthly e-mail-based peer comparisons were directed against all oral antibiotic prescribing for discharged patients. Oral antibiotic prescriptions were compared in baseline (June 2016 to December 2017) and intervention (January to June 2018) periods using an interrupted time series regression model. Prescribing appropriateness was compared during January to June 2017 and the intervention period. During the intervention period, antibiotic prescriptions decreased monthly by 10.4 prescriptions per 1,000 ED visits (P = 0.07 [95% confidence interval {CI}, -21.7 to 1.0]). The relative decrease in the trend of antibiotic prescriptions during the intervention period compared to baseline was 9.9 prescriptions per 1,000 ED visits per month (P = 0.07 [95% CI, -20.9 to 1.0]). The intervention was associated with a significant decrease and increase in amoxicillin-clavulanate and cephalexin prescriptions, respectively (P < 0.001, P = 0.004). Decreasing trends in ciprofloxacin prescriptions during the baseline period were maintained during the intervention. Unnecessary antibiotic prescribing (i.e., antibiotic not indicated) decreased from 55.6% to 38.7% during the intervention (30.4% decrease, P = 0.003). Optimal antibiotic prescribing (i.e., antibiotics were indicated, and a guideline-concordant agent was prescribed for guideline-concordant duration) increased by 36% (21.6% to 29.3%, P = 0.12). A peer comparison-based stewardship intervention directed at ED clinicians was associated with reductions in overall and unnecessary oral antibiotic prescribing. There is potential to further improve antibiotic use as suboptimal prescribing remained common.
Collapse
|
17
|
Cotterill S, Tang MY, Powell R, Howarth E, McGowan L, Roberts J, Brown B, Rhodes S. Social norms interventions to change clinical behaviour in health workers: a systematic review and meta-analysis. HEALTH SERVICES AND DELIVERY RESEARCH 2020. [DOI: 10.3310/hsdr08410] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023] Open
Abstract
Background
A social norms intervention seeks to change the clinical behaviour of a target health worker by exposing them to the values, beliefs, attitudes or behaviours of a reference group or person. These low-cost interventions can be used to encourage health workers to follow recommended professional practice.
Objective
To summarise evidence on whether or not social norms interventions are effective in encouraging health worker behaviour change, and to identify the most effective social norms interventions.
Design
A systematic review and meta-analysis of randomised controlled trials.
Data sources
The following databases were searched on 24 July 2018: Ovid MEDLINE (1946 to week 2 July 2018), EMBASE (1974 to 3 July 2018), Cumulative Index to Nursing and Allied Health Literature (1937 to July 2018), British Nursing Index (2008 to July 2018), ISI Web of Science (1900 to present), PsycINFO (1806 to week 3 July 2018) and Cochrane trials (up to July 2018).
Participants
Health workers took part in the study.
Interventions
Behaviour change interventions based on social norms.
Outcome measures
Health worker clinical behaviour, for example prescribing (primary outcome), and patient health outcomes, for example blood test results (secondary), converted into a standardised mean difference.
Methods
Titles and abstracts were reviewed against the inclusion criteria to exclude any that were clearly ineligible. Two reviewers independently screened the remaining full texts to identify relevant papers. Two reviewers extracted data independently, coded for behaviour change techniques and assessed quality using the Cochrane risk-of-bias tool. We performed a meta-analysis and presented forest plots, stratified by behaviour change technique. Sources of variation were explored using metaregression and network meta-analysis.
Results
A total of 4428 abstracts were screened, 477 full texts were screened and findings were based on 106 studies. Most studies were in primary care or hospitals, targeting prescribing, ordering of tests and communication with patients. The interventions included social comparison (in which information is given on how peers behave) and credible source (which refers to communication from a well-respected person in support of the behaviour). Combined data suggested that interventions that included social norms components were associated with an improvement in health worker behaviour of 0.08 standardised mean differences (95% confidence interval 0.07 to 0.10 standardised mean differences) (n = 100 comparisons), and an improvement in patient outcomes of 0.17 standardised mean differences (95% confidence interval 0.14 to 0.20) (n = 14), on average. Heterogeneity was high, with an overall I
2 of 85.4% (primary) and 91.5% (secondary). Network meta-analysis suggested that three types of social norms intervention were most effective, on average, compared with control: credible source (0.30 standardised mean differences, 95% confidence interval 0.13 to 0.47); social comparison combined with social reward (0.39 standardised mean differences, 95% confidence interval 0.15 to 0.64); and social comparison combined with prompts and cues (0.33 standardised mean differences, 95% confidence interval 0.22 to 0.44).
Limitations
The large number of studies prevented us from requesting additional information from authors. The trials varied in design, context and setting, and we combined different types of outcome to provide an overall summary of evidence, resulting in a very heterogeneous review.
Conclusions
Social norms interventions are an effective method of changing clinical behaviour in a variety of health service contexts. Although the overall result was modest and very variable, there is the potential for social norms interventions to be scaled up to target the behaviour of a large population of health workers and resulting patient outcomes.
Future work
Development of optimised credible source and social comparison behaviour change interventions, including qualitative research on acceptability and feasibility.
Study registration
This study is registered as PROSPERO CRD42016045718.
Funding
This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 41. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Sarah Cotterill
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Mei Yee Tang
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Rachael Powell
- Manchester Centre for Health Psychology, Division of Psychology and Mental Health, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Elizabeth Howarth
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Laura McGowan
- Manchester Centre for Health Psychology, Division of Psychology and Mental Health, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Jane Roberts
- Outreach and Evidence Search Service, Library and E-learning Service, Northern Care Alliance, NHS Group, Royal Oldham Hospital, Oldham, UK
| | - Benjamin Brown
- Health e-Research Centre, Farr Institute for Health Informatics Research, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Centre for Primary Care, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Sarah Rhodes
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| |
Collapse
|
18
|
Dieujuste N, Johnson‐Koenke R, Christopher M, Gunzburger EC, Emmendorfer T, Kessler C, Haukoos J, Smith J, Sasson C. Feasibility Study of a Quasi-experimental Regional Opioid Safety Prescribing Program in Veterans Health Administration Emergency Departments. Acad Emerg Med 2020; 27:734-741. [PMID: 32239558 DOI: 10.1111/acem.13980] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2019] [Revised: 03/03/2020] [Accepted: 03/27/2020] [Indexed: 12/01/2022]
Abstract
BACKGROUND The Veterans Health Administration (VHA) Opioid Safety Initiative (OSI) was implemented in 2013 and was associated with a 25% relative decrease in the dispensing of opioids. Although emergency department (ED) providers play a role in the initiation and continuation of opioids, the incumbent OSI did not target EDs. OBJECTIVE The goal of this feasibility study was to leverage the existing VHA OSI and test a novel ED-based quality improvement (QI) program to decrease opioid prescribing in multiple ED settings. METHODS This was a quasi-experimental study of phased-in implementation of a QI ED-based OSI. The general setting for this pilot were four VHA EDs across the Veterans Integrated Services Network (VISN) region 19: Denver, Oklahoma City, Muskogee, and Salt Lake City. We developed and disseminated a dashboard to assess ED-specific prescribing rates and an ED-tailored toolkit to implement the program. Academic detailing pharmacists provided focused audits and feedback with the highest prescribing providers. We measured change in ED-provider prescribing rate of opioids for patients discharged from the ED, by provider and aggregated up to facility level, pre- and postimplementation. RESULTS Interrupted time-series analysis of provider-level data from the program implementation sites indicated a significant decrease in the trend for proportion of opioid prescriptions relative to the preintervention trend. The results of the analysis suggest that the intervention was associated with accelerating the rate at which ED provider prescribing rates decreased. CONCLUSION Due to the high volume of patients and the vital role the ED plays in patient treatment and hospital admissions, it is evident that the ED is an important site for QI programs as well as the implementation of opioid safety measures. Given the findings of this pilot, we believe that implementation of a national Veterans Affairs ED OSI implementation is feasible practice.
Collapse
Affiliation(s)
- Nathalie Dieujuste
- From the VA Eastern Colorado Health Care System, Veterans Health Administration Aurora CO
| | - Rachel Johnson‐Koenke
- From the VA Eastern Colorado Health Care System, Veterans Health Administration Aurora CO
| | - Melissa Christopher
- National Academic Detailing Services, Veterans Health Administration San Diego CA
| | - Elise C. Gunzburger
- From the VA Eastern Colorado Health Care System, Veterans Health Administration Aurora CO
| | - Thomas Emmendorfer
- National Pharmacy Benefits Management, Veterans Health Administration Washington DC
| | - Chad Kessler
- Durham VA Medical Center National Emergency Medicine Program Office, Veterans Health Administration Durham NC
| | - Jason Haukoos
- and the Denver Health and Hospital Authority Denver CO
- and the Colorado School of Public Health Aurora CO
- the University of Colorado School of Medicine Aurora CO
| | - Jason Smith
- and VA Eastern Colorado Health Care System, Veterans Health Administration Denver CO
| | - Comilla Sasson
- From the VA Eastern Colorado Health Care System, Veterans Health Administration Aurora CO
- and the Colorado School of Public Health Aurora CO
- the University of Colorado School of Medicine Aurora CO
| |
Collapse
|
19
|
Kim DA, Sanchez LD, Chiu D, Brown IP. Social Determinants of Hallway Bed Use. West J Emerg Med 2020; 21:949-958. [PMID: 32726269 PMCID: PMC7390564 DOI: 10.5811/westjem.2020.4.45976] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Accepted: 04/01/2020] [Indexed: 11/16/2022] Open
Abstract
Introduction Hallway beds in the emergency department (ED) produce lower patient satisfaction and inferior care. We sought to determine whether socioeconomic factors influence which visits are assigned to hallway beds, independent of clinical characteristics at triage. Methods We studied 332,919 visits, across 189,326 patients, to two academic EDs from 2013–2016. We estimated a logistic model of hallway bed assignment, conditioning on payor, demographics, triage acuity, chief complaint, patient visit frequency, and ED volume. Because payor is not generally known at the time of triage, we interpreted it as a proxy for other observable characteristics that may influence bed assignment. We estimated a Cox proportional hazards model of hallway bed assignment on length of stay. Results Median patient age was 53. 54.0% of visits were by women. 42.1% of visits were paid primarily by private payors, 37.1% by Medicare, and 20.7% by Medicaid. A total of 16.2% of visits were assigned to hallway beds. Hallway bed assignment was more likely for frequent ED visitors, for lower acuity presentations, and for psychiatric, substance use, and musculoskeletal chief complaints, which were more common among visits paid primarily by Medicaid. In a logistic model controlling for these factors, as well as for other patient demographics and for the volume of recent ED arrivals, Medicaid status was nevertheless associated with 22% greater odds of assignment to a hallway bed (odds ratio 1.22, [95% confidence interval, CI, 1.18–1.26]), compared to private insurance. Visits assigned to hallway beds had longer lengths of stay than roomed visits of comparable acuity (hazard ratio for departure 0.91 [95% CI, 0.90–0.92]). Conclusion We find evidence of social determinants of hallway bed use, likely involving epidemiologic, clinical, and operational factors. Even after accounting for different distributions of chief complaints and for more frequent ED use by the Medicaid population, as well as for other visit characteristics known at the time of triage, visits paid primarily by Medicaid retain a disproportionate association with hallway bed assignment. Further research is needed to eliminate potential bias in the use of hallway beds. [West J Emerg Med. 2020;21(4)949–958.]
Collapse
Affiliation(s)
- David A Kim
- Stanford University, Department of Emergency Medicine, Palo Alto, California
| | - Leon D Sanchez
- Beth Israel Deaconess Medical Center, Department of Emergency Medicine, Boston, Massachusetts
| | - David Chiu
- Beth Israel Deaconess Medical Center, Department of Emergency Medicine, Boston, Massachusetts
| | - Ian P Brown
- Stanford University, Department of Emergency Medicine, Palo Alto, California
| |
Collapse
|
20
|
Navathe AS, Volpp KG, Bond AM, Linn KA, Caldarella KL, Troxel AB, Zhu J, Yang L, Matloubieh SE, Drye EE, Bernheim SM, Oshima Lee E, Mugiishi M, Endo KT, Yoshimoto J, Emanuel EJ. Assessing The Effectiveness Of Peer Comparisons As A Way To Improve Health Care Quality. Health Aff (Millwood) 2020; 39:852-861. [DOI: 10.1377/hlthaff.2019.01061] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
Affiliation(s)
- Amol S. Navathe
- Amol S. Navathe is a core investigator at the Corporal Michael J. Cresencz Veterans Affairs (VA) Medical Center; and an assistant professor in the Department of Medical Ethics and Health Policy, Perelman School of Medicine, and a senior fellow at the Leonard Davis Institute of Health Economics, University of Pennsylvania, all in Philadelphia
| | - Kevin G. Volpp
- Kevin G. Volpp is a professor of medicine in the Department of Medicine at the Perelman School of Medicine and of health care management at the Wharton School, vice chair for health policy in the Department of Medical Ethics and Health Policy, and director of the Center for Health Incentives and Behavioral Economics, all at the University of Pennsylvania, and a staff physician at the Corporal Michael J. Crescenz VA Medical Center
| | - Amelia M. Bond
- Amelia M. Bond is an assistant professor of health care policy and research at Weill Cornell Medical College, in New York City
| | - Kristin A. Linn
- Kristin A. Linn is an assistant professor of biostatistics in the Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania
| | - Kristen L. Caldarella
- Kristen L. Caldarella is a project manager in the Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania
| | - Andrea B. Troxel
- Andrea B. Troxel is director of the Division of Biostatistics, New York University School of Medicine, in New York City
| | - Jingsan Zhu
- Jingsan Zhu is associate director of data analytics in the Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania
| | - Lin Yang
- Lin Yang is a programmer analyst in the Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania
| | - Shireen E. Matloubieh
- Shireen E. Matloubieh is a research coordinator in the Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania
| | - Elizabeth E. Drye
- Elizabeth E. Drye is a research scientist in the Department of Pediatrics, Yale University School of Medicine, in New Haven, Connecticut
| | - Susannah M. Bernheim
- Susannah M. Bernheim is director of quality measurement at the Center for Outcomes Research and Evaluation at Yale–New Haven Hospital and an assistant clinical professor in the Department of Internal Medicine at Yale University School of Medicine
| | - Emily Oshima Lee
- Emily Oshima Lee is assistant vice president of health strategy at the Hawaii Medical Services Association (HMSA), in Honolulu
| | | | - Kimberly Takata Endo
- Kimberly Takata Endo is a health strategist in the Department of Payment Transformation, HMSA
| | - Justin Yoshimoto
- Justin Yoshimoto is a health strategist in the Department of Payment Transformation, HMSA
| | - Ezekiel J. Emanuel
- Ezekiel J. Emanuel is the Diane V. S. Levy and Robert M. Levy University Professor, chair of the Department of Medical Ethics and Health Policy, and vice provost for global initiatives, all at the University of Pennsylvania
| |
Collapse
|
21
|
Strayer RJ, Hawk K, Hayes BD, Herring AA, Ketcham E, LaPietra AM, Lynch JJ, Motov S, Repanshek Z, Weiner SG, Nelson LS. Management of Opioid Use Disorder in the Emergency Department: A White Paper Prepared for the American Academy of Emergency Medicine. J Emerg Med 2020; 58:522-546. [DOI: 10.1016/j.jemermed.2019.12.034] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2019] [Revised: 12/19/2019] [Accepted: 12/24/2019] [Indexed: 11/28/2022]
|
22
|
Ogdie A, Asch DA. Changing health behaviours in rheumatology: an introduction to behavioural economics. Nat Rev Rheumatol 2019; 16:53-60. [DOI: 10.1038/s41584-019-0336-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/31/2019] [Indexed: 11/09/2022]
|
23
|
McCarthy DM, Curtis LM, Courtney DM, Cameron KA, Lank PM, Kim HS, Opsasnick LA, Lyden AE, Gravenor SJ, Russell AM, Eifler MR, Hur SI, Rowland ME, Walton SM, Montague E, Kim KYA, Wolf MS. A Multifaceted Intervention to Improve Patient Knowledge and Safe Use of Opioids: Results of the ED EMC 2 Randomized Controlled Trial. Acad Emerg Med 2019; 26:1311-1325. [PMID: 31742823 DOI: 10.1111/acem.13860] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
OBJECTIVES Despite increased focus on opioid prescribing, little is known about the influence of prescription opioid medication information given to patients in the emergency department (ED). The study objective was to evaluate the effect of an Electronic Medication Complete Communication (EMC2 ) Opioid Strategy on patients' safe use of opioids and knowledge about opioids. METHODS This was a three-arm prospective, randomized controlled pragmatic trial with randomization occurring at the physician level. Consecutive discharged patients at an urban academic ED (>88,000 visits) with new hydrocodone-acetaminophen prescriptions received one of three care pathways: 1) usual care, 2) EMC2 intervention, or 3) EMC2 + short message service (SMS) text messaging. The ED EMC2 intervention triggered two patient-facing educational tools (MedSheet, literacy-appropriate prescription wording [Take-Wait-Stop]) and three provider-facing reminders to counsel (directed to ED physician, dispensing pharmacist, follow-up physician). Patients in the EMC2 + SMS arm additionally received one text message/day for 1 week. Follow-up at 1 to 2 weeks assessed "demonstrated safe use" (primary outcome). Secondary outcomes including patient knowledge and actual safe use (via medication diaries) were assessed 2 to 4 days and 1 month following enrollment. RESULTS Among the 652 enrolled, 343 completed follow-up (57% women; mean ± SD age = 42 ± 14.0 years). Demonstrated safe opioid use occurred more often in the EMC2 group (adjusted odds ratio [aOR] = 2.46, 95% confidence interval [CI] = 1.19 to 5.06), but not the EMC2 + SMS group (aOR = 1.87, 95% CI = 0.90 to 3.90) compared with usual care. Neither intervention arm improved medication safe use as measured by medication diary data. Medication knowledge, measured by a 10-point composite knowledge score, was greater in the EMC2 + SMS group (β = 0.57, 95% CI = 0.09 to 1.06) than usual care. CONCLUSIONS The study found that the EMC2 tools improved demonstrated safe dosing, but these benefits did not translate into actual use based on medication dairies. The text-messaging intervention did result in improved patient knowledge.
Collapse
Affiliation(s)
| | - Laura M Curtis
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
| | - D Mark Courtney
- Department of Emergency Medicine, Northwestern University, Chicago, IL
| | - Kenzie A Cameron
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
- Department of Preventive Medicine, Northwestern University, Chicago, IL
| | - Patrick M Lank
- Department of Emergency Medicine, Northwestern University, Chicago, IL
| | - Howard S Kim
- Department of Emergency Medicine, Northwestern University, Chicago, IL
| | - Lauren A Opsasnick
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
| | - Abbie E Lyden
- College of Pharmacy, Rosalind Franklin School of Medicine and Science, Chicago, IL
| | | | - Andrea M Russell
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
| | - Morgan R Eifler
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
| | - Scott I Hur
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
| | - Megan E Rowland
- Department of Emergency Medicine, Northwestern University, Chicago, IL
| | - Surrey M Walton
- Department of Pharmacy Administration, University of Illinois at Chicago, Chicago, IL
| | - Enid Montague
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
- School of Computing, DePaul University, Chicago, IL
| | - Kwang-Youn A Kim
- Department of Preventive Medicine, Northwestern University, Chicago, IL
| | - Michael S Wolf
- Division of General Internal Medicine and Geriatrics, Northwestern University, Chicago, IL
| |
Collapse
|
24
|
Haegerich TM, Jones CM, Cote PO, Robinson A, Ross L. Evidence for state, community and systems-level prevention strategies to address the opioid crisis. Drug Alcohol Depend 2019; 204:107563. [PMID: 31585357 PMCID: PMC9286294 DOI: 10.1016/j.drugalcdep.2019.107563] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/20/2019] [Revised: 08/07/2019] [Accepted: 08/12/2019] [Indexed: 11/23/2022]
Abstract
BACKGROUND Practitioners and policy makers need evidence to facilitate the selection of effective prevention interventions that can address the ongoing opioid overdose epidemic in the United States. METHODS We conducted a systematic review of publications reporting on rigorous evaluations of systems-level interventions to address provider and patient/public behavior and prevent prescription and illicit opioid overdose. A total of 251 studies were reviewed. Interventions studied included 1) state legislation and regulation, 2) prescription drug monitoring programs (PDMPs), 3) insurance strategies, 4) clinical guideline implementation, 5) provider education, 6) health system interventions, 7) naloxone education and distribution, 8) safe storage and disposal, 9) public education, 10) community coalitions, and 11) interventions employing public safety and public health collaborations. RESULTS The quality of evidence supporting selected interventions was low to moderate. Interventions with the strongest evidence include PDMP and pain clinic legislation, insurance strategies, motivational interviewing in clinical settings, feedback to providers on opioid prescribing behavior, intensive school and family-based programs, and patient education in the clinical setting. CONCLUSIONS Although evidence is growing, further high-quality research is needed. Investigators should aim to identify strategies that can prevent overdose, as well as influence public, patient, and provider behavior. Identifying which strategies are most effective at addressing prescription compared to illicit opioid misuse and overdose could be fruitful, as well as investigating synergistic effects and unintended consequences.
Collapse
Affiliation(s)
- Tamara M. Haegerich
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, 4770 Buford Highway NE, Atlanta, GA, 30341, USA,Corresponding author: (T.M. Haegerich)
| | - Christopher M. Jones
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, 4770 Buford Highway NE, Atlanta, GA, 30341, USA
| | - Pierre-Olivier Cote
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, 4770 Buford Highway NE, Atlanta, GA, 30341, USA.
| | - Amber Robinson
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, 4770 Buford Highway NE, Atlanta, GA, 30341, USA.
| | | |
Collapse
|
25
|
Smalley CM, Willner MA, Muir MR, Meldon SW, Borden BL, Delgado FJ, Fertel BS. Electronic medical record-based interventions to encourage opioid prescribing best practices in the emergency department. Am J Emerg Med 2019; 38:1647-1651. [PMID: 31718956 DOI: 10.1016/j.ajem.2019.158500] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2019] [Revised: 10/01/2019] [Accepted: 10/02/2019] [Indexed: 11/30/2022] Open
Abstract
OBJECTIVE Overdose from opioids has reached epidemic proportions. Large healthcare systems can utilize existing technology to encourage responsible opioid prescribing practices. Our study measured the effects of using the electronic medical record (EMR) with direct clinician feedback to standardize opioid prescribing practices within a large healthcare system. METHODS This retrospective multicenter study compared a 12 month pre- and post-intervention in 14 emergency departments after four interventions utilizing the EMR were implemented: (1) deleting clinician preference lists, (2) defaulting dose, frequency, and quantity, (3) standardizing formulary to encourage best practices, and (4) creating dashboards for clinician review with current opioid prescribing practices. Outlying clinicians received feedback through email and direct counseling. Total number of opioid prescriptions per 100 discharges pre- and post-intervention were recorded as primary outcome. Secondary outcomes included number of prescriptions per 100 discharges/clinician exceeding 3-day supply (defined as 12 tablets), number exceeding 30 morphine equivalent daily dose (MEDD)/day, and number of non-formulary prescriptions. RESULTS There were >700,000 discharges during pre- and post-intervention periods. Percentage of total number opioid prescriptions per 100 discharges decreased from 14.4% to 7.4%, a 7.0% absolute reduction, (95% CI,6.9%-7.2%). There was a 5.9% to 0.7% reduction in prescriptions exceeding 3-days, (95% CI, 5.1%-5.3%), a 4.3% to 0.3% reduction in prescriptions exceeding 30 MEDD, (95% CI, 3.9%-4.0%), and a 0.3% to 0.1% reduction in non-formulary prescriptions, (95% CI, 0.2%-0.3%). CONCLUSIONS A multi modal approach using EMR interventions which provide real time data and direct feedback to clinicians can facilitate appropriate opioid prescribing.
Collapse
Affiliation(s)
- Courtney M Smalley
- Emergency Services Institute, Cleveland Clinic Health System, Cleveland, OH, USA; Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, Cleveland, OH, USA
| | - Marc A Willner
- Department of Pharmacy, Cleveland Clinic Health System, Cleveland, OH, USA
| | - McKinsey R Muir
- Emergency Services Institute, Cleveland Clinic Health System, Cleveland, OH, USA
| | - Stephen W Meldon
- Emergency Services Institute, Cleveland Clinic Health System, Cleveland, OH, USA; Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, Cleveland, OH, USA
| | - Bradford L Borden
- Emergency Services Institute, Cleveland Clinic Health System, Cleveland, OH, USA; Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, Cleveland, OH, USA
| | - Fernando J Delgado
- Emergency Services Institute, Cleveland Clinic Health System, Cleveland, OH, USA
| | - Baruch S Fertel
- Emergency Services Institute, Cleveland Clinic Health System, Cleveland, OH, USA; Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, Cleveland, OH, USA; Enterprise Quality and Safety, Cleveland Clinic Health System, Cleveland, OH, USA
| |
Collapse
|
26
|
A Quality Improvement Initiative Featuring Peer-Comparison Prescribing Feedback Reduces Emergency Department Opioid Prescribing. Jt Comm J Qual Patient Saf 2019; 45:669-679. [PMID: 31488343 DOI: 10.1016/j.jcjq.2019.07.008] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2019] [Revised: 07/22/2019] [Accepted: 07/24/2019] [Indexed: 11/27/2022]
Abstract
BACKGROUND Opioid prescribing in the United States nearly tripled from 1999 to 2015, and opioid overdose deaths doubled in the same time frame. Emergency departments (EDs) may play a pivotal role in the opioid epidemic as a source of first-time opioid exposure; however, many prescribers are generally unaware of their prescribing behaviors relative to their peers. METHODS All 117 ED prescribers at an urban academic medical center were provided with regular feedback on individual rates of opioid prescribing relative to their de-identified peers. To evaluate the effect of this intervention on the departmental rate of opioid prescribing, a statistical process control (SPC) chart was created to identify special cause variation, and an interrupted time series analysis was conducted to evaluate the immediate effect of the intervention and any change in the postintervention trend due to the intervention. RESULTS The aggregate opioid prescribing rate in the preintervention period was 8.6% (95% confidence interval [CI]: 8.3%-8.9%), while the aggregate postintervention prescribing rate was 5.8% (95% CI: 5.5%-6.1%). The SPC chart revealed special cause variation in both the pre- and postintervention periods, with an overall downtrend of opioid prescribing rates across the evaluation period and flattening of rates in the final four blocks. Interrupted time series analysis demonstrated a significant immediate downward effect of the intervention and a nonsignificant additional decrease in postintervention trend. CONCLUSION Implementation of peer-comparison opioid prescribing feedback was associated with a significant immediate reduction in the rate of ED discharge opioid prescribing.
Collapse
|
27
|
Reduction of opioid prescribing through the sharing of individual physician opioid prescribing practices. Am J Emerg Med 2019; 37:118-122. [DOI: 10.1016/j.ajem.2018.09.052] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2018] [Revised: 09/28/2018] [Accepted: 09/29/2018] [Indexed: 11/22/2022] Open
|
28
|
Hays GP, Mycyk MB. Confronting the Opioid Crisis by Taking a Long Look in the Mirror … and at Our Peers. Acad Emerg Med 2018; 25:594-596. [PMID: 29498140 DOI: 10.1111/acem.13397] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
|