A high value care curriculum for interns: a description of curricular design, implementation and housestaff feedback

Creation of a Didactic Clinical Pharmacology Lecture Series for Internal Medicine Residents

INTRODUCTION

One of the noted areas of weakness for internal medicine residents is pharmacology. However, there is little data documenting the creation and effect of a comprehensive pharmacology didactic program. Our goal was to create a two-year clinical pharmacology didactic program focused on areas of stated weakness and to evaluate this program for an increase in knowledge and prescribing confidence of the participants.

METHODS

From August 2020 to June 2022, a two-year pharmacology program was developed, which included 20 didactic lectures on a variety of topics. Pre- and post-tests were given for 15 of the lectures, and four surveys were given, two during each year of the program. Four questions on each survey were the same and asked about confidence in choosing an appropriate medication based on current guidelines, patient-specific factors, primary literature, and pharmacokinetics.

RESULTS

 Over the two years, participation in the pharmacology sessions ranged from 17 to 29 residents (65-74% of the residency class). The average pre- and post-test scores increased by an average of 25.1%, which was a statistically significant increase (p<0.001, 95% CI [17.5, 32.8]). A Kruskal-Wallis H test showed a statistically significant difference in resident-reported confidence adjusting medications based on primary literature between the different survey groups, χ2 = 9.871, p = 0.02.

CONCLUSION

A two-year, didactic pharmacology program improved the knowledge of resident participants and confidence in their ability to choose an appropriate medication based on primary literature.

Choosing Words Wisely: Residents' Use of Rhetorical Appeals in Conversations About Unnecessary Tests

PURPOSE

To characterize how residents employ rhetorical appeals (i.e., the strategic use of communication to achieve specifiable goals) when discussing unnecessary diagnostic tests with patients.

METHOD

In 2015, senior hematology residents from 10 Canadian universities participating in a national formative objective structured clinical examination (OSCE) completed a resource stewardship communication station. In this communication scenario, a standardized patient (SP) portrayed a patient requesting unnecessary thrombophilia testing following early pregnancy loss. The authors performed a thematic analysis of audio transcripts using a qualitative description approach to identify residents' rhetorical appeals to logic (rational appeals), credibility, and emotion.

RESULTS

For persuasive communication, residents (n = 27) relied primarily on rational appeals that fit into 3 categories (with themes) focused on medical evidence (poor utility, professional guidelines and recommendations), avoidance of harm (insurance implications, unnecessary or potentially harmful interventions, patient anxiety), and reassurance to patient (normalizing, clinical pretest probability, criteria for reconsidering testing). Appeals to credibility and emotion were rarely used.

CONCLUSIONS

In an OSCE setting, residents relied predominantly on rational appeals when engaging SPs in conversations about unnecessary tests. These observations yield insights into how recent emphasis within residency education on appropriate test utilization may manifest when residents put recommendations into practice in conversations with patients. This study's framework of rational appeals may be helpful in designing communication curricula about unnecessary testing. Future studies should explore rhetoric about unnecessary testing in the clinical environment, strategies to teach and coach residents leading these conversations, and patients' preferences and responses to different appeals.

Prevalence and Predictability of Low-Yield Inpatient Laboratory Diagnostic Tests

IMPORTANCE

Laboratory testing is an important target for high-value care initiatives, constituting the highest volume of medical procedures. Prior studies have found that up to half of all inpatient laboratory tests may be medically unnecessary, but a systematic method to identify these unnecessary tests in individual cases is lacking.

OBJECTIVE

To systematically identify low-yield inpatient laboratory testing through personalized predictions.

DESIGN, SETTING, AND PARTICIPANTS

In this retrospective diagnostic study with multivariable prediction models, 116 637 inpatients treated at Stanford University Hospital from January 1, 2008, to December 31, 2017, a total of 60 929 inpatients treated at University of Michigan from January 1, 2015, to December 31, 2018, and 13 940 inpatients treated at the University of California, San Francisco from January 1 to December 31, 2018, were assessed.

MAIN OUTCOMES AND MEASURES

Diagnostic accuracy measures, including sensitivity, specificity, negative predictive values (NPVs), positive predictive values (PPVs), and area under the receiver operating characteristic curve (AUROC), of machine learning models when predicting whether inpatient laboratory tests yield a normal result as defined by local laboratory reference ranges.

RESULTS

In the recent data sets (July 1, 2014, to June 30, 2017) from Stanford University Hospital (including 22 664 female inpatients with a mean [SD] age of 58.8 [19.0] years and 22 016 male inpatients with a mean [SD] age of 59.0 [18.1] years), among the top 20 highest-volume tests, 792 397 were repeats of orders within 24 hours, including tests that are physiologically unlikely to yield new information that quickly (eg, white blood cell differential, glycated hemoglobin, and serum albumin level). The best-performing machine learning models predicted normal results with an AUROC of 0.90 or greater for 12 stand-alone laboratory tests (eg, sodium AUROC, 0.92 [95% CI, 0.91-0.93]; sensitivity, 98%; specificity, 35%; PPV, 66%; NPV, 93%; lactate dehydrogenase AUROC, 0.93 [95% CI, 0.93-0.94]; sensitivity, 96%; specificity, 65%; PPV, 71%; NPV, 95%; and troponin I AUROC, 0.92 [95% CI, 0.91-0.93]; sensitivity, 88%; specificity, 79%; PPV, 67%; NPV, 93%) and 10 common laboratory test components (eg, hemoglobin AUROC, 0.94 [95% CI, 0.92-0.95]; sensitivity, 99%; specificity, 17%; PPV, 90%; NPV, 81%; creatinine AUROC, 0.96 [95% CI, 0.96-0.97]; sensitivity, 93%; specificity, 83%; PPV, 79%; NPV, 94%; and urea nitrogen AUROC, 0.95 [95% CI, 0.94, 0.96]; sensitivity, 87%; specificity, 89%; PPV, 77%; NPV 94%).

CONCLUSIONS AND RELEVANCE

The findings suggest that low-yield diagnostic testing is common and can be systematically identified through data-driven methods and patient context-aware predictions. Implementing machine learning models appear to be able to quantify the level of uncertainty and expected information gained from diagnostic tests explicitly, with the potential to encourage useful testing and discourage low-value testing that incurs direct costs and indirect harms.

Thrombophilia testing in the inpatient setting: impact of an educational intervention

Background

Thrombophilia testing is frequently ordered in the inpatient setting despite its limited impact on clinical decision-making and unreliable results in the setting of acute thrombosis or ongoing anticoagulation. We sought to determine the effect of an educational intervention in reducing inappropriate thrombophilia testing for hospitalized patients.

Methods

During the 2014 academic year, we implemented an educational intervention with a phase implementation design for Internal Medicine interns at Stanford University Hospital. The educational session covering epidemiology, appropriate thrombophilia evaluation and clinical rationale behind these recommendations. Their ordering behavior was compared with a contemporaneous control (non-medicine and private services) and a historical control (interns from prior academic year). From the analyzed data, we determined the proportion of inappropriate thrombophilia testing of each group. Logistic generalized estimating equations were used to estimate odds ratios for inappropriate thrombophilia testing associated with the intervention.

Results

Of 2151 orders included, 934 were deemed inappropriate (43.4%). The two intervention groups placed 147 orders.

A pooled analysis of ordering practices by intervention groups revealed a trend toward reduction of inappropriate ordering (p = 0.053). By the end of the study, the intervention groups had significantly lower rates of inappropriate testing compared to historical or contemporaneous controls.

Conclusion

A brief educational intervention was associated with a trend toward reduction in inappropriate thrombophilia testing. These findings suggest that focused education on thrombophilia testing can positively impact inpatient ordering practices.

Electronic supplementary material

The online version of this article (10.1186/s12911-019-0889-6) contains supplementary material, which is available to authorized users.

Internal medicine trainees' knowledge and confidence in using the American Society of Hematology Choosing Wisely guidelines in hemostasis, thrombosis, and non-malignant hematology

Background

Several specialty societies participate in the Choosing Wisely (CW) campaign in an attempt to reduce waste in health care spending. We surveyed internal medicine (IM) residents with an objective of classifying knowledge of and confidence in using the American Society of Hematology (ASH) CW principles in hemostasis, thrombosis, and non-malignant hematology.

Methods

Multi-institutional study of IM residents at 5 academic training programs in the United States. A 10-question, case-based multiple choice test, with each question accompanied by a 5-point Likert-scale confidence assessment, was distributed electronically. Responses were summarized with frequencies and percentages or medians and ranges, as appropriate. Two sample t-tests or Wilcoxon rank-sum tests were used to compare confidence and knowledge scores.

Results

Of 892 IM residents, 174 (19.5%) responded to all questions. Overall, residents answered a median of 7 of 10 questions correctly (range 2–10) and median resident confidence in their responses was 3.1 (on a 5-point scale). Correct responses were significantly associated with higher confidence for all but one question. Having a hematology rotation experience was significantly associated with more correct responses and with higher confidence (p = 0.001 and p<0.001, respectively).

Conclusions

IM residents at several academic hospitals have variable knowledge of ASH-CW guidelines in thrombosis and hemostasis/non-malignant hematology. Residents who have done hematology rotations, particularly a hematology consult rotation, were more likely to answer questions correctly and to be more confident that their answers were correct. Adequate clinical exposure and training in cost-effective care is essential to train clinicians who are cost-conscious in any specialty.