Anesthesia myTIPreport: A Web-Based Tool for Real-Time Evaluation of Accreditation Council for Graduate Medical Education’s Milestone Competencies and Clinical Feedback to Residents

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents

OBJECTIVE

Feedback is critical to the growth of learners. However, feedback quality can be variable in practice. Most feedback tools are generic, with few targeting emergency medicine. We created a feedback tool designed for emergency medicine residents, and this study aimed to evaluate the effectiveness of this tool.

METHODS

This was a single-center, prospective cohort study comparing feedback quality before and after introducing a novel feedback tool. Residents and faculty completed a survey after each shift assessing feedback quality, feedback time, and the number of feedback episodes. Feedback quality was assessed using a composite score from seven questions, which were each scored 1 to 5 points (minimum total score, 7 points; maximum, 35 points). Preintervention and postintervention data were analyzed using a mixed-effects model that took into account the correlation of random effects between study participants.

RESULTS

Residents completed 182 surveys and faculty members completed 158 surveys. The use of the tool was associated with improved consistency in the summative score of effective feedback attributes as assessed by residents (P=0.040) but not by faculty (P=0.259). However, most of the individual scores for attributes of good feedback did not reach statistical significance. With the tool, residents perceived that faculty spent more time providing feedback (P=0.040) and that the delivery of feedback was more ongoing throughout the shift (P=0.020). Faculty felt that the tool allowed for more ongoing feedback (P=0.002), with no perceived increase in the time spent delivering feedback (P=0.833).

CONCLUSION

The use of a dedicated tool may help educators provide more meaningful and frequent feedback without impacting the perceived required time needed to provide feedback.

Feasibility and Utility of Mobile Applications for the Evaluation of Urology Residents' Surgical Competence

OBJECTIVE

To provide real-time assessment and feedback on the competency of urology residents' surgical skill via mobile applications and examine their feasibility and utility.

MATERIALS AND METHODS

Two mobile application-based systems (SIMPL and myTIPreport) were sequentially implemented for the case-by-case assessment of residents' performance of surgical skills at a single institution. Data was collected regarding residents' perception of their feedback pre- and post-implementation of the applications. Faculty were surveyed after their implementation to determine their feasibility and utility.

RESULTS

297 individual evaluations were completed with SIMPL and 822 with myTIPreport over four and eleven months respectively. Post-implementation, residents showed significantly improved perceptions regarding the quantity and personalization of surgical skill feedback (P = .043 and .005 respectively). A majority (75%) of the faculty found the mobile applications feasible to use, an improvement compared to prior methods of resident evaluation, and would recommend continued use.

CONCLUSION

This study represents the first documented use of real-time surgical competency assessment in urology. The use of mobile applications to evaluate urology residents' surgical competency in clinical practice is both feasible and useful. Their use may allow for more individualized surgical skill teaching during training and for the verification of the surgical skills necessary to practice autonomously.

A Guide to Performance Evaluation for the Intensivist: Ongoing Professional Practice Evaluation and Focused Professional Practice Evaluation in the ICU

OBJECTIVES

In 2008, The Joint Commission implemented a new standard mandating a detailed evaluation of a provider's performance. The Ongoing Professional Practice Evaluation was designed to provide ongoing performance evaluation as opposed to periodic evaluation. The Focused Professional Practice Evaluation was designed to evaluate the performance of providers new to the medical staff or providers who are requesting new privileges. To date, we are unable to find critical care specific literature on the implementation of Ongoing Professional Practice Evaluation/Focused Professional Practice Evaluation. The purpose of this concise definitive review is to familiarize the reader with The Joint Commission standards and their application to Ongoing Professional Practice Evaluation/Focused Professional Practice Evaluation design and implementation, literature review in the noncritical care setting, and future process optimization and automation.

DATA SOURCES

Studies were identified through MEDLINE search using a variety of search phrases related to Ongoing Professional Practice Evaluation, Focused Professional Practice Evaluation, critical care medicine, healthcare quality, and The Joint Commission. Additional articles were identified through a review of the reference lists of identified articles.

STUDY SELECTION

Original articles, review articles, and systematic reviews were considered.

DATA EXTRACTION

Manuscripts were selected for inclusion based on expert opinion of well-designed or key studies and review articles.

DATA SYNTHESIS

There is limited data for the process of Ongoing Professional Practice Evaluation and Focused Professional Practice Evaluation implementation in critical care medicine. Key recommendations exist from The Joint Commission but leave it up to healthcare institutions to realize these. The process and metrics can be tailored to specific institutions and departments.

CONCLUSIONS

Currently, there is no standard process to develop Ongoing Professional Practice Evaluation and Focused Professional Practice Evaluation processes in critical care medicine. Departments and institutions can tailor metrics and processes but it might be useful to standardize some metrics to assure the overall quality of care. In the future utilization of newer technologies like applications might make this process less time-intensive.