A New Teaching Tool for Peer Review of Charting and Care in the Emergency Department

Poor documentation, incomplete medical decision-making, missing progress notes, and inappropriate care play a major role in medical malpractice cases. We introduced a new quality improvement (QI) process focused on evaluating and improving documentation and clinical care. We hypothesized that a modified, simplified QI scoring rubric would demonstrate inter-rater reliability among attending physicians and provide a useful new standardized tool for both QI departmental review and peer review. We modified a previously developed rubric template that demonstrated high inter-rater reliability for a more streamlined, simpler, and more generalized application. We developed a new system using three discrete templated sections with choices limited to five options. Eight experienced attending physicians evaluated the same 10 charts using our scoring rubrics. Consistency among raters was assessed using the Shrout-Fleiss relative: fixed set mean kappa scores. Our statistical analysis found excellent consistency among our experienced raters for both the documentation (κ = 0.91) and clinical care (κ = 0.84) scoring tools. We conclude that a modified, simplified QI scoring rubric demonstrates inter-rater reliability among experienced attending physicians. We believe this tool can be used as a standardized tool for a departmental review process by experienced quality leaders as well as by faculty to provide peer review while improving their own charting prowess. We further used this tool for peer review by having the attending staff participate in reviewing a specified number of charts using our modified template with explicit criteria so they could provide feedback as well, while gaining a better understanding of the elements of a "good" chart and of opportunities for improved care and resource utilization. By using this tool, we were able to provide more than 50 attendings summative feedback on their charting by a group of their peers that was both numerical and descriptive.

A Comprehensive Departmental Care Review Model: Requirements, Structure, and Flow

BACKGROUND

The Institute of Medicine, the National Patient Safety Foundation, and The Joint Commission have advocated for increased systematic care review to inform future quality improvement. Developing a system to efficiently gather meaningful feedback, review care, and identify areas for improvement can take years to construct. Yet, these systems are vital to reducing future medical error.

CONTEXT, HISTORY, AND DEVELOPMENT

In this article, the authors present a refined intradepartmental system of retrospective care review. The team created and iteratively improved this model for more than 10 years. Herein, key aspects and benefits of the system are described.

CARE REVIEW SYSTEM

A successful care review system should include a broad catchment for cases to review, direct input from multidisciplinary staff involved in each case, a standardized evaluation and feedback process, a system to translate identified gaps into practice improvement, and development of a psychologically safe space for discussions to occur. Resources required to build this system include a quality specialist, a panel of physician and nurse reviewers, and administrative assistance. Blinding cases and electronic blinded polling technology can enhance participation and reduce bias in case assessment.

CONCLUSION

The authors believe that this process for care review can help hospital systems of varying resource levels produce high-quality case review and thereby activate practice improvement to prevent downstream medical errors.

Comparison of Two Simulated Procedural Assessment Formats in Attending Emergency Physicians

Background

Emergency physicians must be proficient at inserting central venous catheters and performing lumbar punctures to provide life-saving therapies to critically ill patients. An assessment of procedural skill is rarely performed after an emergency physician has completed residency. Current board certification exams for emergency medicine focus only on verbal descriptions of procedures to assess skill. We compared two methods of procedural skill assessment, simulated task trainer and verbal description, to assess the range of skill in central venous catheter insertion and lumbar punctures of emergency attending physicians at a large, urban, academic tertiary care institution.

Methodology

This is a prospective cohort study of simulated internal jugular central venous catheter insertion and lumbar puncture skill by emergency attending physicians on a task trainer versus verbal description. A total of 17 attending emergency medicine physicians consented to participate in the study during a yearly procedural skills session. For each subject, two expert raters used previously published checklists to assess procedural skill and give a global rating score.

Results

More checklist items were performed correctly on the task trainer than on verbal assessment for central line (task trainer = 78.4% ± 8.32% and verbal = 68.26% ± 8.9%) and lumbar puncture (task trainer = 85.57% ± 7.6% and verbal = 73.53%4 ± 10.34%) procedures, both with significant differences (p < 0.001). Of the participants, 82% strongly preferred the task trainer format to the verbal description assessment format.

Conclusions

The higher scores on the simulated format compared to the current verbal format imply that a shift towards simulated procedural assessment techniques may benefit examinees. More work is needed to determine if objective checklist scores for practicing attending emergency physicians correlate with subjective expert assessments of their procedural skills.

Ongoing Professional Practice Evaluation for Emergency Medicine Physicians in a Large Health Care System

OBJECTIVE

Ongoing professional practice evaluation (OPPE) is designed to identify professional practice trends that affect quality and safety of practicing physicians. Focused professional practice evaluation (FPPE) is employed when physician nonconformance is identified. The goal of this novel OPPE initiative was threefold: (1) meet The Joint Commission's accreditation standards, (2) assess documentation for compliance and risk, and (3) maintain procedural competency to provide optimal patient care.

METHODS

A quality assurance project with OPPE program development was initiated in 18 emergency departments across a large health care system. First, a monthly comprehensive peer review meeting assessed cases across the system for medicolegal risk. Physicians with reasonable practice concerns were identified and referred to FPPE. Second, a standardized OPPE chart review was performed biannually by a quality assurance committee assessing all physician charts for clinical care, medicolegal risk, and quality. Last, completion of a procedure lab every three years was required to maintain competency.

OUTCOMES

For systemwide peer review in 2019, 47 cases were referred and 12.8% had quality concerns. For standardized OPPE chart review, 221 physicians were reviewed on 1,219 charts on the following metrics: insufficient medical decision making, diagnoses not medical/legally supported, and charts with red flags. Nine physicians (4.1%) and 17 charts (1.4%) were deficient in all three measures, and 8 physicians (3.6%) had deficiencies in ≥ 50% of their charts. For procedure lab competency, 19.0% of physicians completed the lab in 2019 with no quality concerns.

CONCLUSION

A structured OPPE algorithm can aid large health care systems in identifying deviations from practice standards for which additional FPPE can be beneficial.

Quality Assurance in the Emergency Department

Quality assurance (QA) of care in the emergency department encompasses activities ensuring that the care provided meets applicable standards. Health care delivery is complex and many factors affect quality of care. Thus, quantification of health care quality is challenging, especially with regard to attribution of outcomes to various factors contributing to such care. A critical component of the process of QA is determination of quality health care and the concept of (unjustified) deviation from the reference applicable standard of care.