Efficacy of an EMS quality improvement program in improving documentation and performance

Types and effects of feedback for emergency ambulance staff: a systematic mixed studies review and meta-analysis

BACKGROUND

Extensive research has been conducted into the effects of feedback interventions within many areas of healthcare, but prehospital emergency care has been relatively neglected. Exploratory work suggests that enhancing feedback and follow-up to emergency medical service (EMS) staff might provide staff with closure and improve clinical performance. Our aim was to summarise the literature on the types of feedback received by EMS professionals and its effects on the quality and safety of patient care, staff well-being and professional development.

METHODS

A systematic review and meta-analysis, including primary research studies of any method published in peer-reviewed journals. Studies were included if they contained information on systematic feedback to emergency ambulance staff regarding their performance. Databases searched from inception were MEDLINE, Embase, AMED, PsycINFO, HMIC, CINAHL and Web of Science, with searches last updated on 2 August 2022. Study quality was appraised using the Mixed Methods Appraisal Tool. Data analysis followed a convergent integrated design involving simultaneous narrative synthesis and random effects multilevel meta-analyses.

RESULTS

The search strategy yielded 3183 articles, with 48 studies meeting inclusion criteria after title/abstract screening and full-text review. Interventions were categorised as audit and feedback (n=31), peer-to-peer feedback (n=3), postevent debriefing (n=2), incident-prompted feedback (n=1), patient outcome feedback (n=1) or a combination thereof (n=4). Feedback was found to have a moderate positive effect on quality of care and professional development with a pooled effect of d=0.50 (95% CI 0.34, 0.67). Feedback to EMS professionals had large effects in improving documentation (d=0.73 (0.00, 1.45)) and protocol adherence (d=0.68 (0.12, 1.24)), as well as small effects in enhancing cardiac arrest performance (d=0.46 (0.06, 0.86)), clinical decision-making (d=0.47 (0.23, 0.72)), ambulance times (d=0.43 (0.12, 0.74)) and survival rates (d=0.22 (0.11, 0.33)). The between-study heterogeneity variance was estimated at σ2=0.32 (95% CI 0.22, 0.50), with an I2 value of 99% (95% CI 98%, 99%), indicating substantial statistical heterogeneity.

CONCLUSION

This review demonstrated that the evidence base currently does not support a clear single point estimate of the pooled effect of feedback to EMS staff as a single intervention type due to study heterogeneity. Further research is needed to provide guidance and frameworks supporting better design and evaluation of feedback interventions within EMS.

PROSPERO REGISTRATION NUMBER

CRD42020162600.

The Feedback Form and Its Role in Improving the Quality of Trauma Care

Background: One of the tasks of a level I trauma center is quality improvement of level II and level III regional hospitals and emergency medical services by means of continuous education and learning processes. One of the tools for this, which provides constant monitoring of the quality of treatment, is feedback. The purpose of the study was to evaluate the effect of feedback on the quality of trauma care. Methods: Retrospective cohort study comprising two periods of time, 2012-2013 and 2017-2018. The study group included physicians and pre-hospital staff who treated patients prior to referral to the level I center. Upon arrival when the trauma teams identified issues requiring improvement, they were asked to fill in feedback forms. Data on patients treated in the trauma shock room for whom feedback forms were filled out were also extracted. Results: A total of 662 feedback forms were completed, showing a significant improvement (p ˂ 0.0001). The majority of the medical personnel who received the most negative feedback were the pre-hospital staff. A significant increase was revealed in the number of feedbacks with reference to mismanagement of backboard spinal fixation, of the pre-hospital staff, in 2012-2013 compared to 2017-2018 (p < 0.001). Improvement in reducing the time of treatment in the field was also revealed, from 15.2 ± 8.3 min in 2012-2013 to 13.4 ± 7.9 min in 2017-2018. Conclusion: The findings show that feedback improves the treatment of injured patients. Furthermore, constantly monitoring the quality of treatment provided by the trauma team is vital for improvement.

Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study

Background

Prehospitalization documentation is a challenging task and prone to loss of information, as paramedics operate under disruptive environments requiring their constant attention to the patients.

Objective

The aim of this study is to develop a mobile platform for hands-free prehospitalization documentation to assist first responders in operational medical environments by aggregating all existing solutions for noise resiliency and domain adaptation.

Methods

The platform was built to extract meaningful medical information from the real-time audio streaming at the point of injury and transmit complete documentation to a field hospital prior to patient arrival. To this end, the state-of-the-art automatic speech recognition (ASR) solutions with the following modular improvements were thoroughly explored: noise-resilient ASR, multi-style training, customized lexicon, and speech enhancement. The development of the platform was strictly guided by qualitative research and simulation-based evaluation to address the relevant challenges through progressive improvements at every process step of the end-to-end solution. The primary performance metrics included medical word error rate (WER) in machine-transcribed text output and an F1 score calculated by comparing the autogenerated documentation to manual documentation by physicians.

Results

The total number of 15,139 individual words necessary for completing the documentation were identified from all conversations that occurred during the physician-supervised simulation drills. The baseline model presented a suboptimal performance with a WER of 69.85% and an F1 score of 0.611. The noise-resilient ASR, multi-style training, and customized lexicon improved the overall performance; the finalized platform achieved a medical WER of 33.3% and an F1 score of 0.81 when compared to manual documentation. The speech enhancement degraded performance with medical WER increased from 33.3% to 46.33% and the corresponding F1 score decreased from 0.81 to 0.78. All changes in performance were statistically significant (P<.001).

Conclusions

This study presented a fully functional mobile platform for hands-free prehospitalization documentation in operational medical environments and lessons learned from its implementation.

Disparities in Feedback Provision to Emergency Medical Services Professionals

INTRODUCTION

Feedback to EMS professionals is a critical component for optimizing patient care and outcomes in the prehospital setting. There is a paucity of data concerning the feedback received by prehospital providers.

OBJECTIVES

The objective of this study was to describe the prevalence of feedback received by EMS professionals in the past 30 days including the types, sources, modes, and utility of feedback. The secondary objective was to identify factors associated with receiving any feedback and, specifically, feedback regarding medical care provided.

METHODS

This was a cross-sectional survey examining currently practicing nationally certified EMS patient care providers (EMT or higher) in non-military and non-tribal settings. Data were collected on provider characteristics along with feedback received. Descriptive statistics were calculated, and multivariable logistic regression models were constructed to assess the relationship between EMS provider characteristics and receiving feedback. A non-respondent survey was administered to assess for non-response bias.

RESULTS

Responses from 32,314 EMS providers were received (response rate = 10.4%) with 15,766 meeting inclusion criteria. In the 30 days preceding the survey, 69.4% (n = 10,924) of respondents received at least one type of feedback with 54.7% (n = 8,592) reporting receiving medical care feedback. Multivariable logistic regression modeling indicated that higher certification level, fewer years of experience in EMS, working for a hospital-based agency, air medical service, and higher weekly call volumes were significantly associated with increased odds of having received at least one type of feedback, and specifically medical care feedback. Additionally, providing primarily medical/convalescent transport and more years of EMS experience were significantly associated with decreased odds of receiving feedback.

CONCLUSION

Feedback to EMS providers is critical to improving prehospital care. In this study, nearly a third of providers did not receive any feedback in a 30-day period, and nearly half reported not receiving medical care feedback. Disparities in the frequency of feedback exist between different provider levels and service settings, while reported feedback decreased with years of experience in the profession. Future work is needed to assess the content of feedback and role in improving patient care and safety.

Impact of implementing an EMR on physical exam documentation by ambulance personnel

OBJECTIVES

Georgetown University has a student run Emergency Medical Services (EMS) organization with over 100 emergency medical technicians (EMTs). We set out to determine whether implementing an electronic patient care report (ePCR) system was associated with improved physical exam documentation.

METHODS

This study evaluated documentation of the physical exam on prehospital patient care reports (PCRs). An ePCR system was implemented. ePCR documentation was compared to that of the previously used paper PCRs. This study looked retrospectively at 154 PCRs. 77 were hand written PCRs from before the electronic system. The PCRs involved chief complaints that were primarily respiratory, neurologic, or both. 77 ePCRs of matching chief complaint categories were used for comparison. Each chart was reviewed for completion of certain physical exam findings. The mean percentage of documented components from the ePCRs was compared to that of the hand written PCRs. The null hypothesis was that the absolute increase in the mean was not more than 20 percent. The two exclusion criteria were PCRs completed by study investigators after the design of the project and partially or completely missing PCRs.

RESULTS

The absolute increase in mean physical exam component documentation was 36% (95% CI = 29-43%). A weighted kappa of 0.894 showed very good agreement between chart reviewers.

CONCLUSIONS

This study rejected the null hypothesis that the ePCR system was associated with a mean increase of no more than 20%. It observed increase in physical exam documentation. Limitations of this study included the inability to determine whether documentation of physical exam findings reflected performance of the physical exam, and what components of the ePCR system bundle were responsible for the increase in physical exam component documentation.

Lack of emergency medical services documentation is associated with poor patient outcomes: a validation of audit filters for prehospital trauma care

BACKGROUND

Our previous Delphi study identified several audit filters considered sensitive to deviations in prehospital trauma care and potentially useful in conducting performance improvement, a process currently recommended by the American College of Surgeons Committee on Trauma. This study validates 2 of those proposed audit filters.

STUDY DESIGN

We studied 4,744 trauma patients using the electronic records of the Central Region Trauma registry and Emergency Medical Services (EMS) patient logs for the period January 1, 2002, to December 31, 2004. We studied whether requests by on-scene Basic Life Support (BLS) for Advanced Life Support (ALS) assistance or failure by EMS personnel to record basic patient physiology at the scene was associated with increased in-hospital mortality. We performed multivariate analyses, including a propensity score quintile approach, adjusting for differences in case mix and clustering by hospital.

RESULTS

Overall mortality was 6.1%. A total of 28.2% (n = 1,337) of EMS records were missing patient scene physiologic data. Multivariate analysis revealed that patients missing 1 or more measures of patient physiology at the scene had increased risk of death (adjusted odds ratio = 2.15; 95% CI, 1.13 to 4.10). In 17.4% (n = 402) of cases BLS requested ALS assistance. Patients for whom BLS requested ALS had a similar risk of death as patients for whom ALS was initially dispatched (odds ratio = 1.04; 95% CI, 0.51 to 2.15).

CONCLUSIONS

Failure of EMS to document basic measures of scene physiology is associated with increased mortality. This deviation in care can serve as a sensitive audit filter for performance improvement. The need by BLS for ALS assistance was not associated with increased mortality.

Development and implementation of an education and credentialing programme to provide safe paediatric procedural sedation in emergency departments

OBJECTIVE

In the conduct of paediatric procedural sedation (PPS) within the ED the combination of powerful drugs, variable competency levels and high staff turnover carry the potential for sedation-associated adverse events. Yet, currently, there is no set programme for education and accreditation of ED staff in PPS. We set out to develop such a programme.

METHODS

We outline the development process of a comprehensive multidisciplinary PPS programme and present its key educational elements (sedation manual, lecture, treatment order form and checklist, parent information handout) and credentialing through multiple-choice questions and competency assessments. We describe issues associated with the implementation of the programme at a metropolitan mixed ED and the ED of a major tertiary paediatric centre.

RESULTS

Since its inception a total of 294 emergency staff have either completed or have partially completed the programme. Staff feedback showed that the majority of staff scored the elements of the programme as very good to excellent, and felt that their sedation skills had improved and their practice was safer. The development and implementation of the PPS programme raised many issues and posed a number of challenges. We describe the strategies we used to overcome such challenges and barriers.

CONCLUSION

We present the development and implementation of a comprehensive PPS programme for emergency staff. As a result of the multicentre development process involving a community and a tertiary paediatric ED the programme will likely have broad applicability in different types of ED caring for children.

Complaints against an EMS system

Complaints against Emergency Medical Services (EMS) agencies represent a concerning and potentially time-consuming problem for all involved in the delivery of prehospital emergency medical care. The objective of this study was to identify the source of complaints against an EMS system to help focus quality and performance improvement and customer service efforts. We conducted a retrospective review of complaints filed against a busy urban EMS agency over a 6-year period. All complaints were included, totaled by season and by year, and categorized by originator and nature of the complaint. A total of 286 complaints were registered during the 6-year period, with an average of 48 per year and 9.3 per 10,000 responses. The most common originators of complaints were patients (53%) followed by medical personnel (19%) and family members or friends (12%). Rude behavior accounted for 23% of the complaints registered, followed by technical skills (20%), transport problems (18%), and loss of belongings (13%). The identification of areas of dissatisfaction will allow focused quality and performance improvement programs directed at customer service and risk management.

A continuous quality improvement approach to IL-372 documentation compliance in an academic emergency department, and its impact on dictation costs, billing practices, and average patient length of stay

OBJECTIVE

To determine whether continuous quality improvement (CQI) methodology could improve and maintain IL-372 documentation compliance in an academic emergency department (ED). The impact on transcription costs, billing practices, and average patient length of stay was also analyzed.

METHODS

Baseline IL-372 compliance data were collected and shared with staff during a multidisciplinary educational session. Faculty dictation became mandatory. Pocket-sized dictation templates were provided. A Documentation Improvement Committee monitored outcomes. Each month of the study period, a compliance officer reviewed approximately 100 records. The following indicators were monitored: IL-372 compliance rates, dictation rates, transcription costs, down-coding rates, percentage of billable records, and average patient length of stay. Individualized results were provided to faculty.

RESULTS

During the ten-month study period, compliance rates increased from 60% to 100% (p-trend < 0.001), while dictation rates increased from 69% to 100% (p < 0.001). Rates of down-coding adjustments improved from 54% to 2% (p-trend < 0.001). The percentage of billable records increased from 65% to 100% (p-trend < 0.001). Transcription costs increased a modest 16%. The average patient length of stay remained unchanged.

CONCLUSION

The application of CQI methodology, combined with the availability of dictation, resulted in sustained improvement in IL-372 compliance. This was associated with a parallel increase in dictation rates, although concurrent transcription costs increased only modestly. The percentage of billable records increased, while the number of charts requiring down-coding decreased, both beneficial outcomes. Average length of stay was not adversely impacted by this added documentation requirement.