The value of personal professional monitoring performance data and open disclosure policies in anaesthetic practice: a case report

Evaluation of a continuous monitoring and feedback initiative to improve quality of anaesthetic care: a mixed-methods quasi-experimental study

Background

This study evaluated the impact of a continuous quality monitoring and feedback initiative in anaesthesia.

Objectives

To conduct a quasi-experimental evaluation of the feedback initiative and its effect on quality of anaesthetic care and perioperative efficiency. To understand the longitudinal effects of passive and active feedback and investigate the mechanisms and interactions underpinning those effects.

Design

Mixed-methods evaluation with analysis and synthesis of data from longitudinal qualitative interviews, longitudinal evaluative surveys and an interrupted time series study.

Intervention

Continuous measurement of a range of anaesthetic quality indicators was undertaken in a London teaching hospital alongside monthly personal feedback from case summary data to a cohort of anaesthetists, with follow-up roll-out to the whole NHS trust. Basic feedback consisted of the provision of passive monthly personalised feedback reports containing summary case data. In the enhanced phase, data feedback consisted of more sophisticated statistical breakdown of data, comparative and longitudinal views, and was paired with an active programme of dissemination and professional engagement.

Methods

Baseline data collection began in March 2010. Implementation of basic feedback took place in October 2010, followed by implementation of the enhanced feedback protocol in July 2012. Weekly aggregated quality indicator data, coupled with surgical site infection and mortality rates, was modelled using interrupted time series analyses. The study anaesthetist cohort comprised 50,235 cases, performed by 44 anaesthetists over the course of the study, with 22,670 cases performed at the primary site. Anaesthetist responses to the surveys were collected pre and post implementation of feedback at all three sites in parallel with qualitative investigation. Seventy anaesthetists completed the survey at one or more time points and 35 health-care professionals, including 24 anaesthetists, were interviewed across two time points.

Results

Results from the time series analysis of longitudinal variation in perioperative indicators did not support the hypothesis that implementation of basic feedback improved quality of anaesthetic care. The implementation of enhanced feedback was found to have a significant positive impact on two postoperative pain measures, nurse-recorded freedom from nausea, mean patient temperature on arrival in recovery and Quality of Recovery Scale scores. Analysis of survey data demonstrated that anaesthetists value perceived credibility of data and local relevance of quality indicators above other criteria when assessing utility of feedback. A significant improvement in the perceived value of quality indicators, feedback, data use and overall effectiveness was observed between baseline and implementation of feedback at the primary site, a finding replicated at the two secondary sites. Findings from the qualitative research elucidated processes of interaction between context, intervention and user, demonstrating a positive response by clinicians to this type of initiative and willingness to interact with a sustained and comprehensive feedback protocol to understand variations in care.

Conclusions

The results support the potential of quality monitoring and feedback interventions as quality improvement mechanisms and provide insight into the positive response of clinicians to this type of initiative, including documentation of the experiences of anaesthetists that participated as users and codesigners of the feedback. Future work in this area might usefully investigate how this type of intervention may be transferred to other areas of clinical practice and further explore interactions between local context and the successful implementation of quality monitoring and feedback systems.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Using quality indicators in anaesthesia: feeding back data to improve care

After recent UK policy developments, considerable attention has been focused upon how clinical specialties measure and report on the quality of care delivered to patients. Defining the right indicators alone is insufficient to close the feedback loop. This narrative review aims to describe and synthesize a diverse body of research relevant to the question of how information from quality indicators can be fed back and used effectively to improve care. Anaesthesia poses certain challenges in the identification of valid outcome indicators sensitive to variations in anaesthetic care. Metrics collected during the immediate post-anaesthetic recovery period, such as patient temperature, patient-reported quality of recovery, and pain and nausea, provide potentially useful information for the anaesthetist, yet this information is not routinely fed back. Reviews of the effects of feeding back performance data to healthcare providers suggest that this may result in small to moderate positive effects upon outcomes and professional practice, with stronger effects where feedback is integrated within a broader quality improvement strategy. The dominant model for use of data within quality improvement is based upon the industrial process control approach, in which care processes are monitored continuously for process changes which are rapidly detectable for corrective action. From this review and experience of implementing these principles in practice, effective feedback from quality indicators is timely, credible, confidential, tailored to the recipient, and continuous. Considerable further work is needed to understand how information from quality indicators can be fed back in an effective way to clinicians and clinical units, in order to support revalidation and continuous improvement.

Clinical factors associated with the non-utilization of an anaesthesia incident reporting system

BACKGROUND

Incident reporting is a widely recommended method to measure undesirable events in anaesthesia. Under-utilization is a major weakness of voluntary incident reporting systems. Little is known about factors influencing reporting practices, particularly the clinical environment, anaesthesia team composition, severity of the incident, and perceived risk of litigation. The purpose of this study was to assess each of these, using an existing anaesthesia database.

METHODS

We performed a retrospective cohort study and analysed 46 207 surgical patients. We used multivariate analysis to identify factors associated with the non-utilization of the reporting system.

RESULTS

We found that in 7022 (15.1%) of the procedures performed, the incident reporting system was not used. Factors associated with the non-use of the system were regional anaesthesia/local anaesthesia, odds ratio (OR) 1.64 [95% confidence interval (CI) 1.03-2.62], emergency procedures OR 1.15 (95% CI: 1.05-1.27), and a consultant anaesthetist working without a trainee, OR 1.71 (95% CI: 1.03-2.82). In contrast, factors such as longer duration of surgery, OR 0.85 (95% CI: 0.76-0.94), the presence of a senior anaesthesia trainee, OR 0.86 (95% CI: 0.81-0.92), and the occurrence of severe complications with a high risk of litigation (i.e. death, nerve injuries) were less associated with a non-use of the reporting system, OR 0.65 (95% CI: 0.44-0.97). Team composition and time of day had no measurable impact on reporting practices.

CONCLUSIONS

Clinical factors play a significant role in the utilization of an anaesthesia incident reporting system and more particularly, severity of complications and higher liability risks which appear more as incentives than barriers to incident reporting.

Monitoring the rate of re-exploration for excessive bleeding after cardiac surgery in adults

BACKGROUND

The monitoring of adverse events in clinical care can be an important part of quality assurance. There is little evidence on the monitoring of re-exploration after cardiac surgery.

OBJECTIVE

To apply statistical monitoring techniques to the rate of re-exploration for excessive bleeding in adult patients undergoing cardiac surgery procedures using cardiopulmonary bypass at Geelong Hospital, Victoria, Australia, between 1997 and 2003.

METHODS

Shewhart charts, moving average plots and cumulative sum (CUSUM) charts were used to demonstrate changes in the rate of re-exploration over time.

RESULTS

A CUSUM chart was used retrospectively at a time of perceived deteriorating clinical outcomes in patients of the cardiac surgery service. At this time, an intervention aimed at reducing the re-exploration rate was performed, and subsequent CUSUM charts indicated an improvement in this rate. The CUSUM chart has become an important part of the quality feedback of clinical care outcomes within the Anaesthesia & Pain Management unit of Geelong Hospital.

CONCLUSION

Statistical monitoring techniques for quality assurance can identify important changes in clinical performance, and their adoption by clinicians is recommended.

Voluntary incident reporting by anaesthetic trainees in an Australian hospital

OBJECTIVE

To assess the reporting of critical incidents by anaesthetic trainees using personal digital assistants. The project also identified the reporting of 'near miss' incidents by anaesthetic trainees.

DESIGN

Comparison of electronic incident reporting with retrospective case note review of cases in which no incident was reported.

SETTING

A 400-bed university teaching hospital in Victoria.

PARTICIPANTS

Fourteen accredited Australian and New Zealand College of Anaesthetists (ANZCA) registrars and their training supervisors.

INTERVENTIONS

Registrars and supervisors underwent initial training for 1 hour and were provided with ongoing support. The cases and incidents reported to the database using the portable digital assistants were analysed.

MAIN OUTCOME MEASURES

These were the total number of anaesthetics reported to the database; the number of incidents reported to the database; the outcome severity of incidents reported; and the number of incidents detected in the case note review that were not reported to the database.

RESULTS

An incident was reported for 156 (3.5%) of 4441 anaesthetic procedures reported to the database. Of these incidents, 72 (46.2%) were 'near misses'. One incident was identified in a review of 208 case notes, which had no incidents reported electronically, and was not reported to the database electronically. This gives a reporting rate of 99.52% [95% confidence interval (CI) 96.9-100%].

CONCLUSIONS

ANZCA trainees in routine anaesthetic practice can reliably use mobile computing technology to report critical incidents and 'near miss' incident data.

New technology to enable personal monitoring and incident reporting can transform professional culture: the potential to favourably impact the future of health care

There have been recent exposures of poor health care performance in many countries with western health care systems. The poor performance has either related to poor or criminal practices routinely going undetected or to organizational indifference or hostility to staff raising concerns about perceived poor standards of care. The demonstration that routine performance data monitoring would have detected and prevented many of the deaths attributed to poor surgical standards in the Bristol Royal Infirmary paediatric cardiac surgery scandal and criminal behaviour in the Harold Shipman scandal has highlighted the need for routine data collection to demonstrate to both health care administrators and patients that minimum standards of clinical practice are being achieved. The recent proposal that surgical report cards represent an important minimum ethical standard for health care consent will force the medical profession to engage in the debate surrounding routine data collection for performance monitoring and other purposes. This article considers the cultural background to data collection in the medical profession and the cost implications of failing to improve data collection in the areas of performance monitoring and incident reporting. A potential solution developed by the Geelong hospital group and in use in Australia is proposed as a novel, technologically appropriate and working example of practical data collection. This model is endorsed by the professional specialties and supported by modern regulatory theory. The individual, local and system wide benefits of such personal professional data collection are outlined and the necessary prerequisites are detailed.

Practical virtue ethics: healthcare whistleblowing and portable digital technology

Medical school curricula and postgraduate education programmes expend considerable resources teaching medical ethics. Simultaneously, whistleblowers' agitation continues, at great personal cost, to prompt major intrainstitutional and public inquiries that reveal problems with the application of medical ethics at particular clinical "coalfaces". Virtue ethics, emphasising techniques promoting an agent's character and instructing their conscience, has become a significant mode of discourse in modern medical ethics. Healthcare whistleblowers, whose complaints are reasonable, made in good faith, in the public interest, and not vexatious, we argue, are practising those obligations of professional conscience foundational to virtue based medical ethics. Yet, little extant virtue ethics scholarship seriously considers the theoretical foundations of healthcare whistleblowing. The authors examine whether healthcare whistleblowing should be considered central to any medical ethics emphasising professional virtues and conscience. They consider possible causes for the paucity of professional or academic interest in this area and examine the counterinfluence of a continuing historical tradition of guild mentality professionalism that routinely places relationships with colleagues ahead of patient safety.Finally, it is proposed that a virtue based ethos of medical professionalism, exhibiting transparency and sincerity with regard to achieving uniform quality and safety of health care, may be facilitated by introducing a technological imperative using portable computing devices. Their use by trainees, focused on ethical competence, provides the practical face of virtue ethics in medical education and practice. Indeed, it assists in transforming the professional conscience of whistleblowing into a practical, virtue based culture of self reporting and personal development.

Supporting whistleblowers in academic medicine: training and respecting the courage of professional conscience

Conflicts between the ethical values of an organisation and the ethical values of the employees of that organisation can often lead to conflict. When the ethical values of the employee are considerably higher than those of the organisation the potential for catastrophic results is enormous. In recent years several high profile cases have exposed organisations with ethical weaknesses. Academic medical institutions have exhibited such weaknesses and when exposed their employees have almost invariably been vindicated by objective inquiry. The mechanisms that work to produce such low ethical standards in what should be exemplary organisations are well documented and have been highlighted recently. The contribution of elements of medical training in eroding ethical standards of medical students have also been emphasised recently and strategies proposed to reduce or reverse this process. The ability to rapidly change the ethical and professional culture of graduate medical trainees may help to deal with some of the perceived problems of declining ethical standards in academic medicine.