Statistical process control for hospitals: methodology, user education, and challenges

Control Charts Usage for Monitoring Performance in Surgery: A Systematic Review

OBJECTIVES

The control chart is a graphical tool for data interpretation that detects aberrant variations in specific metrics, ideally leading to the identification of special causes that can be resolved. A clear assessment of control chart utilization and its potential impact in surgery is required to justify recommendations for its dissemination. This review aims to describe how performance monitoring using control charts was used over time in surgery.

METHODS

A systematic search of PubMed regarding statistical process control in surgery from its inception until December 2019 was performed using Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. Information extracted from selected publications included study aim and population setting, monitored indicators, control charts methodological parameters, and implementation strategy.

RESULTS

One hundred thirteen studies met the selection criteria with a median of 1916 monitored patients. Overall, 57.5% of studies focused on control chart methodology, 24.8% aimed at evaluating performance changes using control charts retrospectively, and 17.7% implemented control charts for continuous quality improvement prospectively. Although there was a great diversity of used indicators and charting tools, the evaluation of patient safety (72.6%) or efficiency (15.9%) metrics based on Shewhart control chart (33.6%) or cumulative sum chart (54.9%) were common. To foster control charts implementation, 14 studies promoted their periodic review, but only three assessed their impact on patient outcomes.

CONCLUSIONS

The scientific literature supports the feasibility and utility of control chart to improve patient safety in multiple surgical settings. Additional studies are necessary to reveal the optimal manner in which to implement this affordable tool in surgical practice.

Impact of providing patient information leaflets prior to hospital discharge to patients with acute kidney injury: a quality improvement project

Acute kidney injury (AKI) is a common health issue. It is a sudden episode of kidney failure that is almost entirely associated with episodes of acute illness. AKI is common with as many as 20% of patients arriving at hospital having an AKI, with up to 15% of patients developing AKI in a postoperative period. Patients who have an episode of AKI are more likely to have a further episode of AKI and require readmission to hospital. This project aimed to provide patients with AKI education for self-care and management, with the hope of reducing AKI readmissions.

Using quality improvement methodology, the AKI patient discharge and readmission pathway was reviewed, and information about AKI was given to patients. This was in the form of verbal information and a patient information leaflet. This information was provided on discharge from acute care.

Baseline data were collected that showed more than 80% of patients reported that they were not given information about AKI prior to their discharge from hospital. Due to higher readmission rates, the focus of this improvement project was on acute medical wards. Following implementation, there was a sustained reduction in AKI patient readmission rates. This reduction led to a significant reduction of inpatient bed days and a shorter length of stay for those patients who were readmitted. Quality improvement methods have facilitated a successful reduction in acute AKI readmission to hospital.

Productivity evaluation of radiologists interpreting computed tomography scans using statistical process control charts

Radiology service managers search for efficient ways to monitor productivity and improve capacity. One way to assess radiologists' productivity is by measuring their time to complete reports. Radiology reporting times (RRTs) may be monitored using statistical tools, such as process control charts (CCs). This study was carried out in the radiology sector of a University-based general hospital with 850 inward beds. Productivity was monitored using CCs. The selected control variable was RRTs, and process capability was calculated using C_p and C_pk indices. Only chest computed tomography scans were analyzed, totaling 2862 exams over a 6-month period. Our objective was to develop a simple tool to monitor radiologist performance, as given by RRT, over time. For that, we constructed CCs using data from 10 radiologists to monitor the stability of their RRTs. Only 3 radiologists presented mean times below the group average; 6 displayed a trend in RRTs that characterized performance improvement, while 4 displayed the opposite trend. Capability measures for the group indicated a process that is not capable. We demonstrate that CCs may be a useful tool for monitoring radiologists' performances in CT scans interpretation. Results demonstrated that in the individual CT reporting process, common cause variability is the type of variability most frequently observed, being most likely related to natural variations in features of the images analyzed. Lastly, CCs may also assist in decision making in the sector, such as establishing minimum productivity goals based on historical performance.

Control Charts in Healthcare Quality Improvement

Objectives: Use of Shewhart control charts in quality improvement (QI) initiatives is increasing. These charts are typically used in one or more phases of the Plan Do Study Act (PDSA) cycle to monitor summaries of process and outcome data, abstracted from clinical information systems, over time. We summarize methodological criteria of Shewhart control charts and investigate adherence of published QI studies to these criteria.

Methods: We searched Medline, Embase and CINAHL for studies using Shewhart control charts in QI processes in direct patient care. We extracted methodological criteria for Shewhart control charts, and for the use of these charts in PDSA cycles, from textbooks and methodological literature.

Results: We included 34 studies, presenting 64 control charts of which 40 control charts plotted two phases of the PDSA cycle. The criterion to use 10–35 data points in a control chart was least adhered to (48.4% non-adherence). Other criteria were: transformation of the data in case of a skewed distribution (43.7% non adherence), when comparing data from two phases of the PDSA cycle the Plan phase (the first phase) needs to be stable (40.0% non-adherence), using a maximum of four different rules to detect special cause variation (14.1% non-adherence), and setting control limits at three standard deviations from the mean (all control charts adhered).

Conclusion: There is room for improvement with regard to the methodological construction of Shewhart control charts used in QI processes. Higher adherence to all methodological criteria will decrease the risk of incorrect conclusions about the process being monitored.

Development of Process Control Methodology for Tracking the Quality and Safety of Pain, Agitation, and Sedation Management in Critical Care Units

OBJECTIVE

To develop sedation, pain, and agitation quality measures using process control methodology and evaluate their properties in clinical practice.

DESIGN

A Sedation Quality Assessment Tool was developed and validated to capture data for 12-hour periods of nursing care. Domains included pain/discomfort and sedation-agitation behaviors; sedative, analgesic, and neuromuscular blocking drug administration; ventilation status; and conditions potentially justifying deep sedation. Predefined sedation-related adverse events were recorded daily. Using an iterative process, algorithms were developed to describe the proportion of care periods with poor limb relaxation, poor ventilator synchronization, unnecessary deep sedation, agitation, and an overall optimum sedation metric. Proportion charts described processes over time (2 monthly intervals) for each ICU. The numbers of patients treated between sedation-related adverse events were described with G charts. Automated algorithms generated charts for 12 months of sequential data. Mean values for each process were calculated, and variation within and between ICUs explored qualitatively.

SETTING

Eight Scottish ICUs over a 12-month period.

PATIENTS

Mechanically ventilated patients.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The Sedation Quality Assessment Tool agitation-sedation domains correlated with the Richmond Sedation Agitation Scale score (Spearman ρ = 0.75) and were reliable in clinician-clinician (weighted kappa; κ = 0.66) and clinician-researcher (κ = 0.82) comparisons. The limb movement domain had fair correlation with Behavioral Pain Scale (ρ = 0.24) and was reliable in clinician-clinician (κ = 0.58) and clinician-researcher (κ = 0.45) comparisons. Ventilator synchronization correlated with Behavioral Pain Scale (ρ = 0.54), and reliability in clinician-clinician (κ = 0.29) and clinician-researcher (κ = 0.42) comparisons was fair-moderate. Eight hundred twenty-five patients were enrolled (range, 59-235 across ICUs), providing 12,385 care periods for evaluation (range 655-3,481 across ICUs). The mean proportion of care periods with each quality metric varied between ICUs: excessive sedation 12-38%; agitation 4-17%; poor relaxation 13-21%; poor ventilator synchronization 8-17%; and overall optimum sedation 45-70%. Mean adverse event intervals ranged from 1.5 to 10.3 patients treated. The quality measures appeared relatively stable during the observation period.

CONCLUSIONS

Process control methodology can be used to simultaneously monitor multiple aspects of pain-sedation-agitation management within ICUs. Variation within and between ICUs could be used as triggers to explore practice variation, improve quality, and monitor this over time.

Statistical process control: separating signal from noise in emergency department operations

BACKGROUND

Statistical process control (SPC) is a visually appealing and statistically rigorous methodology very suitable to the analysis of emergency department (ED) operations.

OBJECTIVE

We demonstrate that the control chart is the primary tool of SPC; it is constructed by plotting data measuring the key quality indicators of operational processes in rationally ordered subgroups such as units of time. Control limits are calculated using formulas reflecting the variation in the data points from one another and from the mean. SPC allows managers to determine whether operational processes are controlled and predictable. We review why the moving range chart is most appropriate for use in the complex ED milieu, how to apply SPC to ED operations, and how to determine when performance improvement is needed.

DISCUSSION

SPC is an excellent tool for operational analysis and quality improvement for these reasons: 1) control charts make large data sets intuitively coherent by integrating statistical and visual descriptions; 2) SPC provides analysis of process stability and capability rather than simple comparison with a benchmark; 3) SPC allows distinction between special cause variation (signal), indicating an unstable process requiring action, and common cause variation (noise), reflecting a stable process; and 4) SPC keeps the focus of quality improvement on process rather than individual performance.

CONCLUSION

Because data have no meaning apart from their context, and every process generates information that can be used to improve it, we contend that SPC should be seriously considered for driving quality improvement in emergency medicine.

Clinical Practice Improvement Payments: incentives for delivery of quality care

OBJECTIVE

In 2008, Queensland Health mental health services participated in a quality incentive payment scheme referred to as the Clinical Practice Improvement Payment. Services across the state engaged in local improvement projects with the collective aim of improving the number of consumers, diagnosed with schizophrenia, followed up within seven days post discharge. This paper describes the application of this approach over two and a half years.

METHOD

Sixteen mental health services across Queensland participated and were provided with the opportunity to receive incentive payments during the period between January 2009 and June 2011. Data collection was conducted using information available on existing Queensland Health databases. Services were provided with regular updates on their progress through a secure intranet site, state-wide forums and individual service presentations, enabling them to compare their individual service data with peer and state data.

RESULTS

State-wide results showed steady and continual improvement in the indicator over the reporting period.

CONCLUSIONS

The results suggest that the implementation of incentive payments for routine clinical work in mental health can assist with state-wide service improvement. The impact of target setting and supporting activities remains unclear and improvements appeared to be robust to administrative challenges and unexpected external events.

The Queensland Mental Health Clinical Collaborative and the management of schizophrenia

OBJECTIVE

This paper describes the development of a collaborative group of mental health clinicians who have come together to improve practice in adult acute mental health settings for the inpatient management of schizophrenia.

METHOD

Sixteen acute adult mental health inpatient services across Queensland worked together to develop clinical indicators related to the inpatient treatment of schizophrenia. Data collection was conducted by using information available on existing databases and through statewide chart audits using scannable form technology. Through a secure intranet site, and statewide forums, clinicians were able to access information on clinical indicators enabling them to compare their site data to peer and state data.

RESULTS

Available data from 15 of the 16 sites provided information on clinical indicators including average length of stay, 28-day readmission rates, antipsychotic prescribing, medication dose and the use of multiple antipsychotic medications at discharge.

CONCLUSIONS

The formation of the Mental Health Clinical Collaborative has brought together clinicians across the State to develop clinical indicators and openly discuss ideas to inform and improve clinical practice. This process has been effective in improving the quality of routinely collected information across the State and in engaging clinicians in using health information to drive clinical practice.

Local hospital perspective on a nationwide outbreak of Pseudomonas aeruginosa infection in Norway

OBJECTIVE

To implement a system for monitoring of rare events based on statistical process control charts.

DESIGN

Statistical process control plotting by g chart of clinical microbiology laboratory data.

SETTING

Primary and secondary care Norwegian hospital with a 9-bed intensive care unit.

RESULTS

During the winter of 2001-2002 in Norway, there was a national monoclonal nosocomial outbreak of Pseudomonas aeruginosa infection mainly affecting patients in intensive care units. In the present work, we demonstrate how the use of SPC at one of the affected hospitals would have detected this outbreak several weeks before the alert from the Norwegian National Public Health Institute (NIPH). By plotting the monthly incidence rate of P. aeruginosa infection (with a c chart), we found that the hospital would have been alerted in February; by plotting the number of days between events (with a g chart), we found that the hospital would have detected a process already out of control in early January 2002. Not until 9 weeks later (ie, mid-March) did the NIPH declare the P. aeruginosa outbreak to be national, and a commercially produced mouth swab contaminated during the manufacturing process was found to be the source.

CONCLUSION

The plotting of rare events, such as an outbreak of nosocomial infection, with a g chart may be used for early detection of a process out of control.

Safety is part of quality: a proposal for a continuum in performance measurement

OBJECTIVES

Safer care is a strategic priority for health care organizations worldwide. Yet, the measurement and evaluation of key processes and outcomes associated with safer care remains challenging, even with existing performance measurement indicators. The multi-national Quality Indicator Project (QI Project) data are analysed to [1] document the patterns of safety indicators used between 1999 and 2006 among hospitals in Asia, Europe and the USA; and [2] to identify trends in using both organization-level and patient-level data in hospital performance improvement.

DESIGN AND SETTING

Retrospective data are used to ascertain how the use of safety indicators has changed in comparison to other QI Project indicators. 'Continent' rather than 'hospital' is used as the unit of analysis and P-values of the differences in use percentages across Asia, Europe and the USA are calculated.

RESULTS

There was a significant increase in the use of QI Project indicators in Asia between 1999 and 2006. Measured as the mean percentage of usage, the safety versus 'all other' indicators' increase in Asia was 43.7% versus 27% (P < 0.05) and 37.2% versus 24.4% (P < 0.05), respectively, during the study's time period. The European participants used both safety and all other indicators less frequently, 14.7% versus 18% (P < 0.05) and 9.5% versus 19.8% (P < 0.05), respectively. Finally, USA hospitals demonstrated a larger difference in the decrease of QI Project indicator use than European hospitals between the 'safety' and 'all other' indicators, 12.7% decrease for safety indicators and 7.1% for all others (P < 0.05). These findings are consistent with trends reported in a previous study.

CONCLUSION

Traditional performance measures continue to assist hospitals in identifying crucial aspects of safety in the delivery of care. Building on the findings of a previous study, there are emerging trends in the type of measures used in hospitals in Asia, Europe and the USA pursuing the improvement of overall performance. The increasing use of patient-level data specifically, in tandem with organizational level indicators, may signal the continuum of measurement strategies, now still predominantly in the USA but anticipated to be adopted both in Europe and Asia.