Physician-reviewers' perceptions and judgments about quality of care

A case for revisiting peer review: Implications for professional self-regulation and quality improvement

Background

Quality improvement in healthcare has often been promoted as different from and more valuable than peer review and other professional self-regulation processes. In spite of attempts to harmonize these two approaches, the perception of dichotomous opposition has persisted. A sequence of events in the troubled California prison system fortuitously isolated workforce interventions from more typical quality improvement interventions. Our objectives were to (1) evaluate the relative contributions of professional accountability and quality improvement interventions to an observed decrease in population mortality and (2) explore the organizational dynamics that potentiated positive outcomes.

Methods

Our retrospective mixed-methods case study correlated time-series analysis of mortality with the timing of reform interventions. Quantitative and qualitative evidence was drawn from court documents, public use files, internal databases, and other archival documents.

Results

Change point analysis reveals with 98% confidence that a significant improvement in age-adjusted natural mortality occurred in 2007, decreasing from 138.7 per 100,000 in the 1998–2006 period to 106.4 in the 2007–2009 period. The improvement in mortality occurred after implementation of accountability processes, prior to implementation of quality improvement interventions. Archival evidence supports the positive impact of physician competency assessments, robust peer review, and replacement of problem physicians.

Conclusions

Our analysis suggests that workforce accountability provides a critical quality safeguard, and its neglect in scholarship and practice is unjustified. As with quality improvement, effective professional self-regulation requires systemic implementation of enabling policies, processes, and staff resources. The study adds to evidence that the distribution of physician performance contains a heterogeneous left skew of dyscompetence that is associated with significant harm and suggests that professional self-regulation processes such as peer review can reduce that harm. Beyond their responsibility for direct harm, dyscompetent professionals can have negative impacts on group performance. The optimal integration of professional accountability and quality improvement systems merits further investigation.

Peer Review and Surgical Innovation: Robotic Surgery and Its Hurdles

The peer review processes as outlined in the Health Care Quality Improvement Act (HCQIA) is meant ensure quality standard of care through a self-policing mechanism by the medical community. This process grants immunity for people filing a peer review, which is meant to protect whistleblowers. However, it also creates a loophole that can be used maliciously to hinder competition. This is accentuated when surgeons are integrating new technologies, such as robotic surgery, into their practice. With more than 2000 da Vinci robots in use and more than 300 new units being shipped each year, robotic surgery has become a mainstay in the surgical field. The applications for robots continue to expand as surgeons discover their expanding capability. We need a better peer review process. That ensures the peer review is void of competitive bias. Peer reviewers need to be familiar with the procedure and the technology. The current process could stymie innovation in the name of competition.

Applying trigger tools to detect adverse events associated with outpatient surgery

OBJECTIVE

The objective of this study is to evaluate the performance of 5 triggers to detect adverse events (AEs) associated with outpatient surgery. Triggers use surveillance algorithms derived from clinical logic to flag cases where AEs have most likely occurred. Current efforts to detect AEs have focused primarily on the inpatient setting, despite the increase in outpatient surgery in all health care settings.

METHODS

Using trigger logic, we retrospectively evaluated data from 3 large health care systems' electronic medical records. Patients were eligible for inclusion if they had an outpatient (same-day) surgery in 2007 and at least 1 clinical note in the 6 months after the surgery. Two nurse abstractors reviewed a sample of trigger-flagged cases from each health care system. After reaching interrater reliability targets (κ > 0.60), we calculated the positive predictive value (PPV) of each trigger and the confidence interval of the estimate.

RESULTS

The surgical triggers flagged between 1% and 22% of the outpatient surgery cases, with a wide range in PPVs (6.0%-62.0%). The pulmonary embolism and deep vein thrombosis and emergency department triggers had the lowest proportion of flagged cases along with the highest PPVs, showing the most promise for screening cases with a high probability of AE occurrence.

CONCLUSIONS

Triggers may be useful in identifying a narrow set of surgeries for further review to determine if a surgical AE occurred, complementing existing tools and initiatives used to detect AEs. Improved detection of AEs in outpatient surgery should help target potential areas for quality improvement.

A new implicit review instrument for measuring quality of care delivered to pediatric patients in the emergency department

Background

There are few outcomes experienced by children receiving care in the Emergency Department (ED) that are amenable to measuring for the purposes of assessing of quality of care. The purpose of this study was to develop, test, and validate a new implicit review instrument that measures quality of care delivered to children in EDs.

Methods

We developed a 7-point structured implicit review instrument that encompasses four aspects of care, including the physician's initial data gathering, integration of information and development of appropriate diagnoses; initial treatment plan and orders; and plan for disposition and follow-up. Two pediatric emergency medicine physicians applied the 5-item instrument to children presenting in the highest triage category to four rural EDs, and we assessed the reliability of the average summary scores (possible range of 5–35) across the two reviewers using standard measures. We also validated the instrument by comparing this mean summary score between those with and without medication errors (ascertained independently by two pharmacists) using a two-sample t-test.

Results

We reviewed the medical records of 178 pediatric patients for the study. The mean and median summary score for this cohort of patients were 27.4 and 28.5, respectively. Internal consistency was high (Cronbach's alpha of 0.92 and 0.89). All items showed a significant (p < 0.005) positive correlation between reviewers using the Spearman rank correlation (range 0.24 to 0.39). Exact agreement on individual items between reviewers ranged from 70.2% to 85.4%. The Intra-class Correlation Coefficient for the mean of the total summary score across the two reviewers was 0.65. The validity of the instrument was supported by the finding of a higher score for children without medication errors compared to those with medication errors which trended toward significance (mean score = 28.5 vs. 26.0, p = 0.076).

Conclusion

The instrument we developed to measure quality of care provided to children in the ED has high internal consistency, fair to good inter-rater reliability and inter-rater correlation, and high content validity. The validity of the instrument is supported by the fact that the instrument's average summary score was lower in the presence of medication errors, which trended towards statistical significance.