Combining ratings from multiple physician reviewers helped to overcome the uncertainty associated with adverse event classification

Effect of contextual factors on the prevalence of diagnostic errors among patients managed by physicians of the same specialty: a single-centre retrospective observational study

BACKGROUND

There has been growing recognition that contextual factors influence the physician's cognitive processes. However, given that cognitive processes may depend on the physicians' specialties, the effects of contextual factors on diagnostic errors reported in previous studies could be confounded by difference in physicians.

OBJECTIVE

This study aimed to clarify whether contextual factors such as location and consultation type affect diagnostic accuracy.

METHODS

We reviewed the medical records of 1992 consecutive outpatients consulted by physicians from the Department of Diagnostic and Generalist Medicine in a university hospital between 1 January and 31 December 2019. Diagnostic processes were assessed using the Revised Safer Dx Instrument. Patients were categorised into three groups according to contextual factors (location and consultation type): (1) referred patients with scheduled visit to the outpatient department; (2) patients with urgent visit to the outpatient department; and (3) patients with emergency visit to the emergency room. The effect of the contextual factors on the prevalence of diagnostic errors was investigated using logistic regression analysis.

RESULTS

Diagnostic errors were observed in 12 of 534 referred patients with scheduled visit to the outpatient department (2.2%), 3 of 599 patients with urgent visit to the outpatient department (0.5%) and 13 of 859 patients with emergency visit to the emergency room (1.5%). Multivariable logistic regression analysis showed a significantly higher prevalence of diagnostic errors in referred patients with scheduled visit to the outpatient department than in patients with urgent visit to the outpatient department (OR 4.08, p=0.03), but no difference between patients with emergency and urgent visit to the emergency room and outpatient department, respectively.

CONCLUSION

Contextual factors such as consultation type may affect diagnostic errors; however, since the differences in the prevalence of diagnostic errors were small, the effect of contextual factors on diagnostic accuracy may be small in physicians working in different care settings.

How safe are paediatric emergency departments? A national prospective cohort study

BACKGROUND

Despite the high number of children treated in emergency departments, patient safety risks in this setting are not well quantified. Our objective was to estimate the risk and type of adverse events, as well as their preventability and severity, for children treated in a paediatric emergency department.

METHODS

Our prospective, multicentre cohort study enrolled children presenting for care during one of 168 8-hour study shifts across nine paediatric emergency departments. Our primary outcome was an adverse event within 21 days of enrolment which was related to care provided at the enrolment visit. We identified 'flagged outcomes' (such as hospital visits, worsening symptoms) through structured telephone interviews with patients and families over the 21 days following enrolment. We screened admitted patients' health records with a validated trigger tool. For patients with flags or triggers, three reviewers independently determined whether an adverse event occurred.

RESULTS

We enrolled 6376 children; 6015 (94%) had follow-up data. Enrolled children had a median age of 4.3 years (IQR 1.6-9.8 years). One hundred and seventy-nine children (3.0%, 95% CI 2.6% to 3.5%) had at least one adverse event. There were 187 adverse events in total; 143 (76.5%, 95% CI 68.9% to 82.7%) were deemed preventable. Management (n=98, 52.4%) and diagnostic issues (n=36, 19.3%) were the most common types of adverse events. Seventy-nine (42.2%) events resulted in a return emergency department visit; 24 (12.8%) resulted in hospital admission; and 3 (1.6%) resulted in transfer to a critical care unit.

CONCLUSION

In this large-scale study, 1 in 33 children treated in a paediatric emergency department experienced an adverse event related to the care they received there. The majority of events were preventable; most were related to management and diagnostic issues. Specific patient populations were at higher risk of adverse events. We identify opportunities for improvement in care.

Incidence, origins and avoidable harm of missed opportunities in diagnosis: longitudinal patient record review in 21 English general practices

BACKGROUND

Diagnostic error is a global patient safety priority.

OBJECTIVES

To estimate the incidence, origins and avoidable harm of diagnostic errors in English general practice. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available (missed diagnostic opportunities, MDOs).

METHOD

Retrospective medical record reviews identified MDOs in 21 general practices. In each practice, two trained general practitioner reviewers independently conducted case note reviews on 100 randomly selected adult consultations performed during 2013-2014. Consultations where either reviewer identified an MDO were jointly reviewed.

RESULTS

Across 2057 unique consultations, reviewers agreed that an MDO was possible, likely or certain in 89 cases or 4.3% (95% CI 3.6% to 5.2%) of reviewed consultations. Inter-reviewer agreement was higher than most comparable studies (Fleiss' kappa=0.63). Sixty-four MDOs (72%) had two or more contributing process breakdowns. Breakdowns involved problems in the patient-practitioner encounter such as history taking, examination or ordering tests (main or secondary factor in 61 (68%) cases), performance and interpretation of diagnostic tests (31; 35%) and follow-up and tracking of diagnostic information (43; 48%). 37% of MDOs were rated as resulting in moderate to severe avoidable patient harm.

CONCLUSIONS

Although MDOs occurred in fewer than 5% of the investigated consultations, the high numbers of primary care contacts nationally suggest that several million patients are potentially at risk of avoidable harm from MDOs each year. Causes of MDOs were frequently multifactorial, suggesting the need for development and evaluation of multipronged interventions, along with policy changes to support them.

Adverse events in the paediatric emergency department: a prospective cohort study

BACKGROUND

Understanding adverse events among children treated in the emergency department (ED) offers an opportunity to improve patient safety by providing evidence of where to focus efforts in a resource-restricted environment.

OBJECTIVE

To estimate the risk of adverse events, their type, preventability and severity, for children seen in a paediatric ED.

METHODS

This prospective cohort study examined outcomes of patients presenting to a paediatric ED over a 1-year period. The primary outcome was the proportion of patients with an adverse event (harm to patient related to healthcare received) related to ED care within 3 weeks of their visit. We conducted structured telephone interviews with all patients and families over a 3-week period following their visit to identify flagged outcomes (such as repeat ED visits, worsening symptoms) and screened admitted patients' health records with a validated trigger tool. For patients with flagged outcomes or triggers, three ED physicians independently determined whether an adverse event occurred.

RESULTS

Of 1567 eligible patients, 1367 (87.2%) were enrolled and 1319 (96.5%) reached in follow-up. Median patient age was 4.34 years (IQR 1.5 to 10.57 years) and most (n=1281; 93.7%) were discharged. Among those with follow-up, 33 (2.5%, 95% CI 1.8% to 3.5%) suffered an adverse event related to ED care. None experienced more than one event. Twenty-nine adverse events (87.9%, 95% CI 72.7% to 95.2%) were deemed preventable. The most common types of adverse events (not mutually exclusive) were management issues (51.5%), diagnostic issues (45.5%) and suboptimal follow-up (15.2%).

CONCLUSION

One in 40 children suffered adverse events related to ED care. A high proportion of events were preventable. Management and diagnostic issues warrant further study.

Diagnostic Errors Are Common in Acute Pediatric Respiratory Disease: A Prospective, Single-Blinded Multicenter Diagnostic Accuracy Study in Australian Emergency Departments

Background: Diagnostic errors are a global health priority and a common cause of preventable harm. There is limited data available for the prevalence of misdiagnosis in pediatric acute-care settings. Respiratory illnesses, which are particularly challenging to diagnose, are the most frequent reason for presentation to pediatric emergency departments. Objective: To evaluate the diagnostic accuracy of emergency department clinicians in diagnosing acute childhood respiratory diseases, as compared with expert panel consensus (reference standard). Methods: Prospective, multicenter, single-blinded, diagnostic accuracy study in two well-resourced pediatric emergency departments in a large Australian city. Between September 2016 and August 2018, a convenience sample of children aged 29 days to 12 years who presented with respiratory symptoms was enrolled. The emergency department discharge diagnoses were reported by clinicians based upon standard clinical diagnostic definitions. These diagnoses were compared against consensus diagnoses given by an expert panel of pediatric specialists using standardized disease definitions after they reviewed all medical records. Results: For 620 participants, the sensitivity and specificity (%, [95% CI]) of the emergency department compared with the expert panel diagnoses were generally poor: isolated upper respiratory tract disease (64.9 [54.6, 74.4], 91.0 [88.2, 93.3]), croup (76.8 [66.2, 85.4], 97.9 [96.2, 98.9]), lower respiratory tract disease (86.6 [83.1, 89.6], 92.9 [87.6, 96.4]), bronchiolitis (66.9 [58.6, 74.5], 94.3 [80.8, 99.3]), asthma/reactive airway disease (91.0 [85.8, 94.8], 93.0 [90.1, 95.3]), clinical pneumonia (63·9 [50.6, 75·8], 95·0 [92·8, 96·7]), focal (consolidative) pneumonia (54·8 [38·7, 70·2], 86.2 [79.3, 91.5]). Only 59% of chest x-rays with consolidation were correctly identified. Between 6.9 and 14.5% of children were inappropriately prescribed based on their eventual diagnosis. Conclusion: In well-resourced emergency departments, we have identified a previously unrecognized high diagnostic error rate for acute childhood respiratory disorders, particularly in pneumonia and bronchiolitis. These errors lead to the potential of avoidable harm and the administration of inappropriate treatment.

Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review

OBJECTIVES

Record review is the most used method to quantify patient safety. We systematically reviewed the reliability and validity of adverse event detection with record review.

DESIGN

A systematic review of the literature.

METHODS

We searched PubMed, EMBASE, CINAHL, PsycINFO and the Cochrane Library and from their inception through February 2015. We included all studies that aimed to describe the reliability and/or validity of record review. Two reviewers conducted data extraction. We pooled κ values (κ) and analysed the differences in subgroups according to number of reviewers, reviewer experience and training level, adjusted for the prevalence of adverse events.

RESULTS

In 25 studies, the psychometric data of the Global Trigger Tool (GTT) and the Harvard Medical Practice Study (HMPS) were reported and 24 studies were included for statistical pooling. The inter-rater reliability of the GTT and HMPS showed a pooled κ of 0.65 and 0.55, respectively. The inter-rater agreement was statistically significantly higher when the group of reviewers within a study consisted of a maximum five reviewers. We found no studies reporting on the validity of the GTT and HMPS.

CONCLUSIONS

The reliability of record review is moderate to substantial and improved when a small group of reviewers carried out record review. The validity of the record review method has never been evaluated, while clinical data registries, autopsy or direct observations of patient care are potential reference methods that can be used to test concurrent validity.

Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

IMPORTANCE

Diagnostic errors are common and harmful, but difficult to define and measure. Measurement of diagnostic errors often depends on retrospective medical record reviews, frequently resulting in reviewer disagreement.

OBJECTIVES

We aimed to test the accuracy of an instrument to help detect presence or absence of diagnostic error through record reviews.

DESIGN

We gathered questions from several previously used instruments for diagnostic error measurement, then developed and refined our instrument. We tested the accuracy of the instrument against a sample of patient records (n = 389), with and without previously identified diagnostic errors (n = 129 and n = 260, respectively).

RESULTS

The final version of our instrument (titled Safer Dx Instrument) consisted of 11 questions assessing diagnostic processes in the patient-provider encounter and a main outcome question to determine diagnostic error. In comparison with the previous sample, the instrument yielded an overall accuracy of 84 %, sensitivity of 71 %, specificity of 90 %, negative predictive value of 86 %, and positive predictive value of 78 %. All 11 items correlated significantly with the instrument's error outcome question (all p values ≤ 0.01). Using factor analysis, the 11 questions clustered into two domains with high internal consistency (initial diagnostic assessment, and performance and interpretation of diagnostic tests) and a patient factor domain with low internal consistency (Cronbach's alpha coefficients 0.93, 0.92, and 0.38, respectively).

CONCLUSIONS

The Safer Dx Instrument helps quantify the likelihood of diagnostic error in primary care visits, achieving a high degree of accuracy for measuring their presence or absence. This instrument could be useful to identify high-risk cases for further study and quality improvement.

Diagnostic errors related to acute abdominal pain in the emergency department

OBJECTIVE

Diagnostic errors in the emergency department (ED) are harmful and costly. We reviewed a selected high-risk cohort of patients presenting to the ED with abdominal pain to evaluate for possible diagnostic errors and associated process breakdowns.

DESIGN

We conducted a retrospective chart review of ED patients >18 years at an urban academic hospital. A computerised 'trigger' algorithm identified patients possibly at high risk for diagnostic errors to facilitate selective record reviews. The trigger determined patients to be at high risk because they: (1) presented to the ED with abdominal pain, and were discharged home and (2) had a return ED visit within 10 days that led to a hospitalisation. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available during the first ED visit, regardless of patient harm, and included errors that involved both ED and non-ED providers. Errors were determined by two independent record reviewers followed by team consensus in cases of disagreement.

RESULTS

Diagnostic errors occurred in 35 of 100 high-risk cases. Over two-thirds had breakdowns involving the patient-provider encounter (most commonly history-taking or ordering additional tests) and/or follow-up and tracking of diagnostic information (most commonly follow-up of abnormal test results). The most frequently missed diagnoses were gallbladder pathology (n=10) and urinary infections (n=5).

CONCLUSIONS

Diagnostic process breakdowns in ED patients with abdominal pain most commonly involved history-taking, ordering insufficient tests in the patient-provider encounter and problems with follow-up of abnormal test results.

The challenges in defining and measuring diagnostic error

Diagnostic errors have emerged as a serious patient safety problem but they are hard to detect and complex to define. At the research summit of the 2013 Diagnostic Error in Medicine 6th International Conference, we convened a multidisciplinary expert panel to discuss challenges in defining and measuring diagnostic errors in real-world settings. In this paper, we synthesize these discussions and outline key research challenges in operationalizing the definition and measurement of diagnostic error. Some of these challenges include 1) difficulties in determining error when the disease or diagnosis is evolving over time and in different care settings, 2) accounting for a balance between underdiagnosis and overaggressive diagnostic pursuits, and 3) determining disease diagnosis likelihood and severity in hindsight. We also build on these discussions to describe how some of these challenges can be addressed while conducting research on measuring diagnostic error.

Application of a trigger tool in near real time to inform quality improvement activities: a prospective study in a general medicine ward

Background

Retrospective record review using trigger tools remains the most widely used method for measuring adverse events (AEs) to identify targets for improvement and measure temporal trends. However, medical records often contain limited information about factors contributing to AEs. We implemented an augmented trigger tool that supplemented record review with debriefing front-line staff to obtain details not included in the medical record. We hypothesised that this would foster the identification of factors contributing to AEs that could inform improvement initiatives.

Method

A trained observer prospectively identified events in consecutive patients admitted to a general medical ward in a tertiary care academic medical centre (November 2010 to February 2011 inclusive), gathering information from record review and debriefing front-line staff in near real time. An interprofessional team reviewed events to identify preventable and potential AEs and characterised contributing factors using a previously published taxonomy.

Results

Among 141 patients, 14 (10%; 95% CI 5% to 15%) experienced at least one preventable AE; 32 patients (23%; 95% CI 16% to 30%) experienced at least one potential AE. The most common contributing factors included policy and procedural problems (eg, routine protocol violations, conflicting policies; 37%), communication and teamwork problems (34%), and medication process problems (23%). However, these broad categories each included distinct subcategories that seemed to require different interventions. For instance, the 32 identified communication and teamwork problems comprised 7 distinct subcategories (eg, ineffective intraprofessional handovers, poor interprofessional communication, lacking a shared patient care, paging problems). Thus, even the major categories of contributing factors consisted of subcategories that individually related to a much smaller subset of AEs.

Conclusions

Prospective application of an augmented trigger tool identified a wide range of factors contributing to AEs. However, the majority of contributing factors accounted for a small number of AEs, and more general categories were too heterogeneous to inform specific interventions. Successfully using trigger tools to stimulate quality improvement activities may require development of a framework that better classifies events that share contributing factors amenable to the same intervention.

How safe are our paediatric emergency departments? Protocol for a national prospective cohort study

INTRODUCTION

Adverse events (AEs), defined as unintended patient harm related to healthcare provided rather than an underlying medical condition, represent a significant threat to patient safety and public health. The emergency department (ED) is a high-risk patient safety setting for many reasons including presentation 'outside of regular hours', high patient volumes, and a chaotic work environment. Children have also been identified as particularly vulnerable to AEs. Despite the identification of the ED as a high-risk setting and the vulnerability of the paediatric population, little research has been conducted regarding paediatric patient safety in the ED. The study objective is to generate an estimate of the risk and type of AEs, as well as their preventability and severity, for children seen in Canadian paediatric EDs.

METHODS AND ANALYSIS

This multicentre, prospective cohort study will enrol patients under 18 years of age from nine paediatric EDs across Canada. A stratified cluster random sampling scheme will be used to ensure patients recruited are representative of the overall ED population. A rigorous, standardised two-stage process will be used for AE identification. The primary outcome will be the proportion of children with AEs associated with ED care in the 3 weeks following the ED visit. Secondary outcomes will include the proportion of children with preventable AEs and the types and severity of AEs. We will aim to recruit 5632 patients over 1 year and this will allow us to detect a proportion of patients with an AE of 5% (to within an absolute margin of error of 0.6%).

ETHICS AND DISSEMINATION

Ethics approval has been obtained from participating sites. Results will be disseminated through presentations, peer review publications, linkages with emergency research network and a webinars for key knowledge user groups.

TRIAL REGISTRATION NUMBER

This study is registered at Clinicaltrials.gov (NCT02162147; https://clinicaltrials.gov/ct2/show/NCT02162147).

A comparative assessment of adverse event classification in the out-of-hospital setting

OBJECTIVES

We sought to test reliability of two approaches to classify adverse events (AEs) associated with helicopter EMS (HEMS) transport.

METHODS

The first approach for AE classification involved flight nurses and paramedics (RN/Medics) and mid-career emergency physicians (MC-EMPs) independently reviewing 50 randomly selected HEMS medical records. The second approach involved RN/Medics and MC-EMPs meeting as a group to openly discuss 20 additional medical records and reach consensus-based AE decision. We compared all AE decisions to a reference criterion based on the decision of three senior emergency physicians (Sr-EMPs). We designed a study to detect an improvement in agreement (reliability) from fair (kappa = 0.2) to moderate (kappa = 0.5). We calculated sensitivity, specificity, percent agreement, and positive and negative predictive values (PPV/NPV).

RESULTS

For the independent reviews, the Sr-EMP group identified 26 AEs while individual clinician reviewers identified between 19 and 50 AEs. Agreement on the presence/absence of an AE between Sr-EMPs and three MC-EMPs ranged from κ = 0.20 to κ = 0.25. Agreement between Sr-EMPs and three RN/Medics ranged from κ = 0.11 to κ = 0.19. For the consensus/open-discussion approach, the Sr-EMPs identified 13 AEs, the MC-EMP group identified 18 AEs, and RN/medic group identified 36 AEs. Agreement between Sr-EMPs and MC-EMP group was (κ = 0.30 95%CI -0.12, 0.72), whereas agreement between Sr-EMPs and RN/medic group was (κ = 0.40 95%CI 0.01, 0.79). Agreement between all three groups was fair (κ = 0.33, 95%CI 0.06, 0.66). Percent agreement (58-68%) and NPV (63-76%) was moderately dissimilar between clinicians, while sensitivity (25-80%), specificity (43-97%), and PPV (48-83%) varied.

CONCLUSIONS

We identified a higher level of agreement/reliability in AE decisions utilizing a consensus-based approach for review rather than independent reviews.

Strengths and weaknesses of working with the Global Trigger Tool method for retrospective record review: focus group interviews with team members

OBJECTIVES

The aim was to describe the strengths and weaknesses, from team member perspectives, of working with the Global Trigger Tool (GTT) method of retrospective record review to identify adverse events causing patient harm.

DESIGN

A qualitative, descriptive approach with focus group interviews using content analysis.

SETTING

5 Swedish hospitals in 2011.

PARTICIPANTS

5 GTT teams, with 5 physicians and 11 registered nurses.

INTERVENTION

5 focus group interviews were carried out with the five teams. Interviews were taped and transcribed verbatim.

RESULTS

8 categories emerged relating to the strengths and weaknesses of the GTT method. The categories found were: Usefulness of the GTT, Application of the GTT, Triggers, Preventability of harm, Team composition, Team tasks, Team members' knowledge development and Documentation. Gradually, changes in the methodology were made by the teams, for example, the teams reported how the registered nurses divided up the charts into two sets, each being read respectively. The teams described the method as important and well functioning. Not only the most important, but also the most difficult, was the task of bringing the results back to the clinic. The teams found it easier to discuss findings at their own clinics.

CONCLUSIONS

The GTT method functions well for identifying adverse events and is strengthened by its adaptability to different specialties. However, small, gradual methodological changes together with continuingly developed expertise and adaption to looking at harm from a patient's perspective may contribute to large differences in assessment over time.

Measuring adverse events in helicopter emergency medical services: establishing content validity

INTRODUCTION

We sought to create a valid framework for detecting adverse events (AEs) in the high-risk setting of helicopter emergency medical services (HEMS).

METHODS

We assembled a panel of 10 expert clinicians (n = 6 emergency medicine physicians and n = 4 prehospital nurses and flight paramedics) affiliated with a large multistate HEMS organization in the Northeast US. We used a modified Delphi technique to develop a framework for detecting AEs associated with the treatment of critically ill or injured patients. We used a widely applied measure, the content validity index (CVI), to quantify the validity of the framework's content.

RESULTS

The expert panel of 10 clinicians reached consensus on a common AE definition and four-step protocol/process for AE detection in HEMS. The consensus-based framework is composed of three main components: (1) a trigger tool, (2) a method for rating proximal cause, and (3) a method for rating AE severity. The CVI findings isolate components of the framework considered content valid.

CONCLUSIONS

We demonstrate a standardized process for the development of a content-valid framework for AE detection. The framework is a model for the development of a method for AE identification in other settings, including ground-based EMS.

Improving patient safety through the systematic evaluation of patient outcomes

Despite increased advocacy for patient safety and several large-scale programs designed to reduce preventable harm, most notably surgical checklists, recent data evaluating entire health systems suggests that we are no further ahead in improving patient safety and that hospital complications are no less frequent now than in the 1990s. We suggest that the failure to systematically measure patient safety is the reason for our limited pro gress. In addition to defining patient safety outcomes and describing their financial and clinical impact, we argue why the failure to implement patient safety measurement systems has compromised the ability to move the agenda forward. We also present an overview of how patient safety can be assessed and the strengths and weaknesses of each method and comment on some of the consequences created by the absence of a systematic measurement system.

Reliability of the peer-review process for adverse event rating

BACKGROUND

Adverse events are poor patient outcomes caused by medical care. Their identification requires the peer-review of poor outcomes, which may be unreliable. Combining physician ratings might improve the accuracy of adverse event classification.

OBJECTIVE

To evaluate the variation in peer-reviewer ratings of adverse outcomes; determine the impact of this variation on estimates of reviewer accuracy; and determine the number of reviewers who judge an adverse event occurred that is required to ensure that the true probability of an adverse event exceeded 50%, 75% or 95%.

METHODS

Thirty physicians rated 319 case reports giving details of poor patient outcomes following hospital discharge. They rated whether medical management caused the outcome using a six-point ordinal scale. We conducted latent class analyses to estimate the prevalence of adverse events as well as the sensitivity and specificity of each reviewer. We used this model and Bayesian calculations to determine the probability that an adverse event truly occurred to each patient as function of their number of positive ratings.

RESULTS

The overall median score on the 6-point ordinal scale was 3 (IQR 2,4) but the individual rater median score ranged from a minimum of 1 (in four reviewers) to a maximum median score of 5. The overall percentage of cases rated as an adverse event was 39.7% (3798/9570). The median kappa for all pair-wise combinations of the 30 reviewers was 0.26 (IQR 0.16, 0.42; Min = -0.07, Max = 0.62). Reviewer sensitivity and specificity for adverse event classification ranged from 0.06 to 0.93 and 0.50 to 0.98, respectively. The estimated prevalence of adverse events using a latent class model with a common sensitivity and specificity for all reviewers (0.64 and 0.83 respectively) was 47.6%. For patients to have a 95% chance of truly having an adverse event, at least 3 of 3 reviewers are required to deem the outcome an adverse event.

CONCLUSION

Adverse event classification is unreliable. To be certain that a case truly represents an adverse event, there needs to be agreement among multiple reviewers.

Electronic health record-based surveillance of diagnostic errors in primary care

BACKGROUND

Diagnostic errors in primary care are harmful but difficult to detect. The authors tested an electronic health record (EHR)-based method to detect diagnostic errors in routine primary care practice.

METHODS

The authors conducted a retrospective study of primary care visit records 'triggered' through electronic queries for possible evidence of diagnostic errors: Trigger 1: A primary care index visit followed by unplanned hospitalisation within 14 days and Trigger 2: A primary care index visit followed by ≥1 unscheduled visit(s) within 14 days. Control visits met neither criterion. Electronic trigger queries were applied to EHR repositories at two large healthcare systems between 1 October 2006 and 30 September 2007. Blinded physician-reviewers independently determined presence or absence of diagnostic errors in selected triggered and control visits. An error was defined as a missed opportunity to make or pursue the correct diagnosis when adequate data were available at the index visit. Disagreements were resolved by an independent third reviewer.

RESULTS

Queries were applied to 212 165 visits. On record review, the authors found diagnostic errors in 141 of 674 Trigger 1-positive records (positive predictive value (PPV)=20.9%, 95% CI 17.9% to 24.0%) and 36 of 669 Trigger 2-positive records (PPV=5.4%, 95% CI 3.7% to 7.1%). The control PPV of 2.1% (95% CI 0.1% to 3.3%) was significantly lower than that of both triggers (p≤0.002). Inter-reviewer reliability was modest, though higher than in comparable previous studies (к=0.37 (95% CI 0.31 to 0.44)).

CONCLUSIONS

While physician agreement on diagnostic error remains low, an EHR-facilitated surveillance methodology could be useful for gaining insight into the origin of these errors.

Using prospective clinical surveillance to identify adverse events in hospital

Background

To improve patient safety, organisations must systematically measure avoidable harms. Clinical surveillance—consisting of prospective case finding and peer review—could improve identification of adverse events (AEs), preventable AEs and potential AEs. The authors sought to describe and compare findings of clinical surveillance on four clinical services in an academic hospital.

Methods

Clinical surveillance was performed by a nurse observer who monitored patients for prespecified clinical events and collected standard information about each event. A multidisciplinary, peer-review committee rated causation for each event. Events were subsequently classified in terms of severity and type.

Results

The authors monitored 1406 patients during their admission to four hospital services: Cardiac Surgery Intensive Care (n=226), Intensive Care (n=211), General Internal Medicine (n=453) and Obstetrics (n=516). The authors detected 245 AEs during 9300 patient days of observation (2.6 AEs per 100 patient days). 88 AEs (33%) were preventable. The proportion of patients experiencing at least one AE, preventable AE or potential AE was 13.7%, 6.1% and 5.3%, respectively. AE risk varied between services, ranging from 1.4% of Obstetrics to 11% of Internal Medicine and Intensive Care patients experiencing at least one preventable AE. The proportion of patients experiencing AEs resulting in permanent disability or death varied between services: ranging from 0.2% on Obstetrics to 4.9% on Cardiac Surgery Intensive Care. No services shared the most frequent AE type.

Conclusions

Using clinical surveillance, the authors identified a high risk of AE and significant variation in AE risks and subtypes between services. These findings suggest that institutions will need to evaluate service-specific safety problems to set priorities and design improvement strategies.

Adverse events among patients registered in high-acuity areas of the emergency department: a prospective cohort study

OBJECTIVE

To enhance patient safety, it is important to understand the frequency and causes of adverse events (defined as unintended injuries related to health care management). We performed this study to describe the types and risk of adverse events in high-acuity areas of the emergency department (ED).

METHODS

This prospective cohort study examined the outcomes of consecutive patients who received treatment at 2 tertiary care EDs. For discharged patients, we conducted a structured telephone interview 14 days after their initial visit; for admitted patients, we reviewed the inpatient charts. Three emergency physicians independently adjudicated flagged outcomes (e.g., death, return visits to the ED) to determine whether an adverse event had occurred.

RESULTS

We enrolled 503 patients; one-half (n = 254) were female and the median age was 57 (range 18-98) years. The majority of patients (n = 369, 73.3%) were discharged home. The most common presenting complaints were chest pain, generalized weakness and abdominal pain. Of the 107 patients with flagged outcomes, 43 (8.5%, 95% confidence interval 8.1%-8.9%) were considered to have had an adverse event through our peer review process, and over half of these (24, 55.8%) were considered preventable. The most common types of adverse events were as follows: management issues (n = 18, 41.9%), procedural complications (n = 13, 30.2%) and diagnostic issues (n = 10, 23.3%). The clinical consequences of these adverse events ranged from minor (urinary tract infection) to serious (delayed diagnosis of aortic dissection).

CONCLUSION

We detected a higher proportion of preventable adverse events compared with previous inpatient studies and suggest confirmation of these results is warranted among a wider selection of EDs.

Description of the development and validation of the Canadian Paediatric Trigger Tool

Objective

To describe the process of developing and validating the Canadian Association of Paediatric Health Centres Trigger Tool (CPTT).

Methods

Five existing trigger tools were consolidated with duplicate triggers eliminated. After a risk analysis and modified Delphi process, the tool was reduced from 94 to 47 triggers. Feasibility of use was tested, reviewing 40 charts in three hospitals. For validation, charts were randomly selected across four age groups, half medical/half surgical diagnoses, from six paediatric academic health sciences centres. 591 charts were reviewed by six nurses (for triggers and adverse events (AEs)) and three physicians (for AEs only). The incidence of trigger- and AE-positive charts was documented, and the sensitivity and specificity of the tool to identify charts with AEs were determined. Identification of AEs by nurses and physicians was compared. The positive predictive value (PPV) of each trigger was calculated and the ratio of false- to true-positive AE predictors analysed for each trigger.

Results

Nurses rated the CPTT easy to use and identified triggers in 61.1% (361/591; 95% CI 57.2 to 65.0) of patient charts; physicians identified AEs in 15.1% (89/ 591, 95% CI 0.23 to 0.43). Over a third of patients with AEs were neonates. The sensitivity and specificity were 0.88 and 0.44, respectively. Nurse and physician AE assessments correlated poorly. The PPV for each trigger ranged from 0 to 88.3%. Triggers with a false/true-positive ratio of >0.7 were eliminated, resulting in the final 35-trigger CPTT.

Conclusions

The CPTT is the first validated, comprehensive trigger tool available to detect AEs in children hospitalised in acute care facilities.

Using Medical Emergency Teams to detect preventable adverse events

Introduction

Medical Emergency Teams (METs), also known as Rapid Response Teams, are recommended as a patient safety measure. A potential benefit of implementing an MET is the capacity to systematically assess preventable adverse events, which are defined as poor outcomes caused by errors or system design flaws. We describe how we used MET calls to systematically identify preventable adverse events in an academic tertiary care hospital, and describe our surveillance results.

Methods

For four weeks we collected standard information on consecutive MET calls. Within a week of the MET call, a multi-disciplinary team reviewed the information and rated the cause of the outcome using a previously developed rating scale. We classified the type and severity of the preventable adverse event.

Results

We captured information on all 65 MET calls occurring during the study period. Of these, 16 (24%, 95% confidence interval [CI] 16%–36%) were felt to be preventable adverse events. The most common cause of the preventable adverse events was error in providing appropriate therapy despite an accurate diagnosis. One service accounted for a disproportionate number of preventable adverse events (n = 5, [31%, 95% CI 14%–56%]).

Conclusions

Our method of reviewing MET calls was easy to implement and yielded important results. Hospitals maintaining an MET can use our method as a preventable adverse event detection system at little additional cost.

Relationship Between Preventability of Death After Coronary Artery Bypass Graft Surgery and All-Cause Risk-Adjusted Mortality Rates

Background— The goal of this study was to determine the relationship between all-cause, risk-adjusted, in-hospital mortality after coronary artery bypass graft surgery and the proportion of preventable in-hospital deaths as a measure of quality of care at an institution level.

Methods and Results— We conducted a retrospective analysis of 347 randomly selected in-hospital deaths after isolated coronary artery bypass graft surgery at 9 institutions in Ontario over the period of 1998 to 2003. Nurse-abstracted chart summaries were reviewed by 2 experienced cardiac surgeons who were blinded to patient, surgeon, and hospital and used a standardized implicit tool to identify preventable death. A third reviewer reassessed all cases in which the first 2 reviewers disagreed. Rates of preventable deaths were estimated for each hospital and compared with all-cause mortality rates. A structured adverse event audit completed by each surgeon-reviewer was used to identify quality improvement opportunities for the preventable deaths. A total of 111 of 347 deaths (32%) were judged preventable despite a low risk-adjusted mortality range (1.3% to 3.1%) across hospitals. No significant correlation was found between all-cause, risk-adjusted in-hospital mortality rates and the proportion of preventable deaths at the hospital level (Spearman coefficient, −0.42; P =0.26). A large proportion of preventable deaths were related to problems in the operating room (86%) and intensive care unit (61%). Many deaths were associated with deviations in perioperative care (32% based on concurrence of 2 reviewers, and another 42% in cases in which 1 reviewer reached that opinion).

Conclusions— Approximately one third of in-hospital coronary artery bypass graft deaths were judged preventable by surgeon reviewers. All-cause risk-adjusted mortality rates are convenient measures of institutional quality of care but were not correlated with preventable mortality in our jurisdiction. Providers should conduct detailed adverse event audits to drive meaningful improvements in quality.