Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data

Patient Safety Indicators at an Academic Veterans Affairs Hospital: Addressing Dual Goals of Clinical Care and Validity

BACKGROUND

Hospital-acquired complications add to patient morbidity and mortality, costs, length of stay, and negative patient experience. Patient Safety Indicators (PSIs) are a validated and widely used metric to evaluate hospital administrative data on preventing these events. Although many studies have addressed PSI validity, few have aimed to reduce PSI through clinical care. The authors aimed to reduce PSI events by addressing both validity and clinical care.

METHODS

Frontline clinicians used a deep dive template to provide input on all PSI cases, which were then reviewed by a PSI task force to identify performance gaps. After analyzing the frequency of gaps and cost-vs.-impact of potential solutions, five interventions were implemented to address the three most common, highly weighted PSIs: pressure ulcers, postoperative venous thromboembolism (VTE), and postoperative sepsis. Clinical care interventions included increasing patient mobility by creating a specialized mobility technician position, skin care audits to prevent pressure ulcers, and increasing use of pharmacologic VTE prophylaxis. Administrative interventions addressed improving clinician-coding concordance for sepsis and increasing documentation of comorbidities.

RESULTS

After interventions, the number of PSI events for composite PSI, VTE, and sepsis decreased by 41.3% (p = 0.039), 85.2% (p = 0.0091), and 51.5% (p = 0.063), respectively, relative to the preintervention period. Pressure ulcers increased by 33.3% (p = 0.0091).

CONCLUSION

Hospital complications cause substantial burden to hospitals, patients, and caregivers. Addressing administrative and clinical factors with targeted interventions led to reduction in composite PSI. Further efforts are needed locally to reduce the pressure ulcer PSI.

Evidence for clinician underprescription of and patient non-adherence to guideline-recommended cardiovascular medications among adults with peripheral artery disease: protocol for a systematic review and meta-analysis

INTRODUCTION

International guidelines recommend that adults with peripheral artery disease (PAD) be prescribed antiplatelet, statin and antihypertensive medications. However, it is unclear how often people with PAD are underprescribed these drugs, which characteristics predict clinician underprescription of and patient non-adherence to guideline-recommended cardiovascular medications, and whether underprescription and non-adherence are associated with adverse health and health system outcomes.

METHODS AND ANALYSIS

We will search MEDLINE, EMBASE and Evidence-Based Medicine Reviews from 2006 onwards. Two investigators will independently review abstracts and full-text studies. We will include studies that enrolled adults and reported the incidence and/or prevalence of clinician underprescription of or patient non-adherence to guideline-recommended cardiovascular medications among people with PAD; adjusted risk factors for underprescription of/non-adherence to these medications; and adjusted associations between underprescription/non-adherence to these medications and outcomes. Outcomes will include mortality, major adverse cardiac and limb events (including revascularisation procedures and amputations), other reported morbidities, healthcare resource use and costs. Two investigators will independently extract data and evaluate study risk of bias. We will calculate summary estimates of the incidence and prevalence of clinician underprescription/patient non-adherence across studies. We will also conduct subgroup meta-analyses and meta-regression to determine if estimates vary by country, characteristics of the patients and treating clinicians, population-based versus non-population-based design, and study risks of bias. Finally, we will calculate pooled adjusted risk factors for underprescription/non-adherence and adjusted associations between underprescription/non-adherence and outcomes. We will use Grading of Recommendations, Assessment, Development and Evaluation to determine estimate certainty.

ETHICS AND DISSEMINATION

Ethics approval is not required as we are studying published data. This systematic review will synthesise existing evidence regarding clinician underprescription of and patient non-adherence to guideline-recommended cardiovascular medications in adults with PAD. Results will be used to identify evidence-care gaps and inform where interventions may be required to improve clinician prescribing and patient adherence to prescribed medications.

PROSPERO REGISTRATION NUMBER

CRD42022362801.

Estimates of 30-day postoperative pulmonary complications after gastrointestinal endoscopic procedures: A retrospective cohort analysis of a health system population

The incidence of 30-day postoperative pulmonary complications (PPC) of gastrointestinal endoscopic procedures (GIEP) are not well characterized in the literature. The primary aim of this study was to identify the incidence of 30-day PPC after GIEP within a large healthcare system. We conducted a retrospective cohort study of 5377 patients presenting for GIEP between January 2013 and January 2022. Our primary outcome was the Agency for Healthcare Research and Quality PPC composite (AHRQ-PPC). Secondary outcomes were sub-composites derived from the AHRQ-PPC; including pneumonia (AHRQ-PNA), respiratory failure (AHRQ-RF), aspiration pneumonia/ pneumonitis (AHRQ-ASP) and pulmonary emboli (AHRQ-PE). We performed propensity score matching (PSM) followed by multivariable logistic regression to analyze primary and secondary outcomes. Inpatients had higher 30-day AHRQ-PPC (6.0 vs. 1.2%, p<0.001), as well as sub-composite AHRQ-PNA (3.2 vs. 0.7%, p<0.001), AHRQ-RF (2.4 vs. 0.5%, p<0.001), and AHRQ-ASP (1.9 vs. 0.4%, p<0.001). After PSM adjustment, pre-procedural comorbidities of electrolyte disorder [57.9 vs. 31.1%, ORadj: 2.26, 95%CI (1.48, 3.45), p<0.001], alcohol abuse disorder [16.7 vs. 6.8%, ORadj: 2.66 95%CI (1.29, 5.49), p = 0.01], congestive heart failure (CHF) [22.3 vs. 8.7%, ORadj: 2.2 95%CI (1.17, 4.15), p = 0.02] and pulmonary circulatory disorders [21 vs. 16.9%, ORadj: 2.95, 95%CI (1.36, 6.39), p = 0.01] were associated with 30-day AHRQ-PPC. After covariate adjustment, AHRQ-PPC was associated with upper endoscopy more than lower endoscopy [5.9 vs. 1.0%, ORadj: 3.76, 95%CI (1.85, 7.66), p<0.001]. When compared to gastroenterologist-guided conscious sedation, anesthesia care team presence was protective against AHRQ-PPC [3.7 vs. 8.4%, ORadj: 0.032, 95%CI (0.01, 0.22), p<0.001] and AHRQ-ASP [1.0 vs. 3.37%, ORadj: 0.002, 95%CI (0.00, 0.55), p<0.001]. In conclusion, we report estimates of 30-day PPC after GIEP across inpatient and outpatient settings. Upper endoscopic procedures confer a higher risk, while the presence of an anesthesia care team may be protective against 30-day PPC.

Validity of 16 AHRQ Patient Safety Indicators to identify in-hospital complications: a medical record review across nine Swiss hospitals

The validity of the Agency for Healthcare Research and Quality's Patient Safety Indicators (PSIs) has been established in the USA and Canada. However, these indicators are also used for hospital benchmarking and cross-country comparisons in other nations with different health-care settings and coding systems as well as missing present on admission (POA) flags in the administrative data. This study sought to comprehensively assess and compare the validity of 16 PSIs in Switzerland, where they have not been previously applied. We performed a medical record review using administrative and electronic medical record data from nine Swiss hospitals. Seven independent reviewers evaluated 1245 cases at various hospitals using retrospective data from the years 2014-18. True positives, false positives, positive predictive values (PPVs), and reasons for misclassification were compared across all investigated PSIs, and the documentation quality of the PSIs was examined. PSIs 6 (iatrogenic pneumothorax), 10 (postoperative acute kidney injury), 11 (postoperative respiratory failure), 13 (postoperative sepsis), 14 (wound dehiscence), 17 (birth trauma), and 18 and 19 (obstetric trauma with or without instrument) showed high PPVs (range: 90-99%) and were not strongly influenced by missing POA information. In contrast, PSIs 3 (pressure ulcer), 5 (retained surgical item), 7 (central venous catheter-related bloodstream infection), 8 (fall with hip fracture), and 15 (accidental puncture/laceration) showed low PPVs (range: 18-49%). In the case of PSIs 3, 8, and 12 (perioperative embolism/thrombosis), the low PPVs were largely due to the lack of POA information. Additionally, it was found that the documentation of PSI 3 in discharge letters could be improved. We found large differences in validity across the 16 PSIs in Switzerland. These results can guide policymakers in Switzerland and comparable health-care systems in selecting and prioritizing suitable PSIs for quality initiatives. Furthermore, the national introduction of a POA flag would allow for the inclusion of additional PSIs in quality monitoring.

Patient safety indicators from a spine surgery perspective: the importance of a specialty specific clinician working with the documentation team and the impact to your hospital

BACKGROUND CONTEXT

The Hospital Acquired Conditions (HAC) Reduction Program supports the Centers for Medicare and Medicaid Services (CMS) effort to prevent harm to patients by providing a financial incentive to reduce HACs. HAC scores are impacted by Patient Safety Indicators (PSIs), potentially preventable hospital-related events associated with harmful patient outcomes. PSIs are identified using International Classification of Diseases (ICD) coding; however, ICD coding does not always reflect the patient's true medical course.

PURPOSE

To evaluate the efficacy of and costs savings associated with a clinical documentation review process in tandem with clinician collaboration in identifying incorrectly generated PSIs.

STUDY DESIGN

Retrospective chart review.

PATIENT SAMPLE

All patients undergoing spine surgery at a single multi-surgeon tertiary spine center.

OUTCOME MEASURES

Occurrence of PSI.

METHODS

Over two 11-month periods, all PSIs attributable to spine surgery were determined. The number and type of spine related PSIs were compared before (Control) and after the implementation of a specialty specific clinical review (Intervention) to identify incorrectly generated PSIs. The financial impact of this intervention was calculated in the form of an annual cost savings to our hospital system.

RESULTS

During the Control phase, 61 PSIs were reported in 3368 spine cases, representing a total of 3.6 PSIs/month. During Intervention phase, 26 PSIs in 4,482 spine cases, resulting in a statistically significant decrease of 1.5 PSIs per month. The percentage of PSIs across all surgical cases attributable to spine surgery had a statistically significant decrease during the Intervention period compared to the Control period (16% vs. 10%, p=.034), resulting in the avoidance of a 1% CMS cost reduction, an annual cost saving of approximately $3-4 million dollars per year.

CONCLUSIONS

The implementation of a clinical documentation review process with clinician collaboration to ensure ICD-10 coding accurately reflects the patient's medical course leads to more accurate PSI reporting, with the potential for substantial cost-savings for hospitals from CMS reimbursement.

The deterrent effect of tort law: Evidence from medical malpractice reform

We examine whether caps on noneconomic damages in medical malpractice cases affect in-hospital patient safety. We use Patient Safety Indicators - measures of adverse events - as proxies for safety. In difference-in-differences ("DiD") analyses of five states that adopt caps during 2003-2005, we find that multiple measures of non-fatal patient safety events worsen after cap adoption relative to control states. DiD inference can be unreliable with a small number of treated units. We therefore develop a randomization inference-based test for inference with few treated units but multiple correlated outcomes and confirm the robustness of our results with this nonparametric approach. We also provide evidence that the decline in patient safety is unlikely to be driven by patient selection.

Association between neuraxial anaesthesia or general anaesthesia for lower limb revascularisation surgery in adults and clinical outcomes: population based comparative effectiveness study

OBJECTIVE

To examine the associations between neuraxial anaesthesia or general anaesthesia and clinical outcomes, length of hospital stay, and readmission in adults undergoing lower limb revascularisation surgery.

DESIGN

Comparative effectiveness study using linked, validated, population based databases.

SETTING

Ontario, Canada, 1 April 2002 to 31 March 2015.

PARTICIPANTS

20 988 patients Ontario residents aged 18 years or older who underwent their first lower limb revascularisation surgery in hospitals performing 50 or more of these surgeries annually.

MAIN OUTCOME MEASURES

Primary outcome was 30 day all cause mortality. Secondary outcomes were in-hospital cardiopulmonary and renal complications, length of hospital stay, and 30 day readmissions. Multivariable, mixed effects regression models, adjusting for patient, procedural, and hospital characteristics, were used to estimate associations between anaesthetic technique and outcomes. Robustness of analyses were evaluated by conducting instrumental variable, propensity score matched, and survival sensitivity analyses.

RESULTS

Of 20 988 patients who underwent lower limb revascularisation surgery, 6453 (30.7%) received neuraxial anaesthesia and 14 535 (69.3%) received general anaesthesia. The percentage of neuraxial anaesthesia use ranged from 0.6% to 90.6% across included hospitals. Furthermore, use of neuraxial anaesthesia declined by 17% over the study period. Death within 30 days occurred in 204 (3.2%) patients who received neuraxial anaesthesia and 646 (4.4%) patients who received general anaesthesia. After multivariable, multilevel adjustment, use of neuraxial anaesthesia compared with use of general anaesthesia was associated with decreased 30 day mortality (absolute risk reduction 0.72%, 95% confidence interval 0.65% to 0.79%; odds ratio 0.68, 95% confidence interval 0.57 to 0.83; number needed to treat to prevent one death=139). A similar direction and magnitude of association was found in instrumental variable, propensity score matched, and survival analyses. Use of neuraxial anaesthesia compared with use of general anaesthesia was also associated with decreased in-hospital cardiopulmonary and renal complications (odds ratio 0.73, 0.63 to 0.85) and a reduced length of hospital stay (-0.5 days, -0.3 to-0.6 days).

CONCLUSIONS

Use of neuraxial anaesthesia compared with general anaesthesia for lower limb revascularisation surgery was associated with decreased 30 day mortality and hospital length of stay. These findings might have been related to reduced cardiopulmonary and renal complications after neuraxial anaesthesia and support the increased use of neuraxial anaesthesia in patients undergoing these surgeries until the results of a large, confirmatory randomised trial become available.

Hospital Length of Stay Reduction Over Time and Patient Readmission for Severe Adverse Events Following Surgery

OBJECTIVE

The aim of the study was to investigate whether patients who undergo surgery in hospitals experiencing significant length of stay (LOS) reductions over time are exposed to a higher risk of severe adverse events in the postoperative period.

SUMMARY BACKGROUND DATA

Surgical care innovation has encouraged hospitals to shorten LOS under financial pressures with uncertain impact on patient outcomes.

METHODS

We selected all patients who underwent elective colectomy or urgent hip fracture repair in French hospitals between 2013 and 2016. For each procedure, hospitals were categorized into 3 groups according to variations in their median LOS as follows: major decrease, moderate decrease, and no decrease. These groups were matched using propensity scores based on patients' and hospitals' potential confounders. Potentially avoidable readmission for severe adverse events and death at 6 months were compared between groups using Cox regressions.

RESULTS

We considered 98,713 patients in 540 hospitals for colectomy and 206,812 patients in 414 hospitals for hip fracture repair before matching. After colectomy, patient outcomes were not negatively impacted when hospitals reduced their LOS [hazard ratio (95% confidence interval): 0.93 (0.78-1.10)]. After hip fracture repair, patients in hospitals with major decreases in LOS had a higher risk of severe adverse events [1.22 (1.11-1.34)] and death [1.17 (1.04-1.32)].

CONCLUSIONS

Patients who underwent surgical procedures in hospitals experiencing major decreases in LOS were demonstrated worse postoperative outcomes after urgent hip fracture repair and not after elective colectomy. Development of care bundles to enhance recovery after emergency surgeries may allow better control of LOS reduction and patient outcomes.

Postoperative respiratory failure: An update on the validity of the Agency for Healthcare Research and Quality Patient Safety Indicator 11 in an era of clinical documentation improvement programs

BACKGROUND

Administrative data can be used to identify cases of postoperative respiratory failure (PRF). We aimed to determine if recent changes to the Agency for Healthcare Research and Quality Patient Safety Indicator 11 (PSI 11) and adoption of clinical documentation improvement programs have improved the validity of PSI 11. We also analyzed reasons why PSI 11 was falsely triggered.

STUDY DESIGN

Cross-sectional study of all eligible discharges using health record data from five academic medical centers between October 1, 2012 and September 30, 2015.

RESULTS

Of 437 flagged records, 434 (99.3%) were accurately coded and 414 (94.7%) represented true clinical PRF. None of the false positive records involved respiratory failure present on admission. Most (78.3%) false positive records required airway protection but did not have respiratory failure.

CONCLUSION

The validity of PSI 11 has improved with recent changes to the code criterion and adoption of clinical documentation improvement programs.

A Comparison of Surgical Quality and Patient Satisfaction Indicators Between VA Hospitals and Hospitals Near VA Hospitals

BACKGROUND

The Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act established a community care program allowing veterans to receive care outside Veteran Affairs Medical Centers (VAMCs). We sought to compare patient safety and satisfaction indicators from VAMCs and surrounding non-VAMCs (non-VAs).

METHODS

We identified VAMCs with at least one non-VA acute care hospital within 25 miles in three geographic regions (West/Southwest, New England, and Deep South). Children's, specialty, and critical access hospitals were excluded. Using publicly available Hospital Compare data, we analyzed VAMC and surrounding non-VA performance in postsurgical patient safety indicator (PSI) events and Hospital Consumer Assessment of Healthcare Providers and Systems patient satisfaction scores and hospital star ratings.

RESULTS

The 34 VAMCs performed better than 319 surrounding non-VAs in rates of wound dehiscence, accidental lacerations, and perioperative hemorrhage/hematoma as well as composite PSI rating (P < 0.05). VAMCs performed significantly better than non-VAs (18.0 versus 51.4 events per 1000 patients, P < 0.001) in composite surgery-specific PSIs. When comparing mean linear Hospital Consumer Assessment of Healthcare Providers and Systems score star ratings (1-5 scale), VAMCs had similar performance in overall hospital rating compared with non-VAs (3.28 versus 3.38, P = 0.48) and summary rating of hospital stays (2.87 versus 2.92, P = 0.69). When compiled patient satisfaction star ratings were compared, there was no difference (2.96 versus 2.97, P = 0.9). VAMCs performed worse than non-VAs in "would recommend" ratings (2.7 versus 3.13, P = 0.007).

CONCLUSIONS

Across disparate regions, VAMCs match or outperform neighboring non-VAs in surgical quality metrics and patient satisfaction ratings. Veterans receiving surgical care at VAMCs may receive equivalent or better care than at non-VAs.

Symptomatic human immunodeficiency virus-infected patients have poorer outcomes following emergency general surgery: A study of the nationwide inpatient sample

BACKGROUND

The impact of human immunodeficiency virus (HIV) infection on outcomes following common emergency general surgery procedures has not been evaluated since the widespread introduction of highly active antiretroviral therapy.

METHODS

A retrospective cohort study was conducted using the Nationwide Inpatient Sample. Records of patients who underwent laparoscopic or open appendectomy, cholecystectomy, or colon resection after emergency admission from 2004 to 2011 were obtained. Outcomes analyzed included in-hospital mortality, length of stay, total charges, and selected postoperative complications. Patients were divided among three groups, HIV-negative controls, asymptomatic HIV-positive patients, and symptomatic HIV/acquired immune deficiency syndrome (AIDS) patients. Data were analyzed using χ and multivariable regression with propensity score matching among the three groups, with p value less than 0.05 significant.

RESULTS

There were 974,588 patients identified, of which 1,489 were HIV-positive and 1,633 were HIV/AIDS-positive. The HIV/AIDS patients were more likely to die during their hospital stay than HIV-negative patients (4.4% vs. 1.6%, adjusted odds ratio, 3.53; 95% confidence interval [CI], 2.67-4.07; p < 0.001). The HIV/AIDS patients had longer hospital stays (7 days vs. 3 days; adjusted difference, 3.66 days; 95% CI, 3.53-4.00; p < 0.001) and higher median total charges than HIV-negative patients (US $47,714 vs. US $28,405; adjusted difference, US $15,264; 95% CI, US $13,905-US $16,623; p < 0.001). The HIV/AIDS patients also had significantly increased odds of certain postoperative complications, including sepsis, septic shock, pneumonia, urinary tract infection, acute renal failure and need for transfusion (p < 0.05 for each). Differences persisted irrespective of case complexity and over the study period. Asymptomatic HIV-positive patients had outcomes similar to HIV-negative patients.

CONCLUSION

The HIV/AIDS patients have a greater risk of death, infectious, and noninfectious complications after emergency surgery regardless of operative complexity and despite advanced highly active antiretroviral therapy. Patients who have not developed advanced disease, however, have similar outcomes to HIV-negative patients.

LEVEL OF EVIDENCE

Prognostic and epidemiologic study, level III.

Opioid Stewardship Program and Postoperative Adverse Events

Background

A 6-month opioid use educational program consisting of webinars on pain assessment, postoperative and multimodal pain opioid management, safer opioid use, and preventing addiction coupled with on-site coaching and monthly assessments reports was implemented in 31 hospitals. The authors hypothesized the intervention would measurably reduce and/or prevent opioid-related harm among adult hospitalized patients compared to 33 nonintervention hospitals.

Methods

Outcomes were extracted from medical records for 12 months before and after the intervention start date. Opioid adverse events, evaluated by opioid overdose, wrong substance given or taken in error, naloxone administration, and acute postoperative respiratory failure causing prolonged ventilation were the primary outcomes. Opioid use in adult patients undergoing elective hip or knee arthroplasty or colorectal procedures was also assessed. Differences-in-differences were compared between intervention and nonintervention hospitals.

Results

Before the intervention, the incidence ± SD of opioid overdose, wrong substance given, or substance taken in error was 1 ± 0.5 per 10,000 discharges, and naloxone use was 117 ± 13 per 10,000 patients receiving opioids. The incidence of respiratory failure was 42 ± 10 per 10,000 surgical discharges. A difference-in-differences of –0.2 (99% CI, –1.1 to 0.6, P = 0.499) per 10,000 in opioid overdose, wrong substance given, or substance taken in error and –13.6 (99% CI, –29.0 to 0.0, P = 0.028) per 10,000 in respiratory failure was observed postintervention in the intervention hospitals; however, naloxone administration increased by 15.2 (99% CI, 3.8 to 30.0, P = 0.011) per 10,000. Average total daily opioid use, as well as the fraction of patients receiving daily opioid greater than 90 mg morphine equivalents was not different between the intervention and nonintervention hospitals.

Conclusions

A 6-month opioid educational intervention did not reduce opioid adverse events or alter opioid use in hospitalized patients. The authors’ findings suggest that despite opioid and multimodal analgesia awareness, limited-duration educational interventions do not substantially change the hospital use of opioid analgesics.

Editor’s Perspective

What We Already Know about This Topic

What This Article Tells Us That Is New

Association between cancer-specific adverse event triggers and mortality: A validation study

Background

As there are few validated measures of patient safety in clinical oncology, creating an efficient measurement instrument would create significant value. Accordingly, we sought to assess the validity of a novel patient safety measure by examining the association of oncology‐specific triggers and mortality using administrative claims data.

Methods

We examined a retrospective cohort of 322 887 adult cancer patients enrolled in commercial or Medicare Advantage products for one year after an initial diagnosis of breast, colorectal, lung, or prostate cancer in 2008‐2014. We used diagnosis and procedure codes to calculate the prevalence of 16 cancer‐specific "triggers"–events that signify a potential adverse event. We compared one‐year mortality rates among patients with and without triggers by cancer type and metastatic status using logistic regression models.

Results

Trigger events affected 19% of patients and were most common among patients with metastatic colorectal (41%) and lung (50%) cancers. There was increased one‐year mortality among patients with triggers compared to patients without triggers across all cancer types in unadjusted and multivariate analyses. The increased mortality rate among patients with trigger events was particularly striking for nonmetastatic prostate cancer (1.3% vs 7.5%, adjusted odds ratio 1.96 [95% CI 1.49‐2.57]) and nonmetastatic colorectal cancer (4.1% vs 11.7%, 1.44 [1.19‐1.75]).

Conclusions

The association between adverse event triggers and poor survival among a cohort of cancer patients supports the validity of a cancer‐specific, administrative claims‐based trigger tool.

Are hospital nurse staffing practices associated with postoperative cardiac events and death? A systematic review

INTRODUCTION

Postoperative cardiac events are frequent complications of surgery, and their occurrence could be associated with suboptimal nurse staffing practices, but the existing evidence remains scattered. We systematically reviewed studies linking nurse staffing practices to postoperative cardiac events and two related outcomes, all-cause mortality and failure-to-rescue.

METHODS

A systematic search of the English/French literature was undertaken in the CINAHL, PsychInfo, and Medline databases. Studies were included if they: a) were published between 1996 and 2018; b) used a quantitative design; c) examined the association between at least one of seven staffing practices of interest (i.e., staffing levels, skill mix, work environment characteristics, levels of education and experience of the registered nurses, and overtime or temporary staff use) and postoperative cardiac events, mortality or failure-to-rescue; and d) were conducted among surgical patients. Data extraction, analysis, and synthesis, along with study methodological quality appraisal, were performed by two authors. High methodological heterogeneity precluded a formal meta-analysis.

RESULTS

Among 3,375 retrieved articles, 44 studies were included (39 cross-sectional, 3 longitudinal, 1 case-control, 1 interrupted time series). Existing evidence shows that higher nurse staffing levels, a higher proportion of registered nurses with an education at the baccalaureate degree level, and more supportive work environments are related to lower rates of both 30-day mortality and failure-to-rescue. Other staffing practices were less often studied and showed inconsistent associations with mortality or failure-to-rescue. Similarly, few studies (n = 10) examined the associations between nurse staffing practices and postoperative cardiac events and showed inconsistent results.

CONCLUSION

Higher nurse staffing levels, higher registered nurse education (baccalaureate degree level) and more supportive work environments were cross-sectionally associated with lower 30-day mortality and failure-to-rescue rates among surgical patients, but longitudinal studies are required to corroborate these associations. The existing evidence regarding postoperative cardiac events is limited, which warrants further investigation.

Identifying and Codifying Complications after Radical Cystectomy: Comparison of Administrative Diagnostic and Procedure Codes, and Clinical Chart Review

PURPOSE

To our knowledge the reliability of administrative claims codes to report postoperative radical cystectomy complications has not been examined. We compared complications identified by claims data to those abstracted from clinical chart review following radical cystectomy.

METHODS

We manually reviewed the charts of 268 patients treated with radical cystectomy between 2014 and 2016 for 30-day complications and queried administrative complication coding using 805 ICD-9/10 codes. Complications were categorized. Using Cohen κ statistics we assessed agreement between the 2 methods of complication reporting for 1 or more postoperative complications overall, categorical complications and complications stratified by the top quartile length of hospital stay and patients who were readmitted.

RESULTS

At least 1 or more complications were recorded in 122 patients (45.5%) through manual chart review and 80 (29.9%) were recorded via claim coding data with a concordance rate of κ=0.16, indicating weak agreement. Concordance was generally weak for categorical complication rates (range 0.05 to 0.36). However, when examining only the top length of stay quartile, 1 or more complications were reported in 32 patients (65%) by the manual chart review and in 12 (25%) via coding data with a concordance rate of κ=-0.2. Agreement was weak, similar to the total cohort.

CONCLUSIONS

Manual chart review and claim code identification of complications are not highly concordant even when stratified by patients with an extended length of stay, who are known to have more frequent complications. Researchers and administrators should be aware of these differences and exercise caution when interpreting complication reports.

Validation of new ICD-10-based patient safety indicators for identification of in-hospital complications in surgical patients: a study of diagnostic accuracy

OBJECTIVE

Administrative data systems are used to identify hospital-based patient safety events; few studies evaluate their accuracy. We assessed the accuracy of a new set of patient safety indicators (PSIs; designed to identify in hospital complications).

STUDY DESIGN

Prospectively defined analysis of registry data (1 April 2010-29 February 2016) in a Canadian hospital network. Assignment of complications was by two methods independently. The National Surgical Quality Improvement Programme (NSQIP) database was the clinical reference standard (primary outcome=any in-hospital NSQIP complication); PSI clusters were assigned using International Classification of Disease (ICD-10) codes in the discharge abstract. Our primary analysis assessed the accuracy of any PSI condition compared with any complication in the NSQIP; secondary analysis evaluated accuracy of complication-specific PSIs.

PATIENTS

All inpatient surgical cases captured in NSQIP data.

ANALYSIS

We assessed the accuracy of PSIs (with NSQIP as reference standard) using positive and negative predictive values (PPV/NPV), as well as positive and negative likelihood ratios (±LR).

RESULTS

We identified 12 898 linked episodes of care. Complications were identified by PSIs and NSQIP in 2415 (18.7%) and 2885 (22.4%) episodes, respectively. The presence of any PSI code had a PPV of 0.55 (95% CI 0.53 to 0.57) and NPV of 0.93 (95% CI 0.92 to 0.93); +LR 6.41 (95% CI 6.01 to 6.84) and -LR 0.40 (95% CI 0.37 to 0.42). Subgroup analyses (by surgery type and urgency) showed similar performance. Complication-specific PSIs had high NPVs (95% CI 0.92 to 0.99), but low to moderate PPVs (0.13-0.61).

CONCLUSION

Validation of the ICD-10 PSI system suggests applicability as a first screening step, integrated with data from other sources, to produce an adverse event detection pathway that informs learning healthcare systems. However, accuracy was insufficient to directly identify or rule out individual-level complications.

Postoperative wound dehiscence after laparotomy: a useful healthcare quality indicator? A cohort study based on Norwegian hospital administrative data

OBJECTIVES

Postoperative wound dehiscence (PWD) is a serious complication to laparotomy, leading to higher mortality, readmissions and cost. The aims of the present study are to investigate whether risk adjusted PWD rates could reliably differentiate between Norwegian hospitals, and whether PWD rates were associated with hospital characteristics such as hospital type and laparotomy volume.

DESIGN

Observational study using patient administrative data from all Norwegian hospitals, obtained from the Norwegian Patient Registry, for the period 2011-2015, and linked using the unique person identification number.

PARTICIPANTS

All patients undergoing laparotomy, aged at least 15 years, with length of stay at least 2 days and no diagnosis code for immunocompromised state or relating to pregnancy, childbirth and puerperium. The final data set comprised 66 925 patients with 78 086 laparotomy episodes from 47 hospitals.

OUTCOMES

The outcome was wound dehiscence, identified by the presence of a wound reclosure code, risk adjusted for patient characteristics and operation type.

RESULTS

The final data set comprised 1477 wound dehiscences. Crude PWD rates varied from 0% to 5.1% among hospitals, with an overall rate of 1.89%. Three hospitals with statistically significantly higher PWD than average were identified, after case mix adjustment and correction for multiple comparisons. Hospital volume was not associated with PWD rate, except that hospitals with very few laparotomies had lower PWD rates.

CONCLUSIONS

Among Norwegian hospitals, there is considerable variation in PWD rate that cannot be explained by operation type, age or comorbidity. This warrants further investigation into possible causes, such as surgical technique, perioperative procedures or handling of complications. The risk adjusted PWD rate after laparotomy is a candidate quality indicator for Norwegian hospitals.

Unexpected discrepancies in hospital administrative databases can impact the accuracy of monitoring thyroid surgery outcomes in France

Objective

To determine the validity of hospital administrative databases compared to prospective collection of medical data assessing thyroid surgery complications.

Background

Administrative data are increasingly used to track surgical outcomes.

Methods

All patients undergoing thyroid surgery at three French university hospitals between April 2008 and April 2009 were prospectively included. Using diagnosis and procedural codes from hospital administrative database, we designed three indicators for measuring complications of thyroid surgery: recurrent laryngeal nerve palsy, postoperative hypoparathyroidism, and postoperative hemorrhage. Gold standard was obtained from a prospective collection of medical data after systematically screening each patient for the above-mentioned complications. Their ability to monitor surgical outcomes over time within individual hospitals was estimated using control charts. Spatial comparison between hospitals was performed by funnel plots.

Results

A total of 1909 patients were included. Complication rates extracted from administrative data were significantly lower compared to medical data (nerve palsy 2.4% vs. 6.7%, hypoparathyroidism 10.6% vs. 22.3%, p<0.0001). Indicator sensitivity was 30.4% for nerve palsy, 45.4% for hypoparathyroidism and 71.4% for postoperative hemorrhage. Corresponding positive predictive values were 84.4%, 95.1% and 68.2%. In two of the three hospitals, administrative data were not able to track temporal variations in complications rates. Regarding inter-hospital comparisons, 2 out of 3 hospitals were considered outliers according to administrative data despite having an average performance based on medical data.

Conclusions

The ability of indicators extracted from administrative databases to measure thyroid surgery outcomes depends on the quality of underlying data coding. Validation in every center should be a prerequisite before implementing such metrics for tracking performance

Detection of Surgical Site Infection Utilizing Automated Feature Generation in Clinical Notes

Postsurgical complications (PSCs) are known as a deviation from the normal postsurgical course and categorized by severity and treatment requirements. Surgical site infection (SSI) is one of major PSCs and the most common healthcare-associated infection, resulting in increased length of hospital stay and cost. In this work, we proposed an automated way to generate keyword features using sublanguage analysis with heuristics to detect SSI from cohort in clinical notes and evaluated these keywords with medical experts. To further valid our approach, we also applied different machine learning algorithms on cohort using automatically generated keywords. The results showed that our approach was able to identify SSI keywords from clinical narratives and can be used as a foundation to develop an information extraction system or support search-based natural language processing (NLP) approaches by augmenting search queries.

Physician derived versus administrative data in identifying surgical complications. Fact versus Fiction

BACKGROUND

Administrative data are widely used as determinants of surgical quality. We compared surgical complications identified in a structured surgical review to coding and billing data of over a 19-month period.

METHODS

A retrospective review of monthly morbidity and mortality conference reports was compared to a report over the same time period generated from hospital coding and billing data.

RESULTS

807 sequential operative procedures were included. Physician derived data compared to administrative data identified a complication of any severity in 205 (25.4%) versus 111 (13.8%) cases (r = 0.39), and major complications in 68 (8.4%) versus 46 (5.7%) cases (r = 0.36). Review of the administrative data regarding major complications identified 80 false negatives, 52 false positives, and 38 true positive designations. Overall sensitivity, specificity, positive and negative predictive values, and accuracy for administrative data in identifying major complications was 0.32, 0.99, 0.42, 0.99, and 0.99.

CONCLUSIONS

The correlation between physician determined and administrative data with regard to identifying surgical complications is poor. Administrative data are insensitive and lack positive predictive value.

The Kaiser Permanente Northern California Enhanced Recovery After Surgery Program: Design, Development, and Implementation

Complications are common after surgery, highlighting the need for innovations that reduce postsurgical morbidity and mortality. In this report, we describe the design, development, and implementation of an Enhanced Recovery After Surgery program in the Kaiser Permanente Northern California integrated health care delivery system. This program was implemented and disseminated in 2014, targeting patients who underwent elective colorectal resection and those who underwent emergent hip fracture repair across 20 Medical Centers. The program leveraged multidisciplinary and broad-based leadership, high-quality data and analytic infrastructure, patient-centered education, and regional-local mentorship alignment. This program has already had an impact on more than 17,000 patients in Northern California. It is now in its fourth phase of planning and implementation, expanding Enhanced Recovery pathways to all surgical patients across Kaiser Permanente Northern California.

Daily review of AHRQ patient safety indicators has important impact on value-based purchasing, reimbursement, and performance scores

BACKGROUND

The Patient Safety Indicators (PSIs) Composite (PSI 90) of the Agency for Healthcare Research and Quality has been found to have low positive predictive values. Because scores can affect hospital reimbursement and ranking, our institution designed a review process to ensure accurate data and incur minimal penalties under the Hospital Value-Based Purchasing Program.

METHODS

A multidisciplinary team was assembled to review PSI 90 within a performance period. The positive predictive value of each PSI was calculated. Weight-adjusted PSI rates were used to recalculate the PSI 90 Performance Period Index Value (PPIV). The adjusted PPIV was used to estimate what the achievement points and financial impact would have been if PSI review had not been implemented. Differences in PPIV, achievement points, and financial impact before and after PSI review were calculated.

RESULTS

A total of 1,470 cases were flagged for PSI over a 2-year period. The positive predictive value was 63.3%. Refuting 36.7% of PSIs resulted in a decrease in the PPIV from 0.696 to 0.508, an increase in achievement points from 5 to 10, resulting in a decreased net loss of $111,773.

CONCLUSION

Multidisciplinary review processes are practical and effective in identifying false-positive patient safety events. The real-time process affects hospital performance and resultant Medicare reimbursement substantially.

Automated Extraction of VTE Events From Narrative Radiology Reports in Electronic Health Records: A Validation Study

BACKGROUND

Surveillance of venous thromboembolisms (VTEs) is necessary for improving patient safety in acute care hospitals, but current detection methods are inaccurate and inefficient. With the growing availability of clinical narratives in an electronic format, automated surveillance using natural language processing (NLP) techniques may represent a better method.

OBJECTIVE

We assessed the accuracy of using symbolic NLP for identifying the 2 clinical manifestations of VTE, deep vein thrombosis (DVT) and pulmonary embolism (PE), from narrative radiology reports.

METHODS

A random sample of 4000 narrative reports was selected among imaging studies that could diagnose DVT or PE, and that were performed between 2008 and 2012 in a university health network of 5 adult-care hospitals in Montreal (Canada). The reports were coded by clinical experts to identify positive and negative cases of DVT and PE, which served as the reference standard. Using data from the largest hospital (n=2788), 2 symbolic NLP classifiers were trained; one for DVT, the other for PE. The accuracy of these classifiers was tested on data from the other 4 hospitals (n=1212).

RESULTS

On manual review, 663 DVT-positive and 272 PE-positive reports were identified. In the testing dataset, the DVT classifier achieved 94% sensitivity (95% CI, 88%-97%), 96% specificity (95% CI, 94%-97%), and 73% positive predictive value (95% CI, 65%-80%), whereas the PE classifier achieved 94% sensitivity (95% CI, 89%-97%), 96% specificity (95% CI, 95%-97%), and 80% positive predictive value (95% CI, 73%-85%).

CONCLUSIONS

Symbolic NLP can accurately identify VTEs from narrative radiology reports. This method could facilitate VTE surveillance and the evaluation of preventive measures.

Enhanced Quality Measurement Event Detection: An Application to Physician Reporting

The wide-scale adoption of electronic health records (EHR)s has increased the availability of routinely collected clinical data in electronic form that can be used to improve the reporting of quality of care. However, the bulk of information in the EHR is in unstructured form (e.g., free-text clinical notes) and not amenable to automated reporting. Traditional methods are based on structured diagnostic and billing data that provide efficient, but inaccurate or incomplete summaries of actual or relevant care processes and patient outcomes. To assess the feasibility and benefit of implementing enhanced EHR- based physician quality measurement and reporting, which includes the analysis of unstructured free- text clinical notes, we conducted a retrospective study to compare traditional and enhanced approaches for reporting ten physician quality measures from multiple National Quality Strategy domains. We found that our enhanced approach enabled the calculation of five Physician Quality and Performance System measures not measureable in billing or diagnostic codes and resulted in over a five-fold increase in event at an average precision of 88 percent (95 percent CI: 83–93 percent). Our work suggests that enhanced EHR-based quality measurement can increase event detection for establishing value-based payment arrangements and can expedite quality reporting for physician practices, which are increasingly burdened by the process of manual chart review for quality reporting.

Geriatric Patient Safety Indicators Based on Linked Administrative Health Data to Assess Anticoagulant-Related Thromboembolic and Hemorrhagic Adverse Events in Older Inpatients: A Study Proposal

Background

Frail older people with multiple interacting conditions, polypharmacy, and complex care needs are particularly exposed to health care-related adverse events. Among these, anticoagulant-related thromboembolic and hemorrhagic events are particularly frequent and serious in older inpatients. The growing use of anticoagulants in this population and their substantial risk of toxicity and inefficacy have therefore become an important patient safety and public health concern worldwide. Anticoagulant-related adverse events and the quality of anticoagulation management should thus be routinely assessed to improve patient safety in vulnerable older inpatients.

Objective

This project aims to develop and validate a set of outcome and process indicators based on linked administrative health data (ie, insurance claims data linked to hospital discharge data) assessing older inpatient safety related to anticoagulation in both Switzerland and France, and enabling comparisons across time and among hospitals, health territories, and countries. Geriatric patient safety indicators (GPSIs) will assess anticoagulant-related adverse events. Geriatric quality indicators (GQIs) will evaluate the management of anticoagulants for the prevention and treatment of arterial or venous thromboembolism in older inpatients.

Methods

GPSIs will measure cumulative incidences of thromboembolic and bleeding adverse events based on hospital discharge data linked to insurance claims data. Using linked administrative health data will improve GPSI risk adjustment on patients’ conditions that are present at admission and will capture in-hospital and postdischarge adverse events. GQIs will estimate the proportion of index hospital stays resulting in recommended anticoagulation at discharge and up to various time frames based on the same electronic health data. The GPSI and GQI development and validation process will comprise 6 stages: (1) selection and specification of candidate indicators, (2) definition of administrative data-based algorithms, (3) empirical measurement of indicators using linked administrative health data, (4) validation of indicators, (5) analyses of geographic and temporal variations for reliable and valid indicators, and (6) data visualization.

Results

Study populations will consist of 166,670 Swiss and 5,902,037 French residents aged 65 years and older admitted to an acute care hospital at least once during the 2012-2014 period and insured for at least 1 year before admission and 1 year after discharge. We will extract Swiss data from the Helsana Group data warehouse and French data from the national health insurance information system (SNIIR-AM). The study has been approved by Swiss and French ethics committees and regulatory organizations for data protection.

Conclusions

Validated GPSIs and GQIs should help support and drive quality and safety improvement in older inpatients, inform health care stakeholders, and enable international comparisons. We discuss several limitations relating to the representativeness of study populations, accuracy of administrative health data, methods used for GPSI criterion validity assessment, and potential confounding bias in comparisons based on GQIs, and we address these limitations to strengthen study feasibility and validity.

Validity of the Agency for Health Care Research and Quality Patient Safety Indicators and the Centers for Medicare and Medicaid Hospital-acquired Conditions: A Systematic Review and Meta-Analysis

BACKGROUND

The Agency for Health Care Research and Quality Patient Safety Indicators (PSIs) and Centers for Medicare and Medicaid Services Hospital-acquired Conditions (HACs) are increasingly being used for pay-for-performance and public reporting despite concerns over their validity. Given the potential for these measures to misinform patients, misclassify hospitals, and misapply financial and reputational harm to hospitals, these need to be rigorously evaluated. We performed a systematic review and meta-analysis to assess PSI and HAC measure validity.

METHODS

We searched MEDLINE and the gray literature from January 1, 1990 through January 14, 2015 for studies that addressed the validity of the HAC measures and PSIs. Secondary outcomes included the effects of present on admission (POA) modifiers, and the most common reasons for discrepancies. We developed pooled results for measures evaluated by ≥3 studies. We propose a threshold of 80% for positive predictive value or sensitivity for pay-for-performance and public reporting suitability.

RESULTS

Only 5 measures, Iatrogenic Pneumothorax (PSI 6/HAC 17), Central Line-associated Bloodstream Infections (PSI 7), Postoperative hemorrhage/hematoma (PSI 9), Postoperative deep vein thrombosis/pulmonary embolus (PSI 12), and Accidental Puncture/Laceration (PSI 15), had sufficient data for pooled meta-analysis. Only PSI 15 (Accidental Puncture and Laceration) met our proposed threshold for validity (positive predictive value only) but this result was weakened by considerable heterogeneity. Coding errors were the most common reasons for discrepancies between medical record review and administrative databases. POA modifiers may improve the validity of some measures.

CONCLUSION

This systematic review finds that there is limited validity for the PSI and HAC measures when measured against the reference standard of a medical chart review. Their use, as they currently exist, for public reporting and pay-for-performance, should be publicly reevaluated in light of these findings.

[Comparison of the "Trigger" tool with the minimum basic data set for detecting adverse events in general surgery]

INTRODUCTION

Surgery is a high risk for the occurrence of adverse events (AE). The main objective of this study is to compare the effectiveness of the Trigger tool with the Hospital National Health System registration of Discharges, the minimum basic data set (MBDS), in detecting adverse events in patients admitted to General Surgery and undergoing surgery.

MATERIAL AND METHODS

Observational and descriptive retrospective study of patients admitted to general surgery of a tertiary hospital, and undergoing surgery in 2012. The identification of adverse events was made by reviewing the medical records, using an adaptation of "Global Trigger Tool" methodology, as well as the (MBDS) registered on the same patients. Once the AE were identified, they were classified according to damage and to the extent to which these could have been avoided. The area under the curve (ROC) were used to determine the discriminatory power of the tools. The Hanley and Mcneil test was used to compare both tools.

RESULTS

AE prevalence was 36.8%. The TT detected 89.9% of all AE, while the MBDS detected 28.48%. The TT provides more information on the nature and characteristics of the AE. The area under the curve was 0.89 for the TT and 0.66 for the MBDS. These differences were statistically significant (P<.001).

CONCLUSIONS

The Trigger tool detects three times more adverse events than the MBDS registry. The prevalence of adverse events in General Surgery is higher than that estimated in other studies.

Clinical validity and relevance of accidental puncture or laceration as a patient safety indicator for children

PURPOSE

Accidental puncture or laceration (APL) has been endorsed as a patient safety indicator and is being used to compare hospital performance and for reimbursement. We sought to determine the positive predictive value (PPV) of APL as a quality metric in a pediatric population.

METHODS

We retrospectively reviewed all cases that met APL administrative criteria over 5years in a quaternary pediatric hospital system. Events were categorized as false positive (FP) or true positive (TP). TP cases were further categorized as "potentially consequential" or "inconsequential". The PPV of APL was calculated, and a z-test was used to provide 95% confidence intervals.

RESULTS

Of the 238 cases identified, 204 were categorized as TP (86%; 95% CI: 80%-90%). Thirty-four of these events (17%) involved injuries that were considered "inconsequential". True events that required repair were identified as "potentially consequential" (n=170). Thus, the PPV of APL was 71% (95% CI: 65%-77%). Extenuating factors such as adhesive disease or abnormal anatomy were present in 39% of TP cases. Thirty-four cases (14%) were categorized as FP because no documented injury was found.

CONCLUSIONS

A large proportion of APL events are either false or clinically irrelevant, thus questioning its usability as a patient safety indicator for children undergoing surgery.

TYPE OF STUDY

Retrospective review.

LEVEL OF EVIDENCE

IV.

Current Quality Measurement Tools Are Insufficient to Assess Complications in Orthopedic Surgery

PURPOSE

The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) is a clinically-derived, validated tool to track outcomes in surgery. The Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) are a set of computer algorithms run on administrative data to identify adverse events. The purpose of this study is to compare complications following orthopedic surgery identified by ACS-NSQIP and AHRQ-PSI.

METHODS

Patients between 2010 and 2012 who underwent orthopedic procedures (arthroplasty, spine, trauma, foot and ankle, hand, and upper extremity) at our tertiary-care, academic institution were identified (n = 3,374). Identification of inpatient adverse events by AHRQ-PSI in the cohort was compared with 30-day events identified by ACS-NSQIP. Adverse events common to both AHRQ-PSI and ACS-NSQIP were infection, sepsis, venous thromboembolism, bleeding, respiratory failure, wound disruption, and renal failure. Concordance between AHRQ-PSI and ACS-NSQIP for identifying adverse events was examined.

RESULTS

A total of 729 adverse events (21.6%) were identified in the cohort using ACS-NSQIP methodology and 35 adverse events (1.0%) were found using AHRQ-PSI. Only 12 events were identified by both methodologies. The most common complication was bleeding in ACS-NSQIP (18.1%) and respiratory failure in AHRQ-PSI (0.53%). The overlap was highest for venous thromboembolic events. There was no overlap in adverse events for 5 of the 7 categories of adverse events.

CONCLUSIONS

A large discrepancy was observed between adverse events reported in ACS-NSQIP and AHRQ-PSI. A large percentage of clinically important adverse events identified in ACS-NSQIP were missed in AHRQ-PSI algorithms. The ability of AHRQ-PSI for detecting adverse events varied widely with ACS-NSQIP.

CLINICAL RELEVANCE

AHRQ-PSI algorithms currently are insufficient to assess the quality of orthopedic surgery.

Do daily ward interviews improve measurement of hospital quality and safety indicators? A prospective observational study

RATIONALE, AIMS AND OBJECTIVES

The aim of this study was to determine if the addition of daily ward interview data improves the capture of hospital quality and safety indicators compared with incident reporting systems alone. An additional aim was to determine the potential characteristics influencing under-reporting of hospital quality and safety indicators in incident reporting systems.

METHODS

A prospective, observational study was performed at two tertiary metropolitan public hospitals. Research assistants from allied health backgrounds met daily with the nurse in charge of the ward and discussed the occurrence of any falls, pressure injuries and rapid response medical team calls. Data were collected from four general medical wards, four surgical wards, an orthopaedic, neurosciences, plastics, respiratory, renal, sub-acute and acute medical assessment unit.

RESULTS

An estimated total of 303 falls, 221 pressure injuries and 884 rapid response medical team calls occurred between 15 wards across two hospitals, over a period of 6 months. Hospital incident reporting systems underestimated falls by 30.0%, pressure injuries by 59.3% and rapid response medical team calls by 17.0%. The use of ward interview data collection in addition to hospital incident reporting systems improved data capture of falls by 23.8% (n = 72), pressure injuries by 21.7% (n = 48) and rapid response medical team calls by 12.7% (n = 112). Falls events were significantly less likely to be reported if they occurred on a Monday (P = 0.04) and pressure injuries significantly more likely to be reported if they occurred on a Wednesday (P = 0.01).

CONCLUSIONS

Hospital quality and safety indicators (falls, pressure injuries and rapid response medical team calls) were under-reported in incident reporting systems, with variability in under-reporting between wards and the day of event occurrence. The use of ward interview data collection in addition to hospital incident reporting systems improved reporting of hospital quality and safety indicators.

Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator for Postoperative Respiratory Failure (PSI 11) does not identify accurately patients who received unsafe care

BACKGROUND

The Agency for Healthcare Research and Quality Patient Safety Indicator 11 is used to identify postoperative respiratory failure events and detect areas for quality improvement. This study examines the accuracy of Patient Safety Indicator 11 in identifying clinically valid patient safety events.

METHODS

All cases flagged for Patient Safety Indicator 11 from July 2013 to July 2015 by Agency for Healthcare Research and Quality QI Version 4.5 including International Classification of Diseases-9 codes were evaluated. Code-confirmed cases underwent independent review by 2 physicians. Inpatient electronic medical records were used to identify clinical factors for postoperative respiratory failure in each case to determine if postoperative respiratory failure was a result of unsafe care. The clinical true-positive rate and positive predictive value were calculated.

RESULTS

A total of 166 postoperative respiratory failure cases were reviewed; 51 were recoded and reversed due to coding or documentation errors; 115 cases met the Agency for Healthcare Research and Quality definition of postoperative respiratory failure. A total of 71 (61.7%) of the 115 cases were false positives and did not reflect unsafe care, while 44 cases were true positives with a positive predictive value of 38.3%. χ(2) analysis did not reveal an association between demographics, clinical characteristics, or operative procedure with true-positive cases.

CONCLUSION

Administrative coding data for Agency for Healthcare Research and Quality Patient Safety Indicator 11 do not identify accurately patients who received unsafe care when taking into account unpreventable clinical factors causing postoperative respiratory failure. The use of Agency for Healthcare Research and Quality Patient Safety Indicator 11 as a hospital performance measure should be reconsidered until inclusion and exclusion criteria are revised.

Quality indicators for pharmaceutical care: a comprehensive set with national scores for Dutch community pharmacies

Background The quality of pharmaceutical care in community pharmacies in the Netherlands has been assessed annually since 2008. The initial set has been further developed with pharmacists and patient organizations, the healthcare inspectorate, the government and health insurance companies. The set over 2012 was the first set of quality indicators for community pharmacies which was validated and supported by all major stakeholders. The aims of this study were to describe the validated set of quality indicators for community pharmacies and to report their scores over 2012. In subanalyses the score development over 5 years was described for those indicators, that have been surveyed before and remained unchanged. Methods Community pharmacists in the Netherlands were invited in 2013 to provide information for the set of 2012. Quality indicators were mapped by categories relevant for pharmaceutical care and defined for structures, processes and dispensing outcomes. Scores for categorically-measured quality indicators were presented as the percentage of pharmacies reporting the presence of a quality aspect. For numerical quality indicators, the mean of all reported scores was expressed. In subanalyses for those indicators that had been questioned previously, scores were collected from earlier measurements for pharmacies providing their scores in 2012. Multilevel analysis was used to assess the consistency of scores within one pharmacy over time by the intra-class correlation coefficient (ICC). Results For the set in 2012, 1739 Dutch community pharmacies (88 % of the total) provided information for 66 quality indicators in 10 categories. Indicator scores on the presence of quality structures showed relatively high quality levels. Scores for processes and dispensing outcomes were lower. Subanalyses showed that overall indicators scores improved within pharmacies, but this development differed between pharmacies. Conclusions A set of validated quality indicators provided insight into the quality of pharmaceutical care in the Netherlands. The quality of pharmaceutical care improved over time. As of 2012 quality structures were present in at least 80 % of the community pharmacies. Variation in scores on care processes and outcomes between individual pharmacies and over time can initiate future research to better understand and facilitate quality improvement in community pharmacies.

Using Both Clinical Registry and Administrative Claims Data to Measure Risk-adjusted Surgical Outcomes

OBJECTIVE

To examine the validity of hybrid quality measures that use both clinical registry and administrative claims data, capitalizing on the strengths of each data source.

BACKGROUND

Previous studies demonstrate substantial disagreement between clinical registry and administrative claims data on the occurrence of postoperative complications. Clinical data have greater validity than claims data for quality measurement but can be burdensome for hospitals to collect.

METHODS

American College of Surgeons National Surgical Quality Improvement Program records were linked to Medicare inpatient claims (2005-2008). National Quality Forum-endorsed risk-adjusted measures of 30-day postoperative complications or death assessed hospital quality for patients undergoing colectomy, lower extremity bypass, or all surgical procedures. Measures use hierarchical multivariable logistic regression to identify statistical outliers. Measures were applied using clinical data, claims data, or a hybrid of both data sources. Kappa statistics assessed agreement on determinations of hospital quality.

RESULTS

A total of 111,984 patients participated from 206 hospitals. Agreement on hospital quality between clinical and claims data was poor. Hybrid models using claims data to risk-adjust complications identified by clinical data had moderate agreement with all clinical data models, whereas hybrid models using clinical data to risk-adjust complications identified by claims data had routinely poor agreement with all clinical data models.

CONCLUSIONS

Assessments of hospital quality differ substantially when using clinical registry versus administrative claims data. A hybrid approach using claims data for risk adjustment and clinical data for complications may be a valid alternative with lower data collection burden. For quality measures focused on postoperative complications to be meaningful, such policies should require, at a minimum, collection of clinical outcomes data.

Patient safety in plastic surgery: identifying areas for quality improvement efforts

Improving quality of health care is a global priority. Before quality benchmarks are established, we first must understand rates of adverse events (AEs). This project assessed risk-adjusted rates of inpatient AEs for soft tissue reconstructive procedures.Patients receiving soft tissue reconstructive procedures from 2005 to 2010 were extracted from the Nationwide Inpatient Sample. Inpatient AEs were identified using patient safety indicators (PSIs), established measures developed by Agency for Healthcare Research and Quality.We identified 409,991 patients with soft tissue reconstruction and 16,635 (4.06%) had a PSI during their hospital stay. Patient safety indicators were associated with increased risk-adjusted mortality, longer length of stay, and decreased routine disposition (P < 0.01). Patient characteristics associated with a higher risk-adjusted rate per 1000 patients at risk included older age, men, nonwhite, and public payer (P < 0.05). Overall, plastic surgery patients had significantly lower risk-adjusted rate compared to other surgical inpatients for all events evaluated except for failure to rescue and postoperative hemorrhage or hematoma, which were not statistically different. Risk-adjusted rates of hematoma hemorrhage were significantly higher in patients receiving size-reduction surgery, and these rates were further accentuated when broken down by sex and payer. In general, plastic surgery patients had lower rates of in-hospital AEs than other surgical disciplines, but PSIs were not uncommon. With the establishment of national basal PSI rates in plastic surgery patients, benchmarks can be devised and target areas for quality improvement efforts identified. Further prospective studies should be designed to elucidate the drivers of AEs identified in this population.

Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review

OBJECTIVE

Measuring the incidence of healthcare-associated infections (HAI) is of increasing importance in current healthcare delivery systems. Administrative data algorithms, including (combinations of) diagnosis codes, are commonly used to determine the occurrence of HAI, either to support within-hospital surveillance programmes or as free-standing quality indicators. We conducted a systematic review evaluating the diagnostic accuracy of administrative data for the detection of HAI.

METHODS

Systematic search of Medline, Embase, CINAHL and Cochrane for relevant studies (1995-2013). Methodological quality assessment was performed using QUADAS-2 criteria; diagnostic accuracy estimates were stratified by HAI type and key study characteristics.

RESULTS

57 studies were included, the majority aiming to detect surgical site or bloodstream infections. Study designs were very diverse regarding the specification of their administrative data algorithm (code selections, follow-up) and definitions of HAI presence. One-third of studies had important methodological limitations including differential or incomplete HAI ascertainment or lack of blinding of assessors. Observed sensitivity and positive predictive values of administrative data algorithms for HAI detection were very heterogeneous and generally modest at best, both for within-hospital algorithms and for formal quality indicators; accuracy was particularly poor for the identification of device-associated HAI such as central line associated bloodstream infections. The large heterogeneity in study designs across the included studies precluded formal calculation of summary diagnostic accuracy estimates in most instances.

CONCLUSIONS

Administrative data had limited and highly variable accuracy for the detection of HAI, and their judicious use for internal surveillance efforts and external quality assessment is recommended. If hospitals and policymakers choose to rely on administrative data for HAI surveillance, continued improvements to existing algorithms and their robust validation are imperative.

Improved coding of postoperative deep vein thrombosis and pulmonary embolism in administrative data (AHRQ Patient Safety Indicator 12) after introduction of new ICD-9-CM diagnosis codes

BACKGROUND

Symptomatic venous thromboembolism is a common postoperative complication. The Agency for Healthcare Research and Quality (AHRQ) has developed a Patient Safety Indicator 12 to assist hospitals, payers, and other stakeholders to identify patients who experienced this complication.

OBJECTIVES

To determine whether newly created and recently redefined ICD-9-CM codes improved the criterion validity of Patient Safety Indicator 12, based on new samples of records dated after October 2009.

RESEARCH DESIGN, SUBJECTS, MEASURES

Two sources of data were used: (1) UHC retrospective case-control study of risk factors for acute symptomatic venous thromboembolism occurring within 90 days after total knee arthroplasty in teaching hospitals; (2) chart abstraction data by volunteer hospitals participating in the Validation Pilot Project of the AHRQ.

RESULTS

In the UHC sample, the positive predictive value (PPV) was 99% (125/126) and the negative predictive value was 99.4% (460/463). In the AHRQ sample, the overall PPV was 81% (126/156).

CONCLUSIONS

The PPV based on both samples shows substantial improvement compared with the previously reported PPVs of 43%-48%, suggesting that changes in ICD-9-CM code architecture and better coding guidance can improve the usefulness of coded data.

Diagnoses and clinical features associated with high risk for unplanned readmission in vascular surgery. A cohort study

Background

Readmission rate is an established health quality indicator. Preventable readmissions bear an unnecessary, high cost on the healthcare system. An analysis performed by the National Centre for Health Outcomes Development (NCHOD) has demonstrated an increasing trend in emergency readmissions in the UK. Vascular surgery has been reported to have high readmission rates second only to congestive heart failure. This study aims to identify diagnoses and other clinical risk factors for high unplanned readmission rates. This may be the first step to sparing both the health care system and patients of unnecessary readmissions.

Results

The overall 30 day readmission rate for Leeds Vascular Institute was 8.8%. The two diagnoses with the highest readmission rates were lower limb ischaemia and diabetic foot sepsis. The readmission rate for medical reasons was overwhelmingly higher than for surgical reasons (6.5% and 2.3% respectively). The most common medical diagnoses were renal disease and COPD. The majority of the patients readmitted under the care of vascular surgery required further surgical treatment.

Conclusion

Vascular units should focus on holistic and multidisciplinary treatment of lower limb ischaemia and diabetic foot sepsis, in order to prevent readmissions. Furthermore, the early involvement and input of physicians in the treatment of vascular patients with renal disease and COPD may be appropriate.

•

Lower limb ischaemia and diabetic foot sepsis are the two diagnoses with the highest readmission rate.

•

Vascular patients are more frequently readmitted for medical rather than surgical health problems.

•

For vascular patients, the most common medical, readmission diagnoses are infection renal disease complications and COPD exacerbation.

•

Most of the patients readmitted under vascular surgery necessitate further surgical treatment.

•

Diabetes may be an independent risk factor for readmission.

A quality indicator can be biased by intra-hospital heterogeneity: the case for quality of patient record keeping in France

BACKGROUND

Since 2008, French health institutions providing medical, surgical and obstetrical care are assessed on the basis of a set of quality indicators. The French National Authority for Health developed a survey design in which 80 records are randomly selected from each institution. The main aim was to assess the effects of internal heterogeneity of a hospital that comprises several units. The survey method is based on the hypothesis of intra-institution homogeneity, which overlooks the fact that in wide hospitals homogeneity is related to departments and thus leads to overall intra-hospital heterogeneity.

METHODS

Simulated databases were created to modelise the heterogeneity of our hospital and computed to assess the reliance of indicator measurement. We used real data from a large teaching hospital having internal heterogeneity related to each department.

RESULTS

Variance under heterogeneity was greater than under homogeneity (3- to 18-fold) leading to an increased size of the confidence interval (CI) (at 95%) from 9 (given Haute Autorité de Santé sources) to 22 (for greatest internal heterogeneity).

CONCLUSIONS

The variations in a quality indicator can be explained by intra-institution heterogeneity and are not related to changes in the quality policy of the hospitals and may lead to errors in terms of pay for performance.