Validation of the rates of adverse event incidence in administrative healthcare data through patient chart review: A scoping review protocol

Background

Patient safety is a key issue for health systems and a growing global public health challenge. Administrative healthcare data provide a coded summary of a patient and their encounter with the healthcare system. These aggregated datasets are often used to inform research and decisions relating to health service planning and therefore it is vital that they are accurate and reliable. Given the reported inaccuracy of these datasets for detecting and recording adverse events, there have been calls for validation studies to explore their reliability and investigate further their potential to inform research and health policy. Researchers have since carried out validation studies on the rates of adverse events in administrative data through chart reviews therefore, it seems appropriate to identify and chart the evidence and results of these studies within a scoping review.

Methods

The scoping review will be conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews. A search of databases such as PubMed, CINAHL, ScienceDirect and Scopus will be conducted in addition to a search of the reference lists of sourced publications and a search for grey literature. Following this, Covidence will be used to screen the sourced publications and subsequently extract data from the included sources. A numerical summary of the literature will be presented in addition to a charting based on the qualitative content analysis of the studies included.

Conclusions

This protocol provides the structure for the conduct of a review to identify and chart the evidence on validation studies on rates of adverse events in administrative healthcare data. This review will aim to identify research gaps, chart the evidence of and highlight any flaws within administrative datasets to improve extraction and coding practices and enable researchers and policy makers to use these data to their full potential.

Assessing the accuracy gap in early postoperative complication surveillance: ICD-10 codes versus manual curation-clinical and economic implications

BACKGROUND AND OBJECTIVE

Assessing surgical outcomes, notably postoperative complications (PCs), is crucial for healthcare systems. However, reliance on International Classification of Diseases, 10th revision (ICD-10) codes, may be suboptimal. This study aims to compare the accuracy of ICD-10 codes against manual curation of electronic healthcare records (EHRs) for identifying 13 individual PCs and evaluate associated resource utilization.

METHODS

EHR data from 11,827 surgical cases across 18 Danish hospitals in November 2021 were analyzed. PCs were identified and extracted through both manual curation and ICD-10 codes. Outcomes such as readmission, admission days, intensive care unit (ICU) stays, reoperations, and radiology procedures were assessed as proxies for resource consumption. Statistical and economic analyses quantified resource utilization and associated costs.

RESULTS

In total, 1047 PCs were found through manual curation and 439 PCs were found through ICD-10 codings. Only 218 of the PCs found through ICD-10 codes were retrieved during manual curation-corresponding to a correct ICD-10 coding of 20.8% of PCs. Patients with PCs experienced significantly higher resource utilization, including a 6.6 times higher readmission rate, 6 additional admission days, 2 extra ICU days, 7.7 times more reoperations. PCs incurred substantial economic costs, with additional admission days alone accounting for €25.5 million annually, over four times higher than estimates from ICD-10 codes.

CONCLUSIONS

ICD-10 codes inadequately capture early PCs highlighting the need for improved detection strategies. The actual costs associated with PCs far exceed current estimates, emphasizing the necessity for enhanced monitoring for informed decision-making. In the Danish healthcare system, ICD-10 codes only capture approximately 21% of PCs, making it inadequate for surgical quality monitoring. The actual costs related to PCs, based on study assumptions, are more than four times higher than estimated from current standard. This calls for novel strategies for PC detection to improve healthcare as well as political and financial decision-making.

Feasibility of Capturing Adverse Events From Insurance Claims Data Using International Classification of Diseases, Tenth Revision, Codes Coupled to Present on Admission Indicators

OBJECTIVE

The aim of the study was to investigate the feasibility of using administrative data to screen adverse events in Korea.

METHODS

We used a diagnosis-related groups claims data set and the information of the checklist of healthcare quality improvement (a part of the value incentive program) to verify adverse events in fiscal year 2018. Adverse events were identified using patient safety indicator (PSI) clusters and a present on admission indicator (POA). The PSIs consisted of 19 clusters representing subcategories of adverse events, such as hospital-acquired infection. Among the adverse events identified using PSI clusters, "POA = N," which means not present at the time of admission, was only deemed as the case in the final stage. We compared the agreement on the occurrence of adverse events from claims data with a reference standard data set (i.e., checklist of healthcare quality improvement) and presented them by PSI cluster and institution.

RESULTS

The cases of global PSI for any adverse event numbered 27,320 (2.32%) among all diagnostic codes in 2018. In terms of institutional distribution, considerable variation was observed throughout the clusters. For example, only 13.2% of institutions (n = 387) reported any global PSI for any adverse event throughout the whole year. The agreement between the reference standard and the claims data was poor, in the range of 2.2% to 10.8%, in 3 types of adverse events. The current claims data system (i.e., diagnostic codes coupled to POA indicators) failed to capture a large majority of adverse events identified using the reference standard.

CONCLUSIONS

Our results imply that the coding status of International Classification of Diseases, Tenth Revision, codes and POA indicators should be refined before using them as quality indicators.

Validity of Acute Cardiovascular Outcome Diagnoses Recorded in European Electronic Health Records: A Systematic Review

BACKGROUND

Electronic health records are widely used in cardiovascular disease research. We appraised the validity of stroke, acute coronary syndrome and heart failure diagnoses in studies conducted using European electronic health records.

METHODS

Using a prespecified strategy, we systematically searched seven databases from dates of inception to April 2019. Two reviewers independently completed study selection, followed by partial parallel data extraction and risk of bias assessment. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value estimates were narratively synthesized and heterogeneity between sensitivity and PPV estimates were assessed using I2.

RESULTS

We identified 81 studies, of which 20 validated heart failure diagnoses, 31 validated acute coronary syndrome diagnoses with 29 specifically recording estimates for myocardial infarction, and 41 validated stroke diagnoses. Few studies reported specificity or negative predictive value estimates. Sensitivity was ≤66% in all but one heart failure study, ≥80% for 91% of myocardial infarction studies, and ≥70% for 73% of stroke studies. PPV was ≥80% in 74% of heart failure, 88% of myocardial infarction, and 70% of stroke studies. PPV by stroke subtype was variable, at ≥80% for 80% of ischaemic stroke but only 44% of haemorrhagic stroke. There was considerable heterogeneity (I2 >75%) between sensitivity and PPV estimates for all diagnoses.

CONCLUSION

Overall, European electronic health record stroke, acute coronary syndrome and heart failure diagnoses are accurate for use in research, although validity estimates for heart failure and individual stroke subtypes were lower. Where possible, researchers should validate data before use or carefully interpret the results of previous validation studies for their own study purposes.

Are hospital nurse staffing practices associated with postoperative cardiac events and death? A systematic review

INTRODUCTION

Postoperative cardiac events are frequent complications of surgery, and their occurrence could be associated with suboptimal nurse staffing practices, but the existing evidence remains scattered. We systematically reviewed studies linking nurse staffing practices to postoperative cardiac events and two related outcomes, all-cause mortality and failure-to-rescue.

METHODS

A systematic search of the English/French literature was undertaken in the CINAHL, PsychInfo, and Medline databases. Studies were included if they: a) were published between 1996 and 2018; b) used a quantitative design; c) examined the association between at least one of seven staffing practices of interest (i.e., staffing levels, skill mix, work environment characteristics, levels of education and experience of the registered nurses, and overtime or temporary staff use) and postoperative cardiac events, mortality or failure-to-rescue; and d) were conducted among surgical patients. Data extraction, analysis, and synthesis, along with study methodological quality appraisal, were performed by two authors. High methodological heterogeneity precluded a formal meta-analysis.

RESULTS

Among 3,375 retrieved articles, 44 studies were included (39 cross-sectional, 3 longitudinal, 1 case-control, 1 interrupted time series). Existing evidence shows that higher nurse staffing levels, a higher proportion of registered nurses with an education at the baccalaureate degree level, and more supportive work environments are related to lower rates of both 30-day mortality and failure-to-rescue. Other staffing practices were less often studied and showed inconsistent associations with mortality or failure-to-rescue. Similarly, few studies (n = 10) examined the associations between nurse staffing practices and postoperative cardiac events and showed inconsistent results.

CONCLUSION

Higher nurse staffing levels, higher registered nurse education (baccalaureate degree level) and more supportive work environments were cross-sectionally associated with lower 30-day mortality and failure-to-rescue rates among surgical patients, but longitudinal studies are required to corroborate these associations. The existing evidence regarding postoperative cardiac events is limited, which warrants further investigation.

Sensitivity and specificity of breast cancer ICD-9-CM codes in three Italian administrative healthcare databases: a diagnostic accuracy study

OBJECTIVES

To assess the accuracy of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes in identifying patients diagnosed with incident carcinoma in situ and invasive breast cancer in three Italian administrative databases.

DESIGN

A diagnostic accuracy study comparing ICD-9-CM codes for carcinoma in situ (233.0) and for invasive breast cancer (174.x) with medical chart (as a reference standard). Case definition: (1) presence of a primary nodular lesion in the breast and (2) cytological or histological documentation of cancer from a primary or metastatic site.

SETTING

Administrative databases from Umbria Region, Azienda Sanitaria Locale (ASL) Napoli 3 Sud (NA) and Friuli VeneziaGiulia (FVG) Region.

PARTICIPANTS

Women with breast carcinoma in situ (n=246) or invasive breast cancer (n=384) diagnosed (in primary position) between 2012 and 2014.

OUTCOME MEASURES

Sensitivity and specificity for codes 233.0 and 174.x.

RESULTS

For invasive breast cancer the sensitivities were 98% (95% CI 93% to 99%) for Umbria, 96% (95% CI 91% to 99%) for NA and 100% (95% CI 97% to 100%) for FVG. Specificities were 90% (95% CI 82% to 95%) for Umbria, 91% (95% CI 83% to 96%) for NA and 91% (95% CI 84% to 96%) for FVG.For carcinoma in situ the sensitivities were 100% (95% CI 93% to 100%) for Umbria, 100% (95% CI 95% to 100%) for NA and 100% (95% CI 96% to 100%) for FVG. Specificities were 98% (95% CI 93% to 100%) for Umbria, 86% (95% CI 78% to 92%) for NA and 90% (95% CI 82% to 95%) for FVG.

CONCLUSIONS

Administrative healthcare databases from Umbria, NA and FVG are accurate in identifying hospitalised news cases of carcinoma of the breast. The proposed case definition is a powerful tool to perform research on large populations of newly diagnosed patients with breast cancer.

Adverse events related to hospital care: a retrospective medical records review in a Swiss hospital

Objective

Retrospective records reviews carried out in several countries have shown substantial rates of adverse events (AE) among hospitalized patients, preventable in half the cases. As no such data have been recorded in Switzerland, we estimated the incidence of AE in one acute care hospital as a basis for a safety improvement program.

Design

A two steps retrospective records review (screening criteria and full review of positively screened records).

Setting

A medium size community hospital.

Participants

A stratified sample of 400 surgical and 600 medical hospitalizations whose records fulfilled a set of information quality criteria.

Intervention(s)

Not applicable.

Main outcome measure(s)

Adverse events, preventable adverse events and extent of resulting harm.

Results

The proportion of hospitalizations with at least one AE was 12.3% (95% CI: 10.4-14.1) whereas the overall hospital incidence rate was 14.1% (95% CI: 12.0-16.2). Nearly half of AE were judged preventable, corresponding to one or more preventable AE in 6.4% of hospitalizations (95% CI: 5.0-7.8). Sixty percent of AE resulted in no or minor impairment at discharge whereas 23% resulted in severe disability. AE were twice more frequent in surgical patients, and preventable AE resulted more often in severe impairment than unpreventable AE. No death was attributed to an AE. The proportion of stays with an AE increased with age and length of stay.

Conclusions

The incidence of preventable AE in patients hospitalized in one Swiss hospital is comparable to previously reported rates. Further, patient safety improvement is needed, especially among older patients, and for surgical procedures.

Assessing the Health-Care Risk: The Clinical-VaR, a Key Indicator for Sound Management

Clinical risk includes any undesirable situation or operational factor that may have negative consequences for patient safety or capable of causing an adverse event (AE). The AE, intentional or unintentionally, may be related to the human factor, that is, medical errors (MEs). Therefore, the importance of the health-care risk management is a current and relevant issue on the agenda of many public and private institutions. The objective of the management has been evolving from the identification of AE to the assessment of cost-effective and efficient measures that improve the quality control through monitoring. Consequently, the goal of this paper is to propose a Key Risk Indicator (KRI) that enhances the advancement of the health-care management system. Thus, the application of the Value at Risk (VaR) concept in combination to the Loss Distribution Approach (LDA) is proved to be a proactive tool, within the frame of balanced scorecard (BSC), in health organizations. For this purpose, the historical events recorded in the Algo-OpData^® database (Algorithmics Inc., Toronto, ON, Canada, IBM, Armonk, NY, USA) have been used. The analysis highlights the importance of risk in the financials outcomes of the sector. The results of paper show the usefulness of the Clinical-VaR to identify and monitor the risk and sustainability of the implemented controls.

Intra-hospital correlations among 30-day mortality rates in 18 different clinical and surgical settings

OBJECTIVE

To examine whether a correlation exists in hospitals among 30-day mortality rates for different types of hospitalizations.

DESIGN

Cross-sectional study of hospital care based on publically available Italian data from the National Outcome Evaluation Program Edition 2015 of the Italian Agency for Regional Health Services.

SETTING AND PARTICIPANTS

Patients hospitalized with a diagnosis of congestive heart failure, acute myocardial infarction, chronic renal failure, chronic obstructive pulmonary disease exacerbation, femoral neck fracture, ischemic stroke and non-variceal upper gastrointestinal bleeding, or those who underwent isolated cardiac valve procedure, isolated coronary artery bypass graft surgery, non-ruptured abdominal aortic aneurysm repair and interventions for the following tumors: colon, kidney, brain, lung, stomach, rectal, liver or pancreatic cancer.

MAIN OUTCOME MEASURES

Condition-specific 30-day crude and risk-adjusted mortality rates.

RESULTS

A total of 808 280 admissions were reported from 844 institutions (median of 4 conditions evaluated per hospital; interquartile range 2-8). Volumes and outcome varied by clinical and surgical conditions across hospitals. Out of 153 pairs of different conditions, 41 were statistically significant in terms of concordance with crude mortality rates and 44 for their adjusted values. The hospital mean percentile rank for 30-day mortality, a composite measure that summarized the multiple indicators, increased significantly alongside number of conditions per hospital with a significant reduction of mortality when most of the studied conditions were treated in the same hospital.

CONCLUSIONS

The variability in 30-day mortality rates at hospital level and the correlation between risk mortality rates suggest that there may be common hospital-wide factors influencing short-term mortality.