International Classification of Diseases, 9th Revision, Clinical Modification codes in discharge abstracts are poor measures of complication occurrence in medical inpatients

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data

Patient safety is a key quality issue for health systems. Healthcare acquired adverse events (AEs) compromise safety and quality; therefore, their reporting and monitoring is a patient safety priority. Although administrative datasets are potentially efficient tools for monitoring rates of AEs, concerns remain over the accuracy of their data. Chart review validation studies are required to explore the potential of administrative data to inform research and health policy. This review aims to present an overview of the methodological approaches and strategies used to validate rates of AEs in administrative data through chart review. This review was conducted in line with the Joanna Briggs Institute methodological framework for scoping reviews. Through database searches, 1054 sources were identified, imported into Covidence, and screened against the inclusion criteria. Articles that validated rates of AEs in administrative data through chart review were included. Data were extracted, exported to Microsoft Excel, arranged into a charting table, and presented in a tabular and descriptive format. Fifty-six studies were included. Most sources reported on surgical AEs; however, other medical specialties were also explored. Chart reviews were used in all studies; however, few agreed on terminology for the study design. Various methodological approaches and sampling strategies were used. Some studies used the Global Trigger Tool, a two-stage chart review method, whilst others used alternative single-, two-stage, or unclear approaches. The sources used samples of flagged charts (n = 24), flagged and random charts (n = 11), and random charts (n = 21). Most studies reported poor or moderate accuracy of AE rates. Some studies reported good accuracy of AE recording which highlights the potential of using administrative data for research purposes. This review highlights the potential for administrative data to provide information on AE rates and improve patient safety and healthcare quality. Nonetheless, further work is warranted to ensure that administrative data are accurate. The variation of methodological approaches taken, and sampling techniques used demonstrate a lack of consensus on best practice; therefore, further clarity and consensus are necessary to develop a more systematic approach to chart reviewing.

Medications and conditions associated with weight loss in patients prescribed semaglutide based on real-world data

OBJECTIVE

Approved by the Food and Drug Administration (FDA) in 2017 for diabetes and in 2021 for weight loss, semaglutide has seen widespread use among individuals who aim to lose weight. The aim of this study was to evaluate weight loss and the influence of clinical factors on semaglutide patients in real-world clinical practice.

METHODS

Using data from 10 health systems within the Greater Plains Collaborative (a PCORnet Clinical Research Network), nearly 4000 clinical factors encompassing demographic, diagnosis, and prescription information were extracted for semaglutide patients. A gradient-boosting, machine-learning classifier was developed for weight-loss prediction and identification of the most impactful factors via SHapley Additive exPlanations (SHAP) value extrapolation.

RESULTS

A total of 3555 eligible patients (539 of whom were observed 52 weeks following exposure) from March 2017 to April 2022 were studied. On average, individuals lost 4.44% (male individuals, 3.66%; female individuals, 5.08%) of their initial weight. History of diabetes mellitus diagnosis was associated with less weight loss, whereas prediabetes and linaclotide use were associated with more pronounced weight loss.

CONCLUSIONS

Weight loss in patients prescribed semaglutide from real-world evidence was strong but attenuated compared with previous clinical trials. Machine-learning analysis of electronic health record data identified factors that warrant further research and consideration when tailoring weight-loss therapy.

Association between Pre-Existing Sleep Disorders and Survival Rates of Patients with Breast Cancer

PURPOSE: To investigate the effects of pre-existing sleep disorders on the survival outcomes of women receiving standard treatments for breast invasive ductal carcinoma (IDC). METHODS: We recruited patients from the Taiwan Cancer Registry Database who had received surgery for clinical stage I–III breast IDC. The Cox proportional hazards model was used to analyze all-cause mortality. We categorized the patients into those with and without sleep disorders (Groups 1 and 2, respectively) through propensity score matching. RESULTS: In the multivariate Cox regression analysis, the adjusted hazard ratio for all-cause mortality for Group 1 compared with Group 2 was 1.51 (95% confidence interval: 1.19, 1.91; p < 0.001). CONCLUSION: Our study demonstrated that the sleep disorder group had poorer survival rates than the non-sleep disorder group in breast cancer. Therefore, patients should be screened and evaluated for pre-existing sleep disorders prior to breast surgery, with such disorders serving as a predictor of survival in patients with breast cancer. Future studies may investigate the survival benefits of pharmacological and behavioral treatments for sleep problems in patients with breast cancer.

Race, ethnicity, and other sociodemographic characteristics of patients with hospital admission for migraine in the United States

BACKGROUND

Despite the growing awareness across the general population, migraine is often underdiagnosed and undertreated in socially and economically marginalized groups. The present study aimed to investigate the differential effects of race and income on other sociodemographic data and hospital length of stay in patients admitted to hospital with a primary diagnosis of migraine headache.

METHODS

We utilized the Nationwide Inpatient Sample (NIS) database to identify patients admitted to the hospital from 2004 to 2017 with primary diagnosis of migraine. Information on demographic and length of stay data was obtained. Only patients older than 18 years were selected and age outliers were excluded. Race groups were identified as "White", "Black", "Asian or Pacific Islander", "Native American", or "Other ethnic group", as originally reported in the NIS database. Income was identified as the estimated median household income of residents in the patient's ZIP Code.

RESULTS

A total of 106,761,737 valid cases were identified. After applying our case inclusion criteria, only 61453 (median age= 42 years, range= 18-78 years) were included. Patients identified as "Black", "Hispanic" or "Native Americans" were more likely to have lower household income (p < 0.001), whereas higher income was found for the patients identified as "White"", even when men and women were considered separately (p < 0.001). No effects of race and/or household income was found on the length of stay in hospital.

IMPLICATIONS

The occurrence of migraine diagnosis on hospital admission in the USA can be impacted by dramatic culturally driven patient-clinician communication differences between ethnic groups.

GASP: Graph-based Approximate Sequential Pattern Mining for Electronic Health Records

Sequential pattern mining can be used to extract meaningful sequences from electronic health records. However, conventional sequential pattern mining algorithms that discover all frequent sequential patterns can incur a high computational and be susceptible to noise in the observations. Approximate sequential pattern mining techniques have been introduced to address these shortcomings yet, existing approximate methods fail to reflect the true frequent sequential patterns or only target single-item event sequences. Multi-item event sequences are prominent in healthcare as a patient can have multiple interventions for a single visit. To alleviate these issues, we propose GASP, a graph-based approximate sequential pattern mining, that discovers frequent patterns for multi-item event sequences. Our approach compresses the sequential information into a concise graph structure which has computational benefits. The empirical results on two healthcare datasets suggest that GASP outperforms existing approximate models by improving recoverability and extracts better predictive patterns.

Detection of adverse drug events in e-prescribing and administrative health data: a validation study

BACKGROUND

Administrative health data are increasingly used to detect adverse drug events (ADEs). However, the few studies evaluating diagnostic codes for ADE detection demonstrated low sensitivity, likely due to narrow code sets, physician under-recognition of ADEs, and underreporting in administrative data. The objective of this study was to determine if combining an expanded ICD code set in administrative data with e-prescribing data improves ADE detection.

METHODS

We conducted a prospective cohort study among patients newly prescribed antidepressant or antihypertensive medication in primary care and followed for 2 months. Gold standard ADEs were defined as patient-reported symptoms adjudicated as medication-related by a clinical expert. Potential ADEs in administrative data were defined as physician, ED, or hospital visits during follow-up for known adverse effects of the study medication, as identified by ICD codes. Potential ADEs in e-prescribing data were defined as study drug discontinuations or dose changes made during follow-up for safety or effectiveness reasons.

RESULTS

Of 688 study participants, 445 (64.7%) were female and mean age was 64.2 (SD 13.9). The study drug for 386 (56.1%) patients was an antihypertensive, and for 302 (43.9%) an antidepressant. Using the gold standard definition, 114 (16.6%) patients experienced an ADE, with 40 (10.4%) among antihypertensive users and 74 (24.5%) among antidepressant users. The sensitivity of the expanded ICD code set was 7.0%, of e-prescribing data 9.7%, and of the two combined 14.0%. Specificities were high (86.0-95.0%). The sensitivity of the combined approach increased to 25.8% when analysis was restricted to the 27% of patients who indicated having reported symptoms to a physician.

CONCLUSION

Combining an expanded diagnostic code set with e-prescribing data improves ADE detection. As few patients report symptoms to their physician, higher detection rates may be achieved by collecting patient-reported outcomes via emerging digital technologies such as patient portals and mHealth applications.

A Review on the Application and Limitations of Administrative Health Care Data for the Study of Acute Kidney Injury Epidemiology and Outcomes in Children

Administrative health care databases contain valuable patient information generated by health care encounters. These "big data" repositories have been increasingly used in epidemiological health research internationally in recent years as they are easily accessible and cost-efficient and cover large populations for long periods. Despite these beneficial characteristics, it is also important to consider the limitations that administrative health research presents, such as issues related to data incompleteness and the limited sensitivity of the variables. These barriers potentially lead to unwanted biases and pose threats to the validity of the research being conducted. In this review, we discuss the effectiveness of health administrative data in understanding the epidemiology of and outcomes after acute kidney injury (AKI) among adults and children. In addition, we describe various validation studies of AKI diagnostic or procedural codes among adults and children. These studies reveal challenges of AKI research using administrative data and the lack of this type of research in children and other subpopulations. Additional pediatric-specific validation studies of administrative health data are needed to promote higher volume and increased validity of this type of research in pediatric AKI, to elucidate the large-scale epidemiology and patient and health systems impacts of AKI in children, and to devise and monitor programs to improve clinical outcomes and process of care.

The Costs of Hepatitis C by Liver Disease Stage: Estimates from the Veterans Health Administration

BACKGROUND

The release of highly effective but costly medications for the treatment of hepatitis C virus combined with a doubling in the incidence of hepatitis C virus have posed substantial financial challenges for many healthcare systems. We provide estimates of the cost of treating patients with hepatitis C virus that can inform the triage of pharmaceutical care in systems with limited healthcare resources.

METHODS

We conducted an observational study using a national US cohort of 206,090 veterans with laboratory-identified hepatitis C virus followed from Fiscal Year 2010 to 2014. We estimated the cost of: non-advanced Fibrosis-4; advanced Fibrosis-4; hepatocellular carcinoma; liver transplant; and post-liver transplant. The former two stages were ascertained using laboratory result data; the latter stages were ascertained using administrative data. Costs were obtained from the Veterans Health Administration's activity-based cost accounting system and more closely represent the actual costs of providing care, an improvement on the charge data that generally characterizes the hepatitis C virus cost literature. Generalized estimating equations were used to estimate and predict costs per liver disease stage. Missing data were multiply imputed.

RESULTS

Annual costs of care increased as patients progressed from non-advanced Fibrosis-4 to advanced Fibrosis-4, hepatocellular carcinoma, and liver transplant (all p < 0.001). Post-liver transplant, costs decreased significantly (p < 0.001). In simulations, patients were estimated to incur the following annual costs: US $17,556 for non-advanced Fibrosis-4; US $20,791 for advanced Fibrosis-4; US $46,089 for liver cancer; US $261,959 in the year of the liver transplant; and US $18,643 per year after the liver transplant.

CONCLUSIONS

Cost differences of treating non-advanced and advanced Fibrosis-4 are relatively small. The greatest cost savings would be realized from avoiding progression to liver cancer and transplant.

Delirium in Postoperative Patients Admitted to the Intensive Care Unit

Postoperative delirium (POD) is common among surgical patients admitted to the intensive care unit (ICU) and is associated with increased resource utilization, morbidity, and death. Our primary aim was to compare rates of POD using administrative International Classification of Diseases, Ninth Revision, records and automated interrogation of electronic health records from Confusion Assessment Method for the ICU (CAM-ICU) screening. The secondary aim was to assess POD risk associated with patient and perioperative characteristics. Electronic health records of surgical patients admitted to the ICU during 2011 through 2014 were abstracted for POD assessment by CAM-ICU and by administrative codes, Charlson comorbidity index, surgical characteristics, and Acute Physiology, Age, Chronic Health Evaluation III scores. Of 6338 patients, CAM-ICU identified 606 (9.6%) and administrative records identified 55 (0.9%) POD cases, with agreement on 50 cases. In multivariable logistic regression based on POD identified with CAM-ICU, preexisting dementia had the strongest association with POD (odds ratio [95% confidence interval], 6.47 [3.68–11.37]; P < 0.001). Other associations found were older age, congestive heart failure, chronic pulmonary disease, increased surgical duration, emergency cases, blood transfusions, postoperative ventilation, and higher Acute Physiology, Age, Chronic Health Evaluation III scores (all P ≤ 0.01). POD cases had lengthier ICU and hospital stays and a higher mortality rate (all P < 0.001). CAM-ICU scores identified higher rates of POD than a search for POD based on administrative codes. Preoperative presence of dementia and major comorbidities were associated with POD. Delirium in surgical patients is associated with worse outcomes.

Appropriateness of ICD-coded Diagnostic Inpatient Hospital Discharge Data for Medical Practice Assessment

Objectives: We performed a systematic review to investigate the quality of diagnostic hospital discharge data (DHDD) in order to gain insight in the usefulness of these data for medical practice assessment. We investigated the methods used to evaluate data quality, factors that determine data quality and its consequences for medical practice assessment.

Methods: We selected studies in which both completeness (or sensitivity: SENS) and correctness (or positive predictive value: PPV) were measured. We used the random-effects model to calculate mean SENS and PPV and to explore the effect of a number of covariates.

Results: The 101 included studies were very heterogeneous. We distinguished six typical study designs. We found a mean SENS of 0.67 (95%CI: 0.62– 0.73) and PPV of 0.76 (95%CI: 0.73– 0.79); SENS was significantly lower for comorbidity and complication studies than for some single disease studies. PPV was significantly higher for Scandinavian countries than for other countries. Recoding compared to re-abstracting of the medical record as a gold standard gave a significantly lower PPV. Diagnostic data were considered appropriate by the authors of the studies for quality of care purposes when both SENS and PPV were at least 0.85. Only 13% of the studies fulfilled this criterion.

Conclusions: Variability in quality of care between settings can easily be overshadowed by variability in data quality. However, the use of DHDD by physicians to evaluate their own medical practice may be useful. But only if physicians are willing to critically interpret the meaning of the information for their medical practice assessment.

Utility of electronic medical record for recruitment in clinical research: from rare to common disease

BACKGROUND

Recruitment for clinical trials is a major challenge. Movement disorders, which do not have associated diagnostic laboratory tests, may be especially prone to inaccuracy in coding. Our objective was to evaluate the accuracy of diagnostic codes such as cervical dystonia (CD) and PD in an electronic medical record.

METHODS

Retrospective chart review was performed to confirm the ICD-9 diagnoses of PD, CD and diabetes mellitus type 2 (DM-2), using published clinical diagnostic criteria (PD, CD) and hemoglobin A1c ≥ 6.5 (DM-2).

RESULTS

421 charts (n=129, n=142, n=150 for PD, CD and DM-2, respectively) were reviewed. The accuracy rate was different between all diseases examined with an overall p<0.001. In post hoc pairwise comparisons, the accuracy of DM-2 diagnosis by ICD-9 (96.6%) was greater than CD (88.0%) and both greater than PD (55.0%) (p≤0.003).

CONCLUSIONS

Using an electronic medical record based screening of clinically diagnosed diseases such as CD may be more accurate than previously thought and may identify potential clinical trial participants even without confirmatory lab tests available.

Medicaid admissions and readmissions: understanding the prevalence, payment, and most common diagnoses

Reducing hospital readmissions is a way to improve care and reduce avoidable costs. However, there have been few studies of readmissions in the Medicaid population. We sought to characterize acute care hospital admissions and thirty-day readmissions in the Medicaid population through a retrospective analysis in nineteen states. We found that Medicaid readmissions were both prevalent (9.4 percent of all admissions) and costly ($77 million per state) and that they represented 12.5 percent of Medicaid payments for all hospitalizations. Five diagnostic groups appeared to drive Medicaid readmissions, accounting for 57 percent of readmissions and 49 percent of hospital payments for readmissions. The most prevalent diagnostic categories were mental and behavioral disorders and diagnoses related to pregnancy, childbirth, and their complications, which together accounted for 31.2 percent of readmissions. This analysis, conducted through the Medicaid Medical Directors Learning Network, allows Medicaid medical directors to better understand the nature and prevalence of hospital use in the Medicaid population and provides a baseline for measuring improvement.

Prevalence of second-eye cataract surgery and time interval after first-eye surgery in Iran: a clinic-based study

Purpose:

To determine the prevalence of second-eye senile cataract surgery (SECS) as a proportion of all senile cataract surgeries and the trend in the interval between first and second cataract operations in a main referral and academic eye hospital.

Materials and Methods:

In this cross-sectional study, a list of patients who underwent senile cataract surgery over four consecutive years (2006-2009) was retrieved from hospital computer-based records as the sampling frame. With a systematic random method, 15% of records were selected (1,585 out of 10,517 records).

Results:

First- and second-eye operations were performed in 1,139 (71.9%; 95% confidence interval [CI], 69.5-74.1) and 446 eyes (28.1%; 95% CI, 25.9-30.35), respectively. The proportion of SECS procedures increased from 24.3% in 2006 to 33.4% in 2009 (P = 0.017). The median (interquartile range) interval between the two operations was 9 (4-24) months, which remained stable during the study period. The SECS rate was 10.4% higher (P = 0.01) and the time interval was 13 months shorter (P = 0.007) in patients who underwent phacoemulsification than extracapsular cataract extraction.

Conclusion:

The number of cataract operations in this tertiary eye care setting increased 1.5 fold over the study period. The proportion of second-eye operations also rose from 1/4 to 1/3 during the same time.

Risk of acute kidney injury associated with the use of fluoroquinolones

BACKGROUND

Case reports indicate that the use of fluoroquinolones may lead to acute kidney injury. We studied the association between the use of oral fluoroquinolones and acute kidney injury, and we examined interaction with renin-angiotensin-system blockers.

METHODS

We formed a nested cohort of men aged 40-85 enrolled in the United States IMS LifeLink Health Plan Claims Database between 2001 and 2011. We defined cases as men admitted to hospital for acute kidney injury, and controls were admitted to hospital with a different presenting diagnosis. Using risk-set sampling, we matched 10 controls to each case based on hospital admission, calendar time (within 6 wk), cohort entrance (within 6 wk) and age (within 5 yr). We used conditional logistic regression to assess the rate ratio (RR) for acute kidney injury with current, recent and past use of fluoroquinolones, adjusted by potential confounding variables. We repeated this analysis with amoxicillin and azithromycin as controls. We used a case-time-control design for our secondary analysis.

RESULTS

We identified 1292 cases and 12 651 matched controls. Current fluoroquinolone use had a 2.18-fold (95% confidence interval [CI] 1.74-2.73) higher adjusted RR of acute kidney injury compared with no use. There was no association between acute kidney injury and recent (adjusted RR 0.87, 95% CI 0.66-1.16) or past (RR 0.86, 95% CI 0.66-1.12) use. The absolute increase in acute kidney injury was 6.5 events per 10 000 person-years. We observed 1 additional case per 1529 patients given fluoroquinolones or per 3287 prescriptions dispensed. The dual use of fluoroquinolones and renin-angiotensin-system blockers had an RR of 4.46 (95% CI 2.84-6.99) for acute kidney injury. Our case-time-control analysis confirmed an increased risk of acute kidney injury with fluoroquinolone use (RR 2.16, 95% CI 1.52-3.18). The use of amoxicillin or azithromycin was not associated with acute kidney injury.

INTERPRETATION

We found a small, but significant, increased risk of acute kidney injury among men with the use of oral fluoroquinolones, as well as a significant interaction between the concomitant use of fluoroquinolones and renin-angiotensin-system blockers.

Association of smoking with cardiovascular and infection-related morbidity and mortality in chronic hemodialysis

BACKGROUND AND OBJECTIVES

Smoking is common in the hemodialysis population and is associated with increased all-cause mortality and development of cardiovascular disease. Cause-specific outcomes have not yet been examined in detail. This study investigated the association of baseline smoking status with all-cause, cardiovascular, and infection-related morbidity and mortality in patients undergoing long-term hemodialysis.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

Post hoc analysis of the HEMO Study in patients with available comorbidity, clinical, and nutritional data. Cox proportional hazards regression models were fit to estimate the association of smoking status with mortality. Poisson and negative binomial regression models were fit to estimate the association of smoking status with hospitalization rate.

RESULTS

Complete data were available for 1842 individuals (44% male, 63% black, 45% diabetic). Mean age was 58 ± 14 years. At baseline, 17% were current smokers and 32% were former smokers. After case-mix adjustment, compared with never smoking, current smoking was associated with greater infection-related mortality (hazard ratio [HR], 2.04; 95% confidence interval [CI], 1.32-3.10) and all-cause mortality (HR, 1.44; 95% CI, 1.16-1.79) and greater cardiovascular (incidence rate ratio [IRR], 1.49; 95% CI, 1.22-1.82), infection-related (IRR, 1.35; 95% CI, 1.11-1.64) and all-cause (IRR, 1.43; 95% CI, 1.24-1.65) hospitalization rates. The population attributable fraction (i.e., fraction of observed deaths that may have been avoided) was 5.3% for current smokers versus never-smokers and 2.1% for current versus former smokers.

CONCLUSIONS

Active smoking is prevalent in the chronic hemodialysis population and is associated with greater all-cause, cardiovascular, and infection-related morbidity and mortality.

Postoperative complications following colectomy for ulcerative colitis: a validation study

Background

Ulcerative colitis (UC) patients failing medical management require colectomy. This study compares risk estimates for predictors of postoperative complication derived from administrative data against that of chart review and evaluates the accuracy of administrative coding for this population.

Methods

Hospital administrative databases were used to identify adults with UC undergoing colectomy from 1996–2007. Medical charts were reviewed and regression analyses comparing chart versus administrative data were performed to assess the effect of age, emergent operation, and Charlson comorbidities on the occurrence of postoperative complications. Sensitivity, specificity, and positive/negative predictive values of administrative coding for identifying the study population, Charlson comorbidities, and postoperative complications were assessed.

Results

Compared to chart review, administrative data estimated a higher magnitude of effect for emergent admission (OR 2.52 [95% CI: 1.80–3.52] versus 1.49 [1.06–2.09]) and Charlson comorbidities (OR 2.91 [1.86–4.56] versus 1.50 [1.05–2.15]) as predictors of postoperative complications. Administrative data correctly identified UC and colectomy in 85.9% of cases. The administrative database was 37% sensitive in identifying patients with ≥ 1Charlson comorbidity. Restricting analysis to active comorbidities increased the sensitivity to 63%. The sensitivity of identifying patients with at least one postoperative complication was 68%; restricting analysis to more severe complications improved the sensitivity to 84%.

Conclusions

Administrative data identified the same risk factors for postoperative complications as chart review, but overestimated the magnitude of risk. This discrepancy may be explained by coding inaccuracies that selectively identifying the most serious complications and comorbidities.

Validity of the International Classification of Diseases, Tenth Revision code for acute kidney injury in elderly patients at presentation to the emergency department and at hospital admission

OBJECTIVE

To evaluate the validity of the International Classification of Diseases, Tenth Revision (ICD-10) code N17x for acute kidney injury (AKI) in elderly patients in two settings: at presentation to the emergency department and at hospital admission.

DESIGN

A population-based retrospective validation study.

SETTING

Southwestern Ontario, Canada, from 2003 to 2010.

PARTICIPANTS

Elderly patients with serum creatinine measurements at presentation to the emergency department (n=36 049) or hospital admission (n=38 566). The baseline serum creatinine measurement was a median of 102 and 39 days prior to presentation to the emergency department and hospital admission, respectively.

MAIN OUTCOME MEASURES

Sensitivity, specificity and positive and negative predictive values of ICD-10 diagnostic coding algorithms for AKI using a reference standard based on changes in serum creatinine from the baseline value. Median changes in serum creatinine of patients who were code positive and code negative for AKI.

RESULTS

The sensitivity of the best-performing coding algorithm for AKI (defined as a ≥2-fold increase in serum creatinine concentration) was 37.4% (95% CI 32.1% to 43.1%) at presentation to the emergency department and 61.6% (95% CI 57.5% to 65.5%) at hospital admission. The specificity was greater than 95% in both settings. In patients who were code positive for AKI, the median (IQR) increase in serum creatinine from the baseline was 133 (62 to 288) µmol/l at presentation to the emergency department and 98 (43 to 200) µmol/l at hospital admission. In those who were code negative, the increase in serum creatinine was 2 (-8 to 14) and 6 (-4 to 20) µmol/l, respectively.

CONCLUSIONS

The presence or absence of ICD-10 code N17× differentiates two groups of patients with distinct changes in serum creatinine at the time of a hospital encounter. However, the code underestimates the true incidence of AKI due to a limited sensitivity.

How valid is the AHRQ Patient Safety Indicator "postoperative physiologic and metabolic derangement"?

BACKGROUND

The Agency for Healthcare Research and Quality Patient Safety Indicator postoperative physiologic and metabolic derangement (PMD) uses ICD-9-CM codes to screen for potentially preventable acute kidney injury (AKI) requiring dialysis plus diabetes-related complications after elective surgery. Data on PMD's accuracy in identifying true events are limited. We examined the indicator's positive predictive value (PPV) in the Veterans Health Administration (VA).

STUDY DESIGN

Trained abstractors reviewed medical records of 119 PSI software-flagged PMD cases. We calculated PPVs overall and separately for renal- and diabetes-related complications. We also examined false positives to determine reasons for incorrect identification, and true positives to determine PMD-related outcomes and risk factors.

RESULTS

Overall 75 cases were true positives (PPV 63%, 95% CI 54% to 72%); 73 of 104 AKI cases were true positives (PPV 70%, 60% to 79%); only 2 of 15 diabetes cases were true positives (PPV 13%, 2% to 40%). Of all false positives, 70% represented nonelective admissions and 23% had the complication present on admission. Of AKI true positives, 37% died and 26% were discharged on dialysis; 55% had chronic kidney disease (≥ stage 3) present on admission. Cardiac surgery represented the largest category of AKI-associated index procedures (30%). AKI was most commonly attributed to perioperative renal hypoperfusion (84% of true positives), followed by nephrotoxins (33%) including contrast (11%).

CONCLUSIONS

Due to its low PPV, we recommend removing diabetes complications from the indicator and focusing on AKI. PMD's PPV could be significantly improved by using present-on-admission codes, and specific to the VA, by introduction of admission status codes. Many PMD-identified cases appeared to be at high risk based on patient- and procedure-related factors. The degree to which such cases are truly preventable events requires further assessment.

Correlates of undefined cause of injury coded mortality data in Australia

The objective of this research was to identify the level of detail regarding the external causes of death in Australia and ascertain problematic areas where data quality improvement efforts may be focused. The 2003 national mortality dataset of 12,591 deaths with an external cause of injury as the underlying cause of death (UCOD) or multiple cause of death (MCOD) based on ICD-10 code assignment from death certificate information was obtained. Logistic regression models were used to examine the precision of coded external cause of injury data. It was found that overall, accidents were the most poorly defined of all intent code blocks with over 30% of accidents being undefined, representing 2,314 deaths in 2003. More undefined codes were identified in MCOD data than for UCOD data. Deaths certified by doctors were more likely to use undefined codes than deaths certified by a coroner or government medical office. To improve the quality of external cause of injuries leading to or associated with death, certifiers need to be made aware of the importance of documenting all information pertaining to the cause of the injury and the intent behind the incident, either through education or more explicit instructions on the death certificate and accompanying instructional materials. It is important that researchers are aware of the validity of the data when they make interpretations as to the underlying causes of fatal injuries and causes of injury associated with deaths.

Validation of CKD and related conditions in existing data sets: A systematic review

BACKGROUND

Accurate classification of individuals with kidney disease is vital to research and public health efforts aimed at improving health outcomes. Our objective is to identify and synthesize published literature evaluating the accuracy of existing data sources related to kidney disease.

STUDY DESIGN

A systematic review of studies seeking to validate the accuracy of the underlying data relevant to kidney disease.

SETTING & POPULATION

US-based and international studies covering a wide range of both outpatient and inpatient study populations.

SELECTION CRITERIA FOR STUDIES

Any English-language study investigating the prevalence or cause of kidney disease, existence of comorbid conditions, or cause of death in patients with chronic kidney disease (CKD). All definitions and stages of CKD, including end-stage renal disease (ESRD), were accepted.

INDEX TESTS

Presence of a kidney disease-related variable in existing data sets, including administrative data sets and disease registries.

REFERENCE TESTS

Presence of a kidney disease-related variable defined using laboratory criteria or medical record review.

RESULTS

30 studies were identified. Most studies investigated the accuracy of kidney disease reporting, comparing coded renal disease with that defined using estimated glomerular filtration rate. The sensitivity of coded renal disease varied widely (0.08-0.83). Specificity was higher, with all studies reporting values ≥0.90. Studies evaluating the cause of CKD, comorbid conditions, and cause of death in patients with CKD used ESRD or transplant populations exclusively, and accuracy was highly variable compared with ESRD registry data.

LIMITATIONS

Only English-language studies were evaluated.

CONCLUSIONS

Given the heterogeneous results of validation studies, a variety of attributes of existing data sources, including the accuracy of individual data items within these sources, should be considered carefully before use in research, quality improvement, and public health efforts.

Optimizing algorithms to identify Parkinson's disease cases within an administrative database

Patients assigned the diagnostic ICD-9-CM code for Parkinson's disease (PD) in an administrative database may not truly carry that diagnosis because of the various error sources. Improved ability to identify PD cases within databases may facilitate specific research goals. Experienced chart reviewers abstracted the working diagnosis of all 577 patients assigned diagnostic code 332.0 (PD) during 1 year at a VA Healthcare System. We then tested the ability of various algorithms making use of PD and non-PD diagnostic codes, specialty of clinics visited, and medication prescription data to predict the abstracted working diagnosis. Chart review determined 436 (75.6%) patients to be PD or Possibly PD, and 141 (24.4%) to be Not PD. Our tiered consensus algorithm preferentially used data from specialists over nonspecialists improved PPV to 83.2% (P = 0.003 vs. baseline). When presence of a PD prescription was an additional criterion, PPV increased further to 88.2% (P = 0.04 vs. without medication criterion), but sensitivity decreased from 87.4 to 77.1% (P = 0.0001). We demonstrate that algorithms provide better identification of PD cases than using a single occurrence of the diagnostic code for PD, and modifications of such algorithms can be tuned to maximize parameters that best meet the goals of a particular database query.

Determining perioperative complications associated with vaginal hysterectomy: code classification versus chart review

BACKGROUND

Improvement in outcomes after vaginal hysterectomy (VH) requires accurate identification of complications. We hypothesized that coded data, commonly used to determine morbidity, would miss more complications than chart review would.

STUDY DESIGN

Medical records of women who underwent VH from January 2004 through December 2005 were reviewed for cardiac or respiratory arrest, congestive heart failure, pulmonary edema, pulmonary embolism, urinary tract infection, ureteral obstruction, hemorrhage, and delirium. Complications were identified with use of coded data, in which diagnoses were classified with a modification of the Hospital Adaptation of the International Classification of Diseases.

RESULTS

Records of 712 patients were reviewed. Of the 161 complications identified, 158 (98.1%) were identified through chart review and 48 (29.8%) through coded data. Codes captured all diagnoses of cardiac arrest, respiratory arrest, and pulmonary embolism but missed other complications.

CONCLUSIONS

Codes captured life-threatening complications, but other complications were underestimated or missed entirely. Reliance on coded data for outcomes assessments can be misleading and should be combined with other methods to maximize validity.

Administrative codes combined with medical records based criteria accurately identified bacterial infections among rheumatoid arthritis patients

OBJECTIVE

To evaluate diagnostic properties of International Classification of Diseases, Version 9 (ICD-9) diagnosis codes and infection criteria to identify bacterial infections among rheumatoid arthritis (RA) patients.

STUDY DESIGN AND SETTING

We performed a cross-sectional study of RA patients with and without ICD-9 codes for bacterial infections. Sixteen bacterial infection criteria were developed. Diagnostic properties of comprehensive and restrictive sets of ICD-9 codes and the infection criteria were tested against an adjudicated review of medical records.

RESULTS

Records on 162 RA patients with and 50 without purported bacterial infections were reviewed. Positive and negative predictive values of ICD-9 codes ranged from 54%-85% and 84%-100%, respectively. Positive predictive values of the medical records based criteria were 84% and 89% for "definite" and "definite or empirically treated" infections, respectively. Positive predictive value of infection criteria increased by 50% as disease prevalence increased using ICD-9 codes to enhance infection likelihood.

CONCLUSION

ICD-9 codes alone may misclassify bacterial infections in hospitalized RA patients. Misclassification varies with the specificity of the codes used and strength of evidence required to confirm infections. Combining ICD-9 codes with infection criteria identified infections with greatest accuracy. Novel infection criteria may limit the requirement to review medical records.

Patient-level analysis of outcomes using structured labor and delivery data

This paper presents methods for identifying and analyzing associations among nursing care processes, patient attributes, and patient outcomes using unit-level and patient-level representations of care derived from computerized nurse documentation. The retrospective, descriptive analysis included documented nursing events for 900 Labor and Delivery patients at three hospitals over the 2-month period of January and February 2006. Two models were used to produce quantified measurements of nursing care received by each patient. The first model considered only the hourly census of nurses and patients. The second model considered the size of nurses' patient loads as represented by computerized nurse-entered documentation. Significant relationships were identified between durations of labor and nursing care scores generated by the second model. In addition to the clinical associations identified, the study demonstrated an approach with global application for representing the amount of nursing care received at the individual patient level in analyses of patient outcomes.

Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database

OBJECTIVE

The goal of this study was to assess the validity of the International Classification of Disease, 10th Version (ICD-10) administrative hospital discharge data and to determine whether there were improvements in the validity of coding for clinical conditions compared with ICD-9 Clinical Modification (ICD-9-CM) data.

METHODS

We reviewed 4,008 randomly selected charts for patients admitted from January 1 to June 30, 2003 at four teaching hospitals in Alberta, Canada to determine the presence or absence of 32 clinical conditions and to assess the agreement between ICD-10 data and chart data. We then re-coded the same charts using ICD-9-CM and determined the agreement between the ICD-9-CM data and chart data for recording those same conditions. The accuracy of ICD-10 data relative to chart data was compared with the accuracy of ICD-9-CM data relative to chart data.

RESULTS

Sensitivity values ranged from 9.3 to 83.1 percent for ICD-9-CM and from 12.7 to 80.8 percent for ICD-10 data. Positive predictive values ranged from 23.1 to 100 percent for ICD-9-CM and from 32.0 to 100 percent for ICD-10 data. Specificity and negative predictive values were consistently high for both ICD-9-CM and ICD-10 databases. Of the 32 conditions assessed, ICD-10 data had significantly higher sensitivity for one condition and lower sensitivity for seven conditions relative to ICD-9-CM data. The two databases had similar sensitivity values for the remaining 24 conditions.

CONCLUSIONS

The validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions was generally similar though validity differed between coding versions for some conditions. The implementation of ICD-10 coding has not significantly improved the quality of administrative data relative to ICD-9-CM. Future assessments like this one are needed because the validity of ICD-10 data may get better as coders gain experience with the new coding system.

Sensitivity of International Classification of Diseases codes for hyponatremia among commercially insured outpatients in the United States

Background

Administrative claims are a rich source of information for epidemiological and health services research; however, the ability to accurately capture specific diseases or complications using claims data has been debated. In this study, the authors examined the validity of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes for the identification of hyponatremia in an outpatient managed care population.

Methods

We analyzed outpatient laboratory and professional claims for patients aged 18 years and older in the National Managed Care Benchmark Database from Integrated Healthcare Information Services. We obtained all claims for outpatient serum sodium laboratory tests performed in 2004 and 2005, and all outpatient professional claims with a primary or secondary ICD-9-CM diagnosis code of hyponatremia (276.1).

Results

A total of 40,668 outpatient serum sodium laboratory results were identified as hyponatremic (serum sodium < 136 mmol/L). The sensitivity of ICD-9-CM codes for hyponatremia in outpatient professional claims within 15 days before or after the laboratory date was 3.5%. Even for severe cases (serum sodium ≤ 125 mmol/L), sensitivity was < 30%. Specificity was > 99% for all cutoff points.

Conclusion

ICD-9-CM codes in administrative data are insufficient to identify hyponatremia in an outpatient population.

Effect of complications on mortality after coronary artery bypass grafting surgery: evidence from New York State

OBJECTIVE

Complications are associated with increased risk of death. The objective of this study is to quantify the increased odds of dying from complications after isolated coronary artery bypass grafting surgery.

METHODS

We conducted a retrospective cohort study using the New York State Coronary Artery Bypass Grafting Surgery Reporting System for all patients undergoing isolated coronary artery bypass grafting surgery in New York State who were discharged between 1997 and 1999 (51,750 patients; 2.20% mortality). We estimated the independent effect of individual postoperative complications on in-hospital mortality after controlling for patient clinical risk factors and demographics.

RESULTS

The mortality rate for patients without complication was 0.77% versus 16.1% for patients with complications (P < .001). After adjusting for preoperative risk factors, transmural myocardial infarction (adjusted odds ratio, 7.90; P < .001), respiratory failure (adjusted odds ratio, 6.02; P < .001), renal failure (adjusted odds ratio, 7.15; P < .001), and stroke within 24 hours (adjusted odds ratio, 4.09; P < .001) were the most strongly associated with mortality.

CONCLUSIONS

There is a strong association between postoperative complications and in-hospital mortality. Complications after isolated coronary artery bypass grafting surgery are associated with a 1.4- to 8-fold increase in the odds of death after adjusting for severity of disease and comorbidities. This information might prove valuable to hospitals in their efforts to design quality improvement initiatives and care protocols to improve mortality after coronary artery bypass grafting surgery.

Incidence and prognostic significance of hypoglycemia in hospitalized non-diabetic elderly patients

BACKGROUND AND AIMS

To assess the incidence and prognostic value of hypoglycemia in hospitalized non-diabetic elderly patients.

METHODS

An observational retrospective study, with a 3-year follow-up, was performed in a series of 678 patients aged over 65 years, admitted between January 1 2001 and December 31 2001 to the Units of Gerontology and Geriatrics of the Careggi University Hospital, Florence, Italy. Patients with diabetes mellitus were excluded. To determine the cumulative incidence of hypoglycemia, all measurements of venous or capillary blood glucose during hospital stay were taken into account. In-hospital mortality was determined from hospital discharge records. Information on all-cause, three-year mortality after hospital admission was obtained from the City of Florence Registry Office.

RESULTS

Hypoglycemia was observed in 8.6% of patients, and was asymptomatic in about 25% of cases. In-hospital mortality was significantly higher in patients with hypoglycemia (41.4% vs 14.3%; p<0.001), even after adjustment for potential confounders, including comorbidity, indices of malnutrition, and pharmacological treatment (adjusted OR 2.17[1.25;3.85]). 3-year mortality was significantly higher in patients with hypoglycemia during hospital stay, but the difference was not significant after adjustment for confounders.

CONCLUSIONS

Hypoglycemia is a prognostic marker of in-hospital mortality in non-diabetic hospitalized patients, even after adjustment for comorbidity and indices of malnutrition. Instead, it does not seem to have any relevant independent prognostic value in the longer term.

Performance of International Classification of Diseases, 9th Revision, Clinical Modification Codes as an Adverse Drug Event Surveillance System

BACKGROUND

Adverse drug events (ADEs) are one of the most frequent causes of iatrogenic injury. Because International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes are routinely assigned to inpatient discharges, they could provide a method to detect ADEs within a hospital, a state, and the nation.

OBJECTIVE

The objective of this study was to determine validity of selected ICD-9-CM codes in identifying inpatient ADEs.

RESEARCH DESIGN

An expert panel identified 416 ICD-9-CM codes to represent ADEs (flagged ADE codes). Retrospective chart review using a structured tool was performed to ascertain code performance in detecting ADEs.

SUBJECTS

Subjects included 3103 inpatients from all 41 acute care hospitals in Utah in 2001: 1961 inpatients sampled randomly (random sample) and 1142 inpatients sampled from the discharge records with at least one flagged ADE code (flagged sample).

MEASURES

Measures were ADEs identified by structured review.

RESULTS

The flagged sample yielded 1122 flagged ADE codes recorded in patient charts with 704 representing ADEs (63%). Two hundred eighty-six of the 704 verified ADE codes (41%) were determined to be inpatient ADEs. In the random sample, 32 of 58 ADEs (55%) causing hospital admission were detected by the ADE-flagged codes. Only 23 of 224 inpatient ADEs had been assigned a flagged ADE code (10%).

CONCLUSIONS

Flagged ADE codes have an overall positive predictive value of 63% and detect just over half of ADEs causing hospital admission. These codes have a positive predictive value of 25% for inpatient ADEs but detect only 10% of overall inpatient ADEs. Flagged ADE codes provide an imperfect but immediately available ADE surveillance system.

Validity of International Classification of Diseases, Ninth Revision, Clinical Modification Codes for Acute Renal Failure

Administrative and claims databases may be useful for the study of acute renal failure (ARF) and ARF that requires dialysis (ARF-D), but the validity of the corresponding diagnosis and procedure codes is unknown. The performance characteristics of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes for ARF were assessed against serum creatinine-based definitions of ARF in 97,705 adult discharges from three Boston hospitals in 2004. For ARF-D, ICD-9-CM codes were compared with review of medical records in 150 patients with ARF-D and 150 control patients. As compared with a diagnostic standard of a 100% change in serum creatinine, ICD-9-CM codes for ARF had a sensitivity of 35.4%, specificity of 97.7%, positive predictive value of 47.9%, and negative predictive value of 96.1%. As compared with review of medical records, ICD-9-CM codes for ARF-D had positive predictive value of 94.0% and negative predictive value of 90.0%. It is concluded that administrative databases may be a powerful tool for the study of ARF, although the low sensitivity of ARF codes is an important caveat. The excellent performance characteristics of ICD-9-CM codes for ARF-D suggest that administrative data sets may be particularly well suited for research endeavors that involve patients with ARF-D.

Accuracy of obstetric diagnoses and procedures in hospital discharge data

OBJECTIVE

The objective of the study was to estimate the validity of obstetric procedures and diagnoses in California patient discharge data.

STUDY DESIGN

We randomly sampled 1611 deliveries from 52 of 267 California hospitals that performed more than 678 eligible deliveries in 1992 to 1993. We compared hospital-reported procedures and diagnoses against our recoding of the same records.

RESULTS

Cesarean, forceps, and vacuum delivery were accurately reported, with sensitivities and positive predictive values exceeding 90%. Episiotomy was underreported (70% sensitivity). Cesarean indications were reported with at least 60% sensitivity, except uterine inertia, herpes, and long labor. Among comorbidities, sensitivity exceeded 60% for chorioamnionitis, diabetes, premature labor, preeclampsia, and intrauterine death. Sensitivity was poor (less than 60%) for anemia, asthma, thyroid disorders, mental disorders, drug abuse, genitourinary infections, obesity, fibroids, excessive fetal growth, hypertension, premature rupture, polyhydramnios, and postdates.

CONCLUSION

The validity of hospital-reported obstetric procedures and diagnoses varies, with moderate to high accuracy for some codes but poor accuracy for others.

Coding of Perineal Lacerations and Other Complications of Obstetric Care in Hospital Discharge Data

OBJECTIVE

To assess the validity of obstetric complications, including the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) Core Measure on perineal lacerations, in the California Patient Discharge Data Set.

METHODS

We randomly sampled 1,611 deliveries from 52 of the 267 hospitals that performed more than 678 eligible deliveries in California in 1992-1993. We compared hospital-reported complications against our recoding of the same records.

RESULTS

Third- and fourth-degree perineal lacerations were reported accurately, with estimated sensitivities exceeding 90% and positive predictive values exceeding 65% (weighted to account for the stratified sampling design) or 85% (unweighted). Based on in-depth review of discrepant cases, we estimate the actual positive predictive value at over 90%. Most coding discrepancies were between no injury and first degree, or between first and second degree. Most postpartum complications, including urinary tract and wound infections, endometritis, anesthesia complications, and postpartum hemorrhage were reported with less than 70% sensitivity, but at least 80% positive predictive value. Composite measures from HealthGrades and Solucient, which include these complication codes, also suffer from high false-negative rates.

CONCLUSION

Third- and fourth-degree perineal lacerations are accurately reported on hospital discharge abstracts, confirming the validity of related quality indicators sponsored by the Agency for Healthcare Research and Quality and JCAHO. Administrative data seem less useful for monitoring other in-hospital postpartum complications.

Pressure ulcers in secondary care: incidence, prevalence, and relevance

OBJECTIVE

To compare point and period prevalence rates.

DESIGN

Descriptive, cohort, cross-sectional survey.

PARTICIPANTS

From a cohort of 25,075 cases, information on pressure ulcer status on admission was recorded for 20,283 cases. From 3237 selected cases, the pressure ulcer team made 2234 assessments.

MAIN OUTCOME MEASURES

Point prevalence, period prevalence, and incidence rates.

MAIN RESULTS

The cohort showed a period prevalence rate of 1.4% and an incidence rate of 0.6%. Patients with a pressure ulcer were older, were more likely to have had surgery, had longer hospital stays, and had a higher cost weight. The cross-sectional survey revealed a point prevalence rate of 5.3%. Patients within the cross-sectional survey had longer lengths of stay, were more likely to have had surgery, and presented a higher cost weight in comparison with the cohort.

CONCLUSIONS

In an unselected hospital sample one can expect a period prevalence rate of 2% and a point prevalence rate of 10%. As demonstrated by the present study, differences between the 2 prevalence measurements are mainly due to the confounding of point prevalence rates by length of stay. Length of stay determines the probability of inclusion in a cross-sectional study and should be considered in pressure ulcer trials in the future.

Identifying and distinguishing cases of parkinsonism and Parkinson's disease using ICD-9 CM codes and pharmacy data

Administrative databases have the potential to assess quality and cost of care for parkinsonism and Parkinson's disease. However, the validity of findings is limited by our understanding of how cases are identified. Patient records listing International Classification of Diseases, Version 9, Clinical Modification (ICD-9 CM) codes for parkinsonism (n = 2,076) and dopaminergic medications (n = 2,798) were pulled from fiscal years 1999 to 2001 for patients in the Pacific Northwest Veterans Administration. Samples of these records (n = 397) and records without these ICD-9 CM codes (n = 500) were reviewed, and clinical data were extracted. The accuracy of administrative data to identify and distinguish between Parkinson's disease and parkinsonism was calculated. A total of 37.9% of parkinsonism cases were detected using pharmacy data and ICD-9 CM codes compared to 18.7% by using ICD-9 CM codes alone. The ICD-9 CM code for paralysis agitans (332.0) did not distinguish between probable Parkinson's disease and other causes of parkinsonism, whereas the ICD-9 CM code for degenerative basal ganglia disorder (333.0) predicted having secondary parkinsonism (odds ratio [OR] = 5.0) as well as dopa-responsiveness in patients without secondary parkinsonism (OR = 4.5). Administrative data are limited in the ability to identify parkinsonism. The ICD-9 CM code, 332.0, which is generally considered the code to identify Parkinson's disease, did not distinguish between parkinsonism and Parkinson's disease.

The evolving science of quality measurement for hospitals: implications for studies of competition and consolidation

The literature on hospital competition and quality is young; most empirical studies have focused on few conditions and outcomes. Measures of in-hospital mortality and complications are susceptible to bias from unmeasured severity and transfer/discharge practices. Only one research team has evaluated related process and outcome measures, and none has exploited chart-review or patient survey-based data. Prior studies have generated inconsistent findings, suggesting the need for additional research. We describe the strengths and limitations of various approaches to quality measurement, summarize how quality has been operationalized in studies of hospital competition, outline three mechanisms by which competition may affect hospital quality, and propose measures appropriate for testing each mechanism.

Validity of procedure codes in International Classification of Diseases, 9th revision, clinical modification administrative data

BACKGROUND

Administrative hospital discharge data are widely used to assess quality of care in patients undergoing certain procedures. However, little is known about the validity of administrative coding of procedure data. We conducted a detailed chart review to evaluate the accuracy and completeness of information on procedures in administrative data.

METHODS

We randomly selected 1200 hospital separations in the period April 1, 1996, to March 31, 1997, from administrative discharge data of 3 acute adult hospitals in Calgary, Alberta, Canada. Each separation record in administrative data contains up to 10 procedure coding fields. The corresponding medical charts were reviewed for recording presence or absence of procedures. We then determined sensitivity to quantify the accuracy of coding presence of procedures in administrative data when these are present in the chart data (criterion standard).

RESULTS

The agreement between the 2 databases varied greatly across 35 procedures studied. The sensitivity ranged from 0% to 94%. Of 6 major procedures studied, validity of coding was generally good, with 5 procedures having coding sensitivity of 69% and over and only 1 (lysis of peritoneal adhesion) with a low sensitivity of 41%. In contrast, many minor procedures had low sensitivities. Of 29 minor procedures studied, sensitivity was lower than 50% for 15 procedures, between 50% and 79% for 10, and 80% and over for 4.

CONCLUSION

Validity of information on procedures in administrative discharge data appears to be related to type of procedures. Major procedures that are usually performed in operating rooms are reasonably well-coded. Meanwhile, minor procedures that are routinely performed on wards or in radiology departments are generally undercoded.

How safe is the safety paradigm?

This paper reviews safety initiatives in the health systems of the UK, Canada, Australia, and the US. Initiatives to tackle safety shortcomings involve public-private collaborations. Patient safety agencies (to institute learning, action and safety culture), adverse event reporting and, to a lesser extent, safety related performance indicators are currently used to design safer health systems. Their benefits are mixed, but there is little debate as to their possible side effects. Foreseeable adverse effects of multiple safety organisations stem from them being too many, too vague, too narrowly focused, threatened by the medical practice environment, and too optimistic. Safety related performance indicators are most developed in the US but suffer from inadequacies of administrative data, underreporting, variable indicator definitions, "extended" use, and low sensitivity of the diagnosis coding system, and arguable preventability of the prescribed conditions. A critical appraisal of the implications of these deficiencies is important to assure the safety of current health system safety initiatives and to establish evidence based safety. It is necessary to embed health system safety (as well as patient safety) in the societal culture, structures, and policies which promote effective, user centred, high performance care while allowing for healthy innovation.

How safe is the safety paradigm?

This paper reviews safety initiatives in the health systems of the UK, Canada, Australia, and the US. Initiatives to tackle safety shortcomings involve public-private collaborations. Patient safety agencies (to institute learning, action and safety culture), adverse event reporting and, to a lesser extent, safety related performance indicators are currently used to design safer health systems. Their benefits are mixed, but there is little debate as to their possible side effects. Foreseeable adverse effects of multiple safety organisations stem from them being too many, too vague, too narrowly focused, threatened by the medical practice environment, and too optimistic. Safety related performance indicators are most developed in the US but suffer from inadequacies of administrative data, underreporting, variable indicator definitions, "extended" use, and low sensitivity of the diagnosis coding system, and arguable preventability of the prescribed conditions. A critical appraisal of the implications of these deficiencies is important to assure the safety of current health system safety initiatives and to establish evidence based safety. It is necessary to embed health system safety (as well as patient safety) in the societal culture, structures, and policies which promote effective, user centred, high performance care while allowing for healthy innovation.

Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data

OBJECTIVE

Canadian administrative hospital discharge data contain a diagnosis-type indicator for each coded diagnosis that allows researchers to distinguish complications from pre-existing diagnoses. Given that the validity of diagnosis-type indicators is unknown, we conducted a detailed chart review to evaluate the accuracy of diagnosis-type indicators for flagging complications.

STUDY DESIGN AND SETTING

We obtained administrative hospital discharge data for 1,200 randomly selected adult inpatient separations in Calgary, Alberta, occurring between April 1, 1996 and March 31, 1997. Each discharge record contains up to 16 diagnoses and 16 corresponding diagnosis-type indicators (value of "2"=complication). The corresponding medical charts were reviewed for evidence of diagnoses and complications. A complication was defined as a new diagnosis arising after the start of hospitalization. We determined the extent to which the diagnosis-type indicator in the administrative data agreed with the chart reviewer's assessment (criterion standard) of whether a diagnosis was a complication or not.

RESULTS

The agreement for complications between the two databases varied greatly across 12 conditions studied (kappa range: 0-0.72) and was often low (kappa <0.20 for six conditions). Sensitivity ranged from 0 to 57.1% (higher than 50% for only two conditions), indicating a tendency for complications to often be miscoded as baseline comorbidities. In contrast, specificity was generally high (range: 99.0-100%), suggesting that pre-existing conditions were usually appropriately coded as such in the administrative data.

CONCLUSION

The validity of diagnosis-type indicators in Canadian administrative discharge data appears to be poor for some types of complications. This is likely to be of greatest concern in studies that rely solely on diagnosis-type indicators to define complications as outcomes.

Do Complication Screening Programs Detect Complications Present at Admission?

BACKGROUND

A study was undertaken to verify the accuracy of computer algorithms on administrative data to identify hospital complications. The assessment was based on a medical records indicator that differentiated hospital-acquired conditions from preexisting comorbidities.

METHODS

The indicators for identifying potential hospital complications were applied to all secondary diagnoses to distinguish hospital-acquired from preexisting conditions for all 1997-1998 discharges.

RESULTS

Of the 95 defined complication types, cases were found with secondary diagnoses that met the criteria for 71 different complications. Sixty-nine of these complications had one or more cases with the trigger diagnosis coded as an acquired condition. Thirty-five complications had at least 30 cases with acquired conditions. Hospital complications add greatly to costs; for example, postoperative septicemia increased the hospital bill by more $25,000, added 13 hospital days to the stay, and increased hospital mortality by 16.6%.

CONCLUSIONS

Current complication algorithms identify many cases where the condition was actually present on hospital admission. This fact, coupled with the known variability in coding between institutions, makes comparisons between hospitals on many of the complications problematic. Collection of the present-on-admission flag significantly reduces the noise in monitoring complication rates.

Measuring hospital quality: can medicare data substitute for all-payer data?

OBJECTIVES

To assess whether adverse outcomes in Medicare patients can be used as a surrogate for measures from all patients in quality-of-care research using administrative datasets.

DATA SOURCES

Patient discharge abstracts from state data systems for 799 hospitals in 11 states. National MedPAR discharge data for Medicare patients from 3,357 hospitals. State hospital staffing surveys or financial reports. American Hospital Association Annual Survey.

STUDY DESIGN

We calculate rates for 10 adverse patient outcomes, examine the correlation between all-patient and Medicare rates, and conduct negative binomial regressions of counts of adverse outcomes on expected counts, hospital nurse staffing, and other variables to compare results using all-patient and Medicare patient data.

DATA COLLECTION/EXTRACTION

Coding rules were established for eight adverse outcomes applicable to medical and surgical patients plus two outcomes applicable only to surgical patients. The presence of these outcomes was coded for 3 samples: all patients in the 11-state sample, Medicare patients in the 11-state sample, and Medicare patients in the national Medicare MedPAR sample. Logistic regression models were used to construct estimates of expected counts of the outcomes for each hospital. Variables for teaching, metropolitan status, and bed size were obtained from the AHA Annual Survey.

PRINCIPAL FINDINGS

For medical patients, Medicare rates were consistently higher than all-patient rates, but the two were highly correlated. Results from regression analysis were consistent across the 11-state all-patient, 11-state Medicare, and national Medicare samples. For surgery patients, Medicare rates were generally higher than all-patient rates, but correlations of Medicare and all-patient rates were lower, and regression results less consistent.

CONCLUSIONS

Analyses of quality of care for medical patients using Medicare-only and all-patient data are likely to have similar findings. Measures applied to surgery patients must be used with more caution, as those tested only in Medicare patients may not provide results comparable to those from all-patient samples or across different samples of Medicare patients.

Validity of hospital discharge International Classification of Diseases (ICD) codes for identifying patients with hyponatremia

Medical diagnosis can be studied using various sources of information, such as medical and hospital discharge records and laboratory measurements. These sources do not always concur. The objective of the present study was to assess the sensitivity, specificity, and positive and negative predictive values of hospital discharge diagnosis compared with clinical laboratory data for the identification of hyponatremia. Patients with hyponatremia were selected from a hospital information system determined by the International Classification of Diseases, 9th edition (ICD-9). The validity parameters for hyponatremia (ICD code 276.1) were estimated by comparison with accurate serum sodium (Na+) levels. A total of 2632 cases of hyponatremia were identified using laboratory measurements (Na+ < or =135 mmol/L). The sensitivity of ICD coding for hyponatremia was maximally about 30% for patients with very severe hyponatremia (Na+ < or =115 mmol/L). Corresponding specificities were high (>99%). In 87% of the cases with severe hyponatremia (Na+ < or =125 mmol/L), other discharge ICD codes reflecting severe morbidity were found. This study suggests that ICD codes for hyponatremia represent only one third of the patients admitted to the hospital and experiencing hyponatremia. About two thirds of the patients with hyponatremia were classified as hospitalized for other reasons. To assess the validity of case finding of patients with hyponatremia, the use of analytical techniques, such as certain laboratory measurements, is advisable.

The effects of nurse staffing on adverse events, morbidity, mortality, and medical costs

BACKGROUND

Nurse staffing levels are an important working condition issue for nurses and believed to be a determinant of the quality of nursing care and patient outcomes.

OBJECTIVES

To examine the effects of nurse staffing on adverse events, morbidity, mortality, and medical costs.

METHODS

Using two existing databases, the study sample included 232 acute care California hospitals and 124,204 patients in 20 surgical diagnosis-related groups. The adverse events included patient fall/injury, pressure ulcer, adverse drug event, pneumonia, urinary tract infection, wound infection, and sepsis. Multilevel analysis was employed to examine, simultaneously, the effects of nurse staffing and patient and hospital characteristics on patient outcomes.

RESULTS

Three statistically significant relationships were found between nurse staffing and adverse events. An increase of 1 hour worked by registered nurses (RN) per patient day was associated with an 8.9% decrease in the odds of pneumonia. Similarly, a 10% increase in RN Proportion was associated with a 9.5% decrease in the odds of pneumonia. Providing a greater number of nursing hours per patient day was associated with a higher probability of pressure ulcers. The occurrence of each adverse event was associated with a significantly prolonged length of stay and increased medical costs. Patients who had pneumonia, wound infection or sepsis had a greater probability of death during hospitalization.

CONCLUSION

Patients are experiencing adverse events during hospitalization. Care systems to reduce adverse events and their consequences are needed. Having appropriate nurse staffing is a significant consideration in some cases.

The effect of pneumonia on mortality among patients hospitalized for acute stroke

OBJECTIVE

To determine the effect of pneumonia on 30-day mortality in patients hospitalized for acute stroke.

METHODS

Subjects in the initial cohort were 14,293 Medicare patients admitted for stroke to 29 greater Cleveland hospitals between 1991 and 1997. The relative risk (RR) of pneumonia for 30-day mortality was determined in a final cohort (n = 11,286) that excluded patients dying or having a do not resuscitate order within 3 days of admission. Clinical data were obtained from chart abstraction and were merged with Medicare Provider Analysis and Review files to obtain deaths within 30 days. A predicted-mortality model (c-statistic = 0.78) and propensity score for pneumonia (c-statistic = 0.83) were used for risk adjustment in logistic regression analyses.

RESULTS

Pneumonia was identified in 6.9% (n = 985) of all patients and in 5.6% (n = 635) of the final cohort. The rates of pneumonia were higher in patients with greater stroke severity and features indicating general frailty. Unadjusted 30-day mortality rates were six times higher for patients with pneumonia than for those without (26.9% vs 4.4%, p < 0.001). After adjusting for admission severity and propensity for pneumonia, RR of pneumonia for 30-day death was 2.99 (95% CI 2.44 to 3.66), and population attributable risk was 10.0%.

CONCLUSION

In this large community-wide study of stroke outcomes, pneumonia conferred a threefold increased risk of 30-day death, adding impetus to efforts to identify and reduce the risk of pneumonia in patients with stroke.

Can administrative data be used to compare postoperative complication rates across hospitals?

BACKGROUND

Several quality assessment systems use administrative data to identify postoperative complications, with uncertain validity.

OBJECTIVES

To determine how accurately postoperative complications are reported in administrative data, whether accuracy varies systematically across hospitals, and whether serious complications are more consistently reported.

DESIGN

Retrospective cohort.

SUBJECTS

Nine hundred ninety-one randomly sampled adults who underwent elective lumbar diskectomies at 30 nonfederal acute care hospitals in California in 1990 to 1991. Hospitals with especially low or high risk-adjusted complication rates, and patients who experienced complications, were over sampled.

MEASURES

Postoperative complications were specified by reviewing medical literature and consulting clinical experts; each complication was mapped to ICD-9-CM. Hospital-reported complications were compared with our independent recoding of the same records.

RESULTS

The weighted sensitivity, specificity, and positive and negative predictive values for reported complications were 35%, 98%, 82%, and 84%, respectively. The weighted sensitivity was 30% for serious, 40% for minor, and 10% for questionable complications. It varied from 21% among hospitals with fewer complications than expected to 45% among hospitals with more complications than expected. Only reoperation, bacteremia/sepsis, postoperative infection, and deep vein thrombosis were reported with at least 60% sensitivity. Half of the difference in risk-adjusted complication rates between low and high outlier hospitals was attributable to reporting variation.

CONCLUSIONS

ICD-9-CM complications were underreported among diskectomy patients, especially at hospitals with low risk-adjusted complication rates. The validity of using coded complications to compare provider performance is questionable, even with careful efforts to identify serious events, although these results must be confirmed using more recent data.